Statistical Inference-Notes-Part6-Wilks' Theorem

Keywords: Likelihood ratio tests, Wilks’ Theorem, profile likelihood

1. Likelihood ratio test on an open subset of $\mathbb{R}^d$

We have shown the likelihood ratio test function is the most powerful test for simple hypothesis. Let’s consider a multiparameter problem in which $\theta = (\theta_1, \dots, \theta_d) \in \Theta$ forming an open subset of $\mathbb{R}^d$ and suppose we want to test of the from:

$H_0: \theta_1 = \theta_1^0, \dots, \theta_m = \theta_m^0$

against the alternative $H_1$ in which $\theta_1, \dots, \theta_d$ are unrestricted. Here $1 \le m \le d$ and $\theta_1^0, \dots, \theta_m^0$ are known prescribed values. (联合假设检验)

Let $L_1$ denote $\sup\{L(\theta): \theta \in \Theta\}$ and $L_0$ denote $\sup\{L(\theta): \theta \in \Theta_0\}$.

Define the likelihood ratio statistic $T_n = 2\log \Big(\frac{L_1}{L_0}\Big)$

The notation indicates dependence on the sample size $n$.

2. Wilks’ Theorem

Let’s assume the $L(\theta)$ be at least twice continuously differentiable in all its components in some neighborhood of the true value of $\theta$, and that the Fisher information matrix be well-defined and invertible. Suppose $H_0$ is true. Then, as $n \xrightarrow{} +\infty$, then $T_n \xrightarrow{} \chi^2_{m}$ (卡方分布的自由度是参数约束的个数)

Proof:

$T_n = 2\{l_n(\hat{\theta}_n) - l_n(\theta_0)\} = 2(\hat{\theta_n} - \theta_0)l_n^{\prime}(\hat{\theta}_n) - \frac{2}{2}(\hat{\theta_n} - \theta_0)^2l^{\prime\prime}(\theta_n^{*})$ (Lagrange remainder of $\log$, where $\theta_n^{*} \in N(\theta_0, |\theta_n - \theta_0|)$)

Since we assume the $\hat{\theta}_n$ is the solution of regular function, we can quickly seen that:

$T_n = n i_1(\theta_0) (\hat{\theta}_n - \theta_0)^2 \frac{l^{\prime\prime}(\theta_n^{*})}{l^{\prime\prime}(\theta_0 )}\frac{l^{\prime\prime}(\theta_0)}{-n i_1(\theta_0)}$.

By the SLLW and strong consistency of likelihood estimator, $\frac{l^{\prime\prime}(\theta_n^{*})}{l^{\prime\prime}(\theta_0 )}, \frac{l^{\prime\prime}(\theta_0)}{-n i_1(\theta_0)} \xrightarrow{} 1$. According to CLT, we can conclude that $n i_1(\theta_0) (\hat{\theta}_n - \theta_0)^2$ follows $\chi^2$ distribution. The Slutsky’s Lemma established the required result.

Reference

Essentials of Statistics Inference