Asymptotic Distribution of $-2\log \Lambda$

The distribution of some function of $\Lambda$ can't always be found as readily as in the previous examples. If $n$ is large and certain conditions are satisfied, there is an approximation to the distribution of $\Lambda$ that is satisfactory in most large-sample applications of the test. We state without proof the following theorem.

Theorem 9..2

Under the proper regularity conditions on $f(x;\theta)$, the random variable $-2\log \Lambda$ is distributed asymptotically as chi-square. The number of degrees of freedom is equal to the difference between the number of independent parameters in $H_0 \cup H_1$ and $H_0$.

[Note that in Example 9.3.1 the distribution of $-2\log \Lambda$ was exactly $\chi^2_1$.]

Example 9..11
(Test for equality of several variances.)

The hypothesis of equality of variances in two normal distributions is tested using the $F$-test. We will now derive a test for the $k$-sample case by the likelihood ratio procedure. Consider independent samples

$$\begin{array}{lll} x_{11},\,x_{12},\,\dots ,x_{1n_1} & \mbox... ... ,\,x_{kn_k} & \dotfill & N(\mu_k,\,\sigma^2_k). \end{array}$$

That is, we have observations $\{x_{ij}, \ j=1,\, \dots ,\, n_i, \ i=1,\,2,\,\dots ,\,k\}$.

We wish to test the hypothesis

$$H_0:\sigma^2_1=\sigma^2_2=\dots = \sigma^2_k \ (=\sigma^2)$$

against the alternative that the $\sigma^2_i$ are not all the same. Let $n=\sum_in_i$. Now the p.d.f. of the random variable $X_{ij}$ is

$$f(x_{ij})=(2\pi )^{-1/2}\sigma^{-1}_{i}\,\exp\left\{ -{\textstyle\frac 12} (x_{ij} -\mu_i)^2 / \sigma^2_i \right\}.$$

So the likelihood function of the samples above is

$$L(\mu,\,\sigma^2)=(2\pi )^{-n/2}\sigma^{-n_1}_1 \dots \sig... ...k_{i=1}\sum^{n_i}_{j=1}(x_{ij}-\mu_i)^2 / \sigma^2_i\right\}.$$ (9.11)

The whole parameter space and restricted parameter space are given by

$$\begin{eqnarray*} H_0 \cup H_1 & = & \{ (\mu_i,\,\sigma^2_i): \ \mu_i\in (- \in... ...,\infty ), \ \sigma^2\in (0,\,\infty ), \ i=1,\,\dots ,\,k\}. \end{eqnarray*}$$

The log of the likelihood is

$$\log L = -\frac n2 \log 2\pi -\frac{n_1}{2} \log \sigma^2_1 -... ... 12 \sum^k_{i=1}\sigma^{- 2}_i\sum^{n_i}_{j=1}(x_{ij}-\mu_i)^2,$$

using $L$ for $L(\mu ,\,\sigma^2)$.

To find $${\max_{H_O\cup H_1} L}$$ we need the MLE's of the $2k$ parameters $\mu_1,\,\dots ,\,\mu_k, \ \sigma^2_1,\,\dots ,\, \sigma^2_k$.

$$\partial\log L / \partial \mu_i=\frac 12\sigma_i^{- 2}2\sum^{n_i}_{j=1}(x_{ij}-\mu_i), \ \ i=1,\,\dots ,\,k$$ (9.12)

$$\partial \log L / \partial \sigma^2_i = -(n_i / 2\sigma^2_i... ...^4} )\sum^{n_i}_{j=1}(x_{ij} -\mu_i )^2, \ i=1,\,\dots ,\,k$$ (9.13)

Equating (9.12) and (9.12) to zero and solving we obtain

$$\hat{\mu}_i = \frac{1}{n_i} \sum^{n_i}_{j=1} x_{ij} =\overline{x}_{i\cdot}\,, \ \ i=1,\,\dots ,\,k$$ (9.14)

$$\hat{\sigma}^2_i = \frac 1{n_i}\sum^{n_i}_{j=1} (x_{ij}- \overline{x}_{i\cdot} )^2\,, \ \ i=1,\,\dots ,\,k.$$ (9.15)

Substituting these in (9.11) we obtain

$$\max_{H_0\cup H_1} L = (2\pi )^{-n / 2}\hat{\sigma}^{-n_1}_1 \dots \hat{\sigma}^{-n_k}_k... ...ne{x}_{i\cdot})^2n_i}{\sum^{n_i}_{j=1}(x_{ij}- \overline{x}_{i\cdot})^2}\right]$$

$$= (2\pi )^{-n / 2}\hat{\sigma}^{-n_1}_1\dots \hat{\sigma}^{-n_k}_k\,e^{-n / 2},\mbox{ since } n=\sum_{i=1}^k n_i.$$ (9.16)

Now in the restricted parameter space $H_0$ there are $(k+1)$ parameters, $\mu_1,\,\dots ,\,\mu_k$ and $\sigma^2$. So we need to find the mle's of these parameters. The likelihood function now is (putting $\sigma^2_i=\sigma^2$, all $i$)

$$L=(2\pi )^{-n/2}(\sigma^2)^{-n/2}\,\exp\left\{ - \frac{1}{2\sigma^2}\sum^k_{i=1}\sum^{n_i}_{j=1}(x_{ij}-\mu_i)^2\right\}$$ (9.17)

and

$$\log L = -\frac n2 \log (2\pi )-\frac n2\log \sigma^2-\frac {1}{2\sigma^2}\sum^k_{i=1}\sum^{n_i}_{j=1}(x_{ij}-\mu_i)^2$$

$$\frac{\partial \log L}{\partial \mu_i} = - \frac{1}{2\sigma^2}(- 2)\sum^{n_i}_{j=1}(x_{ij}-\mu_i), \ \ i=1,\,\dots ,\,k$$ (9.18)

$$\frac{\partial \log L}{\partial \sigma^2} =-\frac{n}{2\sig... ...c{1}{2\sigma^4} \sum^k_{i=1}\sum^{n_i}_{j=1}(x_{ij}-\mu_i)^2$$ (9.19)

Equating (9.18) and (9.19) to zero and solving we obtain

$$\tilde{\mu}_i = \frac 1{n_i}\sum^{n_i}_{j=1} x_{ij}=\overline{x}_{i\cdot} \ (=\hat{\mu_i}), \ \ i=1,\,\dots ,\,k$$ (9.20)

$$\tilde{\sigma}^2 = \frac 1n \sum^k_{i=1}\sum^{n_i}_{j=1}(x_{ij}- \overline{x}_{i\cdot})^2$$

$$= \frac 1n \sum^k_{i=1} n_i \hat{\sigma}^2_i.$$ (9.21)

Substituting (9.21) and (9.21) into (9.17) we obtain

$$\max_{H_0} L = e^{-n/2}(2\pi )^{-n/2} / \left[ \sum^k_{i=1} n_i \hat{\sigma}^2_i / n \right]^{n/2}$$ (9.22)

So

$$\lambda = \frac{\hat{\sigma}^{n_1}_{1}\dots \hat{\sigma}^{... ...hat{\sigma}^2_i)^{n_i/2}\right] / (\tilde{\sigma}^2)^{n /2}$$ (9.23)

Now, using Theorem 9.3.4, the distribution of $-2\log \Lambda$ is asymptotically $\chi^2$. To determine the number of degrees of freedom we note that the number of parameters in $H_0 \cup H_1$ is $2k$ and in $H_0$ is $k+1$. Hence the number of degrees of freedom is $2k-(k+1)=k-1$. Thus

$$-2\log\Lambda = -\sum^k_{i=1} n_i\log \hat{\sigma}^2_i+n\log\tilde{\sigma}^2$$ (9.24)

is distributed approximately $\chi^2_{k-1}$.

Bartlett (1937) modified this statistic by using unbiased estimates of $\sigma^2_i$ and $\sigma^2$ instead of MLE's. That is, he used $(n_i-1)$ and $(n-k)$ as divisors, so the statistic becomes

$$B=-\sum^k_{i=1} \nu_i\log s^2_i+\left( \sum^k_{i=1}\nu_i \right) \log s^2$$

where

$$s^2_i = \sum^{n_i}_{j=1}(x_{ij}-\overline{x}_{i\cdot})^2 / (n... ...s^2=\frac{\nu_1s^2_1+\dots +\nu_ks^2_k}{\nu_1 +\dots + \nu_k}$$

[Investigate the form of this when $k=2$.]

A better approximation still is obtained using as the statistic $Q=B/C$ where the constant $C$ is defined by

$$C=1+\frac{1}{3(k-1)}\left[ \sum_i \frac{1}{\nu_i}-\frac{1}{\sum_i\nu_i} \right]$$

and this statistic is commonly referred to as Bartlett's statistic for testing homogeneity of variances. That is,

$$Q=\frac{\left( \sum_i\nu_i \right)\log s^2 -\sum_i \nu_i\lo... ...c{1}{3(k-1)}\left[ \sum_i(1/\nu_i)-(1 / \sum_i\nu_i)\right]}$$ (9.25)

is distributed approximately as $\chi^2_{k-1}$ under the hypothesis $H_0:\sigma^2_1=\dots =\sigma^2_k$. The approximation is not very good for small $n_i$.