The Likelihood Ratio Test Procedure

The notion of using the magnitude of the ratio of two probability density functions as the basis of a best test or of a uniformly most powerful test can be modified, and made intuitively appealing, to provide a method of constructing a test where either or both of the hypothesis and alternative are composite. The method leads to tests called likelihood ratio tests, and although not necessarily uniformly most powerful, they often have desirable properties. The test involves a comparison of the maximum value the likelihood can take when $\theta$ is allowed to take any value in the parameter space, and the maximum value of the likelihood when $\theta$ is restricted by the hypothesis. Define

$$\Lambda=\max_{H_0}L(\theta)/\max_{H_0 \cup H_1}L(\theta).$$ (9.5)

Note that

(i)

$\theta$ may be a vector of parameters;

(ii)

Both numerator and denominator (and hence $\Lambda$) are functions of the sample values $x_1, \ldots, x_n$, and the right hand side could be written more fully as

$$\max_{\theta \in H_0}f({\bf x}; \theta)/\max_{\theta \in H_0 \cup H_1}f({\bf x}, \theta).$$

Strictly speaking, $\Lambda$ as defined in (3.5) is a function of random variables $X_1, \ldots, X_n$ and so is itself a random variable with a probability distribution. When ${\bf X}$ is replaced by the observed values ${\bf x}$ in the ratio, we will use $\lambda$ for the observed value of $\Lambda$, and both will be called the likelihood ratio.

Clearly, by the definition of maximum likelihood estimates, $\max_{H_0 \cup H_1} L(\theta)$ will be obtained by substituting the mle('s) for $\theta$ into $L(\theta)$. Note that

(i)

$\lambda \geq 0$ since it is a ratio of pdf's;

(ii)

$\max_{H_0}L(\theta) \leq \max_{H_0 \cup H_1}L(\theta)$ since the set $H_0$ over which L($\theta$) is maximized is a subset of $H_0 \cup H_1$. This means that $\lambda \leq 1$.

So the random variable $\Lambda$ has a probability distribution on [0,1]. If, for a given sample $x_1, \ldots, x_n$, $\lambda$ is close to $1$, then $\max_{H_0}L(\theta)$ is almost as large as $\max_{H_0 \cup H_1} L(\theta)$. This means that we can't find an appreciably larger value of the likelihood, L($\theta$), by searching for a value of $\theta$ through the entire parameter space $H_0 \cup H_1$ supports the proposition that $H_0$ is true. On the other hand, if $\lambda$ is small, we note that the observed $x_1, \ldots, x_n$ was unlikely to occur if $H_0$ were true, so the occurrence of it casts doubt on $H_0$. So a value of $\lambda$ near zero implies the unreasonableness of the hypothesis.

Let the random variable $\Lambda$ have probability density function g($\lambda$), $0 \leq \lambda \leq 1$. To carry out the LR test in a given problem involves finding a value $\lambda_0$ ($< \ 1$) so that the critical region for a size $\alpha$ test is $\{\lambda: 0 < \lambda < \lambda_0 \}$. That is,

$$P(\Lambda \leq \lambda_0) = \int_0^{\lambda_0}g(\lambda)d\lambda=\alpha.$$ (9.6)

Since the distribution of $\Lambda$ is generally very complicated, we would appear to have a difficult problem here. But in many cases, a certain function of $\Lambda$ has a well-known distribution and an equivalent test can be carried out. [See Examples 9.3.3- 9.3.3below.] Cases where this is not so are dealt with in sub-section 9.3.4.