Certain Best Tests

When $H_0$ and $H_1$ are both simple, the error sizes $\alpha$ and $\beta$ are uniquely defined. In this section we require that both the null hypothesis and alternative hypothesis are simple, so that in effect, the parameter space is a set consisting of exactly $2$ points. We will define a best test for testing $H_0$ against $H_1$, and in 3.2.3 we will prove a Theorem that provides a method for determining a best test.

Let $f(x;\theta)$ denote the density function of a random variable $X$. Let $X_1, X_2, \ldots, X_n$ denote a random sample from this distribution and consider the simple hypothesis
$H_0$: $\theta= \theta_0$ and the simple alternative $H_1$: $\theta=\theta_a$. So $H_0 \cup H_1=\{\theta_0, \ \theta_a\}.$

One repetition of the experiment will result in a particular n-tuple, ( $x_1,x_2,\ldots,x_n$). Consider a set $C_i$, which is a collection of n-tuples having size $\alpha$ that is, $C_i$ has the property that

$$P[(X_1,X_2,\ldots,X_n) \ \in \ C_i\vert H_0 \mbox{ is true }] = \alpha.$$

It follows that $C_i$ can be thought of as a critical region for the test. Specifically, if the observed n-tuple $(x_1,x_2, \ldots, x_n)$ falls in our pre-selected $C_i$, we will reject $H_0$. However, if $H_A$ were true, then intuitively the `best' critical region would be the one having the highest probability of containing $(x_1,x_2, \ldots, x_n)$. Formalizing this notion, we have the following definition.

Definition 9..5
C is called the best critical region, (BCR) of size $\alpha$ for testing the simple $H_0$ against the simple $H_1$ if,

(a)

$P[(X_1,X_2,\ldots,X_n) \ \in \ C\vert H_0]=\alpha,$

(b)

$P[(X_1,\ldots,X_n) \ \in \ C\vert H_1 ] \geq P[(X_1,\ldots,X_n) \ \in \ C_i\vert H_1]$ for every other $C_i$ (of size $\alpha$).

This definition can be stated in terms of power. Suppose that there is one of these subsets, say C, such that when $H_0$ is true, the power of the test associated with C is at least as great as the power of the test associated with each other $C_i$.

Definition 9..6
A test of the simple hypothesis $H_0$ versus the simple alternative $H_1$ that has the smallest $\beta$ (or equivalently, the largest $\pi(\theta)$) among tests with no larger $\alpha$ is called most powerful.

Example 9..4
Suppose $X \sim\mbox{bin}(5,\theta)$. Let $f(x;\theta)$ denote the probability function of $X$. Consider $H_0:\theta=\frac{1}{2}$, $H_1:\theta=\frac{3}{4}$. The table below gives the values of $f(x;\frac{1}{2})$, $f(x;\frac{3}{4})$ and $f(x;\frac{1}{2})/f(x;\frac{3}{4})$ for $x=0, 1, \ldots,5$.

x	0	1	2	3	4	5
f(x;$\frac{1}{2}$)	$\frac{1}{32}$	$\frac{5}{32}$	$\frac{10}{32}$	$\frac{10}{32}$	$\frac{5}{32}$	$\frac{1}{32}$
f(x;$\frac{3}{4}$)	$\frac{1}{1024}$	$\frac{15}{1024}$	$\frac{90}{1024}$	$\frac{270}{1024}$	$\frac{405}{1024}$	$\frac{243}{1024}$
f(x;$\frac{1}{2}$)/f(x;$\frac{3}{4}$)	32	$\frac{32}{3}$	$\frac{32}{9}$	$\frac{32}{27}$	$\frac{32}{81}$	$\frac{32}{243}$

Using $X$ to test $H_0$ against $H_1$, we shall first assign significance level $\alpha=1/32$ and want a best critical region of this size. Now $C_1=\{x:\,x=0\}$ and $C_2=\{x:\,x=5\}$ are possible critical regions and there is no other subset with $\alpha=1/32$. So either $C_1$ or $C_2$ is the best critical region for this $\alpha$. If we use $C_1$ then P($x \in C_1\vert H_1$)$=$1/1024 and

P(rejecting $H_0\vert H_1$ is true) $\ll$ P(rejecting $H_0\vert H_0$ is true),

an unacceptable situation. On the other hand, if we use $C_2$ then $P(X \in A_2\vert H_A)=243/1024$ and

P(rejecting $H_0\vert H_1$ is true) $\gg$ P(rejecting $H_0\vert H_0$ is true),

a much more desirable state of affairs. So $C_2$ is the best critical region of size $\alpha=1/32$ for testing $H_0$ against $H_1$. It should be noted that, in this problem, the best critical region, C, is found by including in C the point (or points) at which $f(x;\frac{1}{2})$ is small in comparison with $f(x;\frac{3}{4})$. This suggests that in general, the ratio $f(x;H_0)/f(x;H_1)$ provides a tool by which to find a best critical region for a certain given value of $\alpha$.

The theorem in the following section provides the methodology for deriving the most powerful test for testing simple $H_0$ against simple $H_1$.