Randomized Tests

It will be recalled that for hypothesis testing problems involving discrete distributions, it is usually not possible to choose a critical region consisting of realizable values of the statistic of size exactly $\alpha$, where $\alpha$ is some prescribed value.

In the hypothesis testing procedures considered so far, the sample space of observations X is partitioned into 2 regions, C and $\overline{C}$ (its complement). We can express this in terms of a function $\psi$ as follows. Let

$$\psi({\bf x})=P(\mbox{reject }H_0 \mbox{ when }{\bf X}={\bf x}).$$

For a non-randomized test with rejection region C, $\psi$ for a region C is just its indicator function. That is,

$$\psi({\bf x})=\left\{ \begin{array}{ll} 1 & \mbox{if }{\bf x} \in C\\ 0 & \mbox{if }{\bf x} \notin C. \end{array} \right.$$

We will extend this, to allow for some different action (other that ``reject'' and ``accept'') if the outcome x is on the boundary of the critical region. The other action effectively is performing an auxiliary experiment such as tossing a coin with P(heads)$=$p; if heads results, reject $H_0$; if tails results, $H_0$ is accepted. The value of $p$ is chosen to make the P(rejecting $H_0$) the desired value.

More formally, for a test with critical region C and a value of ${\bf X}= {\bf x}_0$ on the boundary, we may define

$$\psi({\bf x})=\left\{ \begin{array}{ll} 1 & \mbox{if }{\bf x} \in... ...if }{\bf x} \neq {\bf x}_0 \mbox{ and } {\bf x} \notin C\\ \end{array} \right.$$ (9.4)

where $p$ ($0<p<1$) is appropriately chosen.

Example 9..2
Suppose $X$ has a Poisson distribution with mean $\lambda$. A sample of size $n=10$ is used to test H$_0$: $\lambda=.1$ against H$_1$: $\lambda>.1$. Noting that $\sum_{i=1}^{10}X_i$ ($= Y$, say) has a Poisson distribution with mean $1$, the test is to reject $H_0$ for large values of $Y$. Suppose we wish to have a significance level of $\alpha=.05$. Now $P(Y \geq 3)=.080$ and $P(Y \geq 4)=.019$. The desired significance level can be achieved by the test

$$\begin{eqnarray*}\psi(y)=\left\{ \begin{array}{ll} 1 & \mbox { if } y \geq 4 \\ ... ...box { if } y = 3 \\ 0 & \mbox { if } y < 3. \end{array} \right. \end{eqnarray*}$$

Now $p$ is found as follows.

$$\begin{eqnarray*}P(H_0 \mbox{ is rejected}) &=& 1 \times P(Y \geq 4) + p \times ... ...(.080 - .019)p\\ &=& .019 + .061p\\ &=&.05 \mbox{ if }p=31/61. \end{eqnarray*}$$

Comments. Not many statisticians like randomized tests, in practice, because the use of them means that $2$ statisticians could make the same assumptions, observe the same data, apply the same test, yet come to different conclusions. This is perhaps another reason why it is desirable to report a P-value for an experiment (as indicated in 3.1.4) rather than arbitrarily choose a value of $\alpha$, and use a somewhat contrived method to achieve a test with exactly this $\alpha$.

Example 9..3

Given a random variable $X$ has a uniform distribution on $(a,b)$. We wish to test the ``simple'' $H_0: a=1, b=3$ against the ``simple'' $H_1:a=2,b=6$, using a sample of size $1$ and a randomized test with $\alpha=.05$. Graphically the two alternative situations are shown below.

Clearly any sensible decision rule would include

If $x \in (1,2)$ $H_0$ should be accepted (since $H_1$ can't possibly be true)
If $x \in (3,6)$ $H_0$ should be rejected

But if $x \in (2,3)$ we will reject $H_0$ sometimes and accept $H_0$ sometimes. That is consider the function $\psi$ as follows:

$$\begin{eqnarray*} \psi&=&1 \ \mbox{ if } x \in (3,6)\\ &=&p \ \mbox{ if } x \in [2,3]\\ &=&0 \ \mbox{ if } x \in (1,2) \end{eqnarray*}$$

where $p$ is chosen so that $\alpha=.05$. Now

$$\begin{eqnarray*} \alpha&=&\mbox{P(rejecting }H_0\vert H_0 \mbox{ is true})\\ &... ...^3 \frac{1}{2} dx\\ &=&\frac{1}{2}p \, = \, .05 \mbox{ if }p=.1 \end{eqnarray*}$$

The power of the test is

$$P(\mbox{rejecting }H_0\vert H_1 \mbox{ is true})=1 \times \in... ... \, + \, .1 \times \int_2^3 \frac{1}{4}dx \, = \, \frac{31}{40}$$