Multinomial Distribution

(HC p121)

Recall that the binomial distribution arises when we observe $X$, the number of successes in $n$ independent Bernoulli trials (experiments with only 2 possible outcomes, success and failure). The multinomial distribution arises when each trial has $k$ possible outcomes. We say that the random vector $(X_1,\,X_2,\,\dots ,\,X_{k-1})$ has a $k$- nomial distribution if the joint probability function of $X_1,\,\dots, \,X_{k-1}$ is

$$P(X_1=x_1,\,\dots ,\,X_{k-1}=x_{k-1})=\frac{n!}{x_1! \dots x_k!}p_1^{x_1}p_2^{x_2}\dots p_k^{x_k}$$ (2.11)

where $x_k=n-\sum^{k-1}_{i=1} x_i, \ \sum^k_{i=1}p_i=1$. Note that, if $k=2$, this reduces to the binomial probability function.

Now the joint mgf of the $k$-nomial distribution is

$$M_{X_1,\,\ldots ,\,X_{k-1}}(t_1,\,\ldots ,\,t_{k-1}) = E(e^{X_1t_1+\cdots +X_{k-1}t_{k-1}})$$

$$= (p_1\,e^{t_1}+\cdots + p_{k-1}\,e^{t_{k-1}}+p_k)^n.$$ (2.12)

To show this, multiply the RHS of (2.11) by $e^{x_1t_1+\cdots +x_{k- 1}t_{k-1}}$ and sum over all $(k-1)$-tuples, $(x_1,\,\ldots ,\,x_{k-1})$. [HC deals with this for $k=3$ on page 122.]

Comments

1. When $k=2$, (2.12) agrees with the familiar form of the mgf of a binomial $(n,p)$ distribution.
2. Note that the marginal mgf of any $X_i$ (obtained by putting the other $t_i$ equal to $0$) is the familiar mgf of the binomial distribution.