Moment Generating Functions (mgf)

Moments are defined as $\mu'_r=E(Y^r)$ and central moments about $\mu$ as $\mu_r=E(Y-\mu)^r$ for $r=1,2 \ldots$. These are entities by which we start reducing data.

$$\begin{eqnarray*} \mu_1& \equiv& \mbox{mean} \\ \mu_2& \equiv& \mbox{variance} ... ... \mu_6 \\ \vdots \end{array} \right\} && \mbox{no special names} \end{eqnarray*}$$

Often $\mu_1$ and $\mu_2$ are enough to summarize the data. However, fourth moments and their counterparts, cumulants, are needed to find the variance of a variance. Moment generating functions give us a way of determining the formula for a particular moment. But they are more versatile than that, see below.

It will be recalled that in the univariate case, random variable $X$ has mgf defined by

$$M_X(t)=E(e^{Xt})$$

for values of $t$ for which the series or the integral converges.

We will now consider the mgf for a random vector ${\bf X}'=(X_1,\,X_2,\,\dots ,\,X_p)$. The moment generating function of ${\bf X}$ is defined by

$$M_{\bf X}(t_1,\,\dots ,\, t_p) = E(e^{X_1t_1+\dots +X_pt_p)}$$

$$= E(e^{{\bf X}'{\bf t}})$$ (2.9)

where ${\bf t}' =(t_1,\,t_2,\,\dots ,\,t_p)$. Of course $E(e^{{\bf X}'{\bf t}})$ could be written $E(e^{{\bf t}'{\bf X}})$.

Read HC 2.4 from Theorem 4 to the end. Note in particular how multivariate mgf's can be used to find moments (including product moments), to find marginal distributions of one or more variables, and to prove independence. These are summarized below.

1. 
   

   $$\left. \frac{\partial^{s_1+s_2}M_{X,Y}(t_1,t_2)}{\partial ... ...tial t^{s_2}_2}\right\vert _{t_1=t_2=0} = E(X^{s_1}Y^{s_2}).$$ (2.10)

   
   

   
   

   The obvious extension can be made to the case of $p \ (>2)$ variables.

2. The marginal distributions for subsets of the $p$ components have mgf's obtained by setting equal to zero those $t_i$'s that correspond to the variables not in the subset. For example, if ${\bf X}'=(X_1,\,X_2,\,X_3,\,X_4)$ has mgf $M_{\bf X}(t_1,\,t_2,\,t_3,\,t_4)$, then
   
   $$M_{X_2,X_3}(t_2,t_3)=M_{\bf X}(0,\,t_2,\,t_3,\,0).$$
   
   

3. If the random variables $X_1,\,X_2,\,\dots ,\,X_p$ are independent, then
   
   $$M_{\bf X}({\bf t})=M_{X_1}(t_1)M_{X_2}(t_2)\dots M_{X_p}(t_p)$$
   
   
   and the converse is also true.