Moment Generating Function

We will now derive the mgf for a $p$-variate normal distribution and see how it can be used in deriving other results.

Theorem 4..2

Given ${\bf X} \sim N_p(\mbox{\boldmath$\mu$}, \, \mbox{\boldmath$\Sigma$})$ and ${\bf t}' =(t_1,\,t_2,\,\dots ,\,t_p)$ a vector of real numbers, then the mgf of ${\bf X}$ is

$$M_{\bf X} ({\bf t}) = e^{{\bf t}'\mbox{\boldmath$\mu$}+\frac 12 {\bf t}'\mbox{\boldmath$\Sigma$}{\bf t}} \ .$$ (4.3)

Proof
There exists a non-singular matrix ${\bf P}$ so that $\mbox{\boldmath$\Sigma$}={\bf PP}'$. Let ${\bf Y}={\bf P}^{-1}({\bf X}- \mbox{\boldmath$\mu$})$. Then ${\bf Y}\sim N_p ({\bf0},\,{\bf I})$ from the proof of Theorem 4.2. That is, each $Y_i \sim N(0,1)$ and we know that $M_{Y_i}(t)=E(e^{Y_it})=e^{\frac 12 t^2}$. Now

$$\begin{eqnarray*} M_{\bf Y}({\bf t}) & = & E(e^{Y_1t_1+\dots +Y_pt_p}) =E(e^{{\... ...e^{\frac 12 \sum t^2_i}\\ & = & e^{\frac 12 {\bf t}'{\bf t}} \end{eqnarray*}$$

Also

$$\begin{eqnarray*} M_{\bf X}(t) & = & E(e^{{\bf t}'{\bf X}})\\ & = & E(e^{{\bf... ...\ \ \mbox{\boldmath$\Sigma$} \ \ \mbox{for} \ \ {\bf PP}' \ . \end{eqnarray*}$$

Comments

1. Note that when $p=1$, Theorem 4.2 reduces to the familiar result for a univariate normal.
2. If X is multivariate normal with diagonal covariance matrix, then the components of X are independent.
3. The marginal distributions of a multivariate normal are all multivariate (or univariate) normal. eg.
   $$\begin{eqnarray*} \left[ \begin{array}{c} x_1 \\ x_2 \end{array} \right] & \sim... ...& N(\mu_1,\Sigma_{11}) \\ x_2 & \sim & N(\mu_2,\Sigma_{22}) \\ \end{eqnarray*}$$
   
   
   
   
   

4. If X is multivariate normal, then AX is multivariate normal for any matrix A (of appropriate dimension).
   $$\begin{eqnarray*} \mbox{For a r.v}\ X \ \mbox{where}& E(X)=\mu, & \mbox{var}(X) = \Sigma \ , \\ &E(AX) = A \mu, & \mbox{var}(AX)= A \Sigma A' \end{eqnarray*}$$
   
   
   
   
   

5. We also note for future reference the conditional distributions,
   $$\begin{eqnarray*} (x_2\vert x_1) & \sim & N(\mu_{2.1}, \Sigma_{22.1}) \\ \mbox{... ...2.1} & = & \Sigma_{22} - \Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12} \end{eqnarray*}$$