The role of c.g.f.

We use the cumulant generating function as a mathematical tool to derive the results. Knowledge of cumulants up to a given order is equivalent to that of the corresponding moments. Although moments have a direct physical or geometric interpretation, cumulants sometimes have an advantage,

• vanishing of the cumulants for the normal distribution,
• their behaviour for sums of independent random variables, and
• especially in the multivariate case, their behaviour under linear transformation of the random variables concerned.

Define $K(\theta)= \log\left( E(e^{\theta x}) \right)$ and consider a density which is formed by tilting another primitive density $g(x)$,

$$f(x) = g(x) \times \exp \left( \theta x - K(\theta) \right) \ \ .$$

Then for that density,

$$\begin{eqnarray*} \mu & = & K'(\theta) \\ V(\mu) & = & K''(\theta) \end{eqnarray*}$$

Such a density is one from the exponential family and we shall encounter this group later on.

Theorem 4..4

Given ${\bf X}$ is a vector of $p$ components, $X_1,\,\dots ,\,X_p$ distributed iid $N(0,1)$, and
$Q={\bf X}'{\bf BX}$ where ${\bf B}$ is a $p \times p$ matrix of rank $r\leq p$, the distribution of $Q$

(i)

has $s$th cumulant, $\kappa_s=2^{s-1}(s-1)!tr(B^s)$

(ii)

is $\chi^2_r$ if and only if ${\bf B}$ is idempotent (that is, $B^2=B$).

Proof
Now there is an orthogonal matrix $P$ which transforms $Q$ into a sum of squares. That is, let ${\bf X} ={\bf PY}$, and

$$Q={\bf X}'{\bf BX}={\bf Y}'{\bf P}'{\bf BPY}={\bf Y}'{\bf\Lambda Y}$$

where ${\bf\Lambda}$ is a diagonal matrix with elements $\lambda_1,\,\lambda_2,\,\dots ,\,\lambda_p$, the eigenvalues of ${\bf B}$. Now exactly $r$ of these are non-zero where $r$ = rank($B$). So

$$Q=\sum^r_{i=1}\lambda_iY^2_i \ .$$ (4.4)

Now if ${\bf X}\sim N_p({\bf0}, {\bf I})$, then ${\bf Y}={\bf P}^{- 1}{\bf X}$ is distributed as $p$-variate normal with

$$E({\bf Y}) = {\bf P}^{-1}E({\bf X})=0$$

and

$$\begin{eqnarray*} \mbox{cov}({\bf Y}) & = & E({\bf YY}')=E({\bf P}^{-1}{\bf XX}... ...} \ \ \ \mbox{since} \ \ E({\bf XX}')={\bf I}\\ & = & {\bf I} \end{eqnarray*}$$

So ${\bf Y} \sim N_p({\bf0},{\bf I})$.

Consider now the $i$th component of ${\bf Y}$. Since $Y_i \sim N(0,1)$ it follows that $Y_i^2 \sim \chi^2_1$ and has mgf

$$M_{Y^2_i}(t)=(1-2t)^{-\frac 12} \ .$$

So $\lambda_i Y^2_i$ has mgf

$$M_{\lambda_i Y_i^2}(t)=(1-2\lambda_it)^{-\frac 12},$$

and $Q$, defined by (4.4), has mgf

$$M_Q(t)=\prod^r_{i=1} (1-2\lambda _it)^{-\frac 12},$$ (4.5)

since the $Y_i$ are independent. The cumulant generating function (cgf) is

$$\begin{eqnarray*} K_Q(t) & = & \log M_Q(t)\\ & = & -\frac 12 \sum^r_{i=1} \lo... ...+\dots + 2^{s-1}\lambda^s_i\frac{t^s}{s!} (s-1)!+ \dots \right] \end{eqnarray*}$$

(i)

So the $s$th cumulant of $Q,\kappa_s$ is

$$\kappa_s=2^{s-1}(s-1)!\sum^r_{i=1}\lambda^s_i \ \ , \ \ s=1,2,3,\dots$$ (4.6)

Now $\sum^r_{i=1}\lambda^s_i$ is the sum of elements of the leading diagonal of $B^s$. That is,
$\sum^r_{i=1}\lambda^s_i=tr(B^s)$. So (4.6) can be written

$$\kappa_s=2^{s-1}(s-1)!\,tr(B^s).$$ (4.7)

(ii)

Now for a $\chi^2_r$ distribution the mgf is $(1-2t)^{- r/2}$, the cgf is $-\frac r2 \log (1-2t)$, and the $s$th cumulant is

$$2^{s-1}(s-1)!r.$$ (4.8)

So if $Q\sim \chi^2_r$ the $s$th cumulant must be given by (4.8).

Comparing with (4.7), we must have

$$tr(B^s)=r=tr(B).$$

That is, $B^s=B$, and $B$ is idempotent.

On the other hand, if $B$ is idempotent, $r$ of the $\lambda_i=1$ and the others are $0$, so from (4.4), $Q=\sum_{i=1}^rY_i^2$, and $Q\sim \chi^2_r$.

The following theorems (stated without proof) cover more general cases.

Theorem 4..5

Let ${\bf X} \sim N_p({\bf0},\sigma^2{\bf I})$ and define $Q={\bf X}'{\bf BX}$ where ${\bf B}$ is symmetric of rank $r$. Then $Q / \sigma^2 \sim \chi^2_r$ if and only if ${\bf B}$ is idempotent.

What form might $B$ take? See if the projection matrices $X(X'X)^{-1}X'$ and $I - X(X'X)^{-1}X'$ are idempotent.

Theorem 4..6

Let ${\bf X} \sim N_p({\bf0},\mbox{\boldmath$\Sigma$})$ where $\mbox{\boldmath$\Sigma$}$ is positive definite. Define $Q={\bf X}'{\bf BX}$ where ${\bf B}$ is symmetric of rank $r$. Then $Q\sim \chi^2_r$ if and only if ${\bf B}\mbox{\boldmath$\Sigma$}{\bf B}={\bf B}$.