Convolutions

Consider the problem of finding the distribution of the sum of 2 independent (but not necessarily identically distributed) random variables. The pdf of the sum can be neatly expressed using convolutions.

Theorem 3..1

Let $X$ and $Y$ be independent random variables with pdf's $f_X, \ f_Y$ respectively, and define $U=X+Y$. Then the pdf of $U$ is

$$f_U(u)=\int^{\infty}_{-\infty} f_X(u-v)f_Y(v)\,dv.$$ (3.5)

Proof Because of independence, the joint pdf of $X$ and $Y$ may be written

$$f_{X,Y}(x,y)=f_{X}(x)f_Y(y) \ .$$

Define $V=Y$ and, noting that the Jacobian of the inverse transformation is $1$, the joint pdf of $U$ and $V$ is

$$f_{U,V}(u,v)=f_X (u-v)f_Y(v),$$

and hence the marginal pdf of $U$, found by integrating with respect to $v$, is as given in (3.5).

Now $f_U(u)$ is called the convolution of $f_X$ and $f_Y$.

The following heuristic explanation may assist.

Equation (3.5) defines a convolution in the mathematical sense. Each single point of $f_U(u)$ is formed by a weighted average of the entire density $f_Y(v)$. The weights are the other density $f_X(u-v)$ where its value depends on how far apart each $v$ is from $u$. Thus each single point of the density $f_U(u)$ arises from all the density $f_Y(v)$.

Example 3..6

Random variables $X$ and $Y$ are identically and independently distributed (iid) uniformly on $[0,1]$. Find the distribution of $U=X+Y$.

We note that $f_X(x)=1, \ 0\leq x\leq 1, \ f_Y(y)=1, \ 0 \leq y \leq 1$ and that the inverse transformation is $x=u-v, \ y=v$ with $\vert J\vert=1$. The range space for $(u,v)$ is determined from

$$\begin{eqnarray*} x\geq 0 & \Rightarrow & u-v\geq 0, \ \ \mbox{ that is, } \ \ ... ... \Rightarrow & v \geq 0\\ y \leq 1 & \Rightarrow & v \leq 1, \end{eqnarray*}$$

and is shown in the diagram below.

So from Theorem 3.5,

$$f_U(u) = \left\{ \begin{array}{ll} \int^u_0 1.dv, & 0\leq u\... ...2cm] \int^1_{u-1} 1.dv, & 1 < u \leq 2 \end{array} \right.$$

resulting in what is sometimes called the triangular distribution,

$$f_U(u) = \left\{ \begin{array}{ll} u, & 0 \leq u \leq 1\\ 2-u, & 1 < u \leq 2 \end{array} \right.$$

as in Example 3.2.