Joint distributions of random variables

• (a) Since \(F_{X,Y}(+\infty,+\infty)\) must be equal to \(1\) we obtain the constant \(C\) from the requirement \(\iint_{\mathbb R^2}f(x,y)\,dxdy=1\). This gives us \begin{eqnarray*} 1 &=& \iint_{\mathbb R^2} f(x,y) dxdy = C \int_0^{+\infty} \int_0^y e^{-\lambda y} dx dy = C \int_0^{+\infty} y e^{-\lambda y} dy = \left. C \left(-\frac{y e^{-\lambda y}}\lambda - \frac{e^{-\lambda y}}{\lambda^2}\right) \right|_0^{+\infty} = \frac C{\lambda^2}. \end{eqnarray*} We conclude that \(C = \lambda^2\).
• (b) Note that outside of the set \( \{0 \leq x \leq y\}\) the function \(f\) is identically equal to \(0\). Therefore for any \(x \geq 0\), \[ f_X(x) = \int_{-\infty}^{+\infty} f(x,y) dy = \lambda^2 \int_x^{+\infty} e^{-\lambda y} dy = \lambda e^{-\lambda x}. \] Hence, \(X\) is exponentially distributed with parameter \(\lambda\). For \(y \geq 0\), \[ f_Y(y) = \int_{-\infty}^{+\infty} f(x,y) dx = \lambda^2 \int_0^y e^{-\lambda y} dx = \lambda^2 y e^{-\lambda y}. \] Hence, \(f_Y(y) = \lambda^2 y e^{-\lambda y} \cdot 1_{[0,\infty)}(y)\).
• (c) We calculate the probability as follows. \begin{eqnarray*} \mathbb P(Y \leq 2X) &=& \int_0^{\infty}\int_{\frac y2}^y f(x,y) dxdy = \lambda^2 \int_0^{\infty}\int_{\frac y2}^y e^{-\lambda y} dxdy = \frac{\lambda^2}2 \int_0^\infty y e^{-\lambda y} dy = \frac12. \\ \end{eqnarray*}

Let us denote \(Z=7X+5Y\). Since \(X\) and \(Y\) are independent the \(7X\) and \(5Y\) are independent as well. Therefore \(m_{7X+5Y}(t)=m_{7X}(t)\cdot m_{5Y}(t)\). Hence \begin{eqnarray*}m_Z(t)&=&m_{7X+5Y}(t) = m_{7X}(t)\cdot m_{5Y}(t)=\mathbb E\left[e^{7tX}\right]\cdot\mathbb E\left[e^{5tY}\right] \\ &=&e^{\frac1249t^2}\cdot e^{\frac1225t^2}=e^{\frac{74 t^2}2}. \end{eqnarray*} If we denote \(\sigma=\sqrt{74}\) we obtain that \(m_Z(t)=e^{\frac12\sigma^2t}\). Therefore \(Z\) has a normal distribution with expectation \(0\) and variance \(\sqrt{74}\).

In the same way as in problem 2 we conclude that \(U\) is a normal random variable with expectation \(0\) and variance \(\alpha^2+\beta^2\). Thus \(U\sim N\left(0,\alpha^2+\beta^2\right)\). Analogously, we conclude that \(V\sim N\left(0,\beta^2+\gamma^2\right)\). Therefore \begin{eqnarray*} f_U(t)&=&\frac1{\sqrt{2\pi\cdot \left(\alpha^2+\beta^2\right)}}\cdot e^{-\frac12\frac{t^2}{\alpha^2+\beta^2}},\\ f_V(t)&=&\frac1{\sqrt{2\pi\cdot \left(\beta^2+\gamma^2\right)}}\cdot e^{-\frac12\frac{t^2}{\beta^2+\gamma^2}}. \end{eqnarray*}

Our next task is to calculate the covariance between \(U\) and \(V\). Since their expectations are \(0\) we have \(\mbox{cov}(U,V)=\mathbb E\left[UV\right]\). Using that \(U=\alpha X+\beta Y\) and \(V=\beta X+\gamma Y\) and that \(X\) and \(Y\) are independent standard normal random variables we obtain \begin{eqnarray*}\mbox{cov}(U,V)&=&\mathbb E\left[(\alpha X+ \beta Y)\cdot (\beta X+ \gamma Y)\right]= \mathbb E\left[\alpha\beta X^2+\alpha\gamma XY+\beta^2XY+\beta\gamma Y^2\right]\\ &=&\alpha\beta\mathbb E\left[X^2\right]+\left(\alpha\gamma+\beta^2\right)\mathbb E\left[XY\right]+\beta\gamma \mathbb E\left[Y^2\right] \\ &=& \alpha\beta+\left(\alpha\gamma+\beta^2\right)\mathbb E\left[X\right]\cdot\mathbb E\left[Y\right]+\beta\gamma \\ &=& \alpha\beta+\beta\gamma. \end{eqnarray*}

The covariance matrix between random variables \(U\) and \(V\) is \begin{eqnarray*}\Sigma(U,V)&=&\left[\begin{array}{cc}\mbox{var}(U)&\mbox{cov}(U,V)\\ \mbox{cov}(V,U)&\mbox{var}(V)\end{array}\right] =\left[\begin{array}{cc}\alpha^2+\beta^2&\alpha\beta+\beta\gamma\\ \alpha\beta+\beta\gamma & \beta^2+\gamma^2\end{array}\right]. \end{eqnarray*} Observe that the last matrix is the same as \(\left[\begin{array}{cc}\alpha&\beta\\ \beta&\gamma\end{array}\right]\cdot\left[\begin{array}{cc}\alpha&\beta\\ \beta&\gamma\end{array}\right]\).

Let us fix real numbers \(s\) and \(t\). The joint cumulative distribution function \(F_{U,V}(s,t)\) satisfies \begin{eqnarray*} F_{U,V}(s,t)&=&\frac1{2\pi}\iint_D e^{-\frac{x^2+y^2}2}\,dxdy, \end{eqnarray*} where \(D\) is the the region in \(\mathbb R^2\) that satisfies \begin{eqnarray*} D=\left\{(x,y): \alpha x+\beta y \leq s,\; \beta x+\gamma y\leq t\right\}. \end{eqnarray*}

Let us introduce the substitution \(u=\alpha x+ \beta y\) and \(v=\beta x+\gamma y\). Then the bounds for integration are \begin{eqnarray*} u&\in&(-\infty,s]\\ v&\in&(-\infty,t].\end{eqnarray*} Let us denote \[A=\left[\begin{array}{cc}\alpha&\beta\\ \beta &\gamma\end{array}\right].\] Then the relation between pairs \((x,y)\) and \((u,v)\) can be written as \[\left[\begin{array}{c}u\\ v\end{array}\right]=A\cdot \left[\begin{array}{c}x\\ y\end{array}\right].\]

The solution to the system is \[\left[\begin{array}{c}x\\ y\end{array}\right]=A^{-1}\left[\begin{array}{c}u\\ v\end{array}\right],\] where \(A^{-1}\) is the inverse of the matrix \(A\). The inverse \(A^{-1}\) of the matrix \(A\) is \begin{eqnarray*}A^{-1}&=&\left[\begin{array}{cc} \frac{\gamma}{\alpha\gamma-\beta^2}& \frac{-\beta}{\alpha\gamma-\beta^2} \\ \frac{-\beta}{\alpha\gamma-\beta^2} & \frac{\alpha}{\alpha\gamma-\beta^2}\end{array}\right].\end{eqnarray*} The variables \(x\) and \(y\) are explicitly expressed in terms of \(u\) and \(v\) as \begin{eqnarray*} x&=& \frac{\gamma}{\alpha\gamma-\beta^2}u- \frac{\beta}{\alpha\gamma-\beta^2}v\\ y&=&- \frac{\beta}{\alpha\gamma-\beta^2}u+ \frac{\alpha}{\alpha\gamma-\beta^2}v. \end{eqnarray*} The Jacobian is the absolute value of the determinant of the matrix \(\left[\begin{array}{cc} \frac{\gamma}{\alpha\gamma-\beta^2}& \frac{-\beta}{\alpha\gamma-\beta^2} \\ \frac{-\beta}{\alpha\gamma-\beta^2} & \frac{\alpha}{\alpha\gamma-\beta^2}\end{array}\right]\) \[J=\left|\mbox{det}\left[\begin{array}{cc} \frac{\gamma}{\alpha\gamma-\beta^2}& \frac{-\beta}{\alpha\gamma-\beta^2} \\ \frac{-\beta}{\alpha\gamma-\beta^2} & \frac{\alpha}{\alpha\gamma-\beta^2}\end{array}\right]\right|= \left|\frac{\alpha\gamma-\beta^2}{(\alpha\gamma-\beta^2)^2}\right|=\frac1{|\alpha\gamma-\beta^2|}.\]

The integral becomes \begin{eqnarray*} F_{U,V}(s,t)&=&\frac1{2\pi}\cdot \int_{-\infty}^{t}\int_{\infty}^se^{-\frac12\left(\left( \frac{\gamma}{\alpha\gamma-\beta^2}u- \frac{\beta}{\alpha\gamma-\beta^2}v\right)^2+\left(- \frac{\beta}{\alpha\gamma-\beta^2}u+ \frac{\alpha}{\alpha\gamma-\beta^2}v\right)^2\right)} \cdot\frac1{\alpha\gamma-\beta^2}\,dudv\\ &=& \frac1{2\pi\cdot\left(\alpha\gamma-\beta^2\right)}\cdot \int_{-\infty}^{t}\int_{\infty}^s e^{-\frac{\left( \left(\alpha^2+\gamma^2\right)u^2-2\beta(\alpha+\gamma)uv+\left(\alpha^2+\beta^2\right)v^2 \right)}{2\left(\alpha\gamma-\beta^2\right)^2}} \,dudv. \end{eqnarray*} Therefore the density function is \begin{eqnarray*}f_{U,V}(s,t)&=&\frac1{2\pi\cdot\left(\alpha\gamma-\beta^2\right)}\cdot e^{-\frac{\left( \left(\alpha^2+\gamma^2\right)s^2-2\beta(\alpha+\gamma)st+\left(\alpha^2+\beta^2\right)t^2 \right)}{2\left(\alpha\gamma-\beta^2\right)^2}}. \end{eqnarray*} Observe that since \(\Sigma=A^2\) we have that \(\mbox{det}(\Sigma)=\mbox{det}(A)^2=\left(\alpha\gamma-\beta^2\right)^2\). Therefore \(\alpha\gamma-\beta^2=\sqrt{\mbox{det}(\Sigma)}\). It remains to prove that \begin{eqnarray*}\frac{\left(\alpha^2+\gamma^2\right)s^2-2\beta(\alpha+\gamma)st+\left(\alpha^2+\beta^2\right)t^2}{\left(\alpha\gamma-\beta^2\right)^2}&=&\left[\begin{array}{cc} s &t \end{array}\right] \cdot \Sigma^{-1}\cdot \left[\begin{array}{c}s \\ t\end{array}\right]. \end{eqnarray*} This follows by simple algebraic calculations from \begin{eqnarray*}\Sigma^{-1}&=&\left(A^2\right)^{-1}=\left(A^{-1}\right)^2 \end{eqnarray*} and from the representation \(A^{-1}=\left[\begin{array}{cc} \frac{\gamma}{\alpha\gamma-\beta^2}& \frac{-\beta}{\alpha\gamma-\beta^2} \\ \frac{-\beta}{\alpha\gamma-\beta^2} & \frac{\alpha}{\alpha\gamma-\beta^2}\end{array}\right]\).

Let us start from two independent standard normal random variables \(Z_1\) and \(W_1\). Define the random variables \(U_1\) and \(V_1\) in the following way \begin{eqnarray*} U_1&=&\mu+p Z_1\\ V_1&=&\nu+q Z_1+ r W_1. \end{eqnarray*} We will choose the real numbers \(p\), \(q\), and \(r\) later. In the similar way as in problems 3 and 4 we prove that \(U_1\) and \(U_2\) have bivariate normal distribution with expectations \(\left[\begin{array}{c}\mu\\ \nu\end{array}\right]\) and covariance matrix \(\left[\begin{array}{cc}p^2&pq\\ pq& q^2+r^2\end{array}\right]\). We will now choose the real numbers \(p\), \(q\), and \(r\) to achieve \[\left[\begin{array}{cc}p^2&pq\\ pq& q^2+r^2\end{array}\right]=\left[\begin{array}{cc}\sigma^2 & \rho\\ \rho & \eta^2 \end{array}\right].\] This is easily done. It suffices to choose \(p=\sigma\), \(q=\frac{\rho}{\sigma}\), and \(r=\frac{\sqrt{\eta^2\sigma^2-\rho^2}}{\sigma}\).

One can prove that the joint probability density function for random variables \(U_1\) and \(V_1\) is \[f_{U_1,V_1}(s,t)=\frac1{2\pi\cdot\sqrt{\mbox{det}(\Sigma)}}\cdot e^{-\frac12 \left[\begin{array}{cc} s-\mu &t-\nu \end{array}\right] \cdot \Sigma^{-1}\cdot \left[\begin{array}{c}s -\mu\\ t-\nu\end{array}\right]}.\] The proof is left as a homework problem (Homework 6, problem 3) and is similar to the solution of Problem 4.

Therefore the random variables \(U_1\) and \(V_1\) have the same joint probability density function as the random variables \(U\) and \(V\). Thus \(\frac{U-\mu}{p}\) is a standard normal random variable that we can call \(Z\). For such \(Z\), the random variable \(\frac1r\left(V-\nu-qZ\right)\) is standard normal independent from \(Z\). It remains to call this random variable \(W\) and we have completed the proof of the theorem.

• (a) Let us denote \(\hat X=X-2\) and \(\hat Y=Y-1\). The random variables \(\hat X\) and \(\hat Y\) have bivariate normal distribution with the same covariance matrix as the random variables \(X\) and \(Y\). However, the random variables \(\hat X\) and \(\hat Y\) have expectation \(0\). We can now express \(X+3Y\) in terms of \(\hat X\) and \(\hat Y\) in the following way \[X+3Y=\left(2+\hat X\right)+3\left(1+\hat Y\right)=5+\hat X+3\hat Y.\] Since \(\hat X\) and \(\hat Y\) have bivariate normal distribution, their linear combination \(\hat X+3\hat Y\) is a normal random variable. Its expectation is \[\mathbb E\left[\hat X+3\hat Y\right]=\mathbb E\left[\hat X\right]+3\mathbb E\left[\hat Y\right]=0+3\cdot 0=0.\] The variance of \(\hat X+3\hat Y\) is \begin{eqnarray*}\mathbb E\left[\left(\hat X+3\hat Y\right)^2\right]=\mathbb E\left[\hat X^2\right]+6\mathbb E\left[\hat X\cdot\hat Y\right]+9\mathbb E\left[\hat Y^2\right]. \end{eqnarray*} Since the covariance matrix between \(\hat X\) and \(\hat Y\) is \[\Sigma\left(\hat X,\hat Y\right)=\left[\begin{array}{cc}9&2\\ 2& 1\end{array}\right],\] the following identities hold \[\mathbb E\left[\hat X^2\right]=9,\quad\mathbb E\left[\hat X\hat Y\right]=2,\quad\mbox{and}\quad \mathbb E\left[\hat Y^2\right]=1.\] Therefore the variance of \(\hat X+3\hat Y\) is \begin{eqnarray*}\mathbb E\left[\left(\hat X+3\hat Y\right)^2\right]=9+6\cdot 2+9\cdot 1=30. \end{eqnarray*} Thus there exist a standard normal random variable \(Z\) such that \[\hat X+3\hat Y=\sqrt{30}Z.\] From the relations \(X=2+\hat X\) and \(Y=1+\hat Y\) we obtain \[X+3Y=5+\hat X+3\hat Y=5+\sqrt{30}Z.\]
• (b) From parts (a) and (b) we conclude that the probability of the event \(\left\{X+3Y\leq 7\right\}\) is \begin{eqnarray*} \mathbb P\left(X+3Y\leq 7\right)&=&\mathbb P\left(5+\sqrt{30}Z\leq 7\right)=\mathbb P\left(\sqrt 30 Z\leq 2\right)=\mathbb P\left(Z\leq \frac{2}{\sqrt{30}}\right)=\Phi\left(\frac{2}{\sqrt{30}}\right). \end{eqnarray*}

There exists \(Z\sim N(0,1)\) independent of \(X\) such that \(Y=\alpha X+ \beta Z\) for some real numbers \(\alpha\) and \(\beta\). We can find these numbers \(\alpha\) and \(\beta\) from the requirements that \(\mathbb E[XY]=\frac12\) and \(\mathbb E\left[Y^2\right]=1\). The first equation gives us \(\alpha=\frac12\) and the second equation gives us \(\beta=\frac{\sqrt3}2\). We now obtain \begin{eqnarray*}&&\mathbb P\left(X\geq 0,\; Y\geq 0\right)\\ &=&\mathbb P\left(X\geq 0, \; \frac12 X+ \frac{\sqrt 3}2Z\geq 0\right) \\&=& \mathbb P\left(X\geq 0, \; Z\geq -\frac1{\sqrt 3}X\right). \end{eqnarray*} We will now use the rotational symmetry of the joint distribution of two independent normal random variables. The region \(\left\{X\geq 0, \; Z\geq -\frac1{\sqrt 3}X\right\}\) is the angle in the \(XZ\) plane between the lines \(Z=0\) and \(Z=-\frac1{\sqrt 3}X\). The measure of the angle is \(\frac{\pi}2+\mbox{arctan}\frac1{\sqrt 3}\) and the required probability is \begin{eqnarray*}\mathbb P\left(X\geq 0, Y\geq 0\right)&=&\frac14+\frac{\arctan\frac1{\sqrt3}}{2\pi}\\&=&\frac14+\frac{1}{12}=\frac13.\end{eqnarray*}