4.1 Joint pmf

Let \(D_{(X,Y)}\) be the set of of discontinuities of the joint cumulative distribution function \(F_{(X,Y)}(x,y)\text{,}\) that is \[ D_{(X,Y)}=\left \{ \left( x,y\right) \in \mathbb{R}^{2}:P(X=x,Y=y)>0\right \} \]

Definition: \((X,Y)\) is a two dimensional discrete random variable if and only if \[ \sum_{\left( x,y\right) \in D_{(X,Y)}}P\left( X=x,Y=y\right) =1. \]

Remark: As in the univariate case, a multivariate discrete random variable can take a finite number of possible values \((x_{i},y_{j}),\) where \(i=1,2,...,k_{1}\text{ and }j=1,2,...,k_{2}\text{,}\) where \(k_{1}\) and \(k_{2}\) are finite integers, or a countably infinite \((x_{i},y_{j}),\) where \(i=1,2,...\) and \(j=1,2,...\). For the sake of generality we consider the latter case. That is \(D_{(X,Y)}=\left \{ (x_{i},y_{j}),i=1,2,...,j=1,2,...\right \}\)

Joint probability mass function: If \(X\) and \(Y\) are discrete random variables, then the function given by \[ f_{X,Y}(x,y)=P\left( X=x,Y=y\right) \] for \((x,y)\in D_{(X,Y)}\) is called the joint probability mass function of \((X,Y)\) (joint pmf) or joint probability distribution of the random variables \(X\) and \(Y\text{.}\)

Theorem: A bivariate function \(f_{X,Y}(x,y)\) can serve as joint probability mass function of the pair of discrete random variables \(X\) and \(Y\) if and only if its values satisfy the conditions:

• \(f_{X,Y}(x,y)\geq 0\) for any \((x,y)\in \mathbb{R}^{2}\)

• \(\sum _{\left (x ,y\right ) \in D_{(X ,Y)}}f_{X ,Y} (x ,y) =\sum _{i =1}^{\infty }\sum _{j =1}^{\infty }f_{X ,Y} (x_{i} ,y_{j}) =1\)

Remark: We can calculate any probability using this function. For instance \(P\left( \left( x,y\right) \in B\right) =\sum_{\left( x,y\right) \in B}f_{X,Y}(x,y)\)

4.2 Joint cdf

Joint cumulative distribution function: If \(X\) and \(Y\) are discrete random variables, the function given by \[ F_{X,Y}(x\;y)=\sum_{s\leq x}\sum_{t\leq y}f_{X,Y}(s,t) \] for \(\left( x,y\right) \in \mathbb{R}^{2}\) is called the joint distribution function or joint cumulative distribution of \(X\) and \(Y\text{.}\)

4.3 Marginal pmf’s

Marginal probability distribution/function: If \(Y\) and \(X\) are discrete random variables and \(f_{X,Y}\) is the value of their joint probability distribution at \((x,y)\) the function given by

\[ \begin{aligned} P\left( X=x\right) &=&\left \{ \begin{array}{cc} \sum_{y\in D_{y}}f(x,y)=\sum_{y\in D_Y}f_{X,Y}(x,y), & \; \text{for}\;x\in D_{x} \\ 0, & \; \text{for}\;x\notin D_{x} \end{array} \right. \; \text{.}\; \\ P\left( Y=y\right) &=&\left \{ \begin{array}{cc} \sum_{x\in D_{x}}f(x,y)=\sum_{x\in D_X}f_{X,Y}(x,y), \; \; & \; \text{for}\;y\in D_{y} \\ 0, & \; \text{for}\;y\notin D_{y} \end{array} \right. \end{aligned} \]

are respectively is the Marginal probability distribution of the r.v. \(X\) and \(Y\), where \(D_{x}\) and \(D_{y}\) are the range of \(X\) and \(Y\) respectively.

4.4 Independence

Independence of random variables: Two discrete random variables \(X\) and \(Y\) are independent if and only if, for all \((x,y)\in D_{X,Y}\), \[ P\left( X=x,Y=y\right) =P\left( X=x\right) P\left( Y=y\right) \text{.} \]

Are these two random variables independent?

\[ \begin{aligned} &P(X=2000,Y=2000)=P(X=2000)\times P(Y=2000)=0.1\\ &\text{Is this sufficient to say that }X\text{ and }Y\text{ are independent?}\textbf{ NO!}\\ &P(X=0,Y=0)=0.05\text{ but }P(X=0)P(Y=0)=0.06\end{aligned} \] thus \(X\) and \(Y\) are not independent.

4.5 Conditional pmf’s

Conditional probability mass function of \(Y\) given \(X\):A conditional probability function of a discrete random variable \(Y\) given another discrete variable \(X\) taking a specific value is defined as \[\begin{aligned} f_{Y|X=x}(y) =&P\left( Y=y{|}X=x\right) =\frac{P\left( Y=y,X=x\right) }{P\left( X=x\right) } \\ =&\frac{f_{X,Y}(x,y)}{f_{X}(x)},\; \text{{}}\;f_{X}(x)>0, \text{ for a fixed $x$.}\end{aligned}\] The conditional probability function of \(X\) given \(Y\) is defined by

\[ f_{X|Y=y}(x)=\frac{f_{X,Y}(x,y)}{f_{Y}(y)},\; \text{{}}\;f_{Y}(y)>0. \]

Remarks:

• The conditional probability functions satisfy all the properties of probability functions, and therefore \(\sum _{i =1}^{\infty }f_{Y\vert X} (y_{i}) =1.\)

• If \(X\) and \(Y\) are independent\(f_{Y|X=x}(y)=f_{y}(y)\) and \(f_{X|Y=y}(x)=f_{X}(x)\)

example Consider the joint probability function

Compute \(P\left( Y=y{|}X=0\right)\).

\[ \begin{aligned} P\left( Y=0{|}X=0\right) &=&\frac{P\left( Y=0,X=0\right) }{P\left( X=0\right) }=\frac{0.05}{0.2}=0.25 \\ P\left( Y=1000{|}X=0\right) &=&\frac{P\left( Y=1000,X=0\right) }{P\left( X=0\right) }=\frac{0.1}{0.2}=0.5\text{.} \\ P\left( Y=2000{|}X=0\right) &=&\frac{P\left( Y=2000,X=0\right) }{P\left( X=0\right) }=\frac{0.05}{0.2}=0.25\text{.}\end{aligned} \]

4.6 Conditional cdf’s

Definition: The conditional CDF of \(Y\) given \(X\) is defined by

\[ F_{Y\vert X=x}(y)=P\left( Y\leq y{|}X=x\right) =\sum_{y'\in D_Y\wedge y'\leq y}\frac{P\left( Y=y',X=x\right) }{P\left( X=x\right) } \] for a fixed \(x\), with \(P(X=x)>0\).

Remark: It can be checked that \(F_{Y\vert X=x}\) is indeed a CDF.

Exercice: Verify that \(F_{Y\vert X=x}\) is non-decreasing and and \(\lim\limits_{y\to+\infty}F_{Y\vert X=x}(y)=1\).

\[ \begin{aligned} 1)~~F_{Y\vert X=x}&(y+\delta)-F_{Y\vert X=x}(y)\\ &~~~~~~~~~~~=\frac{P\left( Y\leq y+\delta,X=x\right)}{P(X=x)}-\frac{P\left( Y\leq y,X=x\right)}{P(X=x)}\geq 0.\\ 2)~~~\lim_{y\to+\infty}&F_{Y\vert X=x}(y)=\lim_{y\to+\infty}P\left( Y\leq y\vert X=x\right)\\ =\lim_{y\to+\infty}&\frac{P\left( Y\leq y,X=x\right)}{P\left( X=x\right)}=\frac{P\left( Y\leq \infty,X=x\right)}{P\left( X=x\right)}=\frac{P\left(X=x\right)}{P\left( X=x\right)}=1. \end{aligned} \]

5.1 Joint pdf and joint cdf

Definition: \(\left( X,Y\right)\) is a two-dimensional continuous random variable with a joint cumulative distribution function \(F_{X,Y}(x,y)\text{,}\) if and only if \(X\) and \(Y\) are continuous random variables and there is a non-negative real function \(f_{X,Y}(x,y)\text{,}\) such that \[ F_{X,Y}(x,y)=P(X\leq x,Y\leq y)=\int_{-\infty }^{y}\int_{-\infty }^{x}f_{X,Y}(t,s)dtds. \]

The function \(f_{X,Y}(x,y)\) is the joint (probability) density of \(X\) and \(Y\).

Remark: Let \(A\) be a set in the \(\mathbb{R}^2\). Then, \[ P\left((X,Y)\in A\right)=\int\int_{A}f_{X,Y}(t,s)dtds. \]

::: {.example} Joint probability density function of the two dimensional random variable \((P_1,S)\) where \(P_1\) represents the price and \(S\) the total sales (in 10000 units).

Joint probability density function: \[ f_{P_1,S}(p,s)=\begin{cases} 5pe^{-ps},& 0.2<p<0.4,~~s>0\\ 0,& \text{otherwise} \end{cases} \]

Joint cumulative distribution function: \[\begin{aligned} F_{P_1,S}(p,s)&=P(P_1\leq p,S\leq s)\\ &=\begin{cases} 0,&p<0.2 \text{ or }s<0\\ -1+5p-5\frac{e^{-0.2s}-e^{-ps}}{s},& 0.2<p<0.4, s\geq 0\\ 1-5\frac{e^{-0.2s}-e^{-0.4s}}{s},& p\geq 0.4, s\geq 0 \end{cases} \end{aligned} \] To get the CDF we need to make the following computations:

Theorem: A bivariate function can serve as a joint probability density function of a pair of continuous random variables \(X\) and \(Y\) if its values, \(f_{X,Y}(x,y)\), satisfy the conditions:

• \(f_{X ,Y} (x ,y) \geq 0\text{,}\) for all \((x ,y) \in \mathbb{R}^{2}\)

• \(\int _{ -\infty }^{ +\infty }\int _{ -\infty }^{ +\infty }f_{X ,Y} (x ,y) d x d y =1.\)

Property: Let \((X,Y)\) be a bivariate random variable and \(B\in\mathbb{R}^2\), then \[ P((X,Y)\in B)=\int\int_{B}f_{X,Y}(x,y)dxdy. \]

::: {.example} Let \((X,Y)\) be a continuous bi-dimensional random variable with density function \(f_{X,Y}\) given by \[ f_{X,Y}(x,y)=\begin{cases} x+y,&0<x<1,0<y<1\\ 0,&\text{otherwise} \end{cases}. \] Compute \(P(X>Y)\).

Solution: Firstly, we notice that

\[ \begin{aligned} P(X>Y)&=\int _{ 0 }^{ 1 }\int _{0 }^{ x }(x+y) d y d x \\ &=\int _{ 0 }^{ 1 }\frac{3}{2}x^2dx=\frac{1}{2} \end{aligned} \]

Properties: Let \((X,Y)\) be a continuous bivariate random variable. If \(f_{X,Y}\) represents the density function of \((X,Y)\) and \(F_{X,Y}\) represents respectively joint CDF of \((X,Y)\). Then,

• \(f_{X,Y}(x,y)=\frac{\partial ^{2}F_{X,Y}(x,y)}{\partial x\partial y}= \frac{\partial ^{2}F_{X,Y}(x,y)}{\partial y\partial x}\text{, almost everywhere.}\)

5.2 Marginal pdf’s

• Marginal density functions of the random variable \(X\) \[ f_{X} \left (x\right ) =\int _{ -\infty }^{ +\infty }f_{X ,Y} (x ,v) d v \text{,} \]

• Marginal density functions of the random variable \(Y\) \[ f_{Y}\left( y\right) =\int_{-\infty }^{+\infty }f_{X,Y}(u,y)du. \]

5.3 Marginal cdf’s

• Marginal CDF of the random variable \(X\) \[ F_{X} \left (x\right ) =\underset{y \rightarrow +\infty }{\lim }F_{X ,Y} (x ,y) =\int _{ -\infty }^{x}\underbrace{\int _{ -\infty }^{ +\infty }f_{X ,Y} (u ,y) d y}_{=f_{X}(u)} d u\text{,} \]

• Marginal CDF of the random variable \(Y\) \[ F_{Y}\left( y\right) =\underset{x\rightarrow +\infty }{\lim } F_{X,Y}(x,y)=\int_{-\infty }^{y}\underbrace{\int_{-\infty }^{+\infty }f_{X,Y}(x,v)dv}_{=f_Y(v)}dx \text{.} \]

5.4 Conditional pdf’s

Definition: If \(f_{X,Y}(x,y)\) is the joint probability density function of the continuous random variables \(X\) and \(Y\) and \(f_{Y}\left( y\right)\) is the marginal density function of \(Y\), the function given by \[ f_{X|Y=y}\left( x\right) =\frac{f_{X,Y}(x,y)}{f_{Y}\left( y\right) },x\in \mathbb{R}\; \text{ (for fixed }y \text{),}\;f_{Y}\left( y\right) > 0 \] is the conditional probability density function of \(\bf X\) given \(\left \{\bf Y=y\right \} \text{.}\) Similarly if \(f_{X}\left( x\right)\) is the marginal density function of \(X\) \[ f_{Y|X=x}\left( y\right) =\frac{f_{X,Y}(x,y)}{f_{X}\left( x\right) },y\in \mathbb{R}\; \text{(for fixed }x\text{),}\;f_{X}\left( x\right) > 0 \] is the conditional probability function of \(Y\) given \(\left \{ X=x\right \}\).

Remark: Note that \[ P\left( X\in B|Y=y\right) =\int_{B}f_{X|Y=y}\left( x\right) dx \] for any \(B\subset \mathbb{R}\).

\[ \begin{aligned} f_X(x)&=\int_0^1(y+x)dy\\ &=x+\frac{1}{2} \end{aligned} \]

\[ \begin{aligned} f_{Y|X=x}\left( y\right) &=\frac{f_{X,Y}(x,y)}{f_{X}(x)} \\ &=\frac{x+y}{x+\frac{1}{2}}, \, y\in(0,1) \end{aligned} \]

Probability of \(Y\geq 0.7|X=0.5\) \[ \begin{aligned} P(Y\geq 0.7|X=0.5)&=\int_{0.7}^{1}f_{Y|X=0.5}\left( y\right) dy\\ &=\int_{0.7}^{1}\left( y+0.5\right) dy=0.405. \end{aligned} \]

Remark:

• The conditional density functions of \(X\) and \(Y\) verify all the properties of a density function of a univariate random variable.

• Note that we can always decompose a joint density function in the following way \[ f_{X,Y}(x,y)=f_{X}(x)f_{Y|X=x}\left( y\right) =f_{Y}\left( y\right) f_{X|Y=y}\left( x\right) . \]

5.5 Independence

• If \(X\) and \(Y\) are independent \(f_{Y|X=x}\left( y\right) =f_{Y}\left( y\right)\) and \(f_{X|Y=y}\left( x\right) =f_{X}\left( x\right) \text{.}\)

5.6 Conditional cdf’s

Definition: The conditional CDF of \(Y\) given \(X\) by defined by \[F_{Y\vert X=x}(y)=\int_{-\infty}^yf_{Y\vert X=x}(s)ds=\int_{-\infty}^y\frac{f_{Y, X}(s,x) }{f_X(x) }ds \] for a fixed \(x\), with \(f_X(x)>0\).

Remark: It can be checked that \(F_{Y\vert X=x}\) is indeed a CDF.