2. Random Variables

1 Random Variables
2 Cumulative Distribution Function
- 2.1 Cumulative distribution function
- 2.2 Types of random variables
3 Discrete Random Variables
4 Continuous Random Variables
- 4.1 Continuous Random Variables
- 4.2 Probability Density Function
5 Mixed random variables
6 The Distribution of Functions of Random Variables
- 6.1 Functions of Continuous Random Variables
- 6.2 Functions of Discrete Random Variables

1 Random Variables

Random variable, informally, is a variable that takes on numerical values and has an outcome that is determined by an experiment.

Random Variable: Let $S$ be a sample space with a probability measure. A random variable (or stochastic variable) $X$ is a real-valued function defined over the elements of S. $\begin{aligned} X : & S \to R \\ s \to X (s) \end{aligned}$

Important convention: Random variables are always expressed in capital letters. On the other hand, particular values assumed by the random variables are always expressed by lowercase letters.

Remark: Although a random variable is a function of $s$ ; usually we drop the argument, that is we write $X$ ; rather than $X (s)$ .

Remark:

Once the random variable is defined, R is the space in which we work with;
The fact that the definition of a random variable is limited to real-valued functions does not impose any restrictions;
If the outcomes of an experiment are of the categorical type, we can arbitrarily make the descriptions real-valued by coding the categories, perhaps by representing them with the numbers.

Example 1.1 One flips a coin and observes if a head or tail is obtained.

Sample Space: $S = {H, T}$

Random Variable: $X : S \to {0, 1} with X (H) = 0 and X (T) = 1.$

The definition of random variable does not rely explicitly on the concept of probability, it is introduced to make easier the computation of probabilities. Indeed, if $B \subset R$ , then $\begin{aligned} P (X \in B) = P (A), where A = {s \in S : X (s) \in B} \end{aligned}$

Is now clear that: $\begin{aligned} P (X \in B) = 1 - P (X \notin B) . \end{aligned}$ In particular, $\begin{aligned} P (X \leq x) = 1 - P (X > x); \\ P (X < x) = 1 - P (X \geq x) \end{aligned}$

2 Cumulative Distribution Function

2.1 Cumulative distribution function

Let $X$ be a random variable. The cumulative distribution function $F_{X}$ is a real function of real variable given by: $F_{X} (x) = P (X \leq x) = P (X \in (- \infty, x])$

Properties of CDFs:

$0 \leq F_{X} (x) \leq 1;$
$F_{X} (x)$ is non-decreasing: $\forall Δ_{x} > 0 :$
$lim_{x \to - \infty} F_{X} (x) = 0$ and
$P (a < X \leq b) = F_{X} (b) - F_{X} (a),$ for $b > a$
$lim_{x \to a^{+}} F_{X} (x) = F_{X} (a);$ therefore is right continuous
$P (X = a) = F_{X} (a) - lim_{x \to a^{-}} F_{X} (x)$ for any real finite number.

Example 2.1 One flips a coin and observes if a head or tail is obtained.

Sample Space: $S = {H, T}$

Random Variable: $X : S \to {0, 1} with X (H) = 0 and X (T) = 1.$

$X$ counts the number of tails obtained.

It is easy to see that: $P (X = 0) = 1 / 2$ , $P (X = 1) = 1 / 2$ . Since we have $F_{X} (x) = P (X \leq x)$ , then

$\begin{aligned} F_{X} (x) = & P (X \leq x) \\ = & {\begin{cases} 0, & x < 0 \\ \frac{1}{2}, & 0 \leq x < 1 \\ 1, & x \geq 1 \end{cases} \end{aligned}$

Example 2.2 One flips a coin twice and counts the number of tails obtained.

Sample Space: $S = {(H, T), (H, H), (T, H), (T, T)}$

Random Variable:

$X : S \to {0, 1, 2}$ $X ((H, T)) = 1, X ((H, H)) = 0$ , $X ((T, H)) = 1, X ((T, T)) = 2.$

It is easy to see that: $P (X = s) = 1 / 4$ , for $s = 0, 2$ and $P (X = 1) = 1 / 2$ . Since we have $F_{X} (x) = P (X \leq x)$ , then

$F_{X} (x) = {\begin{cases} 0, & x < 0 \\ \frac{1}{4}, & 0 \leq x < 1 \\ 3 / 4, & 1 \leq x < 2 \\ 1, & x \geq 2 \end{cases}$

Further properties:

$P (X < b) = F_{X} (b) - P (X = b)$
$P (X > a) = 1 - F_{X} (a)$
$P (X \geq a) = 1 - F_{X} (a) + P (X = a)$
$P (a < X < b) = F_{X} (b) - F_{X} (a) - P (X = b)$
$P (a \leq X < b) = F_{X} (b) - F_{X} (a) - P (X = b) + P (X = a)$
$P (a \leq X \leq b) = F_{X} (b) - F_{X} (a) + P (X = a)$

Prove the previous properties!

Proof: To prove that $P (X \geq a) = 1 - F_{X} (a) + P (X = a)$ , one notes that: $\begin{aligned} P (X \geq a) & = 1 - P (X < a) = 1 - P (X \leq a) + P (X = a) \\ = 1 - F_{X} (a) + P (X = a) \end{aligned}$

The set of discontinuities of the cumulative distribution function $D_{X}$ is given by $D_{X} = {x \in R : P (X = x) > 0} .$ Note that by property 6 this the same as

$D_{X} = {a \in R : F_{X} (a) - lim_{x \to a^{-}} F_{X} (x) > 0} .$

2.2 Types of random variables

Discrete Random Variable: $X$ is a discrete random variable if $\begin{aligned} D_{X} \neq \emptyset and \sum_{x \in D_{x}} P (X = x) = 1. \end{aligned}$

Continuous Random Variable: $X$ is a continuous random variable if $D_{X} = \emptyset$ and there is a non-negative function $f$ such that $\begin{aligned} F_{X} (x) = \int_{0}^{x} f (s) d s . \end{aligned}$

Mixed Random Variable: $X$ is a mixed random variable if

$\begin{aligned} D_{X} \neq \emptyset, \sum_{x \in D_{x}} P (X = x) < 1 and \\ \exists λ \in (0, 1) such that F_{X} (x) = λ F_{X_{1}} (x) + (1 - λ) F_{X_{2}} (x) \end{aligned}$

where $X_{1}$ is a discrete random variable and $X_{2}$ is a continuous random variable.

3 Discrete Random Variables

$X$ is a discrete random variable if

$\begin{aligned} D_{X} \neq \emptyset and \sum_{x \in D_{x}} P (X = x) = 1. \end{aligned}$ Additionally, the function $f_{X} : R \to [0, 1]$ defined by

$f_{X} (x) = {\begin{cases} P (X = x), & x \in D_{X} \\ 0, & x \in D_{X} \end{cases} .$ is called the probability mass function (pmf).

Theorem: A function can serve as the probability function of a discrete random variable $X$ if and only if its values, $f_{X} (x)$ , satisfy the conditions

$0 \leq f_{X} (x_{j}) \leq 1,$ $j = 1, 2, 3, . . .$
$\sum_{j = 1}^{\infty} f_{X} (x_{j}) = 1.$

For discrete random variables, the cumulative distribution function (cdf) is given by :

$F_{X} (x) = P (X \leq x) = \sum_{x_{j} \leq x} f_{X} (x_{j}) .$

Generally,

$P (X \in B) = \sum_{x_{j} \in B \cap D_{X}} f_{X} (x_{j}) .$

Theorem: If the range of a random variable X consists of the values $x_{1} < x_{2} < \dots < x_{n}$ , then $\begin{aligned} f_{X} (x_{1}) = F_{X} (x_{1}), and f_{X} (x_{i}) = F_{X} (x_{i}) - F_{X} (x_{i - 1}), \end{aligned}$ $i = 2, 3, \dots n .$

Example 3.1 Check whether the function given by $f (x) = \frac{x + 2}{25}$ , for $x = 1, 2, 3, 4, 5$ can serve as the probability function of a discrete random variable $X$ . Compute the cumulative distribution function of $X$ .

4 Continuous Random Variables

4.1 Continuous Random Variables

$X$ is a continuous random variable if $D_{X} = \emptyset$ and there is a function $f_{X} : R \to R_{0}^{+}$ such that

$\begin{aligned} F_{X} (x) = \int_{- \infty}^{x} f_{X} (s) d s . \end{aligned}$

Additionally, $f_{X}$ is called the probability density function.

Remark:

Continuity of $F_{X}$ is necessary, but not sufficient to guarantee that $X$ is a continuous random variable;
Note that $P (X \in D_{X}) = P (X \in \emptyset) = 0$ ;
The function $f_{X}$ provides information on how likely the outcomes of the random variable are.

4.2 Probability Density Function

Theorem. A function can serve as a probability density function of a continuous random variable $X$ if its values, $f_{X} (x)$ , satisfy the conditions:

$f_{X} (x) \geq 0$ for $- \infty < x < + \infty$ ;
$\int_{- \infty}^{+ \infty} f_{X} (x) d x = 1$ .

Example 4.1 Let $X$ be a continuous random variable with a probability density function $f_{X}$ given by

$f_{X} (x) = {\begin{cases} 1 / 5, & x \in [3, a] \\ 0, & x \in R ∖ [3, a] \end{cases}$

Find the value of the parameter $a$ .

According to the previous theorem, we know that $\begin{aligned} f_{X} (x) \geq 0, for - \infty < x < + \infty \\ \int_{- \infty}^{+ \infty} f_{X} (x) d x = 1 \end{aligned}$

From the second condition, we get that

$\frac{a}{5} - \frac{3}{5} = 1 \Leftrightarrow a = 8$ .

Theorem. If $f_{X} (x)$ and $F_{X} (x)$ are the values of the probability density and the distribution function of $X$ at $x$ , then $\begin{aligned} P (a & \leq X \leq b) = F_{X} (b) - F_{X} (a) = \int_{a}^{b} f_{X} (t) d t \end{aligned}$

for any real constants $a$ and with $a \leq b$ , and

$f_{X} (x) = \frac{d F_{X} (x)}{d x}, almost everywhere.$

Remarks:

At the points $x$ where there is no derivative of the CDF, $F_{X}$ , it is agreed that $f_{X} (x) = 0$ . In fact, it does not matter the value that we give to $f_{X} (x)$ as it does not affect the computation of $F_{X}$ .
The probability density function is not a probability and therefore it can assume values bigger than one.
If $X$ is a continuous random variable $P (X = a) = \int_{a}^{a} f_{X} (t) d t = 0.$

Example 4.2 Consider the continuous random variable $X$ with a probability density function $f_{X}$ and cumulative distribution function given by $f_{X} (x) = {\begin{cases} 0, & x < 0 \\ 4 x, & 0 \leq x \leq \frac{1}{2} \\ 4 - 4 x, & \frac{1}{2} \leq x \leq 1 \\ 0, & x > 1 \end{cases}$

Cumulative density function:

$F_{X} (x) = {\begin{cases} 0, & x < 0 \\ 2 x^{2}, & 0 \leq x < \frac{1}{2} \\ - 1 + 4 x - 2 x^{2}, & \frac{1}{2} \leq x < 1 \\ 1, & x \geq 1 \end{cases}$

Is this function $F_{X}$ differentiable?

Theorem: If $X$ is a continuous random variable and $a$ and $b$ are real constants with $a \leq b$ , then $\begin{aligned} P (a & \leq & X \leq b) = P (a \leq X < b) \\ = & P (a < X \leq b) \\ = & P (a < X < b) \end{aligned}$

Proof: To prove the previous theorem one needs notice that: $\begin{aligned} P (a \leq X \leq b) = & P (a < X < b) + P (X = a) + P (X = b) \\ = & P (a < X \leq b) + P (X = a) \\ = & P (a \leq X < b) + P (X = b) \end{aligned}$

Additionally, for $c = a$ or $c = b$ we have

$\begin{aligned} P (X = c) = P (c & \leq X \leq c) = \int_{c}^{c} f_{X} (t) d t = 0 \end{aligned}$

Remark: The previous inequalities are not necessarily true for discrete random variables.

5 Mixed random variables

Mixed Random Variable: $X$ is a mixed random variable if

$\begin{aligned} D_{X} \neq \emptyset, \sum_{x \in D_{x}} P (X = x) < 1 and \\ \exists λ \in (0, 1) tal que F_{X} (x) = λ F_{X_{1}} (x) + (1 - λ) F_{X_{2}} (x) \end{aligned}$

where $X_{1}$ is a discrete r.v. and $X_{2}$ is a continuous r.v..

Example 5.1 A company has received 1 million € to invest in a new business. With probability $\frac{1}{2}$ , the firm does nothing but with probability $\frac{1}{2}$ the money is invested. If it does not invest the money, $1$ million € is kept. Otherwise, the firm gets back a random amount uniformly distributed between $0$ and $3$ million €.

Let $X$ be the following random variable: $X = ‘ ‘ Amount received by the company in millions "$ What type of random variable is $X$ ?

$S = [0, 3] and X = {\begin{cases} 1, & with probability \frac{1}{2} (Scenario 1) \\ [0, 3], & with probability \frac{1}{2} (Scenario 2) \end{cases}$

$X$ is not a discrete r.v. because it takes values in a continuous set;
$X$ is not a continuous random variable because $P (X = 1) = 1 / 2$ (For continuous random variables the probability to take one single point is equal to $0$ ).
$X$ is a mixed random variable?

We can define two random variables:

$\begin{aligned} X_{1} = & ‘ ‘ Amount received by the \\ company in millions in S1 " \end{aligned}$

$\begin{aligned} X_{2} = ‘ ‘ Amount received by the \\ company in millions in S2 " \end{aligned}$

Since $P (X_{1} = 1) = 1$ , then $F_{X_{1}} (x) = {\begin{cases} 0, & x < 1 \\ 1, & x \geq 1 \end{cases}$

On the other hand, in scenario 2, the firm gets back a random amount uniformly distributed between $0$ and $3$ million €. Therefore,

$f_{X_{2}} (x) = {\begin{cases} \frac{1}{3}, & x \in [0, 3] \\ 0, & otherwise \end{cases}, and F_{X_{2}} (x) = {\begin{cases} 0, & x < 0 \\ \frac{x}{3}, & 0 \leq x < 3 \\ 1, & x \geq 3, \end{cases}$

Since S1 holds with probability $\frac{1}{2}$ and S2 holds with probability $\frac{1}{2}$ , we have that

$\begin{aligned} F_{X} (x) & = \frac{1}{2} F_{X_{1}} (x) + \frac{1}{2} F_{X_{2}} (x) = {\begin{cases} 0, & x < 0 \\ \frac{x}{6}, & 0 \leq x < 1 \\ \frac{1}{2} + \frac{x}{6}, & 1 \leq x < 3 \\ 1, & x \geq 3, \end{cases} \end{aligned}$

$D_{X} = {1}$ , because

$\begin{aligned} F_{X} (1) - F_{X} (1^{-}) = \frac{2}{3} - \frac{1}{2} \\ = \frac{1}{2} = P (X = 1) < 1 \end{aligned}$

Exercise: Let

$F_{X} (x) = {\begin{array}{cc} 0 & x < 0 \\ \frac{1}{12} + \frac{3}{4} (1 - e^{- x}) & 0 \leq x < 1 \\ \frac{1}{4} + \frac{3}{4} (1 - e^{- x}) & x \geq 1 \end{array},$

Compute $P (X = 0),$ $P (X = 1),$ $P (0.5 < X < 1)$ and $P (0.5 < X < 2)$ .

Answer: $\begin{aligned} P (X = 0) = \frac{1}{12}, P (X = 1) = \frac{2}{12} \\ P (0.5 < X < 1) = F_{X} (1) - F_{X} (0.5) - P (X = 1) = \frac{3}{4} (e^{- 0.5} - e^{- 1}) \\ P (0.5 < X < 2) = F_{X} (2) - F_{X} (0.5) = \frac{2}{12} + \frac{3}{4} (e^{- 0.5} - e^{- 2}) \end{aligned}$

6 The Distribution of Functions of Random Variables

Motivation: Assume that the random variable $D$ represents the demand of a given product in a store. The profit of this store is represented by the random variable $L = 4 D - 5$ . If the probability function of $D$ is given by

$P (D = d) = {\begin{cases} 0.3, & d = 0 \\ 0.2, & d = 1 \\ 0.3, & d = 2 \\ 0.2, & d = 3 \end{cases},$ what is the probability of having $L > 2$ ?

$P (L > 2) = P (D > \frac{7}{4}) = P (D = 2) + P (D = 3) = 0.5$ Since $L$ is a random variable, it should be possible to find its distribution. How to do it?

Let $X$ be a known random variable with known cumulative distribution function .
Consider a new random variable $Y = g (X)$ , where $g : R \to R$ is a known function. Let be the cumulative distribution function of $Y .$ How can we derive $F_{Y} (y)$ from $F_{X} (x) ?$ .
The derivation of $F_{Y} (y)$ is based on the equality

$\begin{aligned} F_{Y} (y) = P (Y \leq y) = P (g (X) \leq y) = P (X \in A_{y}^{*}) \end{aligned}$ where $A_{y}^{*} = {x : g (x) \leq y}$

Example 6.1 Derive the cumulative distribution functions of $Y = a X + b,$ where $a > 0$ and $Z = X^{2}$ .

$Y = a X + b$

$\begin{aligned} F_{Y} (y) & = P (Y \leq y) = P (a X + b \leq y) \\ = P (X \leq \frac{y - b}{a}) = F_{X} (\frac{y - b}{a}) \end{aligned}$

$Z = X^{2}$

For $z \geq 0$ ,

$\begin{aligned} F_{Z} (z) & = P (Z \leq z) = P (X^{2} \leq z) \\ = P (- \sqrt{z} \leq X \leq \sqrt{z}) \\ = F_{X} (\sqrt{z}) - F_{X} (- \sqrt{z}) + P (X = - \sqrt{z}) \end{aligned}$

6.1 Functions of Continuous Random Variables

Assume that in the previous example $X$ is a continuous random variable such that

$F_{X} (x) = {\begin{cases} 0, & x < 0 \\ x, & 0 \leq x < 1 \\ 1, & x \geq 1 \end{cases},$

then the following holds:

$Y = a X + b$

$\begin{aligned} F_{Y} (y) = & F_{X} (\frac{y - b}{a}) = {\begin{cases} 0, & \frac{y - b}{a} < 0 \\ \frac{y - b}{a}, & 0 \leq \frac{y - b}{a} < 1 \\ 1, & \frac{y - b}{a} \geq 1 \end{cases} \\ = & {\begin{cases} 0, & y < b \\ \frac{y - b}{a}, & b \leq y < a + b \\ 1, & y \geq a + b \end{cases} \end{aligned}$

Example 6.2 Assume that in the previous example $X$ is a continuous random variable such that

$F_{X} (x) = {\begin{cases} 0, & x < 0 \\ x, & 0 \leq x < 1 \\ 1, & x \geq 1 \end{cases},$ then the following holds:

$Z = X^{2}$

If $z < 0$ then $F_{Z} (z) = P (Z \leq z) = 0$ . When $z \geq 0$

$\begin{aligned} F_{Z} (z) & = F_{X} (\sqrt{z}) - F_{X} (- \sqrt{z}) + \underset{= 0, because X is continuous}{\underset{⏟}{P (X = - \sqrt{z})}} \\ = F_{X} (\sqrt{z}) - \underset{= 0 because - \sqrt{z} is negative}{\underset{⏟}{F_{X} (- \sqrt{z})}} \\ = {\begin{cases} 0, & z < 0 \\ \sqrt{z}, & 0 \leq z < 1 \\ 1, & z \geq 1 \end{cases} \end{aligned}$

6.2 Functions of Discrete Random Variables

When $X$ is a discrete random variable, it is easier to find the distribution of $Y = g (X)$ . In this case, we will derive the probability function.
Let $D_{X} = {x_{1}, x_{2}, x_{3} . . .}$ be the set of discontinuities of $F_{X} (x),$ then $D_{Y} = {g (x_{1}), g (x_{2}), g (x_{3}) . . .}$ is the set of discontinuities of $F_{Y} (y) .$
The probability function of $Y$ is given by

$\begin{aligned} f_{Y} (y) & = P (Y = y) = P (g (X) = y) \\ = P (X \in {x \in D_{X} : g (x) = y}) \\ = \sum_{x_{i} \in {x \in D_{X} : g (x) = y}} f (x_{i}) \end{aligned}$

Example 6.3 Consider the discrete random variable $X$ with probability function

x	-2	-1	0	1	2
$f_{X} (x)$	12/60	15/60	10/60	6/60	17/60

Let $Y = X^{2},$ what is $f_{Y} (y) ?$

Firstly: The set of discontinuities $D_{Y}$ is $D_{Y} = {0, 1, 4}$

x	-2	-1	0	1	2
$y = x^{2}$	4	1	0	1	4

Consequently

$f_{Y} (0) = P (Y = 0) = P (X^{2} = 0) = P (X = 0) = \frac{10}{60}$ .
$f_{Y} (1) = P (Y = 1) = P (X^{2} = 1) = P (X = 1) + P (X = - 1) = 6 / 60 + 15 / 60 = 21 / 60.$
$f_{Y} (4) = P (Y = 4) = P (X^{2} = 4) = P (X = 2) + P (X = - 2) = 17 / 60 + 12 / 60 = 29 / 60.$

Last updated on Jan 1, 0001

Edit this page