Discrete Random Variables

Tags

math

If we conduct an experiment with a discrete (finite) number of outcomes \( x \) then these outcomes can be thought of as a random variable \( X \).

Note: We use the uppercase letter \( X \) to refer to the random variable itself and the lower-case variant \( x \) to refer to a particular outcome of \( X \).

For example in a game 3 coins are thrown and the number of heads counted. You can see the probability distribution (possible outcomes of this game and likelihood of each) in tbl:prob-dist-coin.

| \(x\) || 0 | 1 | 2 | 3 | |--------------|---------------------|---------------------|---------------------|---------------------| | \(p(X=x)\) || \( \frac{1}{8} \) | \( \frac{3}{8} \) | \( \frac{2}{8} \) | \( \frac{1}{8} \) |

Expectation & Variance

Expectation

We define the expected value of a random variable as the expected outcome (\( x \)) of the game when repeated many times. This is the average reward we get from the game (also referred to as the mean of the distribution) and is the sum of each possible value weighted by its probability.

\begin{align} E(X) = \mu = \sum x_i p(X = x_i) \label{eq:rand-var-expected} \end{align}

Variance

We define the variance of a random variable \( X \) as

\begin{align} Var(x) &= (\sum x_i^2 p(X = x_i)) - \mu^2 \\

     &= E(X^2) - [E(X)]^2 \label{eq:rand-var-variance}

\end{align}

Linearity of Expectation & Variance

From these we can also define the expectation and variance of a linear function of a random variable:

\begin{align} E(aX) &= aE(X) & Var(aX) &= a^2 Var(x) \
E(aX + b) &= aE(X) + b & Var(aX + b) &= a^2Var(x) \end{align}

Note: these show that adding or multiplying affect the mean of a distribution, but it has no affect on the variance. This should make sense when you consider that the variance relates to the deviation of a distribution and translating or scaling all the elements of a distribution doesn't change the relative gap between them.