Variance

Definition

Let $X$ be a Random Variable with expected value $\mu =\mathbb{E}[X]$. The variance of $X$ is

$$\mathrm{Var}(X)=\mathbb{E}[(X-\mu {)}^2].$$

It measures how far $X$ typically deviates from its mean.

Standard deviation

The standard deviation of $X$ is the square root of the variance:

$${\sigma }_X=\sqrt{\mathrm{Var}(X)}.$$

Unlike the variance, the standard deviation has the same unit as $X$.

Useful identity

Expanding the square gives

Important

$$\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2.$$

Short proof.

$$\mathrm{Var}(X)=\mathbb{E}[(X-\mu {)}^2]=\mathbb{E}[X^2-2\mu X+{\mu }^2]=\mathbb{E}[X^2]-2\mu \mathbb{E}[X]+{\mu }^2=\mathbb{E}[X^2]-{\mu }^2.$$

Since $\mu =\mathbb{E}[X]$, this becomes $\mathrm{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X{]}^2$

Affine transformations

For constants $a,b\in \mathbb{R}$,

$$\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X).$$

So shifting by $b$ does not change the variance, and scaling by $a$ multiplies the variance by $a^2$.

For the standard deviation,

$${\sigma }_{aX+b}=|a|{\sigma }_X.$$

Discrete and continuous forms

If $X$ is discrete, then

$$\mathrm{Var}(X)=\sum _a(a-\mu {)}^2\mathrm{Pr}[X=a].$$

If $X$ is continuous with density $f_X$, then

$$\mathrm{Var}(X)={\int }_{-\infty }^{\infty }(x-\mu {)}^2{f}_X(x)dx.$$

In both cases, $\mu =\mathbb{E}[X]$.