Expectation, Variance & Linear Combinations

Figure 4.2: Distribution Center of Mass: Mean $E(X)$ vs Standard Deviation $\sigma$

1. Expected Value (Mean) of a Continuous Random Variable

The expected value $E(X)$ (or population mean $\mu$) is the center of mass of the probability density function:

$$E(X) = \mu = \int_{-\infty}^\infty x f(x)\,dx$$

For any continuous function $g(X)$ (such as $X^2$, $e^X$, etc.):

$$E(g(X)) = \int_{-\infty}^\infty g(x) f(x)\,dx$$

2. Variance and Standard Deviation

The variance $\text{Var}(X) = \sigma^2$ measures the dispersion of the distribution around its mean:

$$\text{Var}(X) = E\left((X - \mu)^2\right) = E(X^2) - [E(X)]^2$$

where the second moment is computed by:

$$E(X^2) = \int_{-\infty}^\infty x^2 f(x)\,dx$$

The standard deviation is the positive square root: $\sigma = \sqrt{\text{Var}(X)}$.

3. Linear Transformations of a Single Variable $aX + b$

Expectation: $E(aX + b) = aE(X) + b$.
Variance: $\text{Var}(aX + b) = a^2 \text{Var}(X)$. (Adding constant $b$ does not change spread!)
Standard deviation: $\sigma_{aX + b} = |a|\sigma_X$.

4. Linear Combinations of Independent Random Variables (Syllabus 4.15)

Let $X$ and $Y$ be random variables, and let $a, b \in \mathbb{R}$:

Expectation Rule (Always holds): $$E(aX + bY) = aE(X) + bE(Y)$$
Variance Rule (Requires $X$ and $Y$ to be INDEPENDENT): $$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$$ $$\text{Var}(aX - bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$$
Critical Rule: Variances ALWAYS add! Subtracting random variables increases uncertainty, so $\text{Var}(X - Y) = \text{Var}(X) + \text{Var}(Y)$, NEVER minus!

Worked Examples

Worked Example 1: Analytic Mean and Variance of a Continuous Distribution

A continuous random variable $X$ has probability density function defined by:

$$f(x) = \begin{cases} 3x^2 & 0 \le x \le 1 \\ 0 & \text{otherwise} \end{cases}$$

(a) Calculate $E(X)$. [2 marks]
(b) Calculate $E(X^2)$ and hence find the exact value of $\text{Var}(X)$. [3 marks]
(c) Determine the standard deviation $\sigma$ to 3 significant figures. [1 mark]

Full Markscheme Solution:

(a) Expected Value $E(X)$:

$$E(X) = \int_0^1 x f(x)\,dx = \int_0^1 x(3x^2)\,dx = \int_0^1 3x^3\,dx \quad [M1]$$ $$= \left[ \frac{3x^4}{4} \right]_0^1 = \frac{3(1)^4}{4} - 0 = \frac{3}{4} = 0.75 \quad [A1]$$

(b) Second Moment & Variance:

$$E(X^2) = \int_0^1 x^2 f(x)\,dx = \int_0^1 x^2(3x^2)\,dx = \int_0^1 3x^4\,dx \quad [M1]$$ $$= \left[ \frac{3x^5}{5} \right]_0^1 = \frac{3}{5} = 0.6 \quad [A1]$$

Using the variance formula $\text{Var}(X) = E(X^2) - [E(X)]^2$:

$$\text{Var}(X) = \frac{3}{5} - \left(\frac{3}{4}\right)^2 = \frac{3}{5} - \frac{9}{16} = \frac{48 - 45}{80} = \frac{3}{80} = 0.0375 \quad [A1]$$

(c) Standard Deviation:

$$\sigma = \sqrt{\text{Var}(X)} = \sqrt{\frac{3}{80}} = \frac{\sqrt{15}}{20} \approx 0.194 \quad [A1]$$

Examiner Commentary: Notice that because the density $3x^2$ increases toward $x = 1$, the center of mass $E(X) = 0.75$ is pulled toward the right. Always check that $E(X^2) > [E(X)]^2$, as variance must strictly be non-negative.

Worked Example 2: Linear Combinations of Independent Measurements

Two independent random variables $X$ and $Y$ have means and variances given by:
$E(X) = 20, \quad \text{Var}(X) = 4$
$E(Y) = 15, \quad \text{Var}(Y) = 9$
Let $W = 3X - 2Y + 5$.
(a) Find the expected value $E(W)$. [2 marks]
(b) Find the variance $\text{Var}(W)$. [3 marks]
(c) Find the exact standard deviation $\sigma_W$. [1 mark]

Full Markscheme Solution:

(a) Expected Value $E(W)$:

Apply the linearity property of expectation:

$$E(W) = E(3X - 2Y + 5) = 3E(X) - 2E(Y) + 5 \quad [M1]$$ $$= 3(20) - 2(15) + 5 = 60 - 30 + 5 = 35 \quad [A1]$$

(b) Variance $\text{Var}(W)$:

Because $X$ and $Y$ are independent, the variances add with coefficients squared, and constant shifts have zero variance:

$$\text{Var}(W) = \text{Var}(3X - 2Y + 5) = 3^2\text{Var}(X) + (-2)^2\text{Var}(Y) \quad [M1]$$ $$= 9\text{Var}(X) + 4\text{Var}(Y) \quad [A1]$$ $$= 9(4) + 4(9) = 36 + 36 = 72 \quad [A1]$$

(c) Standard Deviation:

$$\sigma_W = \sqrt{\text{Var}(W)} = \sqrt{72} = 6\sqrt{2} \approx 8.49 \quad [A1]$$

Examiner Commentary: A very common student error is writing $\text{Var}(3X - 2Y) = 9\text{Var}(X) - 4\text{Var}(Y) = 36 - 36 = 0$. Variances measure uncertainty and dispersion; subtracting random variables always INCREASES total variability!

Worked Example 3: Sum of $n$ Independent Identical Observations vs $n X$

A random variable $X$ has mean $\mu = 10$ and variance $\sigma^2 = 4$. Compare the following two quantities:
$T = X_1 + X_2 + X_3 + X_4$ (the sum of 4 independent observations of $X$).
$S = 4X$ (a single observation of $X$ scaled by 4).
(a) Find the mean and variance of $T$. [3 marks]
(b) Find the mean and variance of $S$. [3 marks]
(c) Explain why $\text{Var}(S)$ is substantially greater than $\text{Var}(T)$. [1 mark]

Full Markscheme Solution:

(a) Sum of 4 Independent Observations $T$:

$$E(T) = E(X_1) + E(X_2) + E(X_3) + E(X_4) = 4\mu = 4(10) = 40 \quad [A1]$$

Since the 4 observations are independent, their variances add directly:

$$\text{Var}(T) = \text{Var}(X_1) + \text{Var}(X_2) + \text{Var}(X_3) + \text{Var}(X_4) = 4\sigma^2 \quad [M1]$$ $$= 4(4) = 16 \implies \sigma_T = \sqrt{16} = 4 \quad [A1]$$

(b) Scaled Single Variable $S = 4X$:

$$E(S) = 4E(X) = 4(10) = 40 \quad [A1]$$

For a scalar multiple, the scaling constant is squared:

$$\text{Var}(S) = \text{Var}(4X) = 4^2\text{Var}(X) = 16\sigma^2 \quad [M1]$$ $$= 16(4) = 64 \implies \sigma_S = \sqrt{64} = 8 \quad [A1]$$

(c) Examiner Insight & Justification:

In $T$, independent observations cancel out each other's deviations (an unusually large $X_1$ may be offset by an unusually small $X_2$). In $S$, any fluctuation in the single measurement $X$ is amplified fourfold, producing 4 times the variance ($\text{Var}(S) = 4 \times \text{Var}(T)$) and twice the standard deviation $[R1]$.

Worked Example 4: Integration by Parts for Continuous Expectation & Variance

The lifetime $X$ (in hundreds of hours) of a specialized LED lamp has probability density function:

$$f(x) = \begin{cases} 2e^{-2x} & x \ge 0 \\ 0 & x < 0 \end{cases}$$

(a) Using integration by parts, show that $E(X) = \frac{1}{2}$. [3 marks]
(b) Show that $E(X^2) = \frac{1}{2}$. [3 marks]
(c) Hence determine the variance $\text{Var}(X)$ and standard deviation $\sigma_X$. [2 marks]

Full Markscheme Solution:

(a) Calculating $E(X)$:

$$E(X) = \int_0^\infty x(2e^{-2x})\,dx = 2 \int_0^\infty x e^{-2x}\,dx \quad [M1]$$

Use integration by parts with $u = x \implies u' = 1$, and $v' = e^{-2x} \implies v = -\frac{1}{2}e^{-2x}$:

$$\int_0^\infty x e^{-2x}\,dx = \left[ -\frac{x}{2}e^{-2x} \right]_0^\infty - \int_0^\infty \left(-\frac{1}{2}e^{-2x}\right)\,dx \quad [A1]$$

By l'Hôpital's rule, $\lim_{x \to \infty} \frac{x}{e^{2x}} = 0$, so the boundary term vanishes at both limits:

$$= 0 + \frac{1}{2} \left[ -\frac{1}{2}e^{-2x} \right]_0^\infty = \frac{1}{2}\left( 0 - \left(-\frac{1}{2}\right) \right) = \frac{1}{4}$$

Multiply by the prefactor 2:

$$E(X) = 2 \left(\frac{1}{4}\right) = \frac{1}{2} \quad [AG]$$

(b) Calculating $E(X^2)$:

$$E(X^2) = \int_0^\infty x^2(2e^{-2x})\,dx = 2 \int_0^\infty x^2 e^{-2x}\,dx \quad [M1]$$

Parts with $u = x^2 \implies u' = 2x$, and $v' = e^{-2x} \implies v = -\frac{1}{2}e^{-2x}$:

$$\int_0^\infty x^2 e^{-2x}\,dx = \left[ -\frac{x^2}{2}e^{-2x} \right]_0^\infty - \int_0^\infty 2x\left(-\frac{1}{2}e^{-2x}\right)\,dx = 0 + \int_0^\infty x e^{-2x}\,dx \quad [A1]$$

From part (a), $\int_0^\infty x e^{-2x}\,dx = \frac{1}{4}$. Therefore:

$$E(X^2) = 2 \left(\frac{1}{4}\right) = \frac{1}{2} \quad [AG]$$

(c) Variance and Standard Deviation:

$$\text{Var}(X) = E(X^2) - [E(X)]^2 = \frac{1}{2} - \left(\frac{1}{2}\right)^2 = \frac{1}{2} - \frac{1}{4} = \frac{1}{4} \quad [A1]$$ $$\sigma_X = \sqrt{\frac{1}{4}} = \frac{1}{2} \quad [A1]$$

(Remarkable property: For any exponential distribution $f(x) = \lambda e^{-\lambda x}$, the mean and standard deviation are always identical: $\mu = \sigma = \frac{1}{\lambda}$.)

Worked Example 5: Multi-Component Industrial Assembly & Net Difference Modeling

A manufacturing plant produces precision gears $A$ and shafts $B$.
The weight of gear $A$, $W_A$, has mean $\mu_A = 120$ g and standard deviation $\sigma_A = 6$ g.
The weight of shaft $B$, $W_B$, has mean $\mu_B = 85$ g and standard deviation $\sigma_B = 4$ g.
All component weights are mutually independent.
A transmission assembly consists of $3$ gears of type $A$ and $2$ shafts of type $B$, mounted inside a casing of fixed weight $150$ g.
(a) Find the expected total weight $E(T)$ and the standard deviation $\sigma_T$ of the transmission assembly. [4 marks]
(b) Find the mean and variance of the weight difference $D = (A_1 + A_2 + A_3) - (B_1 + B_2)$. [3 marks]
(c) Assuming the total weight $T$ is approximately normally distributed, find $P(T > 700 \text{ g})$. [2 marks]

Full Markscheme Solution:

(a) Total Assembly Weight:

Let $T = A_1 + A_2 + A_3 + B_1 + B_2 + 150$:

$$E(T) = 3E(W_A) + 2E(W_B) + 150 = 3(120) + 2(85) + 150 = 360 + 170 + 150 = 680 \text{ g} \quad [A1]$$

Because the components are independent, their variances add directly (and constant 150 has zero variance):

$$\text{Var}(T) = 3\text{Var}(W_A) + 2\text{Var}(W_B) \quad [M1]$$ $$= 3(6^2) + 2(4^2) = 3(36) + 2(16) = 108 + 32 = 140 \text{ g}^2 \quad [A1]$$ $$\sigma_T = \sqrt{140} = 2\sqrt{35} \approx 11.83 \text{ g} \quad [A1]$$

(b) Weight Difference $D$:

$$E(D) = 3E(W_A) - 2E(W_B) = 3(120) - 2(85) = 360 - 170 = 190 \text{ g} \quad [A1]$$

Variances add regardless of the subtraction:

$$\text{Var}(D) = 3\text{Var}(W_A) + 2\text{Var}(W_B) = 3(36) + 2(16) = 140 \text{ g}^2 \quad [M1][A1]$$

Notice that $\text{Var}(D) = \text{Var}(T)$!

(c) Normal Probability Calculation:

With $T \sim N(680, 140)$:

$$Z = \frac{700 - 680}{\sqrt{140}} = \frac{20}{11.832} \approx 1.690 \quad [M1]$$ $$P(T > 700) = P(Z > 1.690) = 1 - \Phi(1.690) \approx 1 - 0.9545 = 0.0455 \text{ (to 3 s.f.)} \quad [A1]$$

Examiner Commentary: Candidates often mistakenly write $\text{Var}(T) = 3^2\text{Var}(A) + 2^2\text{Var}(B) = 9(36) + 4(16) = 388$. That would represent buying ONE gear and scaling its weight by 3! When assembling 3 distinct items, write $A_1 + A_2 + A_3$ so the variance is $3\sigma^2$, not $9\sigma^2$.

Practice Exercises (10 Tiered Questions)

Try these 10 exam-style questions, from core skills to harder problems. Open each question to view its solution.

Exercise 1 · Expected Value Calculation

A random variable $X$ has PDF $f(x) = 2x$ for $0 \le x \le 1$. Calculate $E(X)$.

Show Full Solution

$E(X) = \int_0^1 x(2x)\,dx = \int_0^1 2x^2\,dx = \left[\frac{2x^3}{3}\right]_0^1 = \frac{2}{3}$. [A1]

Exercise 2 · Variance from Moments

Given that $E(X) = 5$ and $E(X^2) = 29$, calculate $\text{Var}(X)$ and the standard deviation.

Show Full Solution

$\text{Var}(X) = E(X^2) - [E(X)]^2 = 29 - 5^2 = 29 - 25 = 4$. [A1]
Standard deviation $\sigma = \sqrt{4} = 2$. [A1]

Exercise 3 · Linear Scaling

If $E(X) = 12$ and $\text{Var}(X) = 9$, find $E(2X - 5)$ and $\text{Var}(2X - 5)$.

Show Full Solution

$E(2X - 5) = 2E(X) - 5 = 2(12) - 5 = 19$. [A1]
$\text{Var}(2X - 5) = 2^2\text{Var}(X) = 4(9) = 36$. [A1]

Exercise 4 · Variance of Continuous Distribution

For $f(x) = \frac{1}{2}$ on $[0, 2]$ (uniform distribution), calculate $\text{Var}(X)$ by integration.

Show Full Solution

$E(X) = \int_0^2 x(1/2)\,dx = \left[\frac{x^2}{4}\right]_0^2 = 1$. [M1]
$E(X^2) = \int_0^2 x^2(1/2)\,dx = \left[\frac{x^3}{6}\right]_0^2 = \frac{8}{6} = \frac{4}{3}$. [M1]
$\text{Var}(X) = \frac{4}{3} - 1^2 = \frac{1}{3}$. [A1]

Exercise 5 · Variance of Difference of Variables

Independent variables $A$ and $B$ have variances $\text{Var}(A) = 16$ and $\text{Var}(B) = 25$. Calculate $\text{Var}(A - B)$ and $\text{Var}(2A - 3B)$.

Show Full Solution

$\text{Var}(A - B) = \text{Var}(A) + (-1)^2\text{Var}(B) = 16 + 25 = 41$. [A1]
$\text{Var}(2A - 3B) = 2^2\text{Var}(A) + (-3)^2\text{Var}(B) = 4(16) + 9(25) = 64 + 225 = 289$. [A1]

Exercise 6 · Expected Value of Non-linear Function

If $X$ has PDF $f(x) = e^{-x}$ for $x \ge 0$, calculate $E(e^{X/2})$.

Show Full Solution

$E(e^{X/2}) = \int_0^\infty e^{x/2} e^{-x}\,dx = \int_0^\infty e^{-x/2}\,dx$. [M1]
$= \left[ -2e^{-x/2} \right]_0^\infty = 0 - (-2) = 2$. [A1]

Exercise 7 · Elevator Weight Capacity Model

An elevator holds 8 passengers whose weights $W_i$ are independent random variables with mean $70$ kg and standard deviation $10$ kg. Find the mean and standard deviation of total weight $T = \sum_{i=1}^8 W_i$.

Show Full Solution

$E(T) = \sum_{i=1}^8 E(W_i) = 8(70) = 560$ kg. [A1]
Since the weights are independent:
$\text{Var}(T) = \sum_{i=1}^8 \text{Var}(W_i) = 8(10^2) = 8(100) = 800$ kg$^2$. [M1]
Standard deviation $\sigma_T = \sqrt{800} = 20\sqrt{2} \approx 28.3$ kg. [A1]

Exercise 8 · Linear Algebra of Sample Mean

Let $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$ be the sample mean of $n$ independent identical observations with mean $\mu$ and variance $\sigma^2$. Prove that $E(\bar{X}) = \mu$ and $\text{Var}(\bar{X}) = \frac{\sigma^2}{n}$.

Show Full Solution

$E(\bar{X}) = E\left(\frac{1}{n}\sum X_i\right) = \frac{1}{n}\sum E(X_i) = \frac{1}{n}(n\mu) = \mu$. [M1][A1]
$\text{Var}(\bar{X}) = \text{Var}\left(\frac{1}{n}\sum X_i\right) = \left(\frac{1}{n}\right)^2 \text{Var}\left(\sum X_i\right)$. [M1]
Since $X_i$ are independent, $\text{Var}\left(\sum X_i\right) = \sum \text{Var}(X_i) = n\sigma^2$.
$\text{Var}(\bar{X}) = \frac{1}{n^2}(n\sigma^2) = \frac{\sigma^2}{n}$. [A1]

Exercise 9 · Mean of Symmetric Continuous Distribution

Let $f(x)$ be a PDF symmetric about $x = c$, so that $f(c - u) = f(c + u)$. Prove that $E(X) = c$.

Show Full Solution

$E(X) = \int_{-\infty}^\infty x f(x)\,dx$. Let $u = x - c \implies x = c + u$. [M1]
$E(X) = \int_{-\infty}^\infty (c + u)f(c + u)\,du = c\int_{-\infty}^\infty f(c + u)\,du + \int_{-\infty}^\infty u f(c + u)\,du$. [M1]
Since $\int_{-\infty}^\infty f(c + u)\,du = 1$ (total probability) and $u f(c + u)$ is an odd function of $u$ ($(-u)f(c - u) = -u f(c + u)$), its integral is 0! [R1]
Therefore $E(X) = c(1) + 0 = c$. [A1]

Exercise 10 · Profit Optimization Model

A company sells two products with independent profits $X$ and $Y$ having $E(X) = 500, \sigma_X = 50$ and $E(Y) = 800, \sigma_Y = 80$. If the company forms a combined portfolio $P = 2X + 3Y$, find the probability that total profit exceeds $3200$, assuming $P$ is approximately normally distributed.

Show Full Solution

$E(P) = 2(500) + 3(800) = 1000 + 2400 = 3400$. [A1]
$\text{Var}(P) = 2^2(50^2) + 3^2(80^2) = 4(2500) + 9(6400) = 10000 + 57600 = 67600$. [M1]
$\sigma_P = \sqrt{67600} = 260$. [A1]
$z$-score for $3200$: $z = \frac{3200 - 3400}{260} = \frac{-200}{260} \approx -0.769$. [M1]
$P(P > 3200) = P(Z > -0.769) = \Phi(0.769) \approx 0.779$. [A1]

Formula Booklet References

Syllabus 4.14, 4.15: Expectation & Variance

$$E(X) = \int_{-\infty}^\infty x f(x)\,dx$$ $$\text{Var}(X) = \int_{-\infty}^\infty (x - \mu)^2 f(x)\,dx = \int_{-\infty}^\infty x^2 f(x)\,dx - \mu^2$$ $$E(aX + b) = aE(X) + b, \quad \text{Var}(aX + b) = a^2\text{Var}(X)$$ $$\text{For independent } X, Y: \quad E(aX + bY) = aE(X) + bE(Y)$$ $$\text{Var}(aX \pm bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$$

Key Formulas & Identities

1. Variance Always Adds!

$$\text{Var}(X - Y) = \text{Var}(X) + \text{Var}(Y) \quad (\text{NEVER } -)$$

Subtracting random variables compounds uncertainty and widens dispersion. The negative coefficient becomes positive when squared: $(-1)^2 = +1$.

2. Sum of $n$ Independent Observations vs Scaled Single Variable

$$\text{Var}\left(\sum_{i=1}^n X_i\right) = n\sigma^2 \iff \sigma_{\text{sum}} = \sqrt{n}\sigma$$ $$\text{Var}(nX) = n^2\sigma^2 \iff \sigma_{nX} = n\sigma$$

Scaling a single measurement by $n$ produces $n$ times the variance of taking $n$ independent observations.

3. Non-Linear Functions (Jensen's Principle)

$$E(g(X)) = \int_{-\infty}^\infty g(x)f(x)\,dx \neq g(E(X))$$

In general, $E(X^2) \neq [E(X)]^2$, $E(1/X) \neq 1/E(X)$, and $E(e^X) \neq e^{E(X)}$.

Common Exam Pitfalls

Independence Requirement for Variance

$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$ ONLY holds if $X$ and $Y$ are independent! In contrast, expectation $E(aX + bY) = aE(X) + bE(Y)$ always holds regardless of whether the variables are independent or correlated.

Squaring the Scale Factor in Variance

Remember that variance scales with the SQUARE of the coefficient: $\text{Var}(3X) = 9\text{Var}(X)$, NOT $3\text{Var}(X)$. Standard deviation scales linearly: $\sigma_{aX} = |a|\sigma_X$.

Confusing $\sum_{i=1}^n X_i$ with $n X$

In word problems describing batches, packets, or collections of items (e.g., "a box containing 6 apples"), always use $X_1 + X_2 + \dots + X_6$ with variance $6\sigma^2$. Writing $6X$ with variance $36\sigma^2$ is an extremely common error that loses all method marks.

Adding Constants to Variance

Adding a constant amount shifts the distribution without stretching it: $\text{Var}(X + 10) = \text{Var}(X)$. Never add constants to variance calculations!

Figure 4.2: Distribution Center of Mass: Mean $E(X)$ vs Standard Deviation $\sigma$

1. Expected Value (Mean) of a Continuous Random Variable

2. Variance and Standard Deviation

3. Linear Transformations of a Single Variable $aX + b$

4. Linear Combinations of Independent Random Variables (Syllabus 4.15)

Worked Examples

Worked Example 1: Analytic Mean and Variance of a Continuous Distribution

Worked Example 2: Linear Combinations of Independent Measurements

Worked Example 3: Sum of $n$ Independent Identical Observations vs $n X$

Worked Example 4: Integration by Parts for Continuous Expectation & Variance

Worked Example 5: Multi-Component Industrial Assembly & Net Difference Modeling

Practice Exercises (10 Tiered Questions)

Formula Booklet References

Syllabus 4.14, 4.15: Expectation & Variance

Key Formulas & Identities

1. Variance Always Adds!

2. Sum of $n$ Independent Observations vs Scaled Single Variable

3. Non-Linear Functions (Jensen's Principle)

Common Exam Pitfalls

Independence Requirement for Variance

Squaring the Scale Factor in Variance

Confusing $\sum_{i=1}^n X_i$ with $n X$

Adding Constants to Variance

Topic Diagnostic Test

Section A: Diagnostic Multiple Choice Questions

Section B: Examination-Style Free Response Questions