Probability and Statistics Formula Sheet

Probability Basics

• Addition Rule: P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
• Complement Rule: P(A') = 1 - P(A)
• Conditional Probability: P(B | A) = P(A ∩ B) / P(A) (if P(A) > 0)
• Law of Total Probability: P(B) = Σ P(B | A_i)P(A_i)
• Bayes' Theorem: P(A_k | B) = P(B | A_k)P(A_k) / Σ P(B | A_i)P(A_i)
• Independence: P(A ∩ B) = P(A)P(B) → then P(A | B) = P(A)

Counting Principles

• Multiplication Rule: n₁ × n₂ × ...
• Permutation (Order Matters): nPr = n! / (n - r)!
• Combination (No Order): C(n, r) = n! / (r!(n - r)!)
• Distinguishable Permutations: n! / (n₁! n₂! ...)
• Probability (Equal Outcomes): P(A) = (# favorable) / (# total)

Discrete Distributions

Mean: E[X] = Σ x f(x)   Variance: Var(X) = E[X²] - (E[X])²

• Binomial(n, p): P(X = x) = C(n, x)p^x(1 - p)^{n - x}, E = np, Var = np(1 - p)
• Geometric(p): P(X = x) = p(1 - p)^{x - 1}, x = 1, 2, ...   E = 1/p, Var = (1 - p) / p²
• Negative Binomial(r, p): P(X = x) = C(x - 1, r - 1)p^r(1 - p)^{x - r}, E = r/p, Var = r(1 - p) / p²
• Poisson(λ): P(X = x) = e^-λλ^x / x!, E = λ, Var = λ
• Hypergeometric(N1, N2, n): P(X = x) = C(N1, x)C(N2, n - x) / C(N1 + N2, n), E = n(N1 / (N1 + N2))

Continuous Distributions

• Uniform(a, b): f(x) = 1 / (b - a), E = (a + b) / 2, Var = (b - a)² / 12
• Exponential(θ): f(x) = (1/θ)e^-x/θ, E = θ, Var = θ², memoryless
• Gamma(α, θ): f(x) = x^{α - 1}e^-x/θ / (θ^α Γ(α)), E = αθ, Var = αθ²
• Chi-square(ν): Special Gamma(ν/2, 2), E = ν, Var = 2ν
• Normal(μ, σ²): f(x) = 1 / (σ√(2π)) e^{-(x - μ)2 / (2σ2)}, E = μ, Var = σ²

Moment Generating Functions (MGF)

• Definition: M(t) = E[e^tX], then E[Xⁿ] = M⁽ⁿ⁾(0)
• Independent Sum: M_{X+Y}(t) = MX(t) MY(t)}

Two Random Variables

• Joint PMF/PDF: Sum/Integral = 1
• Marginal: f_X(x) = Σ_y f(x, y) or ∫ f(x, y) dy
• Conditional: f_{X|Y}(x|y) = f(x, y) / fY(y)}
• Independence: f(x, y) = f_X(x)f_Y(y) for all x, y.
• Adam's Law (Total Expectation): E[X] = E[E[X | Y]]
• Eve's Law (Total Variance): Var(X) = Var(E[X | Y]) + E[Var(X | Y)]
• Covariance: Cov(X, Y) = E[XY] - E[X]E[Y]
• Correlation: ρ = Cov(X, Y) / (σ_X σ_Y), -1 ≤ ρ ≤ 1
• Note: If independent → Cov = 0 (reverse not true unless bivariate normal).
• Variance of Sum: Var(X ± Y) = Var(X) + Var(Y) ± 2Cov(X, Y)
• Bivariate Normal: X | Y = y ~ N( μ_X + ρ(σ_X / σ_Y)(y - μ_Y), σ_X²(1 - ρ²) )
• Note: If ρ = 0 then X and Y are independent (unique to bivariate normal).

Transformations

• Continuous, Monotone: Y = g(X), inverse X = v(Y) → f_Y(y) = f_X(v(y)) · |v'(y)|
• Two Variables: Jacobian J = determinant of partial derivatives, f_{U,V} = fX,Y} · |J|}

Sample Mean and Variance

For a Normal sample:

• X̄ ~ N(μ, σ² / n)
• (n - 1)S² / σ² ~ χ²(n - 1) with S² = Σ(X_i - X̄)² / (n - 1)
• X̄ and S² are independent.

Central Limit Theorem (CLT)

For i.i.d. variables with mean μ, variance σ², and large n:

• (X̄ - μ) / (σ / √n) ≈ N(0, 1) and (ΣX_i - nμ) / (√n σ) ≈ N(0, 1)
• Continuity Correction: For discrete variables, use ±0.5 around the integer.

Common MGF Reference

• Binomial: (1 - p + pe^t)ⁿ
• Poisson: e^{λ(et - 1)}
• Geometric: pe^t / (1 - (1 - p)e^t)
• Negative Binomial: [pe^t / (1 - (1 - p)e^t)]^r
• Normal: e^{μt + σ2t2 / 2}
• Gamma: (1 - θt)^-α
• Chi-square: (1 - 2t)^-ν/2