Beginner26 min

Bernoulli and Binomial Distributions

Single trials and counts of successes: PMF derivation, moments, and the normal approximation.

Assumes you know

Variance, Moments and Generating Functions

Bernoulli and Binomial Distributions

Intuition first

A Bernoulli trial is one yes/no experiment: a coin flip, a click, a component passing inspection. One parameter, $p$ , the probability of success.

A binomial counts successes across $n$ independent Bernoulli trials with the same $p$ . How many heads in 10 flips, how many of 500 visitors convert, how many of 20 components pass.

Everything about the binomial follows from one observation: it is a sum of independent Bernoullis. The mean follows by linearity, the variance by independence, and the PMF by counting which arrangements of successes are possible. No new machinery is needed — which is why this is the right place to start with named distributions.

The assumptions are worth naming, because they are what break in practice: fixed $n$ , independent trials, constant $p$ . Correlated trials or drifting $p$ produce overdispersion, and a binomial model will then understate uncertainty.

Bernoulli

X \sim \text{Bernoulli}(p), \qquad \Prob(X = 1) = p, \quad \Prob(X = 0) = 1 - p

p(x) = p^x (1-p)^{1-x}, \qquad x \in \{0, 1\}

\E[X] = p, \qquad \Var(X) = p(1-p)

Deriving the Bernoulli momentsAdvanced

\E[X] = 0 \cdot (1-p) + 1 \cdot p = p

For the variance, note that $X^2 = X$ because $0^2 = 0$ and $1^2 = 1$ . So $\E[X^2] = \E[X] = p$ , and

\Var(X) = \E[X^2] - (\E[X])^2 = p - p^2 = p(1-p)

The variance is maximised at $p = 0.5$ , where it equals $0.25$ , and vanishes at $p = 0$ or $p = 1$ — a certain outcome has no variability. This is why estimating a proportion near 0.5 requires the largest sample, and why the sample-size formula in the splits lesson used $p(1-p) \leq 0.25$ as a worst case.

Binomial

X \sim \text{Binomial}(n, p), \qquad p(k) = \binom{n}{k} p^k (1-p)^{n-k}, \qquad k = 0, 1, \dots, n

Notation used in this lesson
Symbol	Meaning	Read aloud
n	Number of independent trials (fixed in advance)	n
p	Probability of success on each trial	p
k	Number of successes observed	k
C(n,k)	Number of ways to arrange k successes among n trials	n choose k

Mean and variance from the Bernoulli decompositionAdvanced

Write $X = \sum_{i=1}^{n} X_i$ with each $X_i \sim \text{Bernoulli}(p)$ independent.

Mean, by linearity — no independence needed:

\E[X] = \sum_{i=1}^n \E[X_i] = np

Variance. Here independence is required, so that all covariance terms vanish:

\Var(X) = \sum_{i=1}^n \Var(X_i) = np(1-p)

Contrast with the direct route, which requires evaluating

\sum_{k=0}^n k \binom{n}{k}p^k(1-p)^{n-k}

and manipulating factorials. The decomposition into indicators avoids all of it — the same technique that solved the envelope problem in the expectation lesson.

MGF, also immediate from the decomposition. A single Bernoulli has $M(t) = (1-p) + pe^t$ , and MGFs of independent sums multiply:

M_X(t) = \big(1 - p + pe^t\big)^n

From which it follows at once that $\text{Bin}(n_1, p) + \text{Bin}(n_2, p) = \text{Bin}(n_1 + n_2, p)$ for independent terms with the same $p$ — the MGFs multiply to give the same form with exponent $n_1 + n_2$ .

\E[X] = np, \qquad \Var(X) = np(1-p), \qquad \sigma = \sqrt{np(1-p)}

Solved problem 1 · Quality inspection

A factory produces components with a 4% defect rate. A sample of 20 is inspected. Find the probability of (a) exactly 1 defective, (b) at most 1 defective, (c) at least 2 defective. Also give the mean and standard deviation.

Step 1 — identify the distribution and check assumptions

Fixed $n = 20$ , constant $p = 0.04$ , and components assumed independent. So $X \sim \text{Binomial}(20, 0.04)$ .

Step 2 — exactly one defective

\Prob(X = 1) = \binom{20}{1}(0.04)^1 (0.96)^{19}

\binom{20}{1} = 20

0.96^{19}: \quad \ln(0.96) = -0.040822, \quad 19 \times (-0.040822) = -0.775618, \quad e^{-0.775618} = 0.46040

\Prob(X = 1) = 20 \times 0.04 \times 0.46040 = 0.8 \times 0.46040 = 0.36832

Step 3 — zero defectives

\Prob(X = 0) = \binom{20}{0}(0.04)^0(0.96)^{20} = 1 \times 1 \times 0.96^{20}

20 \times (-0.040822) = -0.816440, \quad e^{-0.816440} = 0.44200

\Prob(X = 0) = 0.44200

Step 4 — at most one

\Prob(X \leq 1) = \Prob(X = 0) + \Prob(X = 1) = 0.44200 + 0.36832 = 0.81032

Step 5 — at least two, by complement

\Prob(X \geq 2) = 1 - \Prob(X \leq 1) = 1 - 0.81032 = 0.18968

Note carefully: $\Prob(X \geq 2) = 1 - \Prob(X \leq 1)$ , not $1 - \Prob(X \leq 2)$ . The latter would wrongly exclude $X = 2$ .

Step 6 — mean and standard deviation

\E[X] = np = 20 \times 0.04 = 0.8

\Var(X) = np(1-p) = 20 \times 0.04 \times 0.96 = 0.768

\sigma = \sqrt{0.768} \approx 0.8764

Answer

$\Prob(X=1) \approx 0.3683$ ; $\Prob(X \leq 1) \approx 0.8103$ ; $\Prob(X \geq 2) \approx 0.1897$ . Mean $0.8$ defectives, $\sigma \approx 0.876$ .

Solved problem 2 · When to use the normal approximation

A website has a 12% conversion rate. Of 500 visitors, what is the probability that more than 70 convert?

Step 1 — exact setup

$X \sim \text{Binomial}(500, 0.12)$ , and we want $\Prob(X > 70) = \Prob(X \geq 71)$ .

Step 2 — check the approximation conditions

np = 500 \times 0.12 = 60 \geq 5 \;\checkmark

n(1-p) = 500 \times 0.88 = 440 \geq 5 \;\checkmark

Both comfortably satisfied, so a normal approximation is appropriate.

Step 3 — matching moments

\mu = np = 60, \qquad \sigma = \sqrt{np(1-p)} = \sqrt{500 \times 0.12 \times 0.88} = \sqrt{52.8} \approx 7.2664

Step 4 — apply the continuity correction

$X$ is integer-valued and the normal is continuous. $\Prob(X \geq 71)$ corresponds to the normal area above $70.5$ , not above 71 — the correction splits the gap between adjacent integers:

z = \frac{70.5 - 60}{7.2664} = \frac{10.5}{7.2664} \approx 1.4450

Step 5 — look up the tail

\Prob(Z > 1.445) \approx 0.0742

Step 6 — compare with the exact binomial

The exact value is $\Prob(X \geq 71) \approx 0.0751$ . The approximation with continuity correction gives $0.0742$ — an error of about 0.001.

Without the correction, using $z = (71-60)/7.2664 = 1.514$ , we would get $0.0650$ — an error of about $0.010$ , ten times worse.

Answer

About $0.074$ , against an exact $0.075$ . The continuity correction reduces the error roughly tenfold and costs nothing.

When the binomial model is wrong

Violation	What happens	Use instead
Sampling without replacement from a small population	$p$ changes between draws	Hypergeometric
Trials are correlated	Variance exceeds $np(1-p)$	Beta-binomial
$p$ varies across trials	Overdispersion	Beta-binomial, or mixed model
$n$ not fixed in advance	Different sample space	Negative binomial or Poisson
$n$ large, $p$ small, $np$ moderate	Binomial still correct but awkward	Poisson approximation

python

import numpy as np
from scipy import stats

# Worked example 1, exact.
X = stats.binom(n=20, p=0.04)
print(f"P(X=1)   {X.pmf(1):.5f}")
print(f"P(X<=1)  {X.cdf(1):.5f}")
print(f"P(X>=2)  {X.sf(1):.5f}   <- sf(1) = 1 - F(1), NOT sf(2)")
print(f"mean {X.mean():.3f}  sd {X.std():.4f}")

# Worked example 2: exact vs normal, with and without continuity correction.
Y = stats.binom(n=500, p=0.12)
mu, sd = 60, np.sqrt(52.8)
exact = Y.sf(70)
with_cc = stats.norm.sf((70.5 - mu) / sd)
without_cc = stats.norm.sf((71 - mu) / sd)
print(f"\nexact          {exact:.5f}")
print(f"normal + cc    {with_cc:.5f}   error {abs(with_cc-exact):.5f}")
print(f"normal, no cc  {without_cc:.5f}   error {abs(without_cc-exact):.5f}")

# Verify Var = np(1-p) empirically, and show what overdispersion looks like.
rng = np.random.default_rng(0)
indep = rng.binomial(20, 0.04, 200_000)
print(f"\nindependent trials: var {indep.var():.4f}  theory {20*0.04*0.96:.4f}")
p_varying = rng.beta(2, 48, 200_000)          # p drifts around 0.04
over = rng.binomial(20, p_varying)
print(f"p varies by batch:  var {over.var():.4f}  <- overdispersed")

Exercise 1

A basketball player makes 70% of free throws. What is the probability she makes exactly 8 of 10?

Show solution

$X \sim \text{Binomial}(10, 0.7)$ .

\Prob(X = 8) = \binom{10}{8}(0.7)^8(0.3)^2

\binom{10}{8} = \binom{10}{2} = \frac{10 \times 9}{2} = 45

0.7^8 = 0.05764801, \qquad 0.3^2 = 0.09

\Prob(X = 8) = 45 \times 0.05764801 \times 0.09 = 45 \times 0.005188321 \approx 0.2335

About 23.3%. Note this is the single most likely value after $X = 7$ : the mode of a binomial is near $np = 7$ , and $\Prob(X=7) \approx 0.2668$ .

Exercise 2

In 100 coin flips of a fair coin, use the normal approximation to find the probability of between 45 and 55 heads inclusive.

Show solution

$X \sim \text{Binomial}(100, 0.5)$ , so

\mu = 50, \qquad \sigma = \sqrt{100 \times 0.5 \times 0.5} = \sqrt{25} = 5

Conditions: $np = 50 \geq 5$ and $n(1-p) = 50 \geq 5$ ✓.

Applying the continuity correction, "between 45 and 55 inclusive" becomes the interval $[44.5, 55.5]$ :

z_1 = \frac{44.5 - 50}{5} = -1.10, \qquad z_2 = \frac{55.5 - 50}{5} = +1.10

\Prob(-1.10 < Z < 1.10) = 2\Phi(1.10) - 1 = 2(0.8643) - 1 = 0.7286

The exact binomial value is $0.7287$ — agreement to four decimal places, because $p = 0.5$ makes the binomial perfectly symmetric and the normal approximation is at its best.

Without the correction, using $[45, 55]$ , you would get $z = \pm 1.0$ and $0.6827$ — an error of about 4.6 percentage points.

Next: Geometric, Negative Binomial and Hypergeometric, which relax the binomial's assumptions one at a time.