Intermediate26 min

Geometric, Negative Binomial and Hypergeometric

Waiting times, counts until r successes, and sampling without replacement.

Assumes you know

Bernoulli and Binomial Distributions

Geometric, Negative Binomial and Hypergeometric

Intuition first

Three distributions, each obtained by relaxing one assumption of the binomial.

The binomial fixes the number of trials and counts successes. Turn that around — fix the number of successes and count the trials needed — and you get the geometric (one success) and the negative binomial ( $r$ successes).

The hypergeometric keeps trials fixed but drops independence: it samples without replacement, so each draw changes the composition of what remains. Drawing 5 cards from a deck is hypergeometric; the binomial would be wrong because the second card's probabilities depend on the first.

Knowing which of these applies is mostly a matter of two questions: is the number of trials fixed or random, and is sampling with or without replacement?

Choosing among them

Distribution	Fixed	Counts	Replacement
Binomial	$n$ trials	successes	with
Geometric	1 success	trials needed	with
Negative binomial	$r$ successes	trials needed	with
Hypergeometric	$n$ draws	successes	without

The geometric distribution

Number of independent trials until the first success:

X \sim \text{Geometric}(p), \qquad p(k) = (1-p)^{k-1}p, \qquad k = 1, 2, 3, \dots

\E[X] = \frac{1}{p}, \qquad \Var(X) = \frac{1-p}{p^2}

Mean of the geometric, by conditioningAdvanced

The elegant route avoids summing the series. Either the first trial succeeds, or it fails and the situation resets — the memoryless property in action.

\E[X] = \underbrace{1 \cdot p}_{\text{success first try}} + \underbrace{(1-p)\big(1 + \E[X]\big)}_{\text{failure, then start over}}

The second term says: one trial was consumed, and from there the expected additional wait is the same $\E[X]$ as at the beginning. Expand:

\E[X] = p + (1-p) + (1-p)\E[X] = 1 + (1-p)\E[X]

Collect terms:

\E[X] - (1-p)\E[X] = 1 \quad\Longrightarrow\quad p\,\E[X] = 1 \quad\Longrightarrow\quad \E[X] = \frac{1}{p}

Three lines, no series manipulation. The same trick gives the variance and appears again in Markov-chain hitting times and in the RL Bellman equations, which have exactly this self-referential structure.

The geometric is the unique discrete memoryless distribution:

\Prob(X > m + n \mid X > m) = \Prob(X > n)

The negative binomial

Number of trials until the $r$ -th success:

X \sim \text{NegBin}(r, p), \qquad p(k) = \binom{k-1}{r-1}p^r(1-p)^{k-r}, \qquad k = r, r+1, \dots

\E[X] = \frac{r}{p}, \qquad \Var(X) = \frac{r(1-p)}{p^2}

A negative binomial is a sum of $r$ independent geometrics, which gives its mean and variance immediately by linearity and independence.

The hypergeometric

Population of $N$ items containing $K$ successes; draw $n$ without replacement:

p(k) = \frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}}

\E[X] = n\frac{K}{N}, \qquad \Var(X) = n\frac{K}{N}\left(1 - \frac{K}{N}\right)\frac{N-n}{N-1}

Why the correction factor appears, and when it vanishesAdvanced

Draws without replacement are negatively correlated: drawing a success makes the next draw slightly less likely to be a success, since one has been removed. Write $X = \sum_{i=1}^n \mathbb{1}_i$ where $\mathbb{1}_i$ indicates a success on draw $i$ . Then

\Var(X) = \sum_i \Var(\mathbb{1}_i) + \sum_{i \neq j}\Cov(\mathbb{1}_i, \mathbb{1}_j)

Each indicator is Bernoulli $(K/N)$ by symmetry, giving the binomial-like first term. The covariances are all negative and identical, and summing them produces exactly the factor $\frac{N-n}{N-1}$ .

Two limiting checks:

$n = 1$ : the factor is $\frac{N-1}{N-1} = 1$ . A single draw cannot depend on itself, so hypergeometric and binomial coincide.
$n = N$ : the factor is 0. Drawing the whole population always yields exactly $K$ successes, so there is no variability at all.
$N \gg n$ : the factor tends to 1, and the hypergeometric converges to Binomial $(n, K/N)$ . Removing a few items from a huge population barely changes the composition.

The usual rule: use the binomial approximation when $n \leq 0.05N$ .

Solved problem 1 · All three on one scenario

A box holds 20 components, 6 defective.

(a) Draw 5 without replacement — probability exactly 2 are defective. (b) Draw 5 with replacement — same question. (c) Drawing with replacement, probability the first defective appears on draw 4. (d) Drawing with replacement, probability the 3rd defective appears on draw 8.

Step 1 — part (a): hypergeometric

$N = 20$ , $K = 6$ , $n = 5$ , $k = 2$ .

\Prob(X = 2) = \frac{\binom{6}{2}\binom{14}{3}}{\binom{20}{5}}

\binom{6}{2} = 15, \qquad \binom{14}{3} = \frac{14 \times 13 \times 12}{6} = 364, \qquad \binom{20}{5} = 15{,}504

\Prob(X = 2) = \frac{15 \times 364}{15{,}504} = \frac{5{,}460}{15{,}504} \approx 0.35217

Step 2 — part (b): binomial with p = 6/20

p = 0.3

\Prob(X = 2) = \binom{5}{2}(0.3)^2(0.7)^3 = 10 \times 0.09 \times 0.343 = 0.3087

Step 3 — compare (a) and (b)

\text{without replacement}: 0.35217 \qquad \text{with replacement}: 0.30870

Here $n/N = 5/20 = 0.25$ , well above the 0.05 threshold, so the binomial approximation is poor — off by about 14% relative. The finite population correction is

\frac{N-n}{N-1} = \frac{15}{19} \approx 0.789

so the hypergeometric variance is only 79% of the binomial's, concentrating probability nearer the mean of $1.5$ and thus raising $\Prob(X=2)$ .

Step 4 — part (c): geometric

First success on trial 4 means three failures then a success:

\Prob(X = 4) = (0.7)^3(0.3) = 0.343 \times 0.3 = 0.1029

Step 5 — part (d): negative binomial

Third success on trial 8: among the first 7 trials there must be exactly 2 successes, then trial 8 is a success.

\Prob(X = 8) = \binom{7}{2}(0.3)^3(0.7)^5

\binom{7}{2} = 21, \qquad (0.3)^3 = 0.027, \qquad (0.7)^5 = 0.16807

= 21 \times 0.027 \times 0.16807 = 21 \times 0.00453789 \approx 0.09530

Answer

(a) $\approx 0.3522$ hypergeometric; (b) $\approx 0.3087$ binomial; (c) $\approx 0.1029$ ; (d) $\approx 0.0953$ .

With $n/N = 0.25$ the with/without distinction matters materially.

python

from scipy import stats

# (a) hypergeometric: M = population, n = successes in population, N = draws
print(f"(a) hypergeom  {stats.hypergeom(M=20, n=6, N=5).pmf(2):.5f}")
print(f"(b) binomial   {stats.binom(n=5, p=0.3).pmf(2):.5f}")
print(f"(c) geometric  {stats.geom(p=0.3).pmf(4):.5f}")
print(f"(d) negbinom   {stats.nbinom(n=3, p=0.3).pmf(8 - 3):.5f}   # nbinom counts FAILURES")

# Variance: the finite population correction.
H = stats.hypergeom(M=20, n=6, N=5)
B = stats.binom(n=5, p=0.3)
fpc = (20 - 5) / (20 - 1)
print(f"\nhypergeom var {H.var():.5f}")
print(f"binomial var  {B.var():.5f}   x fpc {fpc:.4f} = {B.var()*fpc:.5f}")

# The approximation improves as N grows with K/N fixed.
print(f"\n{'N':>8}  hypergeom P(X=2)   binomial 0.30870")
for N in (20, 100, 1000, 100_000):
    print(f"{N:8d}  {stats.hypergeom(M=N, n=int(0.3*N), N=5).pmf(2):.5f}")

Exercise 1

A salesperson closes 20% of calls. Find the expected number of calls to the first sale, and the probability it takes more than 10 calls.

Show solution

$X \sim \text{Geometric}(0.2)$ .

\E[X] = \frac{1}{0.2} = 5 \text{ calls}

For the tail, "more than 10 calls" means the first 10 all failed:

\Prob(X > 10) = (1 - 0.2)^{10} = 0.8^{10}

0.8^{10} = 0.1073741824 \approx 0.1074

So about a 10.7% chance of needing more than 10 calls.

The survival function has the clean closed form $\Prob(X > k) = (1-p)^k$ , which is often easier than summing the PMF. Note the memorylessness: after 10 failures, the expected additional calls is still 5.

Exercise 2

A shipment of 500 items contains 25 defective. 20 are inspected. Should you use hypergeometric or binomial? Compute the probability of finding no defects both ways.

Show solution

Strictly hypergeometric, since inspection is without replacement. But check the rule of thumb:

\frac{n}{N} = \frac{20}{500} = 0.04 \leq 0.05

so the binomial approximation is acceptable.

Hypergeometric, with $N = 500$ , $K = 25$ , $n = 20$ , $k = 0$ :

\Prob(X = 0) = \frac{\binom{25}{0}\binom{475}{20}}{\binom{500}{20}} = \frac{\binom{475}{20}}{\binom{500}{20}}

This equals the product of sequential probabilities:

\frac{475}{500}\cdot\frac{474}{499}\cdots\frac{456}{481} \approx 0.34122

Binomial, with $p = 25/500 = 0.05$ :

\Prob(X = 0) = (0.95)^{20} \approx 0.35849

The two differ by about 0.017, or 5% relative — small enough for most purposes, and the direction is predictable: the finite population correction

\frac{500-20}{499} \approx 0.962

reduces variance slightly, so the hypergeometric puts marginally less mass in the tail at $k=0$ .

Use the binomial if you want a quick number, the hypergeometric if the result feeds an acceptance decision where 5% matters.

Next: Gamma and Beta Distributions.