Joint, Marginal and Conditional Probability

Intuition first

Most real questions involve two things at once: age and churn, symptom and disease, word and spam. Once there are two attributes, there are three different questions you can ask, and they have three different answers.

Picture a table of 100 people split by whether they exercise and whether they sleep well.

• Joint: pick one person at random — what is the chance they exercise and sleep well? That is one cell of the table divided by 100.
• Marginal: what is the chance they exercise, ignoring sleep entirely? That is a whole row total divided by 100.
• Conditional: given that they exercise, what is the chance they sleep well? That is one cell divided by its row total — you have thrown away everyone who does not exercise.

The only mechanical difference is what you divide by. Get the denominator right and conditional probability stops being confusing.

A worked table

A survey of 100 people records exercise habit and sleep quality.

Let $E$ = "exercises" and $W$ = "sleeps well".

Joint probabilities — cells ÷ grand total

$$\Prob(E \cap W) = \frac{35}{100} = 0.35$$

Each of the four cells gives a joint probability, and the four sum to $1$:

$$0.35 + 0.15 + 0.15 + 0.35 = 1 \;\checkmark$$

Marginal probabilities — totals ÷ grand total

$$\Prob(E) = \frac{50}{100} = 0.5, \qquad \Prob(W) = \frac{50}{100} = 0.5$$

The word marginal is literal: these are the numbers written in the margins of the table.

Marginals are obtained from joints by summing over the variable you want to discard:

$$\Prob(E) = \Prob(E \cap W) + \Prob(E \cap W^c) = 0.35 + 0.15 = 0.5$$

Conditional probabilities — cell ÷ row or column total

$$\Prob(W \mid E) = \frac{35}{50} = 0.7$$

Among the 50 people who exercise, 35 sleep well. The denominator changed from 100 to 50 because conditioning on $E$ discards everyone outside $E$.

The definition of conditional probability

$$\Prob(A \mid B) = \frac{\Prob(A \cap B)}{\Prob(B)}, \qquad \Prob(B) > 0$$

Where the formula comes fromAdvanced

Conditioning on $B$ means restricting the sample space to $B$ — we now treat $B$ as the whole world. Within that restricted world, the outcomes favourable to $A$ are exactly those in $A \cap B$.

For equally likely outcomes this is transparent:

$$\Prob(A \mid B) = \frac{|A \cap B|}{|B|}$$

Divide numerator and denominator by $|\Omega|$:

$$\Prob(A \mid B) = \frac{|A \cap B| / |\Omega|}{|B| / |\Omega|} = \frac{\Prob(A \cap B)}{\Prob(B)}$$

The general definition is taken to be this ratio, which reduces to the counting version when outcomes are equally likely. Dividing by $\Prob(B)$ is precisely what re-normalises the restricted world so its probabilities sum to 1 again.

Check with the table:

$$\Prob(W \mid E) = \frac{\Prob(E \cap W)}{\Prob(E)} = \frac{0.35}{0.5} = 0.7 \;\checkmark$$

The multiplication rule

Rearranging the definition gives a rule for building joints out of conditionals:

$$\Prob(A \cap B) = \Prob(A \mid B)\,\Prob(B) = \Prob(B \mid A)\,\Prob(A)$$

Both forms are valid, which is the seed of Bayes' theorem two lessons from now.

Extended to three events (the chain rule):

$$\Prob(A \cap B \cap C) = \Prob(A)\,\Prob(B \mid A)\,\Prob(C \mid A \cap B)$$

Conditioning is not symmetric

$\Prob(A \mid B)$ and $\Prob(B \mid A)$ are different quantities. From the table:

$$\Prob(W \mid E) = \frac{35}{50} = 0.70, \qquad \Prob(E \mid W) = \frac{35}{50} = 0.70$$

Here they coincide, but only because the margins happen to be equal. Change the table so 80 people exercise and 20 do not, keeping $\Prob(E \cap W) = 0.35$:

$$\Prob(W \mid E) = \frac{0.35}{0.80} = 0.4375, \qquad \Prob(E \mid W) = \frac{0.35}{0.50} = 0.70$$

Solved problem 1 · Reading all three from one table

A shop classifies 200 visits by whether the visitor used a mobile device and whether they purchased.

	Purchased	Did not	Total
Mobile	24	96	120
Desktop	32	48	80
Total	56	144	200

Let $M$ = mobile, $B$ = purchased. Find the joint, both marginals, and both conditionals. Are $M$ and $B$ independent?

Step 1 — joint probability

$$\Prob(M \cap B) = \frac{24}{200} = 0.12$$

Step 2 — marginals from the totals

$$\Prob(M) = \frac{120}{200} = 0.60, \qquad \Prob(B) = \frac{56}{200} = 0.28$$

Step 3 — conversion rate on mobile

$$\Prob(B \mid M) = \frac{\Prob(M \cap B)}{\Prob(M)} = \frac{0.12}{0.60} = 0.20$$

Directly from the table: $24/120 = 0.20$ — 20% of mobile visits convert.

Step 4 — share of purchasers who were on mobile

$$\Prob(M \mid B) = \frac{\Prob(M \cap B)}{\Prob(B)} = \frac{0.12}{0.28} \approx 0.4286$$

Directly: $24/56 \approx 0.4286$ — about 43% of purchasers came from mobile.

Note how different these two numbers are (0.20 versus 0.43) despite sharing the same numerator. They answer different questions.

Step 5 — test independence

$$\Prob(M)\Prob(B) = 0.60 \times 0.28 = 0.168$$$$\Prob(M \cap B) = 0.12 \neq 0.168$$

Not independent. Compare the conditional with the marginal:

$$\Prob(B \mid M) = 0.20 < \Prob(B) = 0.28$$

Mobile visitors convert at 20% against a 28% baseline, so device type carries genuine information about purchasing.

Answer

$\Prob(M \cap B) = 0.12$; $\Prob(M) = 0.60$; $\Prob(B) = 0.28$; $\Prob(B \mid M) = 0.20$; $\Prob(M \mid B) \approx 0.429$. The events are not independent — mobile visitors convert below the overall rate.

Solved problem 2 · Drawing without replacement

An urn holds 5 red and 3 blue balls. Two are drawn without replacement. Find the probability that both are red.

Step 1 — name the events

Let $R_1$ = first ball red, $R_2$ = second ball red. We want $\Prob(R_1 \cap R_2)$.

Step 2 — the first draw

$$\Prob(R_1) = \frac{5}{8}$$

Step 3 — the second draw, conditioned on the first

After removing one red ball, 7 balls remain of which 4 are red:

$$\Prob(R_2 \mid R_1) = \frac{4}{7}$$

This conditional differs from $\Prob(R_2)$, which is what "without replacement" means — the draws are dependent.

Step 4 — multiplication rule

$$\Prob(R_1 \cap R_2) = \Prob(R_2 \mid R_1)\,\Prob(R_1) = \frac{4}{7} \times \frac{5}{8} = \frac{20}{56} = \frac{5}{14} \approx 0.357$$

Step 5 — verify by counting

Choose 2 of 5 reds from 2 of 8 balls:

$$\frac{\binom{5}{2}}{\binom{8}{2}} = \frac{10}{28} = \frac{5}{14} \;\checkmark$$

Answer

$\Prob(\text{both red}) = \dfrac{5}{14} \approx 0.357$.

Conditional probability obeys the axioms

For fixed $B$ with $\Prob(B) > 0$, the function $A \mapsto \Prob(A \mid B)$ is itself a valid probability measure. In particular:

$$\Prob(A^c \mid B) = 1 - \Prob(A \mid B)$$

Verifying the axioms for conditional probabilityAdvanced

Non-negativity. $\Prob(A \mid B) = \Prob(A \cap B)/\Prob(B)$ is a non-negative number divided by a positive one, so it is $\geq 0$.

Normalisation.

$$\Prob(\Omega \mid B) = \frac{\Prob(\Omega \cap B)}{\Prob(B)} = \frac{\Prob(B)}{\Prob(B)} = 1$$

Additivity. For disjoint $A_1, A_2$, the sets $A_1 \cap B$ and $A_2 \cap B$ are also disjoint, so

$$\Prob(A_1 \cup A_2 \mid B) = \frac{\Prob((A_1 \cap B) \cup (A_2 \cap B))}{\Prob(B)} = \frac{\Prob(A_1 \cap B) + \Prob(A_2 \cap B)}{\Prob(B)}$$

which is $\Prob(A_1 \mid B) + \Prob(A_2 \mid B)$.

All three axioms hold, so every theorem proved for $\Prob$ also holds for $\Prob(\cdot \mid B)$. This is why you may freely use the complement rule and inclusion–exclusion inside a conditional.

Exercise 1

From the shop table, compute $\Prob(B \mid M^c)$ — the desktop conversion rate — and confirm that the two conditional rates, weighted by their marginals, recover $\Prob(B)$.

Show solutionHide solution

$$\Prob(B \mid M^c) = \frac{32}{80} = 0.40$$

Weighting each rate by the share of traffic:

$$\Prob(B) = \Prob(B \mid M)\Prob(M) + \Prob(B \mid M^c)\Prob(M^c)$$$$= 0.20(0.60) + 0.40(0.40) = 0.12 + 0.16 = 0.28 \;\checkmark$$

This weighted-average identity is the law of total probability, covered in the next lesson.

Exercise 2

A family has two children. Given that at least one is a girl, what is the probability both are girls? Assume each child is independently a girl with probability $1/2$.

Show solutionHide solution

Sample space with birth order: $\Omega = \{GG, GB, BG, BB\}$, each of probability $1/4$.

Let $A$ = "at least one girl" $= \{GG, GB, BG\}$ and $C$ = "both girls" $= \{GG\}$.

$$\Prob(C \mid A) = \frac{\Prob(C \cap A)}{\Prob(A)} = \frac{1/4}{3/4} = \frac{1}{3}$$

Not $1/2$. Conditioning on "at least one" eliminates only $BB$, leaving three equally likely cases, of which one is $GG$.

The common wrong answer, $1/2$, comes from imagining a specific child is a girl. That is a different condition: "the elder is a girl" restricts to $\{GG, GB\}$ and does give $1/2$.

Exercise 3

Show that if $\Prob(A \mid B) > \Prob(A)$, then $\Prob(B \mid A) > \Prob(B)$.

Show solutionHide solution

Start from the assumption and substitute the definition:

$$\frac{\Prob(A \cap B)}{\Prob(B)} > \Prob(A)$$

Multiply both sides by $\Prob(B) > 0$:

$$\Prob(A \cap B) > \Prob(A)\Prob(B)$$

Now divide both sides by $\Prob(A) > 0$:

$$\frac{\Prob(A \cap B)}{\Prob(A)} > \Prob(B) \quad\Longrightarrow\quad \Prob(B \mid A) > \Prob(B)$$

So "positive association" is symmetric: if $B$ raises the chance of $A$, then $A$ raises the chance of $B$ — even though the two conditional probabilities themselves are generally different numbers.

---

Next: The Law of Total Probability, which formalises the weighted-average trick from Exercise 1 and is the direct prerequisite for Bayes' theorem.