Advanced28 min

Joint Continuous Distributions

Joint densities, conditional density functions, marginalisation by integration and independence.

Assumes you know

Joint, Marginal and Conditional Probability

Joint Continuous Distributions

Intuition first

With one continuous variable, probability is an area under a curve. With two, it becomes a volume under a surface. The joint density $f(x,y)$ is a height above the plane, and the probability of landing in a region is the volume sitting over it.

Everything from the discrete case carries across with sums replaced by integrals. Marginalising means integrating out the variable you do not care about. Conditioning means slicing the surface at a fixed value and renormalising so the slice integrates to 1.

The one genuinely new difficulty is the limits of integration. When the support is not a rectangle — say $0 < x < y < 1$ — getting the limits right is most of the work, and sketching the region first is not optional advice but the fastest route to a correct answer.

Joint density

\Prob\big((X,Y) \in A\big) = \iint_A f(x,y)\,dx\,dy

with

f(x,y) \geq 0, \qquad \int_{-\infty}^{\infty}\!\!\int_{-\infty}^{\infty} f(x,y)\,dx\,dy = 1

Notation used in this lesson
Symbol	Meaning	Read aloud
f(x,y)	Joint density	f of x y
fₓ(x)	Marginal density of X	f sub X of x
f(y\|x)	Conditional density of Y given X = x	f of y given x
∬	Double integral over a region	double integral

Marginals: integrate out

f_X(x) = \int_{-\infty}^{\infty} f(x,y)\,dy, \qquad f_Y(y) = \int_{-\infty}^{\infty} f(x,y)\,dx

Conditionals: slice and renormalise

f(y \mid x) = \frac{f(x,y)}{f_X(x)}, \qquad f_X(x) > 0

The division by $f_X(x)$ is what makes the slice a valid density integrating to 1.

Independence holds exactly when the joint factorises:

X \ind Y \iff f(x,y) = f_X(x)\,f_Y(y) \text{ for all } x, y

Solved problem 1 · A triangular support, worked completely

$f(x,y) = 8xy$ on the region $0 < x < y < 1$ , zero elsewhere.

(a) Verify it is a density. (b) Find both marginals. (c) Find $f(y \mid x)$ . (d) Determine whether $X$ and $Y$ are independent. (e) Find $\Prob(Y > 2X)$ .

Step 1 — sketch the region, then choose an integration order

The support is the triangle with vertices $(0,0)$ , $(0,1)$ , $(1,1)$ — above the line $y = x$ inside the unit square.

Two valid orders:

\text{fix } x: \; x < y < 1, \; 0 < x < 1 \qquad\text{or}\qquad \text{fix } y: \; 0 < x < y, \; 0 < y < 1

Step 2 — part (a): verify normalisation

Integrate with $y$ outer, $x$ inner:

\int_0^1\!\!\int_0^y 8xy\,dx\,dy = \int_0^1 8y\left[\frac{x^2}{2}\right]_0^y dy = \int_0^1 8y\cdot\frac{y^2}{2}\,dy = \int_0^1 4y^3\,dy

= \left[y^4\right]_0^1 = 1 \;\checkmark

Step 3 — part (b): marginal of X

Fix $x$ and integrate $y$ from $x$ to 1:

f_X(x) = \int_x^1 8xy\,dy = 8x\left[\frac{y^2}{2}\right]_x^1 = 4x\big(1 - x^2\big), \qquad 0 < x < 1

Check: $\int_0^1 (4x - 4x^3)dx = [2x^2 - x^4]_0^1 = 2 - 1 = 1 \;\checkmark$

Step 4 — marginal of Y

Fix $y$ and integrate $x$ from 0 to $y$ :

f_Y(y) = \int_0^y 8xy\,dx = 8y\left[\frac{x^2}{2}\right]_0^y = 4y^3, \qquad 0 < y < 1

Check: $\int_0^1 4y^3 dy = 1 \;\checkmark$

Step 5 — part (c): the conditional density

f(y \mid x) = \frac{f(x,y)}{f_X(x)} = \frac{8xy}{4x(1-x^2)} = \frac{2y}{1 - x^2}, \qquad x < y < 1

Verify it integrates to 1 over its range:

\int_x^1 \frac{2y}{1-x^2}\,dy = \frac{1}{1-x^2}\left[y^2\right]_x^1 = \frac{1 - x^2}{1-x^2} = 1 \;\checkmark

Step 6 — part (d): independence

f_X(x)f_Y(y) = 4x(1-x^2)\cdot 4y^3 = 16xy^3(1-x^2)

which is not $8xy$ . So not independent.

The structural reason is decisive on its own: the support is a triangle, so the range of $Y$ depends on $X$ . Knowing $X = 0.9$ forces $Y \in (0.9, 1)$ ; knowing $X = 0.1$ allows $Y \in (0.1, 1)$ .

Step 7 — part (e): P(Y > 2X)

We need the part of the triangle also satisfying $y > 2x$ . Since $y > 2x$ implies $y > x$ for positive $x$ , the region is simply $0 < 2x < y < 1$ , so $0 < x < 1/2$ .

\Prob(Y > 2X) = \int_0^{1/2}\!\!\int_{2x}^{1} 8xy\,dy\,dx

Inner integral:

\int_{2x}^1 8xy\,dy = 8x\left[\frac{y^2}{2}\right]_{2x}^1 = 4x\big(1 - 4x^2\big) = 4x - 16x^3

Outer:

\int_0^{1/2}\big(4x - 16x^3\big)dx = \left[2x^2 - 4x^4\right]_0^{1/2} = 2\cdot\tfrac14 - 4\cdot\tfrac{1}{16} = \tfrac12 - \tfrac14 = \tfrac14

Answer

(a) integrates to 1 ✓; (b) $f_X(x) = 4x(1-x^2)$ , $f_Y(y) = 4y^3$ ; (c) $f(y\mid x) = \dfrac{2y}{1-x^2}$ on $x < y < 1$ ; (d) not independent; (e) $\Prob(Y > 2X) = \tfrac14$ .

Expectations over joint distributions

\E[g(X,Y)] = \iint g(x,y)\,f(x,y)\,dx\,dy

Covariance follows as before:

\Cov(X,Y) = \E[XY] - \E[X]\E[Y]

Solved problem 2 · Covariance from a joint density

For the distribution above, compute $\Cov(X, Y)$ .

Step 1 — E[X] from its marginal

\E[X] = \int_0^1 x\cdot 4x(1-x^2)\,dx = \int_0^1 \big(4x^2 - 4x^4\big)dx = \left[\frac{4x^3}{3} - \frac{4x^5}{5}\right]_0^1

= \frac{4}{3} - \frac{4}{5} = \frac{20 - 12}{15} = \frac{8}{15} \approx 0.53333

Step 2 — E[Y] from its marginal

\E[Y] = \int_0^1 y\cdot 4y^3\,dy = \int_0^1 4y^4\,dy = \left[\frac{4y^5}{5}\right]_0^1 = \frac{4}{5} = 0.8

Step 3 — E[XY] from the joint

\E[XY] = \int_0^1\!\!\int_0^y xy\cdot 8xy\,dx\,dy = \int_0^1\!\!\int_0^y 8x^2y^2\,dx\,dy

Inner:

\int_0^y 8x^2y^2\,dx = 8y^2\left[\frac{x^3}{3}\right]_0^y = \frac{8y^5}{3}

Outer:

\int_0^1 \frac{8y^5}{3}\,dy = \frac{8}{3}\left[\frac{y^6}{6}\right]_0^1 = \frac{8}{18} = \frac{4}{9} \approx 0.44444

Step 4 — assemble

\Cov(X,Y) = \frac{4}{9} - \frac{8}{15}\cdot\frac{4}{5} = \frac{4}{9} - \frac{32}{75}

Common denominator $225$ :

= \frac{100}{225} - \frac{96}{225} = \frac{4}{225} \approx 0.017778

Step 5 — interpret the sign

Positive, as the geometry demands: the constraint $x < y$ means large $x$ forces large $y$ . The magnitude is small because $Y$ is confined to $(0,1)$ and heavily concentrated near 1 — its marginal $4y^3$ puts most mass at the top — leaving little room to vary with $X$ .

Answer

$\Cov(X,Y) = \dfrac{4}{225} \approx 0.0178$ — small and positive.

python

import numpy as np
from scipy import integrate

f = lambda x, y: 8 * x * y

# Normalisation over 0 < x < y < 1, integrating x from 0 to y.
total, _ = integrate.dblquad(f, 0, 1, lambda y: 0, lambda y: y)
print(f"total probability {total:.10f}")

# Marginals, checked against the closed forms.
for x0 in (0.2, 0.5, 0.8):
    num, _ = integrate.quad(lambda y: f(x0, y), x0, 1)
    print(f"f_X({x0}) numeric {num:.6f}   closed form {4*x0*(1-x0**2):.6f}")

# P(Y > 2X).
p, _ = integrate.dblquad(f, 0, 0.5, lambda x: 2*x, lambda x: 1)
print(f"\nP(Y > 2X) {p:.6f}   theory {0.25:.6f}")

# Moments and covariance.
EX, _ = integrate.quad(lambda x: x * 4*x*(1-x**2), 0, 1)
EY, _ = integrate.quad(lambda y: y * 4*y**3, 0, 1)
EXY, _ = integrate.dblquad(lambda x, y: x*y*f(x, y), 0, 1, lambda y: 0, lambda y: y)
print(f"\nE[X] {EX:.6f} (8/15 = {8/15:.6f})")
print(f"E[Y] {EY:.6f} (4/5  = {0.8:.6f})")
print(f"Cov  {EXY - EX*EY:.6f} (4/225 = {4/225:.6f})")

# Rejection sampling as an independent check.
rng = np.random.default_rng(0)
x, y = rng.random(4_000_000), rng.random(4_000_000)
keep = x < y
w = 8 * x[keep] * y[keep]                      # importance weights
w /= w.sum()
idx = rng.choice(keep.sum(), size=500_000, p=w)
xs, ys = x[keep][idx], y[keep][idx]
print(f"\nsampled: E[X] {xs.mean():.4f}  E[Y] {ys.mean():.4f}  Cov {np.cov(xs, ys)[0,1]:.5f}")

Exercise 1

$f(x,y) = x + y$ on the unit square $0 < x, y < 1$ . Find $f_X(x)$ and determine independence.

Show solution

First confirm it is a density:

\int_0^1\!\!\int_0^1 (x+y)\,dx\,dy = \int_0^1\left[\frac{x^2}{2} + xy\right]_0^1 dy = \int_0^1\left(\frac12 + y\right)dy = \frac12 + \frac12 = 1 \;\checkmark

Marginal of $X$ :

f_X(x) = \int_0^1 (x+y)\,dy = \left[xy + \frac{y^2}{2}\right]_0^1 = x + \frac{1}{2}, \qquad 0 < x < 1

By symmetry $f_Y(y) = y + \tfrac12$ .

Test factorisation:

f_X(x)f_Y(y) = \left(x + \tfrac12\right)\left(y + \tfrac12\right) = xy + \tfrac{x}{2} + \tfrac{y}{2} + \tfrac14

which is not $x + y$ . So $X$ and $Y$ are dependent.

Note that here the support is a rectangle, so the rectangle test does not settle it — the dependence comes from the algebraic form, since $x + y$ is a sum rather than a product. A joint density is separable only if it factorises as $g(x)h(y)$ , and a sum never does.

Exercise 2

$X$ and $Y$ are independent $\text{Uniform}(0,1)$ . Find $\Prob(X + Y < 1)$ and $\Prob(XY < 0.5)$ .

Show solution

Independence gives $f(x,y) = 1$ on the unit square, so probabilities are just areas.

First part. The region $x + y < 1$ inside the unit square is the triangle below the diagonal from $(1,0)$ to $(0,1)$ :

\Prob(X+Y<1) = \frac{1}{2}\times 1 \times 1 = \frac{1}{2}

Second part. The region $xy < 0.5$ excludes the area above the hyperbola $y = 0.5/x$ . That curve enters the square at $x = 0.5$ (where $y = 1$ ) and leaves at $x = 1$ (where $y = 0.5$ ).

\Prob(XY \geq 0.5) = \int_{0.5}^{1}\left(1 - \frac{0.5}{x}\right)dx = \Big[x - 0.5\ln x\Big]_{0.5}^{1}

= (1 - 0) - (0.5 - 0.5\ln 0.5) = 0.5 + 0.5\ln 0.5 = 0.5 - 0.5(0.693147) = 0.153426

\Prob(XY < 0.5) = 1 - 0.153426 = 0.846574

The contrast is instructive: the sum splits the square evenly, while the product is below 0.5 nearly 85% of the time. Multiplying two numbers less than 1 usually gives something much smaller, so products of uniforms concentrate near zero — which is why $-\ln(U)$ , used in inverse transform sampling, spreads them back out.

Next: Sums of Random Variables and Convolution.