Calculus and Optimisation

Limits from both sides, the epsilon–delta definition, continuity and the classification of discontinuities.

The derivative as a limit, differentiability vs continuity, and the product, quotient and chain rules.

Assumes: Functions, Limits and Continuity

The theorems that license almost every later bound, with the geometric picture and standard applications.

Assumes: Differentiability and Rules of Differentiation

Polynomial approximation, the remainder term, and why second-order Taylor expansion underpins Newton's method.

Assumes: Rolle's and the Mean Value Theorems

Critical points, first and second derivative tests, and distinguishing local from global extrema.

Assumes: Differentiability and Rules of Differentiation

Closed-form optimisation, boundary cases, and numerical line search methods including golden section.

Assumes: Maxima and Minima of One Variable

Partials, directional derivatives, the gradient as the direction of steepest ascent, and level sets.

Assumes: Differentiability and Rules of Differentiation

First- and second-order derivative matrices for vector-valued functions, and what Hessian definiteness tells you.

Assumes: Partial Derivatives and the Gradient · Quadratic Forms and Definiteness

Composing vector functions, the chain rule in matrix form, and its direct role in backpropagation.

Assumes: The Jacobian and the Hessian

Derivatives with respect to vectors and matrices, layout conventions, and a reference table of standard identities.

Assumes: The Multivariable Chain Rule

Convexity tests, Jensen's inequality, and why convex problems have no bad local minima.

Assumes: The Jacobian and the Hessian

Equality-constrained optimisation, the geometric meaning of multipliers, and shadow prices.

Assumes: Convex Sets and Convex Functions

Inequality constraints, complementary slackness, and the conditions that define the SVM dual.

Assumes: Lagrange Multipliers

The update rule, step-size selection, convergence on convex objectives, and failure modes.

Assumes: Partial Derivatives and the Gradient

Second-order optimisation, Newton's method, and why BFGS and L-BFGS approximate the Hessian instead.

Assumes: Gradient Descent · Taylor and Maclaurin Series

Standard form problems, Lagrangian duality, weak and strong duality, and the duality gap.

Assumes: KKT Conditions

Finite differences and their error, forward vs reverse mode automatic differentiation, and gradient checking.

Assumes: The Multivariable Chain Rule