Imre Nagi
Math for AI

The maths behind machine learning.

Software practitioners comfortable with code, no university maths assumed.

Start lesson 1See the modules
A projection plot: the blue observation x = (3, 1), its green projection p = (2.8, 1.4) on the dashed line through b, and the short residual between them
Drag the point yourself in lesson 50.

51 of 180

lessons published, one connected step at a time.

Latest
Projection Matrices Transform Whole Datasets
Next
Lesson 52: Least squares is a projection problem
Modules
12, from notation to support vector machines

Twelve modules, in order

  1. 1

    Mathematical Foundations

    Notation, functions, sets, sums, logs and reading a derivation.

    10 of 10 published

    1. 1Variables, values, and AI models
    2. 2Functions as reusable mathematical machines
    3. 3Domain, codomain, and valid model inputs
    4. 4Sets, membership, and datasets
    5. 5Indices, subscripts, and tensor addresses
    6. 6Sums, products, and loss aggregation
    7. 7Powers, roots, and model scaling
    8. 8Logarithms turn products into sums
    9. 9Equations, constraints, and solution sets
    10. 10Reading a derivation line by line

  2. 2

    Linear Algebra

    Systems of equations, vectors, matrices, spaces and maps.

    28 of 28 published

    1. 11One linear equation as a geometric line
    2. 12Systems of equations as intersections
    3. 13No solution, one solution, or many solutions
    4. 14Vectors as representations of AI data
    5. 15Vector addition combines signals
    6. 16Scalar multiplication controls strength
    7. 17Linear combinations build new vectors
    8. 18Matrices organize linear rules
    9. 19Matrix-vector products as weighted sums
    10. 20Matrix-matrix products compose transformations and batches
    11. 21Transpose reorients relationships
    12. 22Identity and inverse transformations
    13. 23Gaussian elimination preserves the solution set
    14. 24Row-echelon form reveals structure
    15. 25Numerical solving without explicit inverses
    16. 26Why algebraic structures determine valid operations
    17. 27Vector spaces define legal representations
    18. 28Subspaces capture restricted feature families
    19. 29Linear independence removes redundancy
    20. 30Span describes what a representation can express
    21. 31Bases give compact coordinate systems
    22. 32Dimension counts independent directions
    23. 33Rank measures effective linear information
    24. 34Linear mappings preserve combinations
    25. 35Matrices represent linear mappings
    26. 36Kernel and image explain lost and reachable information
    27. 37Affine spaces add offsets to linear structure
    28. 38Linear algebra in a tiny neural network

  3. 3

    Analytic Geometry

    Norms, inner products, angles, orthogonality and projection.

    13 of 16 published

    1. 39Norms measure vector size
    2. 40L1, L2, and L-infinity see size differently
    3. 41Inner products measure alignment
    4. 42Dot products turn features into scores
    5. 43Distance depends on the chosen geometry
    6. 44Angles and cosine similarity
    7. 45Orthogonality means no linear overlap
    8. 46Orthonormal bases simplify coordinates
    9. 47Gram–Schmidt builds orthogonal directions
    10. 48Orthogonal complements capture what is left
    11. 49Functions can have inner products too
    12. 50Projection finds the nearest point
    13. 51Projection matrices transform whole datasets
    3 upcoming
    1. 52Least squares is a projection problem
    2. 53Rotations preserve length and angle
    3. 54Geometry of similarity search

  4. 4

    Matrix Decompositions

    Determinants, eigenvalues, Cholesky and the SVD.

    0 of 18 published

    Starts at lesson 55.

    18 upcoming
    1. 55Determinants measure oriented volume
    2. 56Singular matrices collapse information
    3. 57Trace summarizes diagonal action
    4. 58Eigenvectors keep their direction
    5. 59Eigenvalues measure directional growth
    6. 60Characteristic polynomials find eigenvalues
    7. 61Eigenvectors in Markov and ranking systems
    8. 62Cholesky factorization exploits positive definiteness
    9. 63Diagonalization decouples a transformation
    10. 64Symmetric matrices have friendly eigenstructure
    11. 65Singular value decomposition factorizes any matrix
    12. 66Singular values rank directions by strength
    13. 67Pseudoinverses solve nonsquare systems
    14. 68Low-rank approximation keeps the strongest patterns
    15. 69Matrix compression for model weights
    16. 70Conditioning predicts numerical sensitivity
    17. 71NumPy factorizations and reconstruction checks
    18. 72How decompositions power modern AI systems

  5. 5

    Vector Calculus

    Derivatives, gradients, the chain rule and backpropagation.

    0 of 26 published

    Starts at lesson 73.

    26 upcoming
    1. 73Change, slopes, and local predictions
    2. 74Limits make instantaneous change precise
    3. 75Derivative rules avoid repeating limits
    4. 76The chain rule traces composed change
    5. 77Derivatives of common activation functions
    6. 78Partial derivatives hold other inputs fixed
    7. 79Gradients collect steepest local change
    8. 80Directional derivatives query any direction
    9. 81Jacobians map vector-to-vector sensitivity
    10. 82The Jacobian chain rule composes layers
    11. 83Matrix gradients track every parameter
    12. 84Trace tricks simplify matrix derivatives
    13. 85Computation graphs make dependencies visible
    14. 86Forward-mode automatic differentiation propagates tangents
    15. 87Reverse-mode automatic differentiation propagates adjoints
    16. 88Backpropagation through a dense layer
    17. 89Gradient checking catches derivative bugs
    18. 90PyTorch autograd records the graph
    19. 91Hessians measure multivariate curvature
    20. 92Second derivatives classify critical points
    21. 93Linearization approximates a model locally
    22. 94Taylor series add curvature information
    23. 95Vanishing gradients from repeated chain rules
    24. 96Exploding gradients and stability controls
    25. 97Attention gradients connect token influence
    26. 98The calculus of a tiny training step

  6. 6

    Probability & Distributions

    Random variables, Bayes, Gaussians and expectations.

    0 of 28 published

    Starts at lesson 99.

    28 upcoming
    1. 99Uncertainty needs a mathematical language
    2. 100Sample spaces, events, and outcomes
    3. 101Probability axioms constrain valid assignments
    4. 102Conditional probability updates the reference set
    5. 103Random variables map outcomes to numbers
    6. 104Discrete distributions assign probability mass
    7. 105Continuous distributions use density, not point mass
    8. 106Cumulative distributions answer threshold questions
    9. 107Sum and product rules build and reduce joint probabilities
    10. 108Bayes' theorem reverses a condition
    11. 109Base rates change what a model alert means
    12. 110Expectation is a probability-weighted average
    13. 111Variance measures spread around expectation
    14. 112Covariance tracks joint movement
    15. 113Covariance matrices summarize many features
    16. 114Independence is stronger than zero covariance
    17. 115Empirical estimates learn statistics from samples
    18. 116Gaussian distributions link location and scale
    19. 117Standardization creates comparable coordinates
    20. 118Multivariate Gaussians shape uncertainty with covariance
    21. 119Gaussian conditionals and marginals
    22. 120Sampling correlated Gaussians with Cholesky
    23. 121Exponential families share a common form
    24. 122Conjugate priors make Bayesian updates tractable
    25. 123Maximum likelihood connects observed data to parameters
    26. 124Change of variables transforms densities
    27. 125Inverse-transform sampling turns uniform noise into data
    28. 126Probability in a classifier pipeline

  7. 7

    Continuous Optimization

    Gradient descent, constraints and convexity.

    0 of 16 published

    Starts at lesson 127.

    16 upcoming
    1. 127Objective functions turn learning into search
    2. 128Gradient descent follows local downhill information
    3. 129Learning rates control step size
    4. 130Loss landscapes contain valleys, plateaus, and saddles
    5. 131Stochastic gradients trade noise for speed
    6. 132Momentum accumulates a velocity
    7. 133Adaptive optimizers rescale coordinate updates
    8. 134Constrained optimization separates goals and rules
    9. 135Lagrange multipliers balance objective and constraint
    10. 136Regularization can act like a soft constraint
    11. 137Convex sets contain the lines between points
    12. 138Convex functions have no bad local minima
    13. 139Convexity gives optimization certificates
    14. 140Optimization in PyTorch without a black box
    15. 141Debugging training with mathematical signals
    16. 142Design an optimizer for a toy model

  8. 8

    Models & Data

    Data, models, learning and evaluation.

    0 of 9 published

    Starts at lesson 143.

    9 upcoming
    1. 143Data, models, and learning rules play different roles
    2. 144Supervised learning approximates an unknown map
    3. 145Empirical risk turns examples into an objective
    4. 146Loss functions encode which errors matter
    5. 147Parameter estimation connects probability and optimization
    6. 148Probabilistic modeling specifies a generative story
    7. 149Inference asks about hidden quantities
    8. 150Directed graphs encode conditional structure
    9. 151Model selection balances fit and complexity

  9. 9

    Linear Regression

    Least squares, MLE, MAP and Bayesian regression.

    0 of 8 published

    Starts at lesson 152.

    8 upcoming
    1. 152Linear regression maps features to numeric targets
    2. 153Least-squares estimation from projection geometry
    3. 154Maximum likelihood explains squared error under Gaussian noise
    4. 155Regularized regression controls unstable weights
    5. 156Bayesian linear regression puts a distribution on weights
    6. 157Posterior predictive uncertainty varies by input
    7. 158Linear regression implementation and diagnostics
    8. 159When a linear model is the right baseline

  10. 10

    Principal Component Analysis

    Dimensionality reduction by projection.

    0 of 8 published

    Starts at lesson 160.

    8 upcoming
    1. 160Dimensionality reduction preserves selected structure
    2. 161PCA finds directions of maximum variance
    3. 162PCA is also a minimum-error projection
    4. 163SVD computes PCA without forming covariance
    5. 164PCA in high-dimensional AI data
    6. 165Practical PCA: centering, scaling, and component choice
    7. 166Probabilistic PCA introduces latent variables
    8. 167PCA for embedding inspection, with caveats

  11. 11

    Gaussian Mixture Models

    Density estimation with EM.

    0 of 6 published

    Starts at lesson 168.

    6 upcoming
    1. 168Mixtures model data with multiple overlapping groups
    2. 169Latent assignments turn mixtures into a hidden-variable model
    3. 170Mixture likelihood couples unknown parameters
    4. 171Expectation–maximization alternates inference and fitting
    5. 172Implement a GMM and watch EM converge
    6. 173GMM failure modes and model selection

  12. 12

    Support Vector Machines

    Margins, duality and kernels.

    0 of 7 published

    Starts at lesson 174.

    7 upcoming
    1. 174Hyperplanes turn dot products into decision boundaries
    2. 175Margin measures classification robustness
    3. 176The primal SVM balances margin and classification errors
    4. 177The dual SVM centers training examples
    5. 178Kernels create nonlinear boundaries through inner products
    6. 179Train and diagnose an SVM
    7. 180From mathematical foundations to ML models

Applied notes

The same maths, put to work on current AI.

Where this came from

The lessons follow the dependency order of Deisenroth, Faisal and Ong’s Mathematics for Machine Learning, with original bridges, visuals and labs, and every lesson ends with worked questions you can check step by step. They were first published on Beyond the Hype, where new lessons still appear first.