中文版: AI 学习路线图 02 · AI 数学基础

02 · Mathematics for AI

The AI Learning Roadmap · Stage 2 of 9

---

Why This Stage Matters

Here is the part most people dread, and here is the secret that makes it easy: you only need four tools, and each one has a plain-English meaning. Machine learning is, at its heart, three operations:

1. Represent things as vectors and matrices (linear algebra).
2. Measure how wrong a model is and improve it using gradients (calculus).
3. Reason about uncertainty and data with probability and statistics.

Optimization — gradient descent — is where all three meet: it is the algorithm that makes every model learn.

You do not need to become a mathematician. You need intuition that is precise enough to reason with. Every formula in this stage is introduced with its "what it actually does" meaning first. The formulas come second, and you will implement the key ones yourself in NumPy.

Core Concepts

1. Linear algebra — the language of data

A vector is just a list of numbers. In AI, one vector = one thing: a data point, a word, an image flattened out. [0.2, 0.8, 0.5] might be one customer's three features. A matrix is a table of vectors — your whole dataset, stored as rows.

The three operations that matter:

• Vector addition and scaling — combine or stretch data. Adding two vectors shifts a point; multiplying by a scalar stretches it.
• The dot product — the workhorse of all of AI. a · b = Σ aᵢbᵢ — multiply matching entries, sum them. Its meaning: how much two vectors point in the same direction — a measure of similarity. Every neural network layer computes dot products. Every embedding similarity is a dot product.
• Matrix multiplication — a compact way to apply many dot products at once. Wx (a weight matrix times an input vector) is exactly what a neural network layer does: each row of W is a set of weights, and the result is a vector of weighted sums. Understand this one sentence and you understand the entire architecture of neural networks: layers are matrix multiplications with learnable weights.

Two more concepts you will meet constantly:

• Norms — the "length" of a vector, ‖v‖. The Euclidean norm is the straight-line distance. Used everywhere for measuring error and for normalization.
• Eigenvalues / eigenvectors and SVD — the deep structure of a matrix: which directions does this transformation stretch the most? You do not need to compute them by hand; you need to know they exist, because they power principal component analysis (PCA) — compressing data by keeping only its most important directions — and many recommendation and compression systems.

Where you will use it: every dataset is a matrix; every embedding (word, image, document) is a vector; every similarity search is dot products; every neural network is matrices.

2. Calculus — the language of improvement

A derivative answers one question: if I nudge this input a tiny bit, how much does the output change? It is a rate of change — a slope.

In machine learning we have a loss function: a number that says how wrong the model is on the data. We want to make it smaller. To do that we need to know: for each knob (parameter) of the model, which direction should I turn it, and how hard?

• The gradient is the generalization to many knobs at once: a vector of partial derivatives, one per parameter, pointing in the direction of steepest increase of the loss. We go the opposite way — hence gradient descent.
• The chain rule is the single most important idea in deep learning. It lets us compute how a change in an early layer's weight affects the final loss, by multiplying rates of change layer by layer. This is literally backpropagation: the chain rule, applied from the output backward through the network. If you truly understand the chain rule, you understand how neural networks learn.
• Partial derivatives — derivatives with respect to one variable while holding others fixed. Every weight in a network gets its own partial derivative.

Intuition to keep: learning = repeatedly (1) compute how wrong the model is, (2) compute the gradient — the direction to improve — and (3) take a small step in that direction.

3. Probability & statistics — the language of uncertainty

AI deals with uncertain predictions: "the image is 87% a cat." Probability is how we express and combine that uncertainty.

• Probability basics — events, P(A), the fact that probabilities sum to 1.
• Conditional probability & Bayes' theorem — P(A|B) = P(B|A)P(A) / P(B). Plain English: when you see new evidence B, how should you update your belief in A? Bayes' theorem is the formal version of "learning": start with a prior belief, observe data, get a posterior belief. It powers Naive Bayes classifiers, and the spirit of Bayes — updating from evidence — underlies how you should think about every model's output.
• Random variables & distributions — a random variable is a value with probabilities attached. The normal (Gaussian) distribution is the bell curve — the default assumption for measurement noise and for the initial randomness in model weights. The Bernoulli/binomial distributions describe yes/no and counts of successes. When someone says "the data is distributed like X," they mean: if I sample many values, here is the shape of the histogram.
• Expectation & variance — expectation is the average you would get after infinite samples (the center of a distribution); variance is how spread out it is. When you average a batch of loss values, you are estimating an expectation.
• Maximum likelihood estimation (MLE) — the principle behind most of statistics and much of training: choose the parameters that make the observed data most probable. Training a model is, mathematically, finding parameters that maximize the probability of the training data (with a twist for deep learning, but the spirit is identical).
• Hypothesis testing basics — p-values and statistical significance, in one paragraph: they ask "how surprising would this result be if nothing were really going on?" You will need enough to read claims critically (and to know that most A/B test results you see online deserve skepticism).

4. Optimization — how learning actually happens

Gradient descent is the algorithm that trains everything. Here is the whole thing:

1. Initialize parameters (randomly, or with a smart scheme).
2. Compute the loss over your data.
3. Compute the gradient of the loss with respect to each parameter.
4. Update each parameter: θ ← θ − η · ∂L/∂θ — step against the gradient.
5. Repeat until the loss stops improving.

Two dials matter more than anything else:

• The learning rate η — step size. Too big: the loss explodes or oscillates. Too small: training crawls. Choosing a good learning rate (and scheduling it to shrink over time) is one of the most important practical skills in all of deep learning.
• Stochastic vs batch — computing the gradient over the whole dataset each step is expensive; using one random sample (stochastic gradient descent, SGD) or a small random batch is noisy but fast, and the noise actually helps escape bad local minima.

You will also hear about convexity — for convex losses (one bowl), gradient descent provably reaches the global minimum; for non-convex losses (many valleys — the real world of deep learning), it finds good local minima. Practice cares about "good," not "provably optimal."

Tools & Skills

• NumPy — the fundamental array library. Vectors and matrices are NumPy arrays. Master: creating arrays, shape, indexing, broadcasting, dot, mean, std, random.
• Matplotlib — plotting. You will visualize data and training curves constantly. Master: line plots, scatter plots, histograms, subplots.
• Optional but recommended: a Jupyter notebook as your scratchpad for all of this.

Hands-On Tasks

1. Vector intuition (day 1–2). In NumPy: create vectors, compute dot products, norms, and angles between vectors. Plot two vectors and their sum. Convince yourself the dot product really measures similarity (dot a vector with itself vs. with its opposite).
2. Matrix as transformation (day 3). Take a 2×2 matrix and apply it to a grid of points; plot the result. See that matrices stretch and rotate space. This is what every layer of a network does to your data.
3. Gradient descent from scratch (days 4–8) — the milestone. Generate synthetic data y = 3x + 2 + noise. Write gradient descent that fits a line by minimizing mean squared error — without any ML library. Plot the loss curve falling and the fitted line converging. This one project will teach you more about how all of AI learns than a month of videos.
4. Bayes in action (day 9). The classic: a disease affects 1% of people; the test is 95% accurate. If you test positive, what's the real probability you have it? Compute it with Bayes' theorem. The (surprising, important) answer is why you should distrust raw test results without base rates.
5. Distributions (day 10). Sample 10,000 values from a normal distribution; plot the histogram. Add more samples and watch the histogram become the bell curve — the law of large numbers, made visible.

Milestone Checkpoint

You are ready for Stage 3 when:

• You implemented gradient descent from scratch in NumPy and it actually converged (loss fell, line fit).
• You can explain the chain rule's role in backpropagation in your own words.
• You can say what a dot product means, not just how to compute it.
• You can compute a Bayes' theorem example by hand and interpret the result.
• You can read θ ← θ − η·∇L and say in plain English what it does.

Curated Resources

• 3Blue1Brown — Essence of Linear Algebra and Essence of Calculus — the best intuition on the planet; watch before reading anything else.
• Mathematics for Machine Learning (Deisenroth, Faisal, Ong) — the exact right depth; free PDF. Read chapters 2–6 (linear algebra, calculus, probability, optimization).
• Khan Academy — Probability and Statistics — for gaps; go straight to the topics above, skip the rest.
• NumPy quickstart — the official 10-minute tour; do it early.

Time Estimate & Pace

• Total: ~4 weeks at 10–15 h/week.
• Suggested split: week 1 → linear algebra + tasks 1–2; week 2 → calculus intuition + gradient descent task; week 3 → probability & statistics + Bayes task; week 4 → optimization depth, review, milestone.

Bridge to the Next Stage

You now hold the three tools every model uses: vectors for representing, gradients for improving, probability for uncertainty. Stage 3 puts them to work — machine learning is simply applying these ideas to data at scale. You will meet gradient descent again in the very first hour, but this time with scikit-learn doing the heavy lifting, and you will understand what it is doing under the hood.