中文版: AI 学习路线图 03 · 机器学习基础

03 · Machine Learning Fundamentals

The AI Learning Roadmap · Stage 3 of 9

---

Why This Stage Matters

Machine learning is the paradigm underneath everything else in this roadmap. Deep learning, large language models, and agents are all instances of the same core idea: learn a function from examples instead of hand-writing rules. If you master the concepts of this stage — the train/test split, overfitting, evaluation — then everything that follows is a matter of scale and architecture, not of new fundamentals.

This stage also gives you your first real toolkit (pandas + scikit-learn), your first real datasets, and your first end-to-end projects. By the end, you will be able to take any tabular dataset and produce a clean, evaluated, defensible model. That is a professional skill in itself.

Core Concepts

1. The learning paradigm

Every ML problem has the same shape:

• Features X — the inputs (a row of numbers describing one example).
• Labels y — the target we want to predict.
• A model — a function f that maps X → y, defined by learnable parameters.
• Training — finding parameters that make f accurate on the data we have.
• Inference — using the trained f on new, unseen data.

The one idea that separates ML from ordinary programming: we never write the rules; the algorithm discovers them from data. A spam filter is not a list of if-statements; it is a model that learned "these patterns → spam" from labeled examples.

Supervised learning uses labeled data (X with known y): predicting a number is regression; predicting a category is classification. Unsupervised learning finds structure in unlabeled data: grouping similar things (clustering) or compressing data (dimensionality reduction).

2. The central problem: generalization and overfitting

Here is the most important concept in all of machine learning: a model that memorizes the training data but fails on new data is worthless. The goal is generalization — performing well on data the model has never seen.

• Overfitting — the model is too flexible; it learns noise along with signal. It nails the training data and fails on new data. (Symptom: train accuracy high, validation accuracy low.)
• Underfitting — the model is too simple to capture the pattern at all. Both scores are poor.
• Bias–variance tradeoff — underfitting is high bias (the model's assumptions are wrong); overfitting is high variance (the model changes wildly with the data). Model selection is finding the sweet spot.

The defenses against overfitting, which you will use forever:

1. Hold-out data. Always split: train / validation / test. Train on train; tune on validation; evaluate once on test (which must stay untouched until the end — evaluating on it repeatedly leaks information).
2. Cross-validation — when data is scarce, repeat train/validate splits in folds (e.g., 5-fold) and average the results for a more reliable estimate.
3. Regularization — explicitly penalize complexity. L1/L2 penalties shrink weights; simpler trees; more data; early stopping. Regularization is the "discipline" that keeps a flexible model honest.
4. Learning curves — plot train vs. validation error vs. training size. They diagnose at a glance: overfit, underfit, or need-more-data.

3. Supervised learning: the classic algorithms

You will implement none of these by hand and understand all of them conceptually. That is the right balance for a practitioner.

• Linear regression — fit a line (or hyperplane) ŷ = wx + b by minimizing squared error. The interpretable baseline: simple, fast, and often good enough. (This is the model you fit with gradient descent in Stage 2.)
• Logistic regression — regression's cousin for classification: it outputs a probability (0–1) via the sigmoid function, then thresholds it. Despite the name, it is a classifier. The go-to strong baseline.
• k-Nearest Neighbors (k-NN) — predict by looking at the k closest training examples and voting. No training at all; slow at inference; a great mental model for "similar inputs → similar outputs."
• Decision trees — recursively split the data on the feature that best separates classes. Interpretable (you can read the rules) but prone to overfitting. The building block of the best classical models.
• Random forests — many trees, each trained on a random subset of data and features; average their votes. The variance of one overfit tree is tamed by averaging many. Robust, no tuning, excellent default.
• Gradient boosting (XGBoost / LightGBM) — trees trained sequentially, each one fixing the errors of the previous. The king of tabular data — for years the algorithm that won most Kaggle competitions. Worth knowing: gradient boosting is conceptually the same "reduce the error iteratively" loop as gradient descent, but over trees.
• Support vector machines (SVM) — find the decision boundary with the widest margin to the data. Elegant, but in practice trees and boosting usually win on tabular data; know what it is, don't obsess over it.

Rule of thumb for tabular data: start with logistic regression (baseline), then random forest, then gradient boosting. The fanciest deep learning model will usually lose to a good gradient-boosted tree on a clean table of numbers — deep learning wins where data is high-dimensional and unstructured (images, text, audio).

4. Unsupervised learning

• k-means clustering — group data into k clusters by iteratively assigning points to the nearest centroid and moving centroids to the mean of their points. The workhorse of segmentation and customer grouping.
• PCA (principal component analysis) — find the directions of maximum variance and project data onto them. Used for: visualization (2D plots), noise reduction, and speeding up models. (This is where those eigenvectors from Stage 2 show up.)

5. Evaluation: the language of "is it good?"

You must know, for every model you ever build, which metric answers the question that matters:

• Regression: MSE/MAE (error in the same units as y), R² (fraction of variance explained).
• Classification: accuracy (correct / total — misleading when classes are imbalanced); precision (of the things predicted positive, how many were right — relevant when false positives are costly); recall (of the actual positives, how many were found — relevant when missing positives is costly); F1 (the harmonic mean, the standard single number for imbalanced problems); ROC-AUC (ranking quality across all thresholds).
• The confusion matrix — the 2×2 table (true/false × positive/negative) that makes precision/recall concrete. Always look at it before trusting an accuracy number.

The habit to build now: define the metric before building the model. "95% accurate" is meaningless without knowing the base rate (if 95% of emails are spam, predicting "spam" always gives 95% accuracy).

6. The end-to-end workflow

This is the shape of every ML project you will ever do:

1. Understand the problem — what would success look like for the business/user? Define the metric.
2. Data acquisition & cleaning — find the data; handle missing values, outliers, duplicate rows, inconsistent types. (80% of real ML work is here.)
3. Exploratory data analysis (EDA) — distributions, correlations, relationships. Plot everything. Understand the data before modeling it.
4. Baseline — train the simplest reasonable model (logistic regression / mean prediction). You cannot know if a fancy model is good until you know what "obviously achievable" looks like.
5. Feature engineering — create features the model can use: interactions, aggregates, encodings. (Trees can use raw features well; linear models need you to engineer more.)
6. Model selection — compare a few candidate algorithms with cross-validation.
7. Hyperparameter tuning — grid or random search over the key knobs (tree depth, learning rate, regularization strength).
8. Final evaluation — one run on the untouched test set; write the report: metric, confusion matrix, what failed, what you'd try next.

Tools & Skills

• pandas — DataFrames: loading CSVs, filtering, grouping, merging, handling missing values. The everyday language of data work. If you remember one thing: df.groupby(...).agg(...).
• scikit-learn — every classic algorithm with one consistent API: fit, predict, score, plus train_test_split, cross_val_score, GridSearchCV, and pipelines. If you can use scikit-learn's API, you can use all of ML.
• Matplotlib / Seaborn — plots for EDA and results.

Hands-On Tasks

1. Warm-up (day 1–2). Load the classic Iris dataset (sklearn.datasets.load_iris). EDA: distributions per feature, scatter matrix, correlations. Train a logistic regression; report accuracy on a proper train/test split.
2. Project A — classification (weeks 2–3). The Titanic dataset (available in many tutorials, or use any public classification dataset). Full workflow: clean the data (missing values, categorical encoding), EDA, baseline, feature engineering (family size, title from name), compare logistic regression / random forest / gradient boosting with cross-validation, tune, evaluate on test. Write a one-page report with your metric and confusion matrix.
3. Project B — regression (week 3–4). Any house-price or similar regression dataset. Repeat the workflow; use R² and MAE; try log-transforming the target if it is skewed; try PCA as a preprocessing step and report whether it helped.
4. Imbalance drill (day ~12). Make an intentionally imbalanced classification (e.g., keep 5% positives). Compare accuracy vs. F1 and see how accuracy lies. Compute precision/recall at different thresholds.
5. Unsupervised (day ~15). Run k-means on a dataset and visualize clusters; run PCA to 2D and color by the true label — see how much structure the compression preserves.

Milestone Checkpoint

You are ready for Stage 4 when:

• You completed two end-to-end projects with clean code, EDA, and written evaluation reports.
• You can explain overfitting in your own words and name three defenses against it.
• You can say why accuracy is the wrong metric for an imbalanced problem and what to use instead.
• You can explain the difference between training, validation, and test sets and why the split exists.
• You have a mental flowchart: given a tabular dataset, what steps do you take first, second, third?

Curated Resources

• Andrew Ng — Machine Learning Specialization — the canonical course; the math is light, the intuition is deep. Watch the videos on regression, overfitting, and evaluation even if you skip the rest.
• Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow (Aurélien Géron) — the best practical book in the field; read chapters 1–7 for this stage.
• scikit-learn User Guide — the reference; every algorithm has a clear page with working examples.
• Kaggle Learn — short practical courses (Python, pandas, ML) with real datasets; also where you'll find your project data.

Time Estimate & Pace

• Total: ~6 weeks at 10–15 h/week (roughly 75 hours).
• Suggested split: week 1 → paradigm, overfitting, evaluation concepts + warm-up task; weeks 2–3 → Project A; week 4 → Project B; week 5 → imbalance, unsupervised, tuning drills; week 6 → polish, review, milestone.

Bridge to the Next Stage

You now understand how machines learn: the paradigm, the pitfalls, the workflow. Stage 4 removes the training wheels — deep learning trades interpretable algorithms for far more powerful ones, and the same paradigm (data → model → loss → gradient → update) now runs on neural networks with millions of parameters. The intuition you built here — especially overfitting and evaluation — transfers one-to-one. The scale just changes.