中文版: AI 学习路线图 06 · 大模型原理

06 · Large Language Models: Inside & Out

The AI Learning Roadmap · Stage 6 of 9

Why This Stage Matters

Stage 5 gave you the engine: the Transformer. This stage gives you the machine built on it. A large language model is a Transformer scaled to billions of parameters and trained on most of the public internet — and at that scale, surprising things happen: general abilities emerge that nobody explicitly programmed, from writing code to multi-step reasoning.

Understanding what these models really are — how they are built, why they sometimes fail, how they are served — is now a core professional skill, not a research interest. Engineers who treat LLMs as magic make products that fail mysteriously; engineers who understand the machinery build systems that survive contact with reality. This stage makes you the second kind.

Core Concepts

1. The anatomy of an LLM

A modern LLM is built in stages, each with a distinct purpose:

1. Pretraining — train a decoder-only Transformer to predict the next token on trillions of tokens of web text (Common Crawl and friends), after heavy cleaning: deduplication, filtering (low-quality, toxic, boilerplate), language balancing, and perplexity-based filtering (keep text that looks like the good stuff). This is where the model's raw knowledge and language ability come from. The compute is staggering — thousands of GPUs for months — which is why pretraining is dominated by a handful of organizations.
2. Supervised fine-tuning (SFT) — the pretrained model is great at next-token prediction but not at following instructions. SFT trains it on (instruction, good response) pairs — tens of thousands of human-written or AI-assisted examples — to convert a text-completer into an assistant.
3. Preference alignment (RLHF / DPO) — SFT makes the model willing; alignment makes it safe and helpful. In RLHF: train a reward model on human preferences ("which answer is better?"), then optimize the LLM against that reward with reinforcement learning (PPO). DPO achieves the same goal with a simpler, direct objective, no separate reward model. Both teach the model to prefer responses humans rate highly. (Stage 8's agents and all modern models are also aligned this way.)
4. Deployment — quantize, serve, and expose via API (below).

The scaling law insight: model quality follows smooth power laws in parameters, data, and compute — bigger models with more data are predictably better, which is why the industry keeps scaling. Emergent abilities are the flip side: some capabilities (few-shot learning, arithmetic, chain-of-thought) appear abruptly past certain scales, like phase transitions. The honest scientific position: we understand the scaling; we do not fully understand what emerges and why.

2. Capabilities — what these models can actually do

• In-context learning — give examples in the prompt, no weight updates; the model pattern-matches to the task. (You will exploit this constantly in Stage 7.)
• Reasoning — with prompting techniques (chain-of-thought) or specialized training (reasoning models that "think" before answering), models can do multi-step math, logic, and planning — imperfectly, and with a strong dependence on problem phrasing.
• Knowledge — factual knowledge is stored in the weights (parametric memory): it is static (cutoff date), can be wrong (hallucination), and cannot be updated without retraining — the key reason RAG exists (Stage 7).
• Code, translation, summarization, dialogue — the everyday abilities that make LLMs the most useful software ever shipped.

3. Failure modes — know the enemy

• Hallucination — fluent, confident, false output. Causes: the model is trained to predict plausible text, not true text; rare facts are under-represented; the model has no external check. Mitigations: RAG, prompting ("answer from the context, say I don't know"), evaluation. You will never eliminate it; you will learn to bound it.
• Knowledge cutoff & staleness — the model only knows its training data. Anything newer requires retrieval.
• Context limits — the model can only "see" a fixed window (today, roughly 8k–2M tokens depending on the model). Inputs beyond the window are truncated; and within the window, attention degrades in the middle of long contexts ("lost in the middle").
• Calibration & sycophancy — models tend to agree with the user and to be overconfident. Ask a leading question and you may get a confident wrong answer.
• Repetition & degeneration — especially in longer generations, models loop. (You saw this in Stage 5.)

Understanding failure modes is not pessimism; it is the prerequisite for engineering. Stage 7 is mostly "how to design around these."

4. Inference engineering — how models are served

You will never pretrain a frontier model; you will serve and call them. The mechanics that matter:

• Autoregressive decoding — generation is one token at a time, each step a full forward pass. This is why LLMs are slow and why output cost scales with length.
• KV cache — at each step, the attention keys/values of previous tokens are cached and reused instead of recomputed. (Memory, not speed, is usually the constraint: a 70B model in FP16 is ~140GB — that's why you don't run it on your laptop.)
• Quantization — store weights in fewer bits (INT8, INT4): ~4× smaller and faster, with modest quality loss. This is why 7–13B models run on consumer hardware.
• Batching — serve many requests in one forward pass; the primary throughput lever.
• Serving stacks — vLLM (PagedAttention, high throughput) and Ollama (the easiest local option) are the two you will meet; the cloud APIs handle all of this for you.

A useful mental model for cost: cost ≈ tokens in + tokens out, and outputs cost more than inputs (roughly 3–4× on most pricing). Every Stage 7 architecture decision is a negotiation between quality and token cost.

5. Open vs. closed models — the landscape

Know the shape of the field, not this month's rankings (they change constantly):

• Closed frontier models — GPT (OpenAI), Claude (Anthropic), Gemini (Google): best general quality, pay-per-token APIs, no weights. When quality and time-to-market matter most.
• Open-weight models — Llama (Meta), Mistral, Qwen (Alibaba), DeepSeek, Gemma (Google): weights downloadable; you can run them yourself (privacy, cost at scale, customization); quality trails the frontier by a modest margin that keeps shrinking. The open/closed gap is the single most important strategic fact in applied LLM work — your architecture should not assume only one exists.
• When to choose what: closed for speed and best quality; open for privacy, cost control, offline, and fine-tuning freedom. Many production systems mix both (open for cheap bulk, closed for hard cases).

6. Evaluating LLMs — the hardest problem in the field

"How good is this model?" is genuinely difficult, and every answer has known failure modes:

• Benchmarks — MMLU (knowledge), GSM8K (math), HumanEval (code), and thousands more. Pros: standardized, comparable. Cons: contamination (benchmarks leak into training data), saturation (frontier models max them out), and they measure tasks, not real usage.
• Human evaluation — the gold standard and the most expensive: humans rank outputs on helpfulness, correctness, safety.
• LLM-as-a-judge — one LLM rates another's outputs against a rubric. Cheap, scalable, and surprisingly correlated with humans — with biases (judges favor longer, more confident answers; judges prefer models like themselves).
• The practical answer: for your application, build a golden set — 50–200 representative inputs with expected behaviors — and evaluate your specific success criteria (accuracy, format compliance, safety, cost). Generic benchmarks tell you about the field; golden sets tell you about your product. Both matter.

Tools & Skills

• Ollama or vLLM — run an open-weight model locally (start with a 7–8B model).
• A model API (OpenAI, Anthropic, or any provider) — for frontier-quality comparisons.
• An eval harness — e.g., lm-evaluation-harness (EleutherAI) for standard benchmarks, or your own golden-set script.
• Hugging Face Hub — model cards, licenses, and usage stats; read the card before you use a model.

Hands-On Tasks

1. Serve a model (week 1). Run a 7–8B open model locally (Ollama or vLLM on Colab). Chat with it; measure tokens/second; watch VRAM usage. Try quantized vs. full precision if you can. This demystifies "inference" forever.
2. Stress the failure modes (week 1–2). Deliberately provoke: hallucination (ask for a specific fake paper), sycophancy (a leading question), repetition (long generation), cutoff (ask about something newer than the model knows). Write down what you observe — these are your engineering constraints.
3. The eval report — the milestone (weeks 2–4). Pick two models (e.g., one closed via API, one open locally). Design a fixed task set: 10–20 questions across categories (facts, reasoning, code, safety, formatting). Define a scoring rubric. Run both, score them, and write a 1–2 page technical report: scores per category, qualitative notes (tone, verbosity, failure types), and cost/latency comparison.
4. Quantization experiment (week 3, if hardware allows). Load a model at FP16 vs. INT4; compare output quality and speed on your task set. Note the tradeoff.
5. Read one model card and one paper (week 4). Read the Llama or Qwen model card (licensing, capabilities, known limitations). Read the RLHF paper or DPO paper — not to implement, to know what "alignment" concretely is.

Milestone Checkpoint

You are ready for Stage 7 when:

• You ran an open-weight model yourself and understand what serving means in practice.
• You wrote a comparison report of two models with a rubric, scores, and qualitative observations.
• You can name three failure modes of LLMs and one engineering mitigation for each.
• You can explain why RAG exists (what parametric memory can and cannot do).
• You can explain open vs. closed tradeoffs and when you would choose each.

Curated Resources

• Andrej Karpathy — "Intro to Large Language Models" — rewatch now; it will land completely differently than at Stage 0.
• Lilian Weng — "LLM Training" and the LLM series — the definitive technical overviews of pretraining, RLHF, and evaluation.
• Hugging Face — "What are Large Language Models?" — a concise official tour.
• Understanding Deep Learning (Simon Prince) — LLM chapters — free textbook; the sequence-model chapters give the cleanest formal treatment of scaling and alignment.
• The DPO paper (Rafailov et al., 2023) — read for the intuition: "aligning a model is simpler than it sounded" — direct preference optimization.

Time Estimate & Pace

• Total: ~4 weeks at 10–15 h/week (roughly 50 hours).
• Suggested split: week 1 → anatomy + serving (tasks 1–2); weeks 2–3 → eval report (task 3) + quantization; week 4 → model cards, papers, review, milestone.

Bridge to the Next Stage

You now understand the machine — what it can do, where it fails, how it's served and evaluated. Stage 7 asks the question that most jobs are actually about: how do you turn this into a product people can rely on? Prompting, retrieval-augmented generation, function calling, evaluation, and cost engineering — the entire applied layer — is next.