Large-language-model learning path

Large language models (LLMs) are usually high-capacity autoregressive or sequence-to-sequence models trained on broad corpora and adapted to many tasks. “Large” does not define a scientific mechanism, and a model name does not identify a complete experiment. This route studies the full model lifecycle.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates. Natural-image multimodal systems now have a separate Vision & Language route; pathology-specific systems remain in the pathology VLM chapter. Every detailed chapter ends with a catalog-generated Complete maintained paper route; its shorter staged table is the teaching spine, not the full bibliography.

Prerequisites

Complete the transformer and evaluation chapters in the NLP route. You should understand tokenization, causal masking, cross-entropy, calibration, retrieval metrics, and leakage. Review ML/deep learning for optimization and uncertainty.

Model-lifecycle map

licensed / documented sources
  → filtering, deduplication, mixtures, tokenizer
  → architecture, objective, optimizer, distributed training
  → base checkpoint
  → instruction data and preference / verifier feedback
  → post-trained checkpoint
  → retrieval, tools, memory, agent loop
  → deployment policy, monitoring, evaluation, incident response

Evidence can fail at every arrow. A benchmark result cannot repair unknown training overlap, an inaccessible checkpoint, or an unsafe system boundary.

Chapters

Order	Chapter	Central questions
1	Pretraining, data, and scaling	What was predicted from which tokens, at what scale and cost? Which architecture and systems choices matter?
2	Post-training and alignment	What changes through SFT, preferences, rewards, verifiers, and reasoning-time compute? Whose behavior is optimized?
3	Retrieval, tools, long context, and agents	Where does evidence come from? Which component fails? What actions can the system take?
4	Evaluation, safety, and efficiency	Which claims are supported under contamination, shift, harmful inputs, uncertainty, and real systems budgets?
5	Code generation and software engineering	Does the claim concern representation, executable synthesis, search, repository repair, dynamic evaluation, or iterative dependency reuse?

The searchable registry and My Library expose these as semantic, independently selectable paper routes.

What to record for every model

Dimension	Minimum record
identity	exact base/instruct checkpoint, revision, tokenizer, license
architecture	dense or MoE, total/active parameters, layers, width, attention/KV design, context method
data	sources, languages, cutoff, mixture, filters, deduplication, rights, synthetic data
training	objective, tokens, batch, optimizer, precision, hardware/compute when disclosed
post-training	SFT and preference sources, reward/verifier, online/offline policy, safety policy
inference	prompt template, system message, decoding, context, tools, quantization
evaluation	benchmark release, contamination check, selection, variance, human protocol, cost

“Same model family” is not enough when tokenizer, checkpoint revision, prompt template, quantization, or hosted service can change.

Three study routes

Reader

Read the transformer, GPT-3, scaling-laws, Chinchilla, InstructGPT, RAG, DPO, and HELM anchors. For each, write the claim, mechanism, data, evaluation, and strongest limitation.

Builder

Train a small causal model; instruction-tune it; compare LoRA and full adaptation; build a retriever; expose a typed tool; and evaluate base, post-trained, and complete system checkpoints under fixed prompts and budgets.

Researcher

Reproduce one claimed gain with:

• identical base checkpoint, data, tokens, and tuning budget;
• evaluation before and after post-training;
• pass@1 and test-time-compute curves rather than only best-of-many;
• contamination and memorization checks;
• retrieval/tool component metrics;
• confidence intervals and documented failure cases;
• measured tokens, latency, memory, energy or hardware, and monetary cost where applicable.

Essential distinctions

• Knowledge versus access: memorized parametric information differs from retrieved evidence.
• Elicitation versus learning: a prompt can reveal an existing behavior without updating weights.
• Outcome versus process: a correct final answer does not validate a displayed reasoning trace.
• Capability versus safety: refusal can hide capability; capability scores do not establish acceptable risk.
• Model versus system: RAG, tools, policies, UI, monitoring, and users change the deployed behavior.
• Open weights versus openness: weights alone do not expose data, code, training process, or rights.

Exit task

Create a model/system card for one LLM answering all fields above. Reproduce one task with the base and post-trained checkpoint, no retrieval and controlled retrieval, fixed decoding, missing-evidence cases, and measured cost. State precisely which component supports each conclusion.

Specialist research routes

Use training and serving systems to study precision, parallelism, kernels, caches, scheduling, and quantization; reasoning and verification to separate answer quality, process validity, search, and compute; and interpretability and model editing to connect attribution, circuits, causal interventions, editing, side effects, and deletion claims; and code generation and software engineering to separate functional synthesis, sampling, execution, contamination, repository repair, and multi-turn code reuse.