ML, AI, and deep-learning foundations

This route builds the background needed to read a modern machine-learning paper as an argument rather than as a collection of architecture names. It begins with data, functions, probability, and evidence; then moves through classical models, neural networks, representation and generative learning, and reliable experimental practice.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates. A review date is a coverage boundary, not a claim that every paper published before that date belongs in the curriculum. New work is integrated into its scientific topic after verification. Every detailed chapter ends with a catalog-generated Complete maintained paper route; its shorter staged table is the teaching spine, not the full bibliography.

What this route teaches

A learning system connects six objects:

population → sampled data → representation → model and objective
           → optimization procedure → claim supported by evaluation

Changing any one of them can change the meaning of a result. A better optimizer does not repair a leaked split; a larger model does not make a target clinically meaningful; and a high validation score does not establish performance on a new institution.

By the end of the route, you should be able to:

1. write a task in terms of inputs, targets, assumptions, objective, and unit of evaluation;
2. derive the loss and gradients of a small model and check them numerically;
3. select a classical or neural baseline for reasons tied to the data regime;
4. distinguish supervised, semi-supervised, weakly supervised, self-supervised, generative, and reinforcement-learning signals;
5. trace information and gradients through a modern architecture;
6. design a split, metric set, uncertainty analysis, and ablation that match the claim;
7. reproduce an experiment from versioned data, code, configuration, and artifacts;
8. explain what the evidence does not establish.

Prerequisite graph

Follow dependencies rather than treating the site as a single long reading list:

Python, arrays, and plots
        │
        ├── linear algebra ── calculus ── optimization
        │          └──────── probability and statistics
        │
        ├── classical supervised and unsupervised learning
        ├── search, reasoning, graphical models, and sequential decisions
        │
        └── neural-network mechanics
                    ├── representation and transfer learning
                    ├── generative modeling
                    ├── sequence and graph models
                    └── scalable training systems

Every branch ───────────────────────→ evaluation and reproducibility

Readers with prior training may use each chapter’s exit task as a diagnostic. If you cannot complete it without notes, study that chapter before depending on it.

The maintained core chapters

Chapter	Core questions	Evidence of mastery
Mathematics, probability, and statistics	What objects are being transformed? What is random? What is estimated, and with what uncertainty?	derive a gradient, use SVD geometrically, form a likelihood, and bootstrap a paired estimate
Classical machine learning	Which inductive bias fits the sample size, feature type, and decision boundary?	compare linear, kernel, tree, and probabilistic baselines with nested model selection
Artificial intelligence and reinforcement learning	How do agents search, reason under uncertainty, plan, and learn from delayed feedback?	solve one problem with search, dynamic programming, temporal-difference learning, and an auditable offline evaluation
Deep learning	How do signals and gradients move through the network, and why can it be trained at scale?	implement and gradient-check an MLP, then audit a modern architecture and optimizer
Generative and representation learning	What supervision shapes a representation or a data distribution?	contrast transfer, self-supervision, latent-variable, adversarial, diffusion, and flow objectives
Knowledge distillation	Which teacher signal is transferred, and what does the student retain under deployment constraints?	distinguish output, feature, relation, sequence, self-, multimodal, and domain-expert transfer
Loss and objective design	Which errors receive gradient, and does that target match the scientific claim?	audit proper scoring, imbalance, metric geometry, overlap, noisy labels, and alignment objectives
Transfer, adaptation, and continual learning	What changes between source and target, which state may adapt, and what must be retained?	distinguish domain, few-shot, semi-supervised, parameter-efficient, and continual-learning contracts
Evaluation and reproducibility	Does the experiment identify the claimed effect, and can another group inspect it?	produce a leakage-resistant evaluation card, uncertainty report, ablation, and reproducibility bundle

Evaluation is not the final chapter chronologically. Apply its contracts from the first baseline onward.

Learning settings before algorithms

Let x denote observed input, y a target when one exists, θ model parameters, and L a loss. The generic empirical objective is

θ̂ = arg minθ  (1/n) Σᵢ L(fθ(xᵢ), yᵢ) + λ Ω(θ).

The symbols look similar across settings, but the origin and meaning of the training signal differ.

Setting	Training signal	Question to ask first
Supervised	an observed target for each training example	Who produced the label, at what unit, and with what error?
Semi-supervised	a small labelled set plus unlabelled examples	Are labelled and unlabelled samples from compatible populations?
Weakly supervised	noisy, coarse, incomplete, or bag-level targets	Which unobserved instance claim is inferred from the coarse label?
Self-supervised	a target constructed from the input itself	Which invariance or predictive relation does the construction impose?
Unsupervised	density, geometry, clusters, or latent structure without task labels	Is the recovered structure identifiable or useful for the downstream question?
Generative	an objective for modelling or transporting a data distribution	Is likelihood, sample quality, coverage, conditional fidelity, or representation quality the goal?
Reinforcement learning	delayed reward from interaction or logged trajectories	What are the state, action, transition, reward, horizon, and data-collection policy?

These labels describe supervision, not model architecture. A Transformer can be trained in any of several settings; “foundation model” does not specify an objective or prove transfer.

Choose a depth

Level	Intended reader	Stop when you can…
L0 · orientation	undergraduate or a reader changing fields	define the input, output, supervision, split unit, and primary metric
L1 · mechanism	advanced undergraduate or early master’s	reconstruct the objective and data flow and implement a small version
L2 · evidence	graduate learner or early-stage researcher	audit model selection, baselines, ablations, uncertainty, and compute
L3 · synthesis	independent researcher	compare incompatible assumptions and design an experiment that could falsify the claim

Difficulty is a prerequisite label, not a quality score. A short classical paper can require more statistical judgment than a large-model technical report.

Guided routes

Undergraduate core

1. Learn tensor shapes, broadcasting, plotting, and automatic differentiation from the official PyTorch tutorials.
2. Complete the linear algebra, derivatives, probability, likelihood, and confidence interval sections of the mathematics chapter.
3. Implement linear and logistic regression, then compare them with a tree and an MLP.
4. Complete one controlled neural-network lab and one evaluation card.
5. Continue to computer vision or natural-language processing.

Master’s research core

1. Diagnose all seven chapters with their exit tasks.
2. Reproduce one mechanism under fixed data, split, metric, tuning budget, and seeds.
3. Compare a feature extractor frozen, partially adapted, and fully fine-tuned.
4. Run an ablation that changes one causal component at a time.
5. Write a short report separating observation, inference, and unresolved alternative explanations.

Computational-pathology research bridge

1. Treat patient, case, specimen, block, slide, region, and tile as distinct units.
2. Learn classical small-data baselines before assuming a deep model is appropriate.
3. Complete the representation-learning branch before comparing pathology encoders.
4. Complete calibration, distribution shift, and uncertainty before making a clinical claim.
5. Continue to the computational-pathology overview and retain the same experimental contracts.

Authoritative resource spine

Purpose	Resource	How to use it
mathematical bridge	Mathematics for Machine Learning	learn the linear algebra, geometry, calculus, probability, and optimization used by ML
applied statistical learning	An Introduction to Statistical Learning	work through the Python labs while keeping a held-out test set untouched
probabilistic unification	Probabilistic Machine Learning: An Introduction and Advanced Topics	connect prediction, latent variables, inference, decision making, and modern generative models
classical ML course	Stanford CS229 and its open notes	add derivations, learning theory, unsupervised learning, and reinforcement learning
neural-network reference	Deep Learning	use as a conceptual and mathematical reference, not a record of every later architecture
executable explanations	Dive into Deep Learning	alternate from-scratch components with framework implementations
broad AI	Stanford CS221	study search, Markov decision processes, games, graphical models, and logic

Links point to official project, author, institution, publisher, or proceedings pages. Exercises and prose in this repository are independently written; course solutions and third-party figures are not copied.

A repeated research loop

For every method:

1. Specify: population, sample, input, target, split unit, and intended decision.
2. Derive: objective, main computation, gradient path, and assumptions.
3. Implement: the smallest version that can test the mechanism.
4. Compare: use the same data, selection budget, metric, seeds, and compute boundary.
5. Stress: inspect leakage, imbalance, calibration, subgroups, and distribution shift.
6. Explain: state the supported claim and at least one plausible alternative.
7. Record: save versions, configuration, failures, uncertainty, and unresolved questions.

How current work is maintained

New papers are not appended to a chronological “latest papers” section. A verified optimizer belongs with optimization; a new sequence architecture belongs with sequence models; a one-step flow model belongs with generative transport. A genuinely new research question earns a semantic subsection only when existing categories cannot describe its assumptions or evidence.

The Daily arXiv Radar is for discovery. Promotion into these chapters requires a primary source, verified publication status, a clear prerequisite role, and a reason the work changes the learning path.

Route exit task

Choose a paper relevant to your research and, without copying its abstract:

1. write its task, population, training signal, and unit of evaluation;
2. reconstruct the objective and architecture as a data-flow diagram;
3. identify the optimizer, regularizers, model-selection procedure, and compute budget;
4. check whether the strongest baseline receives the same information and tuning effort;
5. recover the uncertainty estimate and one important failure hidden by the main metric;
6. propose the smallest controlled experiment that could weaken the central claim.

If you can do this accurately, continue to the domain route. If not, use the missing step to choose the next chapter rather than reading more papers indiscriminately.

Specialist research routes

After the common ladder, continue through the route matching the scientific claim: causal inference for interventions and effects; interpretability and attribution for testable explanations; privacy and federated learning for formal guarantees, distributed protocols, and leakage; or graph and geometric learning for relational and symmetry-aware representations. Use transfer, adaptation, and continual learning whenever the source distribution, available labels, trainable state, or task sequence changes after pretraining.