Natural-language processing learning path

Natural-language processing (NLP) studies how a computational system represents, understands, retrieves, transforms, and generates language. This route begins with the decisions hidden behind “text input”—Unicode, normalization, tokenization, and corpora—then builds through distributional representations, recurrent encoder–decoders, attention, transformers, pretraining, task design, and evaluation.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates. The route is a curated dependency graph, not an attempt to list every NLP paper. New work is integrated into the scientific branch it changes rather than placed in a dated catch-all section. Every detailed chapter ends with a catalog-generated Complete maintained paper route; its shorter staged table is the teaching spine, not the full bibliography.

Where this field sits

raw language
  → corpus and annotation decisions
  → normalization and tokenization
  → token / sequence representations
  → task model or pretrained model
  → decoding, retrieval, or structured prediction
  → task-matched evaluation and error analysis

NLP is broader than large language models. A calibrated linear classifier, a conditional random field, a dense retriever, and a decoder-only model are all NLP systems, but their task contracts and evidence are different. Finish this route before the large-language-model route and the natural-image vision–language route.

Prerequisites

You should be comfortable with vectors, probability, cross-entropy, gradient-based optimization, and train/validation/test separation. Use the ML and deep-learning route when those concepts are not yet operational. Programming prerequisites are Python, array/tensor operations, regular expressions, and a reproducible experiment configuration.

Chapter map

Order	Chapter	What you must be able to explain before moving on
1	Text representations and sequence models	why a tokenizer and corpus define part of the model; sparse versus static versus contextual representations; recurrence, encoder–decoder learning, and attention
2	Transformers and pretraining	Q/K/V shapes, masks, position, residual pathways, encoder/decoder objectives, adaptation choices, and memory cost
3	Tasks, retrieval, and evaluation	the input/output contract, metric limitations, retrieval versus generation evidence, contamination, factuality, and human-evaluation design
4	Large language models	scale, data, post-training, retrieval, tool use, safety, and systems evidence
5	Natural-image vision & language	how text supervision changes visual representation, grounding, multimodal generation, and agents

The maintained searchable literature registry exposes the staged records behind these chapters. In My Library, select all NLP papers or choose one semantic subroute.

Three depth routes

Undergraduate route

1. Implement bag-of-words logistic regression and document the vocabulary.
2. Train or inspect word embeddings; test nearest neighbours and analogy caveats.
3. Build a recurrent sequence classifier and an attention-based encoder–decoder.
4. Implement one self-attention block and causal mask.
5. Evaluate a classifier with macro metrics and a generated output with more than lexical overlap.

Master’s route

Complete the undergraduate route, then compare tokenizers, reproduce a transformer baseline, fine-tune an encoder and an encoder–decoder, build a dense-retrieval baseline, and run input ablations that expose label, template, or retrieval leakage.

Research route

Reconstruct the complete evidence chain for one current system:

source documents → filtering/deduplication → tokenizer → objective → checkpoint
→ adaptation data → decoding/retrieval policy → benchmark version → audit

Report data rights and provenance, exact checkpoint, context policy, training and inference budget, prompt/template, model-selection procedure, uncertainty, subgroup errors, and contamination controls.

Shared notation

Symbol	Meaning
x_1 … x_T	input token sequence of length T
V	tokenizer vocabulary
e_t ∈ R^d	embedding for token position t
h_t	contextual hidden state
`p(x_t	x_1…x_(t−1))`
Q, K, V	query, key, and value matrices in attention
y	task label, output sequence, span, ranking, or structured object

Do not compare perplexities from different tokenizers as though they shared the same atomic unit. Record token-normalized and byte- or character-normalized quantities when the comparison needs them.

Required research habits

• Preserve the untouched raw text and version every transformation.
• Split by the unit that can leak: author, patient, conversation, document family, template, time, or institution—not merely by individual sentence.
• Report tokenizer and normalization rules with the checkpoint.
• Separate retrieval recall from answer generation.
• Test text-only shortcuts, shuffled context, missing evidence, and unanswerable cases.
• Treat a fluent response as an output style, not evidence of correctness.
• Recheck dataset licenses, personally identifiable information, and memorization risk.

Exit task

Choose one transformer NLP paper and produce a one-page reconstruction containing:

1. task and unit of analysis;
2. corpus, split, and contamination boundary;
3. tokenizer and sequence-length policy;
4. architecture, mask, objective, and adaptation;
5. decoding or retrieval procedure;
6. principal metric and a failure it misses;
7. one controlled experiment that could falsify the central claim.

If you cannot answer one item from the paper and official artifacts, record it as missing evidence rather than filling the gap with an assumption.

Specialist research routes

Continue to structured NLP and information extraction for constrained token, span, relation, and graph outputs; multilingual and low-resource NLP for language-by-language coverage and transfer; and generation, translation, and evaluation for decoding, learned metrics, human evidence, and factuality.