Skip to content
Computational Pathology Reading Roadmap

Retrieval, long context, tools, and agents

Retrieval and tools let a language model access changing information or perform actions. They also add failure boundaries. Evaluate the complete system and each component separately.

Latest literature discovery audit: 2 October 2026. Existing paper reviews retain their individual source-check dates.

1. Retrieval-augmented generation

Section titled “1. Retrieval-augmented generation”

A typical system is:

query → rewrite → sparse/dense candidates → rerank → select/compress
→ prompt generator → answer with citations → verify / abstain

Keep the classical sparse baseline explicit. Spärck Jones (1972) derives inverse document frequency as collection-specific term evidence; Okapi at TREC-3 (1994) is the canonical early BM25 record, combining saturating term frequency with document-length normalization; and Järvelin and Kekäläinen (2002) define gain-based ranked evaluation, including normalized discounted cumulative gain. A RAG study should report BM25 beside dense retrieval, Recall@K beside nDCG, and exact gain, cutoff, relevance, and unjudged-item policies.

REALM (2020) integrates retrieval into pretraining. RAG (2020) conditions generation on retrieved passages. Fusion-in-Decoder (2021) lets the decoder combine separately encoded passages. RETRO (2022) retrieves during language-model training.

Measure:

  • corpus coverage and freshness;
  • Recall@K, MRR, nDCG, and labelled-relevance limits;
  • answer correctness independent of retrieval;
  • citation entailment and completeness;
  • robustness to irrelevant, contradictory, stale, or malicious documents;
  • abstention when evidence is absent;
  • storage, indexing, token, latency, and cost budgets.

The exact corpus version, chunking, overlap, embedding, index, query rewrite, top-K, reranker, prompt, and access-control filter are part of the model.

ALCE (2023) separates answer quality, citation correctness, and citation completeness. CRAG (2024) adds realistic, temporally changing questions and explicit retrieval components. Benchmark results do not transfer unless corpus date, access conditions, and evaluation-time retrieval are reproducible.

2. Long context is not retrieval

Section titled “2. Long context is not retrieval”

Putting documents in a long prompt avoids an external index but still requires selection and attention. Test answer position, distractor density, document order, duplicates, conflicting sources, and effective context. Compare long-context, retrieval, and hybrid systems under equal input-token and latency budgets.

LongNet (2023) is an arXiv preprint that replaces dense attention with dilated sparse attention. It splits a sequence into segments, samples positions at increasing dilation rates, combines multiple segment scales, and distributes the long scales across devices. Under the paper’s pattern, attention allocation falls with distance, computation is linear in sequence length, and the dependency path grows logarithmically. The reported one-billion-token run is a systems/scaling demonstration with a constant one-billion-token batch, not evidence that an arbitrary deployed model understands every token in a billion-token prompt. LongNet uses XPOS in its language-model experiment; LongNet itself is not a positional-encoding method. Separate the sparse interaction pattern, position policy, trained length, tested length, hardware, and downstream evidence-use score.

Lost in the Middle (2024) demonstrates position-dependent use of relevant information. LongBench (2024) broadens long-context tasks and languages. Neither a context-window declaration nor a single needle test establishes robust document reasoning.

3. Tools

Section titled “3. Tools”

A tool has a typed schema, authentication and authorization boundary, validation, timeout, retry/idempotency policy, error model, audit log, and side effects. The model proposes calls; the host system must validate them.

Toolformer (2023) learns from self-supervised tool-call examples. ReAct (2023) interleaves reasoning and actions. ToolLLM (2024) studies instruction tuning and evaluation over a large API collection. Tool access does not guarantee correct selection, arguments, interpretation, or authorization.

4. Agents and memory

Section titled “4. Agents and memory”

An agent repeatedly observes state, proposes an action, receives a result, and updates working or persistent memory. Separate planner, executor, verifier, environment, retriever, and human approval.

Evaluate task success plus:

  • valid and authorized action rate;
  • tool-selection and argument accuracy;
  • recovery after errors;
  • unnecessary steps and looping;
  • intervention and escalation;
  • prompt-injection and data-exfiltration resistance;
  • deterministic replay and environment version;
  • observation/action/token/latency/cost budget.

Persistent memory creates privacy, poisoning, stale-state, deletion, and cross-user isolation requirements.

Use environment benchmarks as measurement instruments, not agent definitions. AgentBench (2024) spans several interactive environments; WebArena (2024) uses reproducible websites, and SWE-bench (2024) evaluates repository-level issue resolution. Agent Security Bench (2025) formalizes attacks and defenses across prompts, tools, and memory. Pin environment state and score unauthorized side effects even when the final task succeeds.

Staged route

Section titled “Staged route”
Level Primary reading · year · status Read it for
L1–L2 Dense Passage Retrieval — 2020 · EMNLP dual-encoder retrieval and negative construction
L2 REALM — 2020 · ICML; RAG — 2020 · NeurIPS retrieval in pretraining versus generation
L2 Fusion-in-Decoder — 2021 · EACL multi-passage generation
L2–L3 RETRO — 2022 · ICML; Atlas — 2023 · JMLR retrieval at pretraining/few-shot scale
L2–L3 HyDE — 2023 · ACL convert a query into a hypothetical document before dense retrieval; measure whether fluent hallucinated content helps recall or injects generator bias
L2–L3 Toolformer — 2023 · NeurIPS; ReAct — 2023 · ICLR learned calls and interleaved action
L2–L3 Reflexion — 2023 · NeurIPS verbal feedback stored as episodic memory for trial-to-trial improvement; distinguish inference-time memory from parameter learning and guard against self-reinforced errors
L2–L3 ALCE — 2023 · EMNLP; CRAG — 2024 · NeurIPS Datasets & Benchmarks citation attribution and end-to-end RAG evaluation
L3 Self-RAG and RAPTOR — 2024 · ICLR self-critique and hierarchical retrieval branches
L2–L3 LongNet — 2023 · arXiv preprint multi-scale dilated sparse attention, linear attention computation, logarithmic dependency paths, distributed sequence training, and why architectural capacity is not evidence use
L2–L3 Lost in the Middle — 2024 · TACL; LongBench — 2024 · ACL effective context, position controls, and broad long-context tasks
L2–L3 ToolLLM — 2024 · ICLR; AgentBench — 2024 · ICLR tool-learning data and multi-environment agent evaluation
L3 WebArena — 2024 · ICLR; SWE-bench — 2024 · ICLR reproducible web and software-engineering environments
L3 Agent Security Bench — 2025 · ICLR prompt, tool, memory, attack, defense, and utility/security boundaries

Controlled lab

Section titled “Controlled lab”

Use a versioned small corpus with answer evidence:

  1. compare BM25, dense retrieval, and reranking;
  2. plot retrieval recall against K, latency, and answer quality;
  3. insert irrelevant, contradictory, outdated, and prompt-injection documents;
  4. require sentence-level citations and score entailment/completeness;
  5. add a read-only calculator/search tool, then a sandboxed state-changing tool;
  6. measure invalid calls, recovery, interventions, success, and total cost;
  7. compare no context, full long context, RAG, and hybrid under equal budgets.

Exit task

Section titled “Exit task”

Produce a threat and evaluation diagram showing corpus, permissions, retrieval, prompt construction, model, tool host, user, logs, and persistent state. Mark every place where stale evidence, injection, unauthorized action, privacy loss, or silent failure can enter.

Complete maintained paper route

Section titled “Complete maintained paper route”

32 canonical readings · 1972–2026· route reviewed through October 2, 2026.

This is the complete structured list for this chapter, ordered by an explicit prerequisite-aware teaching sequence. The explanatory ladder above highlights the shortest conceptual route; this list also includes important branches, benchmarks, and current endpoints. It is generated from the same deduplicated catalog used by the AI background registry and My Library, so a future catalog update appears here automatically.

Show all 32 readings in maintained order

  2. #21994TREC-3Technical reportL2–L3

    Okapi at TREC-3

    Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford

    System-description paper in NIST Special Publication 500-225 (TREC-3 proceedings); TREC participation and proceedings inclusion are not represented here as peer review.

    Official artifacts: NIST proceedings record

    Metadata checked

  5. #52025ICLRPeer reviewedL2

    MMTEB: Massive Multilingual Text Embedding Benchmark

    Kenneth Enevoldsen et al.

    Review uses the proceedings PDF and Table 2 benchmark definitions. The landing abstract states 1,000+ languages while the PDF abstract states 250+; Section 5 attributes the broader count to bitext mining. Task-count inconsistencies in prose are retained as caveats, not silently combined.

    Official artifacts: Proceedings PDF · Benchmark implementation · Versioned evaluation results · arXiv

    Metadata checked

  20. #202026arXivPreprintL3

    Sliding-window beats linear attention

    Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron, and Emy Gervais

    arXiv v1 submitted 28 August 2026. No peer-reviewed record, code release, or model artifact was verified on 2 September 2026; it is retained for its unusually important negative baseline evidence rather than publication status.

    Metadata checked