Summarizing Long Documents With LLMs in October 2026: Map-Reduce, Refine, and Hierarchical Summaries That Keep the Key Facts

Long-context models made it tempting to paste a 200-page contract or a year of meeting notes into one prompt and ask for "a summary." Sometimes that works. Often it quietly doesn't: the summary over-represents the first and last sections, drops a number that mattered, or blends two similar clauses into one that never existed. This guide covers the practical patterns for summarizing documents that are long, dense, or both, and how to check that the result still holds the facts your users need.

Photo: Highlighter pen on photocopied text by Guido "random" Alvarez, CC BY 2.0, via Wikimedia Commons

Start by Deciding What the Summary Is For

Most bad summaries come from a vague instruction, not a weak model. "Summarize this" gives the model no way to decide what to keep. Before picking a technique, write down three things:

• The reader and the decision. A lawyer reviewing renewal terms needs different facts than an executive deciding whether to read the contract at all.
• The must-keep facts. Dates, amounts, parties, obligations, open questions, action items. Name them explicitly in the prompt.
• The output shape and length. A fixed set of headings or a JSON object is easier to check than free prose, and a word limit keeps the model from padding.

A summary spec might look like this:

Audience: account manager preparing for a renewal call.
Must include: contract term and end date, renewal and notice
periods, pricing and any escalation clause, termination rights,
SLAs with credits, anything marked as an open issue.
Format: sections "Key dates", "Money", "Obligations", "Risks".
Each bullet ends with a section reference like [§4.2].
Length: under 300 words. Say "not stated" instead of guessing.

That last line matters. Without it, the model will fill gaps with plausible defaults, which is worse than an honest blank.

Option 1: Single Pass in a Long Context

If the whole document fits comfortably in the model's context window with room left for the instructions and the answer, try one call first. It's the simplest pipeline, keeps cross-references intact, and is easy to debug.

Things that make single-pass work better:

• Put the instructions after the document as well as before it, so the request is fresh when the model starts writing.
• Keep structure visible. Preserve headings, section numbers, and page markers when you convert from PDF or HTML. They give the model anchors and let it cite.
• Ask for extraction before synthesis. "First list every date and amount with its section, then write the summary from that list" reduces skipped facts compared with asking for prose directly.

Single-pass starts to break down when the document is near the context limit, when content in the middle of the document matters as much as the ends, or when you need the same summary to be reproducible across reruns. Test it on your own documents rather than assuming the advertised context length means reliable recall across all of it.

Option 2: Map-Reduce

Map-reduce splits the document into chunks, summarizes each chunk independently (map), then combines the chunk summaries into a final summary (reduce). Because map calls don't depend on each other, they run in parallel, which makes this the fastest option for very long inputs.

chunks = split_by_sections(doc, max_tokens=6000)

partials = parallel_map(chunks, lambda c: llm(
    MAP_PROMPT.format(spec=spec, section_ref=c.ref, text=c.text)
))

final = llm(REDUCE_PROMPT.format(spec=spec, partials="\n\n".join(partials)))

Practical rules for map-reduce:

• Split on structure, not raw token counts. Cut at headings or section boundaries so a clause isn't divided across two chunks. Fall back to paragraph boundaries with a small overlap only when sections are huge.
• Make map outputs extractive and structured. Ask each map call for bullets of facts with section references, not a mini essay. Facts survive the reduce step; adjectives don't.
• Pass the same spec to every call. If map calls don't know the must-keep list, they will drop the clause that only matters to your reader.
• Let map calls say "nothing relevant." Boilerplate sections should produce an empty result instead of filler that dilutes the reduce input.

The weak spot is cross-chunk context. A definition in section 1 changes the meaning of a clause in section 9, and the map call for section 9 never saw it. Two mitigations help: prepend a short document header (title, parties, defined terms) to every chunk, and tell the reduce step to flag conflicts between partial summaries instead of silently picking one.

Option 3: Refine (Rolling Summary)

Refine processes chunks in order. The first call summarizes chunk 1; each later call receives the current summary plus the next chunk and returns an updated summary.

summary = llm(FIRST_PROMPT.format(spec=spec, text=chunks[0].text))
for c in chunks[1:]:
    summary = llm(REFINE_PROMPT.format(
        spec=spec, current=summary, section_ref=c.ref, text=c.text))

Refine keeps narrative context, which suits documents that build on themselves: incident timelines, long email threads, transcripts, or a book chapter by chapter. The costs are that it's strictly sequential, so latency grows with document length, and errors compound. If an early call misreads something, every later call inherits the mistake.

Two guards keep refine honest. First, instruct each step to keep existing facts unless the new chunk explicitly changes them, and to record the change ("Price revised from section 3 to section 7"). Second, cap the running summary length so it doesn't balloon, and when it hits the cap, compress older detail rather than dropping new facts.

Photo: Staples, Staple Remover, and Paper Documents by Tony Webster, CC BY-SA 2.0, via Wikimedia Commons

Option 4: Hierarchical Summaries

For very large inputs, such as a whole repository of policies or a multi-volume report, a single reduce step can itself overflow. Hierarchical summarization repeats the reduce: summarize sections, then summarize groups of section summaries into chapter summaries, then chapters into a document summary.

The useful side effect is that you get every level, not just the top. Store them. A user who reads "the vendor can terminate on 30 days' notice" can click through to the chapter summary and then to the original section. That drill-down path is often more valuable than the top-level paragraph, and it gives you a natural citation trail.

Keep each level's prompt aware of its job. Leaf summaries should be extractive and detailed; upper levels should merge, deduplicate, and rank by importance to the reader. If every level uses the same generic "summarize" prompt, detail washes out quickly as you go up.

Choosing Between Them

• Fits easily in context and order doesn't matter much: single pass with extraction-first prompting.
• Very long, mostly independent sections (contracts, reports, policy sets): map-reduce with structured map outputs.
• Narrative or chronological content (transcripts, threads, timelines): refine, with a capped running summary.
• Too big for one reduce, or users need drill-down: hierarchical, and keep every level.

Hybrids are normal. A common one is map-reduce for extraction plus a single long-context call that gets the original document and the extracted facts together to write the final prose.

Checking That the Summary Kept the Facts

Summaries fail silently, so build checks into the pipeline instead of trusting a quick read.

1. Verify citations mechanically. If every bullet ends with a section reference, confirm that the reference exists and that key numbers and dates in the bullet appear in that section's text. A string or regex check catches many invented figures without another model call.
2. Check coverage against the spec. For each must-keep field, confirm the summary either fills it or says "not stated." Then spot-check the "not stated" ones against the source, because a missed fact usually shows up there.
3. Run a question-answer test. Write a handful of questions per document type ("When does the contract end?", "Who owns the action item about billing?") with known answers. Answer them from the summary alone. If the summary can't answer a question its reader cares about, it isn't done.
4. Use a judge model for faithfulness, not style. Ask a second model to list any summary statement not supported by the cited section. Keep the judge's job narrow; "is this a good summary?" produces vague scores.
5. Keep a small golden set. Ten to twenty real documents with reviewed summaries will catch regressions when you change chunk size, prompts, or models.

Cost and Latency Tips

• Cache per chunk. Key map outputs by a hash of the chunk text plus the prompt version. When a document is edited, only changed sections need new calls.
• Use a smaller model for the map step. Extraction from a short chunk is an easier task than synthesis. Evaluate it on your golden set before switching, but it's often a reasonable trade.
• Bound concurrency. Parallel map calls on a 500-page document can hit provider rate limits. Use a worker pool and back off on rate-limit responses.
• Stream only the final step. Users don't need to watch map calls; show progress ("summarizing section 12 of 40") and stream the final summary.

A Short Checklist

• Write a summary spec with audience, must-keep facts, format, and length.
• Preserve headings and section numbers when converting the source.
• Pick single pass, map-reduce, refine, or hierarchical based on length and structure.
• Require section references on every bullet and allow "not stated."
• Verify citations and key numbers in code, then run a narrow faithfulness check.
• Cache chunk-level results and re-run only what changed.

The technique matters less than the discipline around it. A clear spec, structure-aware chunks, and a mechanical check on every number will get you further than any single clever prompt, and they keep working when you swap models next quarter.