How AI reads a large repository: map, reduce, synthesize, write
Read every file once into a short structured summary, combine the summaries folder by folder into one view of the system, then write each part from only the summaries it needs.
A large repository does not fit in one prompt, so any AI tool that explains a codebase has to choose what the model reads at each step. This article shows the approach that works for code, with one small repository measured end to end.
Why not paste the repository into the prompt
Even when a repository fits in a model's context window, asking for everything at once goes badly. The model spends its attention unevenly, the answer leans on whatever it read last, and nothing tells you which file a statement came from.
It is also the expensive way. Input is most of the bill when a model reads code: on our runs over pallets/itsdangerous, input tokens outnumbered output about four to one.
Deciding what the model sees at each step, and in what form, is called context engineering. For code, the approach that holds up is hierarchical summarisation: read the raw code once, turn each file into a compact structured note, and let every later step work from notes instead of source.
The same idea reaches well beyond reading code, and we wrote separately about context engineering for any agent.
Before any model call: filter and rank
The cheapest token is the one never sent, and a lot can be done without a model:
- Filter. Drop vendored folders, build output, lockfiles, minified bundles, binaries and very large files. List each skipped file with its reason, so the reader knows what was not read.
- Rank. Put entry points, manifests, the README and files many others import first, and tests, fixtures and deep paths later. If the budget runs short, the important files have already been read.
- Index the symbols. A parser, or a few patterns per language, can list every function and class with its line. That index later backs citations and lets you check the names the model writes.
- Remove secrets. Keys and tokens in the code should never reach a model, as sending code to an LLM safely explains.
Map: one summary per file
Group files into batches that fit comfortably in the model's window, in ranked order, and ask for a structured answer per file: its role, its purpose, a short summary and its key symbols. JSON is easier to check than prose, because a file missing from the answer is visible at once.
Summarising is extraction, not reasoning, so use the lowest reasoning effort the model offers. Reasoning tokens are billed as output and add nothing here. Measured live on Gemini 3.8 Flash, a low-effort answer used 57 output tokens where high used 502.
Plan for the awkward cases per file, not per job. Split a file too big for one call into chunks and merge their summaries. Split a batch in halves when the provider says the context is too long. When an answer leaves files out, keep the ones that came back and read only the rest again. A model that hits its output limit can return cut JSON that looks like an empty answer, and rereading the whole batch would hit the same limit.
Reduce: folders, deepest first
Walk the folder tree from the deepest level up. Each folder gets one call that sees the summaries of its files and sub-folders, never the code, and writes a summary of its own. A folder with a single child can simply inherit that child's summary.
Save each finished folder as its own unit of work. If the run stops halfway up the tree, the folders already summarised are not paid for again; pause and resume without paying twice shows the pattern.
Synthesize: one view of the system
One call turns the top-level summaries into a description of the whole system: components, how they connect, the stack, the data flow and how to run it. This is the step that needs real reasoning, so give it more effort than the map.
Ask for that description as data, not as a diagram. A diagram drawn from the data in code is always valid and can't carry stray markup from the model.
Write: analyse once, write many
Writing fans out again, one call per section. Each section gets only what it needs: the relevant file and folder summaries, a few facts, and the exact list of file paths it may cite, so it has no reason to invent one. Because the analysis is shared, a second guide for another audience costs only its own sections.
Long shared context is where prompt caching pays. When every section call starts with the same project brief, byte for byte, a provider that caches prompts reads it once at a discount. Measure it per provider before you rely on it. Caching rules differ, and a shared brief that is never read from cache only makes every call bigger.
A worked example: one small repository
Here is that pipeline as Nehemvix runs it on pallets/itsdangerous, a 46-file Python library, writing guides for three audiences: beginner, senior engineer and architect.
| Date | Model | Calls | Tokens in / out | Cost | Scores |
|---|---|---|---|---|---|
| 2026-10-15 | gpt-6-luna, before explicit caching | 37 | 193k / 50k | $0.0467 | 99 / 94 / 94 |
| 2026-10-15 | gemini-3.5-flash-lite, shared brief | 37 | 184k / 67k | $0.22 | 84 / 68 / 88 |
| 2026-11-03 | gpt-6-luna, from PyPI | 38 | $0.0459 | 96 / 97 / 96 |
Two things stand out. The call count is small for three guides, because the analysis is shared and only the sections are written per audience. A rerun of the first job with explicit cache breakpoints cost 16% less, which is the caching point above in practice.
The Flash-Lite line is the counter-example, and it is in the table on purpose. With the shared brief its input grew by about half against the lean per-section layout, the provider reported zero cache reads, and about 34,600 of its output tokens were reasoning. So Nehemvix uses the shared brief on OpenAI and Anthropic models and keeps lean per-section contexts on Gemini.
Memory across runs
A file that hasn't changed doesn't need a new summary. Key each summary by the file's content hash, plus the prompt version and the generation settings, and a second run pays only for what changed. A file copied to several paths is summarised once. Changing the prompt should invalidate the cache on purpose, and changing the model doesn't have to.
Scope the cache like the data it came from: summaries of a private repository belong to whoever ran it. Nehemvix keeps a least-recently-used cache capped at 150 MB. A second run of jonschlinkert/is-odd reused 12 of 12 files and cost $0.0054 instead of $0.0065. The saving is the map stage only, because folders, synthesis and sections are written again for the new job.
What to leave out
- Token-dropping compression. Tools that delete "unimportant" tokens also delete identifiers, paths and line structure, which citations and name checks depend on.
- Embeddings, by default. For questions about a finished guide, full-text search over the guide and its file summaries is often enough. Nehemvix uses SQLite FTS5 and passes the model the top 8 passages.
- An open-ended agent loop. When the work is known up front, a fixed pipeline is cheaper to estimate, cap and resume than a model choosing its next tool.
Checklist
- Filter and rank before the first call, and list what you skipped.
- Read each file once, at low effort, into structured notes.
- Check every answer for missing files and reread only those.
- Summarise folders from notes, deepest first, saving each one.
- Draw diagrams in code from the model's data.
- Give each section only the notes and file paths it may cite.
- Measure prompt caching per provider before relying on it.
- Cache summaries by content hash and prompt version.
How Nehemvix runs these stages for a user is on how it works, and how each guide is then checked against the code is in catching hallucinated names in AI docs.
Related reading: How to understand a new codebase and Estimate and cap LLM costs before a run.
Learning this to build your own agent? AELSHA™ can help you design it.