Pause and resume an agentic AI pipeline without paying twice

Checkpoint every unit of paid work under a stable key, pause at a gate after each fan-out instead of inside it, and keep interrupts for decisions a person makes. That is how Nehemvix resumes a job on any model without redoing or re-billing finished work.

Why long LLM jobs need a resume plan

A Nehemvix job turns a Git repository into a documentation guide with the user's own model key. It summarises every file, then every folder, then the whole system, and then writes the model-written sections of each audience the user picked: 7 for the beginner's guide, 9 for the senior engineer's and 10 for the architect's. On pallets/itsdangerous (46 files) a run for all three audiences takes 37 to 38 model calls, as recorded in our live tests.

Any of those calls can fail for reasons that have nothing to do with the code: a free tier's daily quota, a "503 high demand" from the provider, a wrong key, a deploy that restarts the server, or someone pressing Ctrl-C. If the only recovery is "start again", the user pays a second time for every call that had already succeeded.

The first live run on Gemini 3.8 Flash's free tier stopped on its very first call. Google answered with the quota id GenerateRequestsPerDayPerProjectPerModel-FreeTier and a retry delay of 51,227 seconds, a little over 14 hours. Waiting is not a recovery plan, so the design goal became: a run never dies because one model ran out, and switching to another model continues from the next unfinished unit.

Two layers of durability

The pipeline is a LangGraph StateGraph, compiled with an AsyncSqliteSaver checkpointer and run with at most four branches at a time. LangGraph saves the graph position after every super-step, and the pending writes of parallel branches that finished survive a crash: on recovery only failed or unfinished branches run again.

That alone is too coarse for paid work. One node can make dozens of calls, so Nehemvix keeps a second, finer layer in SQLite:

CREATE TABLE IF NOT EXISTS job_units (
    job_id   TEXT NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
    stage    TEXT NOT NULL,
    unit_key TEXT NOT NULL,
    output   TEXT NOT NULL,
    created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
    PRIMARY KEY (job_id, stage, unit_key)
);

Every folder summary, the system overview, every written section and every chunk of a large file is one row. File summaries live in a separate file_summaries table keyed by the file's content hash, the prompt version and the generation settings, so they also serve as memory across jobs.

Three rules keep this honest:

• Only real results are written. A file whose call failed gets a placeholder in the guide, but the placeholder is never cached, so a resume re-plans exactly those files.
• Graph state stays tiny: flags, counters and lists of file paths. Bulky data lives in SQLite and in a runtime context that is never checkpointed.
• A rerun gets a new thread. "Run again" starts a new LangGraph thread id (<job>:run2), and the unit and file caches make the finished parts free.

Both layers follow one design pattern, and we explain separately why agents are built as graphs with checkpoints.

Use interrupts for people, not for failures

LangGraph's interrupt() is the natural place for a human decision. Nehemvix uses it for exactly one: approving the estimated cost. Unless the high estimate is under the user's auto-approve threshold, the estimate node stops the run before any money is spent:

decision = interrupt({"type": "approve_estimate", "estimate": est})
if not decision.get("approved"):
    raise stages.JobCancelled()

The runner continues it with Command(resume={...}), carrying the approval and the spending cap. A plain ainvoke(None) would only interrupt again, which is what you want for an approval.

An early design also used interrupt() for quota pauses. We changed that: a pause is now an exception raised at a gate node. That gives one resume path for every stage, sequential ones included, and the approval route checks the interrupt's type, so a quota pause can never be mistaken for an approval.

Fan-outs absorb, gates pause

The map stage sends one Send branch per batch of files, and the write stage one per section. We verified that a branch raising an exception cancels its siblings in the middle of their calls. Those calls may already be billed by the provider, and their answers are lost.

So fan-out branches never raise for quota. When every connection is out, a branch defers its unit: no placeholder, nothing cached, and it returns normally. Calls already in flight finish and are settled in the ledger, and no new units are dispatched. A gate node after the fan-in then decides:

def _gate(state, ctx, stage):
    deferred = state.get("swap_deferred", 0)
    if not deferred:
        return {"replan": False}
    if ctx.swap_needed is not None:
        e = ctx.swap_needed
        raise NeedsSwap(str(e), e.kind, e.info | {"stage": stage, "deferred_units": deferred})
    return {"swap_deferred": -deferred, "replan": True}

The runner catches NeedsSwap, marks the job paused_quota, keeps the checkpoint, and stores what ran out (provider, model, quota id, reset time) with the progress so far. An integration test runs out a daily quota in the middle of the map stage and then finishes the job on another provider. It uses a provider double that returns Gemini's real daily-quota error after three calls, then asserts that every ledger row is SUCCEEDED or FAILED: nothing was left in flight, so nothing can be billed twice.

Resume on a different model

On resume the gate runs again with a fresh context, sees the deferred work and routes back. map_gate sends the graph to plan_map, which plans only the unfinished files and sizes the batches for the new model's context window. write_gate sends only the unwritten sections. A large file split into chunks keeps its finished chunks, so even a switch in the middle of a file redoes nothing.

A few details matter in practice:

• Summaries cross models. A file summary written by another model under the same prompt version counts, and the current model's own summary wins when both exist.
• An exhausted connection pauses for free. Resuming with the same out-of-quota connection pauses again before any request, because the exhaustion record is shared.
• Consent first. Sending the rest of the code to a provider the user has not agreed to returns 400 consent_required.
• Provenance in the result. The guide says which model wrote it, for example "gpt-6-luna (you switched from gemini-3.8-flash after its quota ran out)".

Four real interruptions, each resumed without redoing finished work:

What "never pay twice" cannot promise

Be precise about the edges, because users read cost claims closely.

• Calls in flight during a crash. If the process dies mid-call, the ledger row stays IN_FLIGHT and becomes OUTCOME_UNKNOWN on recovery. The provider may have billed it, so its full reservation keeps counting against the spending cap, the job reports how many such calls there were and the most they could have cost, and resuming is the user's explicit choice, never an automatic re-dispatch.
• Cancel works between calls. Usage from a call already in flight can arrive afterwards and is added to the report.
• Test the resume path live. The first live swap, before fixes, "finished" with an empty guide that scored 90. It exposed three bugs at once: gpt-6-luna rejected temperature=0.2, the evaluator counted placeholder sections as content, and repeated bad requests did not pause the job. All three were fixed with regression tests, but no offline test had found them.

A checklist for your own agent pipeline

1. Give every paid unit of work a stable key and store its output before moving on.
2. Never cache a placeholder or a degraded result.
3. Keep graph state small; put bulky data in your own store.
4. Do not raise inside a fan-out. Defer the unit and pause at a gate after the fan-in.
5. Keep interrupt() for decisions a person makes, and check the interrupt type on resume.
6. Record which model produced each unit, and say so in the output.
7. Test with a provider double that fails after N calls, then prove it with a real quota.

How the errors are told apart in the first place (wait, retry, swap or pause) is the subject of rate limits, retries and switching models. What users see when a job pauses is in limits and pauses, where a person decides at each pause is in human in the loop for AI agents, and how to document a LangGraph project shows how to find the same graph, interrupts and checkpoints in someone else's code.

Learning this to build your own agent? AELSHA™ can help you design it.