Claude's Compaction API: Long Sessions Without Context Rot

A long agent run doesn't fail the moment the context window fills. It gets worse gradually, well before that — instructions from turn three stop landing, the model re-litigates decisions it already made, and the useful signal drowns in three hundred tool results nobody needs anymore.

Compaction is Anthropic's server-side answer. When the request's input tokens cross your trigger value, the API runs an extra pass that summarizes the conversation so far, then continues with that summary in place of the history. You get back a content block that looks like this:

{
  "type": "compaction",
  "content": "[summary text]"
}

The important part isn't the block, it's the rule attached to it: the API automatically ignores every content block prior to the compaction block on subsequent requests. You keep appending to the same messages array as always. The server just stops reading the part above the waterline.

Here is the bug you will write, because everyone writes it once. Most chat loops accumulate history like this:

# Wrong — discards the compaction block
text = response.content[0].text
messages.append({"role": "assistant", "content": text})

That works fine for a normal turn and destroys a compacted one. The summary lives in the compaction block, not the textblock. Pull out the string and you've thrown the summary away — so the next request ships the entire uncompacted history again, blows through the trigger again, and compacts again. You pay the compaction pass on every single turn and never once benefit from it. Nothing errors. Your logs look normal.

# Right — the whole content array
messages.append({"role": "assistant", "content": response.content})

Append the array. Every time. If you have a helper that normalizes assistant turns into strings somewhere in your stack, that helper is now a bug.

The whole surface is four fields. Two of them matter more than the docs let on.

The instructions footgun deserves emphasis. It reads like a nudge — “keep the customer's account ID” — and it is actually a full replacement of Anthropic's tuned default prompt. Drop a one-liner in there and you have quietly swapped a well-engineered summarizer for your one-liner. If you need to prioritize specific state, spell out everything you want kept, not just the new thing.

This is the part that will actually cost you money, and it's one sentence in the docs. Top-level usage.input_tokens and usage.output_tokens do not include the compaction iteration. A compacting turn runs two passes, and the top-level fields report only the second one.

The real numbers live in usage.iterations, an array with one entry per pass, each tagged compaction or message. Take the shape straight from Anthropic's documentation — a compaction pass of 180,000 in / 3,500 out, then a message pass of 23,000 in / 1,000 out — and run the arithmetic:

Same turn. Roughly 8x the cost you recorded. If your dashboard reads top-level usage — and most do, because that's what every pre-compaction example used — your spend chart is fiction on exactly the turns that matter most. I flagged something similar when digging into Claude API rate limits: the headers and usage fields carry more than the obvious reads, and the obvious read is usually the wrong one.

Fix it in one place, at the seam where you record usage:

iters = response.usage.iterations or []
total_in  = sum(i.input_tokens  for i in iters) or response.usage.input_tokens
total_out = sum(i.output_tokens for i in iters) or response.usage.output_tokens

The fallback matters: on a turn where compaction didn't fire, treat the top-level fields as the source of truth.

These two get conflated constantly, and it's understandable — they share the context_management parameter. They are separate features with separate beta headers, and mixing the values up gets you a 400.

The practical heuristic: if a tool-heavy agent is drowning in 200 old file reads, context editing (docs) is cheaper and does the job — you're not summarizing garbage, you're deleting it. If a long support or research thread has genuine narrative you'd lose, compaction is worth the extra pass. Plenty of production agents run both: clear the dead tool results continuously, compact the surviving conversation when it gets long.

Compaction rewrites the middle of your prompt, which is exactly the sort of thing that nukes a prompt cache if you're careless. The fix is to cache the stable part on its own breakpoint so it survives the event:

system=[{
    "type": "text",
    "text": SYSTEM_PROMPT,
    "cache_control": {"type": "ephemeral"},   # cached separately
}]

The compaction block takes cache_control too, so after the event only the summary has to be written to cache rather than the whole reconstructed prefix. This stacks with everything in prompt caching — and since cached input tokens don't count against your input-tokens-per-minute ceiling, the two features compound nicely on long agent runs.

Two smaller behaviors worth knowing before you ship:

• ▸Streaming: a compaction block arrives as one content_block_start, one content_block_delta of type compaction_delta carrying the complete summary, then content_block_stop. It does not stream incrementally, so don't build a progress UI expecting it to.
• ▸Token counting: /v1/messages/count_tokens applies existing compaction blocks but never triggers a new compaction. It also returns context_management.original_input_tokens, so you can see the pre-compaction size — useful for deciding whether your trigger is set sensibly.

Compaction is a good default for long-running conversational agents. It is a bad default for everything else, and the beta header being cheap to add makes it tempting to add everywhere.

Skip it when your sessions never approach 50,000 input tokens — the trigger can't go lower, so you've added a beta dependency that will never fire. Skip it on anything where losing verbatim detail is unacceptable; a summary is lossy by construction, and “the model summarized away the exact error string” is a miserable bug to chase. And skip it if your real problem is architectural — an agent that fills a million tokens of context on one task usually needs subagents with their own fresh windows, not a better summarizer bolted onto one enormous thread.

Where it genuinely earns its place: support threads that run for hours, research agents that read a lot and reason across all of it, and anything with a human in the loop coming back the next day. For those, compaction plus real cost controls is the difference between an agent that degrades at turn 30 and one that doesn't.

One last operational note: it's beta on every platform that has it — the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud and Microsoft Foundry. Beta headers change. Pin it, log it, and put a note where your team will find it when a request starts returning 400.