~/experience/freshctx.md

← Home

FreshCtx

FreshCtx icon

FreshCtx is a local context engine for coding agents. Before each model request, it replaces old file reads in the conversation with short markers and adds the current version of the code the agent has read, once, at the end of the request.

Coding agents re-send the whole conversation on every request. When a file changes after the agent read it, the model sees the old version next to tools that work on the new one. In a public dataset of 67,074 OpenHands runs, 78% of runs sent the model at least one stale view of the code, and about 1 in 7 requests carried one.

77.8%

of runs sent the model at least one request with a stale file view

52,151 of 67,074 runs. Each square is 1%.

  • Runs whose last request still carried one 22.1% 14,793 of 67,074 runs
  • All requests that carried one, about 1 in 7 15.0% 645,736 of 4,316,962 requests

How it works

FreshCtx builds on CORVUS, a paper by researchers at Purdue University and AWS AI Labs, and takes on the first item of its future work: refreshing the functions the agent read instead of whole files.

  • No new tool. The agent reads files the way it always has, and any successful read counts.
  • Only the outgoing copy changes. The saved conversation stays exactly as the tools returned it, for debugging and replay.
  • Functions, not files. Tree-sitter parsers for Python, JavaScript, TypeScript, Go, and Rust find each function again by name after edits move or change it. On five real traces, the refreshed code was 1.7 to 8.6 times smaller than whole files.
  • A budget. The current code fills a 128 KiB budget, most recently read first, and a function is never cut in half.
  • Checked before sending. Every inserted piece of code is hashed against the file on disk. If anything changed after the plan, the request is not sent.

Without FreshCtx

  1. userFix the pricing bug
  2. agentview price.js
  3. toolreturn quantity * 10;stale
  4. agentedit price.js
  5. tooledit applied

With FreshCtx

  1. userFix the pricing bug
  2. agentview price.js
  3. tool[a2bc5d45e8a9b6375344a9f4]marker
  4. agentedit price.js
  5. tooledit applied
  6. userprojection
    price.js:symbol:52bytes
function total(quantity) {
  return quantity * 20;
}

It runs as a small Node.js process next to the agent, and a thin bridge inside the agent talks to it. There are bridges for Pi, tested against the real agent, and OpenHands, tested with recorded requests.

Results

I replayed 1,198 runs from the dataset through the engine. Every piece of code it inserted matched the file byte for byte at the moment of its request, with zero stale views.

Whether the model does better is a separate question. In small live experiments, FreshCtx helped when the model could not re-read, giving the current value 7 times out of 8 against 0 out of 8, but did worse when it could re-read, because the markers did not say where the current code went. It is also not cheaper: the old reads it removes are served from the prompt cache, and the current code is billed at full price.

The full write-up covers the method, the failures, and what comes next. The measurements, replays, and live runs are public in the research repository.