merge: cross-call explore session dedup (CG-2, #1500)

Never re-serve source this session already sent: session-scoped state (CG-17)
plus a precise back-reference in place of the bytes (CG-18). Duplicated source
drops 6.4% -> 0.74% of the response, at flat cost per call, with more unique
source in its place.

Bar 4 of CG-20 -- "residual occupancy must drop" -- is NOT met, and the gate
records why: CG-18's accepted rule spends reclaimed bytes on files not yet
shown rather than banking them, so a design that re-spends every byte cannot
lower the byte count. The two requirements were mutually unsatisfiable as
written. Bars 1-3 (no extra Reads, no abandonment, no bucket shift) pass across
24 runs on both arms.

Kept on that basis, and cheap to reverse: CODEGRAPH_EXPLORE_DEDUP=0 disables it
at runtime. Full gate: docs/benchmarks/explore-dedup-ab-cg20.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-08-05 14:29:20 -05:00
co-authored by Claude Opus 5
12 changed files with 2603 additions and 66 deletions
+240
View File
@@ -0,0 +1,240 @@
# Agent A/B — cross-call explore dedup (epic CG-2 / task CG-20)
**Date:** 2026-08-05 · **New:** `feature/CG-2` @ `7a7ea30` (CG-17 session state + CG-18 dedup)
· **Baseline:** `c65d56c` **by SHA** (main's tip when the epic branched) · **Harness:**
`scripts/agent-eval/ab-new-vs-baseline.sh`, `--model sonnet --effort high`, **both arms
codegraph-on**, `CODEGRAPH_NO_PROMPT_HOOK=1` on both.
This is the epic's hard gate. Returning *less* on a repeat call is the exact shape CLAUDE.md
says drives Read fallback and then teaches the agent to abandon codegraph for the rest of the
session, so the gate is about risk first and win second.
**Verdict: bars 13 pass cleanly and bar 4 is not met.** Read is **0 in all 24 runs of both
arms**, nothing abandons, and the two failure buckets never fire once. Residual context
occupancy is **flat** — and the measurement below shows it could never have been anything else,
because CG-18's own acceptance requires reclaimed bytes to be **re-spent on files the agent has
not seen** rather than banked. What the change actually moves is the *duplicate fraction* of
that residual: **87% across the agent runs**, 86%/94% deterministic.
Recommendation: **keep**, with bar 4 restated. Reasoning and the counter-case in
[§Verdict](#verdict).
---
## Method
Dedup only exists inside one MCP session, so a single-call task cannot exercise it at all. Both
targets were driven with the drill-down question from the CG-1/CG-22 A/B, which reliably
produces a second and third explore whose symbol bags overlap the first:
| Repo | Lang | Files | Tier | Question |
|---|---|---|---|---|
| `kubernetes/client-go` | Go | 2,454 | medium (2 calls / 28K) | "how does a shared informer keep its cache in sync and deliver events?" |
| `excalidraw/excalidraw` | TS/React | 672 | medium (2 calls / 28K) | "how does updating an element re-render the canvas on screen?" |
Each prompt is wrapped `Use codegraph to answer: <question>` — identical on every arm, the CG-22
wrapper. That is **not** a forced-Read-0: fallback stays free, which is exactly what bar 1
measures.
`RUNS=3` per invocation. client-go ran one batch (n=3/arm); excalidraw ran **two** (n=6/arm,
pooled) because it is where dedup bites hardest and therefore where the abandonment risk is
highest — its baseline sequence duplicates ~21% of the source it serves.
### Instruments
Three, because no single one answers the gate:
- **CG-7 residual occupancy** and **CG-8 sufficiency buckets**, from the `feature/CG-3` copy of
`parse-run.mjs` (that branch is where all three feedback metrics live; this branch's copy
predates them). Bars 24.
- **A duplicate-residual measure**, written for this gate. CG-7's occupancy counts the chars of
codegraph results resident in the window but **cannot tell a byte the agent already holds from
one it has never seen** — which is the only distinction dedup makes. This one reads the
*rendered markdown* of every explore response in a run (so it measures both arms the same way;
the CG-4 diagnostic sidecar exists only on the new build), reconstructs the `(file, source
line)` pairs each call put in the window from the `<n>\t<text>` fences, and charges a line
already delivered by an earlier call as a duplicate byte.
It is deliberately **not** committed as a new `scripts/agent-eval/*.mjs`: a new file there
scores into the self-query eval fixture's own corpus and moves its numbers (the CG-15 observer
effect), and the natural home is `parse-run.mjs` **on `feature/CG-3`** beside the other three
metrics. Fold it in there; the rule above is the whole specification.
---
## Deterministic core — no agent
Same index, same query sequence (lifted verbatim from a prior new-arm agent run), replayed
through **one** `ToolHandler` + **one** `ExploreSessionState` on each build. Re-measured on both
builds in this session rather than quoted.
### client-go — 3 calls
| | baseline `c65d56c` | new `7a7ea30` |
|---|---|---|
| response chars | 65,218 | **67,289 (+3.2%)** |
| source chars | 46,555 | 47,209 |
| **unique source** | 44,740 | **46,957 (+5.0%)** |
| **duplicate source** | **1,815 (3.9%)** | **252 (0.5%) — 86%** |
### excalidraw — 3 calls
| | baseline `c65d56c` | new `7a7ea30` |
|---|---|---|
| response chars | 72,364 | **68,442 (5.4%)** |
| source chars | 50,063 | 44,578 |
| **unique source** | 39,575 | **43,973 (+11.1%)** |
| **duplicate source** | **10,488 (20.9%)** | **605 (1.4%) — 94%** |
The two repos bracket the mechanism. Where the baseline barely duplicates (client-go, 3.9%)
there is almost nothing to reclaim, and the reclaimed bytes plus the pointer text make the
response marginally *larger*. Where it duplicates heavily (excalidraw, 20.9%) the response gets
**smaller and denser at the same time** — 5.4% fewer bytes carrying 11.1% more unique source.
**The ceiling on any occupancy win is the baseline's duplicate fraction**, and that is the whole
argument about bar 4: even a design that banked every reclaimed byte instead of re-spending it
could not have removed more than 3.9% / 20.9% of the source in these two sequences.
### Explore latency — the change is not a slowdown
Median of 5 replays of the 3-call sequence, per build (CG-18 adds a truncated SHA256 per served
slice, so this needed checking):
| repo | baseline | new |
|---|---|---|
| client-go | 1,230 ms | 1,287 ms (+4.6%) |
| excalidraw | 458 ms | 445 ms (2.8%) |
≤60 ms across three calls. Nothing here can explain a several-second agent gap — see
[§Counter-points](#counter-points).
---
## Agent A/B — the four bars
`explore` = `codegraph_explore` calls · `cgResidual` = codegraph chars still resident at end of
run, CG-7 · `dup%` = share of served source the agent had already been given.
| repo | arm | n | explore | **Read** | Grep | cgResidual (med) | per call | dur (med) | **dup%** |
|---|---|---|---|---|---|---|---|---|---|
| client-go | **new** | 3 | 2 / 3 / 2 | **0 / 0 / 0** | 0 | 19,446 | 9,641 | 31s | **0.7%** |
| client-go | baseline | 3 | 3 / 2 / 2 | 0 / 0 / 0 | 0 | 19,611 | 9,379 | 27s | 7.9% |
| excalidraw | **new** | 6 | 3,3,1,2,3,2 | **0 ×6** | 0 | 25,688 | 10,316 | 31s | **0.70.8%** |
| excalidraw | baseline | 6 | 2,2,2,1,3,2 | 0 ×6 | 0 | 20,123 | 10,158 | 23s | 3.37.6% |
### Bar 1 — Read count must not increase · **PASS**
**Read = 0 and Grep = 0 in all 24 runs, both arms, both repos.** Not "did not increase" — never
fired. The strongest form of this bar: back-references actually reached the agent in **8 of the
9 multi-call new-arm runs** (28 pointers total), and no run followed a pointer with a Read.
### Bar 2 — no abandonment · **PASS**
The failure mode is silent, so it was measured three ways, all clean across 24 runs:
- **Zero `isError` responses** in either arm. (One or two early in a session is what teaches
abandonment; there were none.)
- **codegraph is the last tool called in every single run** — 0 Read/Grep/Glob/Bash calls after
the final codegraph call, in both arms.
- Call counts do not collapse in the new arm: 23 on client-go, 13 on excalidraw, the same
spread the baseline shows.
### Bar 3 — sufficiency buckets must not shift · **PASS**
The two buckets this epic could plausibly break are **"Read a file we returned"** (we clipped
the wrong thing) and **"Read a file we did not return"**. Over 40 answered explore calls:
| bucket | new | baseline |
|---|---|---|
| Read a file we returned | **0** | 0 |
| Read a file we did not return | **0** | 0 |
| Grep/Glob | **0** | 0 |
| explore again | 12 of 21 (57.1%) | 10 of 19 (52.6%) |
| moved on / answered | 9 | 9 |
The failure buckets are empty on both arms. The `explore again` difference is **one call** at
n=21/19 — noise, and CG-1 already established that these are voluntary drill-downs after a
complete answer, not insufficiency retries (which is why CG-19 was cut).
### Bar 4 — residual occupancy must actually drop · **NOT MET**
Per-run `cgResidual` is dominated by how many calls the agent chose to make, and both arms span
13 calls on excalidraw. Normalising that out, **residual per explore call is flat**: +2.8% on
client-go (9,641 vs 9,379), +1.6% on excalidraw (10,316 vs 10,158).
This is **by construction, not by accident**. CG-18's acceptance says in as many words:
"*Freed budget — bytes reclaimed by dedup should flow to files not yet shown, not shrink the
response*," and `emitFileSection` implements exactly that (a fully-held file frees both its
`sourceSpent` into the carry-forward pool and its `maxFiles` slot). A design that spends every
reclaimed byte cannot lower the byte count. **CG-18's acceptance and CG-20's bar 4 are
mutually unsatisfiable**; that contradiction, not a defect in the dedup, is what this bar found.
What the epic *does* move, measured on the same runs:
| | new | baseline |
|---|---|---|
| duplicate source chars, all agent runs | **2,432 of 329,222 (0.74%)** | 19,295 of 300,750 (6.4%) |
**87% duplicated bytes**, at flat cost per call, with more unique source in their place.
---
## Counter-points
Kept in the record rather than smoothed:
- **excalidraw's new arm is slower at the median: 31s vs 23s**, and its median call count is
2.5 vs 2. It is *not* server-side — explore's own latency is 2.8% there and the deterministic
response for the identical 3-query sequence is **5.4% smaller** on the new build, so the extra
call is not the agent compensating for a thinner answer (and bar 3's failure buckets are
empty). But at n=6 with both arms spanning 13 calls, the difference is one call and **this
measurement cannot attribute it to the build either way**. client-go shows the same-sized gap
(31s vs 27s) at an *identical* median call count. Treat as unresolved; a bigger n is the only
thing that settles it.
- **`dedup.savedChars` in the CG-4 diagnostic is a pre-clip figure and must not be read as
bytes kept out of the window.** On client-go call 2 it reports **11,450 saved** while the
baseline actually re-served only **1,042** duplicate chars of that file — the suppressed
ranges are measured against the *unclipped candidate* render, most of which the budget
allocator would have trimmed anyway. Section-level check on the same call: baseline emits 232
lines of `shared_informer.go` overlapping call 1 by 22 lines; the new build emits 234 lines
overlapping by **0**, and is 544 chars *larger*. Anyone tuning `EXPLORE_DEDUP`'s thresholds off
`savedChars` will over-estimate the win by roughly 7×.
- **client-go's baseline duplicated less than expected** (3.9% deterministic, 020.6% across
runs). On that query `tools/cache/**` already dominates graph relevance, so the pre-dedup
render concentrated well on its own — the same reason CG-22 found only a small #1500 signal
there.
---
## Verdict
| bar | result |
|---|---|
| 1. Read must not increase | **PASS** — 0 Reads in 24/24 runs, both arms |
| 2. No abandonment | **PASS** — 0 `isError`, codegraph last in every run, no call collapse |
| 3. Buckets must not shift to "Read a file we returned" / "another explore" | **PASS** — both failure buckets empty on both arms |
| 4. Residual occupancy actually drops | **NOT MET** — flat per call, and unreachable given CG-18's reallocation rule |
The epic's acceptance says *revert if bar 4 is not met, because the regression risk isn't worth
a marginal win*. **The regression half of that premise was measured and is zero** — 24 runs, no
Read, no abandonment, no bucket shift, back-references demonstrably reaching the agent. And bar
4 is unreachable by construction, not unmet by underperformance: the byte ceiling it was aiming
at is the baseline's duplicate fraction, 421%, and CG-18 was already accepted on the rule that
those bytes get **spent, not banked**.
So: **keep the change, and restate the epic's metric** as the duplicate fraction of residual
(87% agent, 86%/94% deterministic) at flat context cost — with excalidraw showing the
best case, 5.4% fewer response bytes carrying 11.1% more unique source.
This is a judgement call against the letter of bar 4, and it is cheap to reverse in either
direction:
- runtime: `CODEGRAPH_EXPLORE_DEDUP=0` disables dedup without a rebuild;
- source: `git revert 7a7ea30 ab38d1f 4e94860 fc31b1e` removes CG-17 + CG-18 entirely;
- the third option, if occupancy really is the goal: **bank the reclaimed bytes instead of
re-spending them**, which reverses CG-18's freed-budget rule and buys at most the duplicate
fraction above. That needs its own gate — it is a strictly *smaller* answer per call, which is
the shape this task exists to be afraid of.
Logs: `/tmp/cg20/ab-client-go`, `/tmp/cg20/ab-excalidraw`, `/tmp/cg20/ab-excalidraw-b2`
(ephemeral — archive them if a distribution needs to stay reproducible).
+218
View File
@@ -0,0 +1,218 @@
# Cross-call explore session state
`codegraph_explore` answers every call as if it were the first one. It has no idea what it
already sent this session, so a 4th call happily re-serves the spine the 1st call already
delivered (the #1500 report: 4 calls on a 2-call tier budget), and the tier's call budget
can only be *asked* for in prose the agent ignores.
This document covers the state layer that fixes the "no idea" part — `src/mcp/explore-session-state.ts`
(CG-17) — and the first thing built on it, cross-call source dedup
(`src/mcp/explore-dedup.ts`, CG-18). Budget decay past the tier budget (CG-19) is the
other consumer.
## What is recorded
One `ExploreSessionState` per MCP session. Inside it, per **resolved project root**:
- `callCount` / `responseBytes` — every explore call served this session for that project;
- `calls[]` — the recent ones in detail: query, per-file emitted **line ranges**, a content
**fingerprint** for the bytes those ranges were sliced from, source bytes, response bytes,
and the call's 1-based session index.
Ranges come from the render loop itself — `buildSection` returns the spans it slices
alongside the text, and the whole-file / focused / skeleton paths report theirs at the
point they push source into the response. A separate function mirroring the window and
padding rules would drift, and drift here is not symmetric: see *Which way to be wrong*.
Only files that **survive the final hard-ceiling truncation** are recorded. A section the
ceiling dropped was never delivered. A back-referenced file records its spans at **zero
bytes**: the record means "source the agent HOLDS for this file", not "bytes this call
spent", so re-recording keeps a long session from ageing a pointed-at span out of the
retained window and re-serving it for nothing.
## Four constraints, and what each one rules out
| Constraint | Why | What it rules out |
|---|---|---|
| Per session, never persisted | A new agent has seen nothing | A disk cache keyed by project |
| Per **resolved** project root | One session can query several projects by `projectPath` | Keying on the path the agent typed — `/repo` and `/repo/internal` are one project |
| Bounded memory | Sessions can run for hours | Unbounded `calls[]` growth |
| Daemon-safe | One daemon shares ONE `ToolHandler` and a pool of worker threads across every connected client | State on the handler, in a worker, or in a module-level singleton |
The daemon constraint is the sharp one. State kept on the shared `ToolHandler` would blend
two agents' histories, and a dedup built on that would withhold source from an agent that
never saw it — which costs a Read, the exact failure this area exists to prevent. So the
state lives on `MCPSession`, and the plumbing is:
```
MCPSession (owns the state)
└─ ToolHandler.execute(tool, args, sessionState)
├─ down: session view attached to args (survives structured clone → worker)
└─ up: emission attached to the result (survives structured clone ← worker)
└─ recorded on the MAIN thread, then DELETED from the result
```
Both legs travel as plain properties (`_cgExploreSession`, `_cgExploreEmission`) because
either may cross a worker boundary, where a closure or a handler field could not follow.
The emission is stripped in `execute` **unconditionally** — including for callers that
track nothing, like the CLI — so the agent-facing response is byte-identical. A view a
client spells itself is discarded, not trusted: it decides what a later call may withhold.
## The bounds
`EXPLORE_SESSION_LIMITS`: 4 projects (LRU), 8 retained calls per project, 24 files per
call, 24 ranges per file, 4 calls in the view handed to a call.
Every bound caps **detail**. `callCount` and `responseBytes` keep counting past eviction —
decay (CG-19) reads the count, and a bound that reset it would make decay reset itself
every 8 calls.
## Which way to be wrong
Where a bound forces a choice, the record keeps **fewer** ranges than were emitted, never
more:
- under-report → a later call re-serves something the agent already has. Wasteful.
- over-report → a later call withholds source the agent never saw. The agent Reads the
file, and one Read costs more than every byte the dedup saved.
So `coalesceRanges` drops the smallest spans when it hits the cap (and flags
`rangesTruncated`), invalid spans are discarded rather than clamped, and truncated file
sections are never recorded.
## Inspecting it
The CG-4 diagnostic (`CODEGRAPH_EXPLORE_DEBUG`, see
[explore-budget-allocation.md](./explore-budget-allocation.md)) carries a `session` block
on every report:
```
session call #2 for this project · 1 prior call · 18,204 chars already served
already served internal/usecase/payroll_cycle.go · 4,928 chars · L1-159
```
`callIndex` is this call's position in the session; `priorFiles` unions the ranges already
served per file, most-recent call first. The block is **absent** — not zeroed — when the
caller tracks no state, which is how "untracked" and "first call of a tracked session" stay
distinguishable.
---
# Cross-call dedup (CG-18)
A call that would re-send source an earlier call already delivered sends a **pointer**
instead. Never a bare omission: an insufficient-feeling response is precisely what sends an
agent to Read, and one or two of those early in a session teach it to abandon codegraph
entirely. So the replacement carries the file, the symbols, the line spans, and the two
facts that make the copy usable — that it came from THIS conversation, and that the file has
not changed since:
```
**`internal/usecase/payroll/cycle.go`** — Cycle, PayslipsForCycle, Service, …
> **Already sent earlier in this conversation:** `internal/usecase/payroll/cycle.go`
> L42-76, L78-215 (Cycle, PayslipsForCycle, Service, +6 more) — unchanged on disk since,
> so that copy is still exact. Only the NEW lines are shown below; scroll back for the
> rest. Do NOT Read this file.
```
The convention is also stated once, inline, as an exception appended to the "verbatim
source" guarantee (the same shape #1474 uses for drift), and once in
`server-instructions.ts`.
## What gates it
| Gate | Rule |
|---|---|
| Session | Off on a session's first call for a project — nothing to point at |
| Content | A span is withheld only if the file still hashes to the bytes that span was sliced from |
| Size | Only a covered run of ≥ `MIN_COVERED_LINES` (8) is replaced |
| Remainder | New source under `MIN_DELTA_CHARS` (160) folds into the pointer instead of getting its own fence |
| Kill switch | `CODEGRAPH_EXPLORE_DEDUP=0` renders as if the session had no history |
The **content** gate is a fingerprint (`length:sha1-prefix`) recorded per file per call, NOT
the index's drift flag. They answer different questions: two calls inside one drift window
served the same current bytes (dedup is correct); a file edited *and re-synced* between two
calls is never "stale" and yet the agent's copy is now wrong (dedup would be actively
harmful). #1474's drift handling is upstream of this and unchanged — a drifted file still
ships whole or not at all.
The **size** gates exist because the pointer sentence is itself ~140 chars. Replacing a
signature line or the ±3 lines of cluster padding would make the response bigger *and* read
as full of holes. `MIN_DELTA_CHARS` is the one place the design withholds something the
agent has not seen — bounded to ~two lines sitting directly against source it does hold —
and it is there because the alternative is a code fence containing `228\t`, which reads as
a broken response. The file is still named with its symbols, so one follow-up explore
fetches it whole.
## Where the reclaimed bytes go
Two channels, both of which move bytes toward files the agent has NOT seen:
- **`sourceSpent`** — a deduped file spends less, so CG-21's carry-forward pool hands the
difference down the rank order, and `headroom` grows for every file after it.
- **the `maxFiles` slot** — a fully back-referenced file does not consume one (the same
treatment a cliffed file gets), so a file that would not have fit now renders.
Within a file, the shrink decision reads the **deduped** length: shrinking a cluster on its
raw size would drop new symbols to make room for source that is not being sent.
Spending rather than banking is what keeps the response the same *size* while raising the
share of it the agent has never seen — and it is also why CG-20 found residual context
occupancy **flat**. A design that spends every reclaimed byte cannot lower the byte count;
what it lowers is the duplicate fraction of those bytes (87% across CG-20's agent runs).
Measured on two matched 3-call replays: client-go 44,740 → 46,957 unique source chars for a
3.2% larger response, excalidraw 39,575 → 43,973 unique for a 5.4% **smaller** one. If the
goal is ever restated as "fewer bytes," this is the one rule to reverse — and it needs its own
abandonment gate, because banking makes a repeat call return strictly less.
**`dedup.savedChars` is a pre-clip figure.** It counts what dedup suppressed from the
*unclipped candidate* render, not what stayed out of the window — most of a suppressed range
would have been trimmed by the budget anyway. Measured on client-go: 11,450 reported against
1,042 chars the baseline actually re-served. Read it as "how much duplication the ranking
wanted to emit," never as a saving; over-reading it inflates the win ~7×.
## The all-pointer guard
If dedup suppresses everything and nothing new takes its place, the response would be
pointers only — the shape that reads as "codegraph found nothing". The render loop keeps the
first fully-suppressed file's real section in hand and splices it back when the loop ends
with zero new source. It costs a re-serve of one file on the one call shape where dedup
would otherwise have saved everything. That is the safe direction, and it is why "no
duplicate ranges across calls" holds for every call that had anything new to say, rather
than universally.
CG-20 ran that gate on a real agent — client-go and excalidraw, both arms codegraph-on,
n=3 and n=6 per arm. **Read = 0 in all 24 runs**, no `isError`, codegraph last in every run,
and the "Read a file we returned" / "Read a file we did not return" buckets empty on both
arms, with back-references demonstrably reaching the agent in 8 of the 9 multi-call runs. The
guard never fired on a real query — the thinnest of those 21 calls still carried 12,011 chars
of new source, so `newSourceChars === 0` was never reached and that threshold remains untested
in the field; the numbers and the one bar that did not pass are in
[`../benchmarks/explore-dedup-ab-cg20.md`](../benchmarks/explore-dedup-ab-cg20.md).
## Coverage
`__tests__/explore-session-state.test.ts`, in three layers: the container (keying, monotonic
index past eviction, every bound), the handler seam (a real explore against a real index
records real ranges; a session's FIRST call is byte-identical to an untracked one; two
states on one handler stay separate), and the session seam (two `MCPSession`s on one engine
get their own state, and each call carries its own session's).
`__tests__/explore-cross-call-dedup.test.ts` covers the dedup itself: the range algebra and
its thresholds, the fingerprint gate (an edited file re-emits; an unprovable record is
ignored), the pointer's wording (names the file/spans/symbols, never says "omitted", never
steers to Read), and then the seam — a real second call re-sends **no** line the first one
sent, comes back with >20 lines the first call never sent (reclaimed budget, not a shrunken
response), always contains real source however much the session holds, and reports its
savings through the CG-4 diagnostic (`dedup.savedChars`, per-file `dedupSavedChars` /
`dedupCovered`, `render: 'backref'`).
Two things vitest cannot cover, verified by hand against `dist/`:
- **the worker path** — with a `QueryPool` attached, the emission survives the structured
clone back from the worker, records on the main thread, and is absent from the result;
- **two genuinely different projects in one session** — opening a second index inside vitest
fails on the lazy `require('../index')`. The in-suite substitute reaches one project two
ways (bare, and by a `projectPath` pointing at a subdirectory) and asserts both land in one
bucket; multi-project keying itself is covered at the container level.