d652c148f6ad263113396734f3a4ed2db36e807f
80
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Colby McHenryandClaude Opus 5
|
d652c148f6 |
docs(cg-30): changelog entry + record the self-query probe flip honestly
The self-query allocation probe fixture's delivered-share gates now fail. The cause is not the new bound: allocation is unchanged between arms (parse-run.mjs 32.3% vs 33.9% on main) and tools.ts delivers the same 8,282 chars in both. What changed is that the incidental file now DELIVERS — on main its whole section was cut by the hard-ceiling truncation, so the fixture passed on truncation luck. Every file on this repo obeys the new bound (max 1.40x of spendable). Recorded as `afterCG30` with that reasoning rather than tuning the bound to restore the pass. The over-reservation it exposes is epic CG-24's subject. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
04c0f8eab5 |
test(agent-eval): sum tokens per turn — result.usage stopped being cumulative
"Tokens processed" was read off result.usage. That was correct when the README figures were measured; in current Claude Code the field reports the LAST turn only. Nothing here changed — the host did, silently — and the harness kept reporting the smaller number. The error is one-sided, which makes it worse than noise: it under-counts whichever arm takes more turns, and that is always the WITHOUT arm. On the 2026-08-05 campaign it turned a real 62% token saving into 19% and invented a token REGRESSION on tokio (-41%) and alamofire (-25%) that does not exist. Those numbers were one push away from the README. Now summed per assistant request and deduped by message.id, the same rule the occupancy timeline already used — Claude Code emits one event per content block carrying identical usage, so summing per event double-counts (~1.7x measured). CLAUDE.md already warned about this field. The code did not follow; it does now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
520ed9d933 |
test(agent-eval): report residual direction by sign, not by hope (CG-13)
The occupancy summary hardcoded "% lower with codegraph". pct(w, wo) is the reduction going with->without, so a negative value means the with-arm's residual is LARGER -- and the line printed "-82% lower with codegraph" for the case where codegraph in fact occupies 82% MORE. A double negative that reads as a win and inverts the headline of the whole metric. Direction now follows the sign in words, and the negative case says what the shape actually is: codegraph front-loads one large verbatim payload that stays resident, where Read/Grep churn many small results that evict. Fewer total tokens processed and a larger persistent footprint are both true at once -- that pair is the axis issue #1500 reported. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
3e8922dfad |
test(agent-eval): report all three feedback metrics per arm, side by side (CG-11)
The three metrics existed but only run-all.sh printed them, one block per run. ab-new-vs-baseline.sh — the harness that actually isolates a retrieval change, both arms codegraph-on — grepped its parse output down to `by type` and `Result`, so occupancy, sufficiency and allocation never reached the maintainer running the A/B they were built for. Both harnesses now print the three blocks under every run and end with one compare-arms.mjs table: median [min–max] per arm across RUNS, sufficiency pooled (it is per-CALL, so median-of-run-percentages would weight a 1-call run like a 5-call one), allocation pooled by bytes and per run. The table is "did it move?"; the per-run blocks stay the "why?" — only they name the query that fell short and the file nothing cited. It reproduces the recorded CG-22 express result off logs already on disk: baseline 3/6 calls in the `Read a file we returned` bucket at 82.0%, new 0/5 at 96.9%. parse-bench-readme.mjs gets the same two metrics as a with-arm table, so the CG-13 campaign aggregates all three rather than occupancy alone. Also folds the CLI-block shim into no-cli-shim.sh and gives it to ab-new-vs-baseline.sh. There it is not a with/without leak but an attribution one, and it breaks all three metrics at once: output arriving through Bash is charged to Bash in the occupancy table, and an explore issued through the CLI is not a tool call at all, so it never reaches the sufficiency classifier or the allocation parse. The run silently drops calls from every number. The daemon pre-warm and the model policy are untouched. Validated on one live gin arm (2 explores, 0 Read, all three blocks + table) and against the cg22/cg15 and ab-readme logs. Selftest 68/68. |
||
|
Colby McHenry
|
db3b8d2a1e |
test(agent-eval): report what share of explore's bytes the answer used (CG-9)
The envelope view needed a human to say which files answer the question (`--answer <glob>`). This reads it off the agent's own final answer and reports one number per run and per call: bytes returned for files the answer cited, over all bytes returned. That is the #1500 defect as a number instead of a hunch. Attribution has two channels, ranked so the weaker one stays separable: the answer naming the file (reported alone as the conservative floor), and the answer citing, in a code span, a symbol only that file DEFINES. Three guards keep the error from leaning optimistic — the direction a tuning metric must not lean: * Only symbols the file defines. Section headers render `name(kind)` for call sites too, and crediting those marked excalidraw's dragElements.ts used because the answer named `mutateElement`. * A definition beats an import alias of the same name (`variable`), or `lib/application.js` gets credit for `require('./utils')`. * A name on 3+ returned files identifies none of them. Bare basenames count as citations (agents write `utils.js:225` in prose) but only for extensions the envelope shipped, so `res.send` and `mime.contentType` — the same token shape — do not read as files. Both the envelope view and this share one parse of the rendered markdown (parseExploreCall), still not the CG-4 sidecar: the sidecar exists only on a post-CG-4 build and so cannot measure a baseline arm. |
||
|
Colby McHenryandClaude Opus 5
|
a2a916e1a5 |
test(agent-eval): bucket every explore by what the agent did next (CG-8)
The agent's next action after a codegraph_explore is free ground truth about
whether the response was enough, and the harness was discarding it. Every run
now bucketed: explored again (insufficient), Read a file we returned
(allocation -- right file, wrong bytes), Read a file we did not return (recall),
Grep/Glob (recall, weak), or moved on (sufficient). The buckets are chosen so
each one names a distinct fix.
The classifier lives in parse-run.mjs next to the occupancy math and takes raw
events, so parse-session.mjs reuses it for interactive runs -- no new
scripts/agent-eval/*.mjs, which would score into the self-query eval fixture's
corpus.
Four rules, three of them found by validating against real transcripts rather
than reasoned up front:
* A call in the SAME assistant message as the explore predates its response,
so it is not a verdict on it. Stepped over, counted as `concurrent`.
* ToolSearch/TodoWrite carry no signal; the call behind them is the verdict.
* SUBAGENTS ARE A SEPARATE THREAD. Claude Code interleaves a subagent's calls
into the same stream under parent_tool_use_id -- verified on a live
excalidraw run where a delegated search's greps landed between the parent's
own calls. Matching reactions across threads scored the subagent's grep as
the parent's verdict on an explore it never saw. A delegation is judged by
what the subagent did FIRST: before that, the same run reported 33%
sufficient while the subagent was off grepping for the file, which is the
one direction of error a tuning metric must not have. In interactive
sessions the subagent is a separate FILE instead, so parse-session.mjs
stitches the threads back with the toolUseId in agent-*.meta.json.
* A re-read of a file an EARLIER explore shipped is still allocation, not
recall -- filing it as recall aims the fix at the wrong end of the pipeline.
Shell file access counts too (`sed -n 100,200p f` reads, `grep`/`find` search),
since both arms have Bash and counting only the Read tool would score those
explores as sufficient. A heredoc or redirect is writing, not reading.
Validated by hand on cg22/ab-express/run-baseline-1 (explore, explore, Read of
lib/response.js which explore #2 returned -- the #1500 allocation bug as a
bucket instead of a hunch; the new-build arm is 100% sufficient) and on
cg15/ab-express/run-new-2 (four explores, the last returning lib/utils.js which
the agent then read at offset 195). Swept over all 76 A/B logs on this machine:
0 crashes, 176 calls bucketed. --selftest covers every bucket, both thread
rules, delegation, shell reads/searches and errored calls: 46/46.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
52b194a6be |
merge main into CG-3: keep the envelope view alongside occupancy
CG-3 branched from main before CG-1 landed and rewrote parse-run.mjs wholesale into an exported parseSession(), which dropped CG-1's --envelope/--answer reporting entirely. That view is the instrument the CG-1/CG-22 allocation gate measures bar 2 with, and it is in that benchmark's documented reproduce steps, so it cannot be lost to the merge. Resolution takes CG-3's rewrite as the structure and ports the envelope feature into it: parseSession now collects codegraph_explore response text in call order, formatEnvelope renders the per-file share, and the CLI parses --envelope/--answer ahead of the positional filter so a glob is never mistaken for a log path. The glob sentinel stays written as a \u0000 escape, never a literal NUL byte -- a raw one makes git treat the whole script as binary, exactly as the comment there warns. Verified: --selftest 18/18, and a synthetic explore transcript reports the expected per-file shares and answer-set total. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
1d333017a8 |
test(agent-eval): count blocked CLI attempts apart from real contamination (CG-7)
The hook denies the invocation, so a denied attempt puts no codegraph output in the window and must not disqualify the run -- only a call that actually returned content does. Attempts are still reported, since an agent hunting for the CLI is worth seeing. |
||
|
Colby McHenry
|
e35d4861e0 |
test(agent-eval): block the codegraph CLI outright — hiding it from PATH was not enough (CG-7)
An agent denied `codegraph` on PATH ran `find / -maxdepth 4 -iname "*codegraph*"`, found the binary, and invoked it by ABSOLUTE PATH — 12 times in one without-arm run. So block the invocation itself with a PreToolUse hook on Bash, written into the run's output dir as an artifact alongside the MCP configs rather than as a repo file. The pattern matches command positions only, so looking is still allowed and only using is denied: `grep codegraph src/`, `ls .codegraph` and `which codegraph` pass through, while `codegraph explore`, `/abs/path/codegraph …`, `cd x && codegraph …` and `VAR=1 codegraph …` are refused. run-all.sh proves both directions at startup and refuses to run if either fails. parse-run.mjs's detector uses the same rule, so prevention and detection cannot drift — and it no longer false-positives on the corpus path, which contains the word codegraph. Verified end-to-end: the without-arm now probes with `ls .codegraph; which codegraph`, finds nothing usable, and falls back to Read/Bash. |
||
|
Colby McHenry
|
d3c01ce8ed |
test(agent-eval): stop the arms reaching codegraph through the shell (CG-7)
The without-arm had no MCP server but still had Bash, and the target repo carries the .codegraph/ index the with-arm needs. Agents found it: 14 of 15 without-arm runs in a 7-repo pass ran `codegraph explore` through Bash, one of them via `ls .codegraph && codegraph explore ...`. That arm was measuring codegraph-over-CLI, not codegraph-absent, so every without-arm number it produced was wrong. It bit the with-arm too -- output arriving through Bash is attributed to Bash, understating what codegraph itself occupies (1 of 15 runs). Both arms now run on a PATH where the CLI is hidden, so the MCP server is the only way to reach codegraph and stays the single variable. The binary shares a directory with tools the run needs -- claude itself sits next to it -- so the directory is substituted in place by one of symlinks to every entry except codegraph, preserving PATH order and precedence. The run aborts if claude or node did not survive the substitution. Prevention alone would fail silently the next time the CLI lands somewhere new, so parse-run.mjs flags any Bash command naming codegraph and parse-bench-readme drops contaminated without-arm runs from the aggregate (CG_INCLUDE_CONTAMINATED=1 keeps them). CG_ARMS re-runs one arm without redoing the other. |
||
|
Colby McHenry
|
257a7b7327 | test(agent-eval): RUN_FROM, to extend a pass without redoing finished runs (CG-7) | ||
|
Colby McHenry
|
af3ce390da | test(agent-eval): drop the duplicated fixed-overhead line (CG-7) | ||
|
Colby McHenry
|
77845c747d |
test(agent-eval): show every run's residual and tool mix, not just the median (CG-7)
A median over 2-3 runs hides swings big enough to flip a repo's sign. On vscode the without-arm ranged 40k to 67k and the with-arm 59k to 65k across two runs; the deciding variable is the tool mix, since a with-arm run that reads files ON TOP of calling explore pays for both. |
||
|
Colby McHenry
|
b93c8d2b6c |
test(agent-eval): self-test the occupancy math, and fix ratio calibration under shedding (CG-7)
parse-run.mjs --selftest runs the math over synthetic transcripts with known answers: attribution, message.id dedupe, compact_boundary, FIFO micro-compaction, and multi-turn stitching. It found a real bug. A gap where the window also SHED content has a delta far below what was added, which reads as absurdly dense text and dragged the whole run's ratio with it -- a shed gap in the fixture pushed 2.5 chars/tok to 4.4 and left the wrong result resident. Shedding can only push a gap's ratio up, so the calibration now takes the lower median as its centre, drops gaps well above it, and pools the rest. Runs that never shed are unaffected (gin and vscode re-measure identically). Also drafts docs/benchmarks/residual-context-occupancy.md -- method, error bar, and the limitations this metric does not settle. Baseline numbers to follow. |
||
|
Colby McHenry
|
4d5f8d371a |
test(agent-eval): report the occupancy metric's own error bar (CG-7)
On a gap that is >=95% one tool result, the measured context delta IS that result's token count, so the spread between it and the run-level ratio is the attribution error. Median over such gaps: +/-1-2% on real runs. |
||
|
Colby McHenry
|
9b4df2133b |
test(agent-eval): price codegraph's fixed context cost alongside its residual (CG-7)
The first request's prompt is system + tool schemas + the question, before any tool has answered, so differencing the arms' ctxBase prices what codegraph occupies whether or not the agent ever calls it. Measured on gin: +775 tokens, small because the tool is deferred -- only its name is in the initial listing. |
||
|
Colby McHenry
|
4080b7501e |
test(agent-eval): measure residual context occupancy, over multi-turn sessions (CG-7)
The A/B arms reported cost, tokens, time and tool counts for one headless question. They could not report what issue #1500 actually measured: how much of the context window a tool's responses still occupy once the question is answered, which every later turn is then charged for. parse-run.mjs now measures that. Tokens are measured, not estimated: for each assistant request, input + cache_read + cache_creation is the exact token count of its whole prompt, so consecutive requests differ by exactly what was appended between them. That delta is priced against the characters in the gap, calibrated on gaps that are >=80% tool result. Explore output lands near 2.3 chars/token, so the usual bytes/4 estimate would have under-counted it by ~40%. Content also leaves the window, so residual is tracked apart from contributed: a compact_boundary clears the resident set, and a mid-run context drop is micro-compaction, which sheds the oldest tool results first and is applied FIFO. run-all.sh takes "Q1||Q2||Q3" and runs them as one resumed session, one segment file per turn; parse-run.mjs stitches the segments back together. bench-readme.sh now runs each README repo as a three-turn session (CG_TURNS=1 restores the single-question form). parse-bench-readme.mjs reports the arms' retrieval residual side by side -- codegraph's responses against the without-arm's Read/Grep/Bash -- in absolute tokens, share of context, and share of window, and says so explicitly when the rows it aggregated were single-turn. Two transcript traps are handled and documented at the call site: Claude Code emits one assistant event per content block, all carrying the same usage (summing per event double-counts every turn with both thinking and a tool_use), and the streamed output_tokens is a partial snapshot. Occupancy lives in parse-run.mjs and is imported by the aggregator rather than extracted to a module -- a new scripts/agent-eval/*.mjs scores into the self-query fixture's own corpus and moves its numbers. |
||
|
Colby McHenryandClaude Opus 5
|
edce18f586 |
test(agent-eval): restore the engine on INT/TERM too (CG-15)
Killing ab-new-vs-baseline.sh mid-baseline-arm left the engine checked out at the baseline ref with the post-baseline files deleted, so every later build in the working tree was silently the OLD code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
48a2309b92 |
test(agent-eval): RUNS knob + explore envelope-share view for new-vs-baseline A/B (CG-15)
ab-new-vs-baseline.sh now builds and indexes once per arm and runs the task RUNS times (default 1), so the >=2-runs-per-arm rule costs one build instead of N. Both arms run with CODEGRAPH_NO_PROMPT_HOOK=1 — the machine's ambient front-load hook resolves to whatever is in dist/, a second uncontrolled channel that confounds the tool-call counts — and point explore's CG-4 diagnostic at a per-arm sidecar. parse-run.mjs gains --envelope/--answer: the per-file share of the explore source envelope, parsed from the rendered markdown so it works on ANY build. The CG-4 sidecar only exists post-CG-4, so it cannot measure the baseline arm; this is the only view that measures both arms the same way. Folded into parse-run.mjs rather than added as a new script on purpose: a new file named after explore's budget scores into the self-query fixture's own corpus and moved its answer share 59.9%% -> 47.9%%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
5f7f5f59df |
feat(mcp): score-proportional byte allocation for explore, with a relative cliff (CG-12, #1500)
The explore envelope used to follow FILE SIZE, not relevance. Every admitted
file was capped at the same flat `maxCharsPerFile`, while the whole-file rule
handed anything under `maxCharsPerFile * 3` its entire contents — a 3x swing
decided by how big a file happened to be:
- self-query: `memory-budget.ts` (score 18) shipped whole and took 51.2% of
the response; `src/mcp/tools.ts` (score 41, 4x the graph mass, 3x the term
hits — it holds the allocator itself) was clipped at 3,800 and got 32.9%.
- #1500 Go fixture: two generated CRUD files shipped whole at ~4.5K each AND
consumed two of the tier's four file slots, so `BuildPayslip` — the
hand-written "calculate" half of the question — ranked #6 and never
rendered at all.
`allocateExploreBudget` now reserves each ranked file a share of the envelope
before anything renders, so the render loop spends a reservation instead of
racing for whatever the files above it left:
- weight = score x worth x (spine ? 2 : 1), where `worth` is `rankPenalty`
applied a SECOND time — ranking answers "is this file about the query",
allocation answers "will these bytes teach the agent anything", and
generated CRUD can legitimately rank while its bytes stay boilerplate;
- a relative cliff at 15% of the top weight (capped at SCORE_FLOOR_MAX, so a
god-file can't silence peers the score floor just admitted) gives a file
ZERO source — path, symbols and line numbers only — and crucially frees its
`maxFiles` slot for a file that earns its bytes;
- every admitted file gets MIN_CHARS, then the remainder splits by weight:
the floor keeps a diffuse survey question returning a spread, the remainder
concentrates a precise one;
- the flat per-file cap is retired as the primary guard, leaving a 70%-of-
envelope safety valve.
Two changes were needed to make the reservation bite: an oversize cluster now
shrinks by whole MEMBER symbol ranges (a single-cluster god-file previously
took ~40% more than allotted, and the file below it was dropped for lack of
room), and the arrival-order budget stops are gone — they cut files by the
order they were reached rather than by merit.
Measured: payroll-go answer group 25.6% -> 78.7%, generated 57.4% -> 0%, and
`func (s *Service) BuildPayslip` now delivered; self-query `tools.ts` 18.5% ->
60.6%, past the epic's >50% bar. Controls hold: cobra/gin diffuse survey
queries keep their file spread (3->3, 3->4), express's middleware query is
byte-identical, and gin's flow query moves its top file from the thin `ginS`
singleton wrapper to `routergroup.go`.
One documented exception to "no previously-unclipped file becomes clipped":
`memory-budget.ts` was unclipped-whole at 5,672 and now clusters within its
3.1K reservation. That is the epic's own diagnosis of the bug — it scored 18
against 58 and was taking the larger slice purely for being small.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
a3898cdc70 |
feat(mcp): relevance scoring overhaul for explore — kill incidental name-collision matches (CG-10, #1500)
Explore's per-file relevance awarded +50/+10/+3/+1 by match class and admitted
anything scoring >= 3. Neither half held up: the tier said HOW a symbol reached
us, never whether the match was evidence, and an absolute floor admits noise on
any repo where the top file scores 50+. Three scripts/agent-eval/*.mjs harnesses
took 63% of this repo's own "how does explore allocate its output budget" answer
on nothing but an unused `const explore` and a `const BUDGET`.
Four levers:
- KIND WEIGHT (RELEVANCE_KIND_WEIGHT): callables and types 1.0, members ~0.5,
variable/constant/parameter 0.15-0.35. A weak-kind symbol with no usage edge
anywhere in the graph (`contains` excluded — nesting is not usage) drops to
0.08. Only weak kinds in the top two tiers pay for the DB probe; the subgraph's
own edges answer most cases free. No measurable latency change (210 vs 211
ms/call, n=12 interleaved).
- PERIPHERAL CAP: nodes >=2 hops from any match accumulate into a bucket capped
at 5. Uncapped they added a flat +1 each, so a file grew more relevant by being
bigger — parse-session.mjs reached 22 off one constant plus twelve unrelated
symbols.
- RANK PENALTY: generated files x0.3, low-value x0.5, applied to the score AND
the graph mass. Score alone would not have fixed #1500 — the generated CRUD
carries MORE graph mass than the hand-written use-case, and graph mass outranks
score in the comparator. Self-normalizing, never a hard exclusion.
- RELATIVE FLOOR: clamp(topScore * 0.2, 1, 10). Capped at one full-strength
direct match so concentration elsewhere can never exclude one (without it a
named-seed-heavy file pushed the floor to 21 and dropped a file the agent had
named by class name). Backfills to 3 candidates when it would leave fewer, and
drops the evidence requirement rather than return nothing at all.
excludeLowValueFiles was dead config — declared per tier, read nowhere; the
test/spec exclusion has been unconditional for a while. Removed. The real gap was
the detector: `isLowValue` anchored on a leading `/`, so a repo-ROOT `test/` dir
(express, cobra, most of npm and Go) never matched — express's routing question
spent 59% of its envelope on three test files. Anchored at `^` too, and the
filter now runs before the floor and judges "are there other candidates?" on the
whole gather.
Measured before/after on the same indexes (baseline
|
||
|
Colby McHenryandClaude Opus 5
|
bd86ad2061 |
test(explore): #1500 regression fixtures for budget allocation (CG-6)
Two permanent fixtures pinning the failure mode from issue #1500 — explore spending its byte envelope on files that merely name-collide with the query. BOTH FAIL TODAY, by design: they document the bug and become the pass gate for CG-10 (scoring) + CG-12 (proportional allocation). __tests__/fixtures/payroll-go/ — a synthetic Go service mirroring the reporter's shape: generated FKIT CRUD beside a hand-written payroll use-case, entered from an HTTP route. Half the generated tree carries ORDINARY names detectable only by their `// Code generated ... DO NOT EDIT.` header (the #1500 case, and end-to-end cover for CG-5); `payrollpb/*.pb.go` covers the path-detectable channel. BuildPayslip, Upsert and Store each exist twice, generated and hand-written. cycle.go sits above the whole-file window so it clips; the generated files sit below it so they ship whole. Asking "how does payroll cycle create and calculate payslips?" — naming none of the answering symbols — the generated CRUD delivers 57.4% of the envelope against the hand-written layer's 25.6%, all of the latter domain types. cycle.go is allocated the single largest slice (30.6%) and delivers ZERO: the hard ceiling drops its whole section. runPayrollCycleAll, the hand-written BuildPayslip and the real Upsert never reach the agent. The second fixture is this repo, "how does explore allocate its output budget across files", where scripts/agent-eval/*.mjs take 71.8% against tools.ts's 18.5% despite scoring 4.6x lower. It reads the live index, so its assertions are relative rather than fixed percentages. - scripts/agent-eval/probe-allocation.mjs — per-file budget-share probe, driving the CG-4 diagnostic through a JSONL sidecar so it measures the shipping allocator. Fixture entries are hermetic (copy + re-index per run, verified byte-identical across runs); exits 1 while any assertion fails. - scripts/agent-eval/allocation-fixtures.json — both fixtures declared, with the 2026-08-03 baselines. - __tests__/explore-allocation-1500.test.ts — fixture-shape assertions green today; the allocation assertions held as `it.fails` so the suite stays green while the bug is open and goes RED the moment it is fixed. Also documented and deliberately left unfixed: runPayrollCycleAll's `s.store.Upsert` edge resolves to the GENERATED Store.Upsert, not the hand-written one — same-name method resolution across two packages picks the wrong receiver. It is upstream of the allocation bug, so it belongs with CG-10's scoring work. Refs #1500 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
d1b75a1a27 |
feat(kernel): R7b Dart walker — dart module, vendored-grammar-C d4d8f3e + wasm byte-copy vendor, dart default-routed (#1386)
R7b batch 4 #4 — the FINAL R7b language (docs/design/dart-kernel-port-checklist.md is the authoritative quirk list). The fourth vendored-grammar-C language, with a twist: production dart resolved its wasm from tree-sitter-wasms, whose dart dependency is an UNPINNED github:UserNobody14/tree-sitter-dart — a routine dependency update would have silently changed dart's grammar. This PR byte-copies the shipping 0.1.13 artifact into src/extraction/wasm/ (VENDORED_WASM_LANGS += dart) and compiles the same-commit (d4d8f3e337d8) parser.c/scanner.c in the kernel — table identity proven by the kernel-grammar-parity row. crates.io tree-sitter-dart is the nielsenko fork (different lineage) — rejected. The center of gravity is THE SIBLING-BODY DOUBLE-WALK, reproduced bug-for-bug: dart attaches every function/method body as a NEXT SIBLING of its signature, and the TS walkers consume each body TWICE — once via resolveBody (attributed to the function/method) and once via the enclosing generic walk (attributed to the file/class). Duplicate local-function nodes with the SAME id under different parents, duplicated calls/instantiates refs, and file/class-attributed fn-ref twins all emit in the exact observed interleave (a dedicated fixture pins the duplicate-id rows; the bloc kind-census spot-check pins the counts). Also preserved (probe-pinned): the extractBareCall selector matrix (the first callTypes=[] language — cascades completely invisible, `?.` encodes like `.`, the `ConfigT.load()` calls+references double emission with no callee-of-call skip, capitalized-chain `Foo.create().run` re-encode, const-object callee names); the constructor hooks (unnamed ctor skipped, named ctors/factories renamed to the CTOR name with the class as returnType, `@override (T) m()` record-misparse rescued by class-name validation); operator methods minting `method "<anonymous>"`; static_final_declaration constants via the visitNode hook while instance fields mint NOTHING; the prefixed-return-type prefix bug (`other.OtherClass f()` → returnType `other`); enum `with` mixins silent vs `implements` working; anonymous extensions named after the ON type; deferred imports invisible; named-argument callbacks NOT fn-ref-captured (the Flutter `onPressed:` idiom — future accuracy PR, TS-side first); `async*`/`sync*` NOT async; value-refs with the LIVE dart sibling-body pull and the `$X`-vs-`${X}` interpolation asymmetry; dartdoc kept in all three comment forms with the annotation-broken chain. Gates: parity sweeps first-run 0-diff on shelf/bloc/flutter — 5,815 clean files byte-parity, deferrals 10/21/1341 ≈ the survey's 10/21/~1340 (both-arm grammar reality: empty object patterns — the sealed-class idiom — and unnamed `library;` dominate; --max-deferral 0.3); full-init dumps byte-identical ×3 (shelf 7,959 / bloc 40,026 / flutter 1,855,319 dump lines); bloc per-kind node census identical across arms (the double-walk duplicate rows survive the store identically); kernel-dart-parity suite (7 fixtures + in-memory CRLF variants + double-walk duplicate-id pin + generated-file skip pin + two defer pins); full suite 2,688 green ×2 with CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += dart (20 langs — R7b COMPLETE). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
bdd687b49f |
feat(kernel): R7b Scala walker — scala module, vendored-grammar-C master@0aca5d0a6f, scala default-routed (#1385)
R7b batch 4 #3 (docs/design/scala-kernel-port-checklist.md is the authoritative quirk list). The third vendored-grammar-C language and the biggest grammar in the tree (35MB parser.c): the vendored wasm is tree-sitter/tree-sitter-scala master@0aca5d0a6f — a post-v0.26.0 generation sync that is not a release (the 0.26.0 crate is 30 states BEHIND, so a crate pin would be a silent downgrade). NO wasm change: production has parsed with this exact revision since #91 — the kernel-grammar-parity row (ABI 15, 26,650 states, 32 fields, id-by-id tables) is the whole alignment proof. Preserved bug-for-bug (all probe-pinned): the leak-through asymmetries — extension methods mint NO nodes (first def's body calls leak to the enclosing scope, later defs invisible, and the braced form resolves its body field to the `{` TOKEN via first-match-wins field lookup → whole extension invisible); anonymous `new T { … }` template_body members leak to the enclosing scope (findAnonymousClassBody misses template_body); the bodied-vs-bodiless class asymmetry (bodiless headers walk class_parameters → default-value calls emit FROM the class; bodied ones never see them) — plus first-segment import names (`import com.example.C` → `com`), the val/var hook keyed on the enclosing-definition NODE TYPE (object vals → constants/value-ref targets, class/trait/enum/given vals → fields) with consumed initializers, every def routed through extractMethod with the top-level function fallback, nested defs in bodies minting NOTHING (the inverse of kotlin) while body-local classes extract fully, curried signatures keeping only the FIRST parameter list (type params win the `parameters` field), enum cases positioned at the CASE node with invisible params/extends tails, extends with-chains via scalaBaseTypeName, `@deprecated(args)` decorates, the #750 capitalized-chain re-encode (`WidgetS.create().render`), literal-receiver silence, static-member reads AND writes, infix invisibility, `derives` silence, scaladoc retention with the CRLF `\r` pin, full value-reference machinery (shadow prune, last-wins same-name targets, `$X`/`${X}` interpolation reads), and SCALA_SPEC fn-refs (bare ids + postfix eta unwrap + varinit, var-init non-capture). Gates: parity sweeps first-run 0-diff on os-lib/cats/scala3-compiler-src/ scala3-library-src — 1,935 clean files byte-parity, deferrals 0/15/57/116 matching the survey's predictions exactly (scala-3's PHANTOM hasError files — flag-true, zero ERROR nodes, capture-checking `^` — defer on the FLAG); full-init dumps byte-identical ×3 (os-lib, cats, scala3 whole-repo 950,889 dump lines); kernel-scala-parity suite (9 fixtures + 9 in-memory CRLF variants incl. Scala-3 indentation through the external scanner + phantom/real-error defer pins + first-segment/namespace/value-ref pins); full suite 2,669 green ×3 with CODEGRAPH_KERNEL_EXPECT=1 (kernel-scaffold's stays-wasm example moved scala → pascal). DEFAULT_ROUTED += scala (19 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e32135171e |
feat(kernel): R7b Lua+Luau walker — one lua module, vendored-grammar-C lua v0.4.1, tree-sitter-luau 1.2.0 pin, both default-routed (#1384)
R7b batch 4 #2 (docs/design/lua-luau-kernel-port-checklist.md is the authoritative quirk list). ONE walker for both dialects (ccpp precedent) — the differences are exactly four: luau's type_definition aliases, the `export `-slice isExported hook, the return-type signature suffix, and the grammar handle. Grammar prep is kernel-side only, no wasm change: lua is the SECOND vendored-grammar-C language (the vendored wasm is the v0.4.1 tag, a revision not on crates.io — tag artifacts compiled via build.rs, shas pinned); luau is a plain crate pin =1.2.0 whose tarball is sha-identical to the tag (the swift tag≠crate divergence does not recur). Grammar-parity rows replace the bump gate entirely. Preserved bug-for-bug (all probe-pinned): the require/visitNode-hook ASYMMETRIES (top-level requires — including inside top-level if/for/while — mint import nodes while the identical body-level statement emits `calls "require"`; top-level `local x = foo()` initializers are invisible while global `x = foo()` calls emit), the BFS string-win inside require args (`require(script:WaitForChild("Kid"))` → import Kid) and Roblox instance paths, receiver-QN methods (`M.sub.deep::chained`, `_G::installed`, stack-QN nested globals like `render::leakedGlobal`), the raw-text callee world (colon forms with `self` never stripped, bracket callees, newline-glued chains byte-verbatim, the `(handler)` paren-conversion), LUA_SPEC function-as-value capture with the `M.cb = cb` param-storage skip and first-occurrence dedupe, LuaDoc `---` keeping a leading `- ` plus `--!strict` joining docstring chains (block-comment docstrings keep interior CRLF bytes), variable nodes at the IDENTIFIER with positional value pairing, duplicate same-(kind,name,line) ids, and the lua↔luau isExported wire divergence (lua functions: flag absent; luau functions: present-false; methods: absent in both; variables: present-false in both; `export type`: true). Gates: parity sweeps first-run 0-diff on kong/lazy.nvim/lua-resty-core (lua) + lune/Fusion (luau) — 1,734 clean files byte-parity, deferrals 1/0/0/3/8 matching the survey's both-arm predictions exactly (kong's 1 = a deliberately invalid fixture; luau's = grammar-inherent generic type packs and default type params); full-init dumps byte-identical kernel-vs-wasm ×4 (kong 157,650 dump lines); kernel-lua-parity suite (both torture fixtures + in-memory CRLF variants + glue-chain, duplicate-id, and cross-dialect defer pins + kernel-arm wire-flag pins); full suite 2,647 green ×2 with CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += lua, luau (18 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
b2f9ab1800 |
feat(kernel): R7b R walker — rlang module, tree-sitter-r 1.2.0 crate pin, r default-routed (#1383)
R7b batch 4 #1 (docs/design/r-kernel-port-checklist.md is the authoritative quirk list; survey + probe record therein). The lightest-shared-surface, heaviest-hook port: languages/r.ts works entirely through the visitNode hook (every type list empty except callTypes:['call']), so the walker is a file node + a faithful hook transcription + the generic extractCall + pre-order recursion — four shared machineries (value-refs, static-member reads, type annotations, fn-ref capture) are dead by language gates and stay dead. Grammar prep is the first true no-op of the arc: the crates.io tree-sitter-r 1.2.0 tarball ships parser.c AND scanner.c sha-identical to the r-lib v1.2.0 tag the vendored wasm was built from — crate pin only, no wasm change, no bump gate; kernel-grammar-parity gains the r row (ABI 14, same-revision). Preserved bug-for-bug (all probe-pinned): calls "return" on every return(x) (named node in v1.2.0), the import quintet's silent dynamic-arg consumption vs class/generic fall-through asymmetry, library(help = pkg) importing the named arg, class-idiom variable suppression by callee name, chained/right- assign/precedence-ghost gaps, env$fn body-leak-to-file, raw-text callees verbatim (pkg::fn, obj$meth, "strfn" quotes kept, (handler) conversion), duplicate same-(kind,name,line) ids, roxygen dropped entirely, UTF-16 columns/slices. Gates: parity sweeps first-run 0-diff on AnomalyDetection/dplyr/ggplot2/ shiny (838 files; deferrals exactly 0/0/0/1 — the 1 is the moustache- template pseudo-R file, both-arm) — kernel-parity.mjs gained lowercased- extension matching so .R files sweep (matches detectLanguage routing); full-init dumps byte-identical kernel-vs-wasm on dplyr/ggplot2/shiny; kernel-r-parity suite (torture fixture + in-memory CRLF + BOM variants + defer pin + kernel-arm quirk pins); full suite 2,638 green ×2 with CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += r (16 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
45a53eb5b5 |
feat(kernel): R7b Kotlin walker — kotlin module, vendored-grammar-C build, kotlin default-routed (#1382)
Sixth R7b port — the T1½ batch finale. Checklist-first recipe
(docs/design/kotlin-kernel-port-checklist.md, 1,121 lines, dist-extractor
ground truth); parity passed FIRST RUN on all three repos.
THE NOVEL MECHANISM — vendored-grammar-C (the §4 tracker's prescription,
first use): the crates.io tree-sitter-kotlin 0.3.8 pins `tree-sitter >= 0.21,
< 0.23` (the kernel links 0.25) and tree-sitter-kotlin-ng is a DIFFERENT
grammar (8 fields vs 0, renamed kinds — extractor-breaking), so no crate dep
is possible. The fwcd 0.3.8 tag's sha-matched parser.c + scanner.c are
vendored into codegraph-kernel/grammars/kotlin and compiled by build.rs (cc),
exposed via tree-sitter-language::LanguageFn. The wasm re-vendor is
behavior-NEUTRAL (0 CST/error disagreements across 1,984 gate-repo files;
old-vs-new full-init dumps byte-identical ×3) — a reproducibility re-vendor,
ABI stays 14.
Walker firsts: extension-function receivers (getReceiverType →
`WidgetK::extend` QN OVERRIDE with no package prefix, the qualified-receiver
`com::qext` first-segment bug, and the owner-contains fallback that excludes
`interface` kinds and is source-order dependent) and extractModifiers
(expect/actual platform modifiers → the node DECORATORS wire field on every
created node — the KMP synthesizer's feed, incl. `actual typealias`).
Preserved bug-for-bug: the FIELD_COUNT-0 dead cluster (no signatures, ZERO
type-annotation refs), hook-consumed property initializers emitting nothing
(incl. `by lazy {}`), the bodiless-vs-bodied class header asymmetry, enum-
entry bodies being invisible, KDoc never a docstring AND chain-breaking,
comment-gluing into import/package extents, `@Anno(args)` emitting nothing
while `@Marker` decorates, zero instantiates refs, the paren-then-lambda
`trailing()` garbage callee, text-includes visibility/suspend false
positives, and the packaged-file value-ref target drop. The fun-interface
misparse-recovery hook is DEFER-SHIELDED (every such file has_error) and
deliberately not ported. The swift-sweep lesson pre-applied: the shared
`assignment` shadow-prune case is implemented alongside the
property_declaration case.
Gates: sweeps 0-diff okio 299/322, okhttp 531/580, kotlinx.coroutines
1031/1082 (deferrals exactly the predicted 23/49/51 — both-arm grammar
reality incl. PHANTOM hasError files with complete CSTs; the kernel trusts
the flag); full-init dumps byte-identical ×3 (46.5k/108.9k/92.3k lines); KMP
expect/actual synthesis IDENTICAL across arms (412 edges on
kotlinx.coroutines — the tracker's KMP validation); kernel-kotlin-parity
suite (torture reflowed off the phantom shapes + .kts script + CRLF variants
+ fun-interface and phantom defer pins) + kotlin grammar-parity row (the
C-build ↔ wasm table identity proof); full suite 2,633 green ×2 under
CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += kotlin (15 langs).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
|
09e301bbfa |
feat(kernel): R7b Swift walker — swift module, tree-sitter-swift 0.7.3 bump, swift default-routed (#1381)
Fifth R7b batch-3 port, checklist-first recipe (docs/design/swift-kernel-port-checklist.md, 1,056 lines — the largest of the arc, with a built-extractor-validated emission pin and a childForFieldName truth table). Grammar bump first, validated standalone: tree-sitter-wasms ^0.4.0 (ABI 13) → crate 0.7.3 — with a provenance twist: the wasm is built from the CRATE TARBALL's src/ (alex-pinkus keeps generated files off main and the 0.7.3-with-generated-files tag ships an older ABI-14 generation that can never sha-match; grammar.json rules are JSON-equal; the tarball is byte-for-byte what the kernel's cargo build compiles — table identity by construction). Older crates evaluated and rejected: clean-parse shapes are byte-identical on 0.7.3 (53-line CST battery diff, all inert), so an older pin buys nothing and loses the macro-era wins. Delta = error-set membership (63 old-error files parse clean: swift-testing #expect, #Preview/#GET macros, package access, typed throws — vapor 23.1%→9.3%; 21 NEW-only regressions in 3 probed construct classes) + two gate-found categories: docstring boundaries near #if directives (7 clean files, docstring-field-only — verified mechanically) and array-literal-callee call refs (2 refs, 1 file). Every hunk classified via the error-union rule + parked-ref↔edge ripple pairing. Walker (the arc's biggest) centers on the #1020 DEDICATED property branch: computed properties → property nodes with the getter walked under the property (SwiftUI body), static let/var → constant/variable, stored → field, decorator/type-annotation/@Siblings-attr-arg refs all attached to the ENCLOSING TYPE, stored initializer calls attributed to the class. Preserved bug-for-bug: the never-resolving 'parameter' field (zero param type refs, zero signatures), present-false isAsync, open→internal visibility, everything-is-extends inheritance (first type_identifier per specifier), no instantiates refs ever, subscript reads as `calls arr`, `defer` as `calls defer`, multi-case enum entries minting only the first case, /** */ block docs ignored AND chain-breaking, init/deinit/subscript minting no nodes with visitNode-routed bodies (calls → class, static reads → nothing), multi- segment extension resolveName, sugar extension names, the #selector shapes, and the value_argument label-forward skip. ONE fix found by the sweep (then pinned in the fixture + checklist): the shared `assignment` shadow-prune case is swift-live — declared-then-assigned `let X: T` prunes X as a value-ref target. Gates: sweeps 0-diff Alamofire 89/98, vapor 224/247, swift-nio 407/554 (--max-deferral 0.3 — swift error incidence is 9–27% on BOTH arms, structural; every deferral count matches the survey's table exactly); full-init dumps byte-identical ×3 (31.9k/20.7k/126.3k lines); the Alamofire census reproduces property=348 (the #1020 number) on the kernel arm; kernel-swift-parity suite (206-line torture + CRLF + the #if-between-enum- cases defer fixture) + swift grammar-parity row; full suite 2,626 green ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += swift (14 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a6c62d77df |
feat(kernel): R7b PHP walker — php module, tree-sitter-php 0.24.2 bump, php default-routed (#1380)
Fourth and final R7b batch-2 port, checklist-first recipe (docs/design/php-kernel-port-checklist.md). Grammar bump first, validated standalone with the diff ENUMERATED + CLASSIFIED (unlike rust/ruby the php bump is NOT graph-neutral): tree-sitter-php ^0.22 (tree-sitter-wasms, 2023) → v0.24.2, the full HTML-interleaving `php` variant (the walker calls LANGUAGE_PHP, never PHP_ONLY) — crate pinned =0.24.2, wasm built from tag 5b5627f's checked-in php/src/parser.c + scanner.c + shared common/scanner.h (all sha-matched against the crates.io tarball, ABI 14→15). Old-vs-new full-init diffs decompose completely into: (1) the anonymous_class wrapper shape (anon-class nodes/methods re-shape — 2,532 rows), (2) grouped nested-clause skip (absent in the gate repos, fixture-pinned), (3) 32 formerly-erroring files parsing clean (monolog Level.php, symfony Request/Response with 8.4 property hooks), (4) a survey-missed category found at gate time: the 8.4 parenthesis-free `new X()->m()` chaining misparse fix (86 garbage instantiates refs disappear, precision-positive), plus resolution RIPPLE proven mechanically (every remaining ref-table flip pairs 1:1 with a resolved edge on the opposite side; node rows byte-stable outside 1/3/4). Walker (java.rs chassis + the php specifics) preserves bug-for-bug: the visitNode hook (const_declaration at ANY scope → bare `constant` nodes, values never walked; trait-use → implements refs WITH filePath via the ruby port's REF_FLAG_FILE_PATH wire slot), FIRST-namespace whole-file scoping (braced namespaces scope nothing; namespaced files DROP top-level const value-ref targets), the import trio (single/aliased/grouped incl. the nested-clause skip, include/require static-literal-only, `Foo\Bar::Baz` use refs), the call-encoding zoo (DOT-joined scoped calls, `this->prop.m` #1251 encoding, `Cls::factory().m` fluent with inner args dropped, nullsafe `?->` emitting nothing, unsuppressed literal receivers), interface multi-extends first-base-only drop, anon-class methods as file-level functions (top) or vanishing (in-body), property type-hints emitting no field refs, the final-modifier-as-type signature quirk, HOF-gated string callables (skipGate) + array callables, and the `name`-node value-ref reader. Gates: sweeps 0-diff monolog 217/217, laravel-framework 3007/3008, symfony 10726/10737 (13,950 files byte-parity; 12 deferrals = exactly the predicted genuinely-broken fixtures, ≈0–0.1%); full-init dumps byte-identical ×3 (16.1k/354.2k/702.8k lines); kernel-php-parity suite (torture + drupal .module + leading-HTML fixtures, CRLF variants, wire-flag pin, defer) + php grammar-parity row; full suite 2,622 green ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += php (13 languages). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
1909931238 |
feat(kernel): R7b Ruby walker — ruby module, tree-sitter-ruby 0.23.1 bump, ref-flag wire slot, ruby default-routed (#1379)
Third R7b port, checklist-first recipe (docs/design/ruby-kernel-port-checklist.md). Grammar bump first, validated standalone (the rust pattern): tree-sitter-ruby ^0.20.1 (tree-sitter-wasms, 2024-02) → v0.23.1 — crate pinned =0.23.1, wasm built from tag 71bd32f's checked-in parser.c/scanner.c (both sha-matched against the crates.io tarball; content bump, ABI stays 14). Old-vs-new full-init dumps: sinatra/jekyll byte-identical; rails = exactly the one classified hunk (the `recv&.!=` safe-nav operator misparse fix, `table_name.!` → `table_name.!=`, precision-positive). Walker (python.rs chassis + the six ruby divergences) preserves bug-for-bug: the importTypes:['call'] funnel (class-body DSL — attr_accessor, has_many, define_method incl. its block, sinatra route blocks — emits NOTHING at non-body scope), hook-handled module multiply-capture (nested modules re-scan their subtree per level after popping — `this.hooked` fn-refs from class AND module AND file), the sibling-scan visibility trio (bare `private` invisible; `private :sym`/`private def` poison all later defs; the inner def stays public), bare-call statements (do…end body_statement emits, brace-block block_body doesn't), `.new` instantiates with last-`::`-segment names, constant-receiver references refs, require/require_relative path refs (posix-normalized, `.rb`-suffixed, `Kernel.require` and interpolated-path quirks included), `=begin` docstring marker survival, and the reverse-order value-ref DFS. Wire v2: the hook's mixin `implements` refs carry `filePath: ctx.filePath` — the ONE extraction-ref denormalized field (php's trait-use refs share the shape). RefRow's first pad byte becomes a flags slot (REF_FLAG_FILE_PATH); decode re-attaches its own filePath parameter; KERNEL_ABI_VERSION 1→2 on both sides (mismatched dist/.node pairs degrade to wasm, as designed). Gates: sweeps 0-diff sinatra 147/147, jekyll 164/164, rails 3452/3452 (3,763 files, 0 deferrals — ruby error incidence 0.00%, any deferral = walker bug); full-init dumps byte-identical ×3 (7.2k/9.4k/375.6k lines); kernel-ruby-parity suite (torture + CRLF + wire-flag pin + defer) + ruby grammar-parity row; full suite 2,613 green ×2 under CODEGRAPH_KERNEL_EXPECT=1 (one unrelated mcp-initialize timing flake under parallel load, passes solo 3/3 ×3). DEFAULT_ROUTED += ruby (12 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
286e9ccc2d |
feat(kernel): R7b C# walker — csharp module, tree-sitter-c-sharp 0.23.5 pin, csharp default-routed (#1378)
Second R7b port, checklist-first recipe (docs/design/csharp-kernel-port-checklist.md; parity passed FIRST RUN again). No grammar bump — the #717 vendored wasm verified table-identical to crate 0.23.5 (ABI 15, STATE_COUNT 8053, node-kind + field tables); first port with no grammar-prep step. The #237 #if-blanking preParse stays TS-side via the existing route-point hoist. Walker preserves bug-for-bug: the single-namespace-node quirks (second namespace nests under the first, nested namespaces leave no trace, import refs hang off the namespace node), raw member-access callee texts (this./base./literal receivers, multi-line fluent chains) with unconditional chain re-encode, deliberate emission holes (property accessor/expression bodies, ctor initializers, delegates/events/ operators/indexers/local functions, top-level locals), garbage extends refs ((repo) primary-ctor args, BaseDto(Name) record bases, enum : byte), the alias- import moduleName quirks, nameof-as-call, CSHARP fn-ref spec (+= subscription, this.X bare-name form, argument layer, initializer lists), C# type-ref engine (nested-generic returnType failure included), and value-ref shadow pruning. Gates: sweeps 0-diff serilog 211/216 / Newtonsoft.Json 914/945 / jellyfin 2104/2105 (deferrals match the survey's per-repo predictions — both-arm #if damage; default --max-deferral 0.1 holds, no c/cpp exemption); full-init dumps byte-identical ×3 (14.0k/109.1k/210.8k lines); kernel-csharp-parity suite (torture ×3 + CRLF variants + 8 micro-pins + defer) + csharp grammar-parity row; full suite 2,608 ×2 under CODEGRAPH_KERNEL_EXPECT=1. DEFAULT_ROUTED += csharp (11 langs). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
f1ca991943 |
feat(kernel): R7b Rust walker — rustlang module, tree-sitter-rust 0.24.2 bump, rust default-routed (#1371)
First R7b language port. Grammar: tree-sitter-rust pinned =0.24.2 + wasm vendored from tag 77a3747 (parser.c/scanner.c sha-matched against the crates.io tarball), replacing the 2023 ABI-14 tree-sitter-wasms build — the bump alone is precision-positive on the wasm path (receiver-qualified instance-method resolutions replace ambiguous bare-name matches; node sections byte-identical on ripgrep/tokio). Walker mirrors the TS reference bug-for-bug per docs/design/rust-lang-kernel-port-checklist.md (survey artifact): dead-code isAsync, impl-pushes-no-scope, the impl-Trait-for-Generic<T> trait-receiver quirk, phantom const identifiers, use-binding triple emission, wildcard-use-emits-nothing, scoped-supertrait drop, chained-call re-encode gated on scoped_identifier, Rocket route macros body-only. Gates: parity sweeps 0 diffs — ripgrep 101/101, tokio 790/790, rust-analyzer 1217/1488 (271 deferrals are token-macro-table sources that error on BOTH arms — grammar-inherent); full-init dump-diffs byte-identical on all three (3,857 / 13,440 / 39,030 nodes); kernel-rustlang-parity suite (torture + CRLF + defer) in npm test; full suite green x2 with CODEGRAPH_KERNEL_EXPECT=1. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2d72891b59 |
feat(kernel): R7a C/C++ walker — dual-lang ccpp module, preParse hoist, 7 new blanks, c/cpp default-routed (#1346)
Parity: 0 diffs on redis/git/fmt/protobuf/ALS sweeps; full-init dumps byte-identical on all five + linux at kernel scale (10.4M dump lines, same sha256 both arms). Linux 2c/6GB envelope: kernel-arm 19.1min vs wasm-arm 22.9min (parse 356s vs 435s) on a much richer graph (the new blanks recover error-swallowed code: git 2x nodes, linux kernel/+mm/ 3x). Deferral guard corrected by measurement (C/C++ error incidence 9-42%; --max-deferral flag); defer-reuse memo kills the 3x re-blank/re-parse cost deferred files paid. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
2a79432b13 |
docs(kernel): R6 — kernel-scale re-validation record (§4f) + parity-harness symlink robustness
cg1212 (Linux kernel, 2 CPU/6GB): completes in 26.4min vs the ~27min baseline with all new machinery active — no regression; graph scale identical (2,048,664 nodes / 6,405,964 edges). The §6 parse expectation (6m → ~2m) was mis-premised: the tree is ~99% C, an unported T2 language, so it transfers to the C/C++ port (R7). Resolution remains the kernel-scale wall (19.2m, 73% — P1). The tree's own Python tooling files: 99/99 byte-parity. kernel-parity.mjs now skips dangling symlinks (Linux dtc fixtures). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c2503e2bee |
feat(kernel): R5 — Python and Go ports, gates passed, default-on
Python (codegraph-kernel/src/python.rs) and Go (src/go.rs) join the native kernel, mirroring the wasm extractors bug-for-bug. Python: decorated_definition docstrings/decorators (decorates refs only for bare-identifier decorators — the call-kind quirk), function-in-class → method, module-level assignments always extract as variable, from-import per-name binding refs, self.x fn-ref candidates as bare names. Go: receiver methods with Recv::name qualified names + contains edges to the first earlier struct of that name, type_spec struct/interface classification with embedding→extends and interface method nodes, composite-literal instantiates keeping the package qualifier, top-level var/const initializer walks attributed to the declared symbol (#693), 2-hop field chains (#1276), New().Method() re-encode (#645/#608), and the GO_SPEC fn-ref layers. Grammars: tree-sitter-python 0.23.6 + tree-sitter-go 0.23.4 crates, with wasm vendored from the same tags (parser.c sha-matched) — both were 2023-era in tree-sitter-wasms. Gates: extraction sweeps 100% (flask 83/83, django 3,035/3,038 +3 error-file deferrals, gin 99/99, prometheus 978/979 +1); full-init dump-diffs byte-identical on flask (10,833 rows), gin (17,540), django (360,794), and prometheus (213,758); torture fixtures enforced in npm test. DEFAULT_ROUTED now covers typescript/tsx/javascript/jsx/java/ python/go. Full suite: 2,471 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
03d54e47a1 |
feat(kernel): R4 — Java port with Lombok synthesis, gate passed, default-on
Java joins the native kernel (codegraph-kernel/src/java.rs), mirroring the wasm extractor's Java paths bug-for-bug: package namespaces, imports, javadoc, annotations→decorates, type_list inheritance, static-final constants, enum constants, anonymous classes (including the TS side's 0-based-line quirk on the extends ref), method_invocation calls with the this.field unwrap and the Foo.getInstance().bar() chain encoding (#645/#608), static-member value reads, method-reference fn-refs (#756), value-reference edges, and the full Lombok member synthesizer (#912: Getter/Setter/Data/Value/Builder/ToString/ EqualsAndHashCode/Slf4j-family with taken-member dedup). The shared docstring/textutil modules moved to crate level. Grammar: tree-sitter-java 0.23.5, with the wasm grammar vendored from the same tag (parser.c sha-matched) replacing tree-sitter-wasms' 2023-era build. Gate (plan §4c): extraction sweeps 100% — gson 262/262, retrofit 341/341, dubbo 4,048/4,048 — plus a Java torture fixture in npm test; full-init dump-diffs byte-identical on gson (49,766 rows), retrofit (62,735), and dubbo (441,266 rows); all R2/R3 repos re-verified; Linux container runs all 23 kernel tests green under CODEGRAPH_KERNEL_EXPECT=1. The gate caught a real cross-language bug: fn-ref dedupe and value-ref self-target checks must compare node ID STRINGS, not node-table rows — ids collide for same-(kind, name, line) nodes, which minified one-line bundles hit routinely (retrofit's website JS exposed it; latent in the TS/JS walker since R2, never released). Fixed in both walkers. Benchmark honesty: dubbo fresh-init on an 11-core Mac is ~flat (parse-loop wall 5,020→4,394ms; total ~11.3s both arms) because that wall is main-thread-bound (reads + store), not worker-CPU-bound — the §6 expectation assumed otherwise. Where worker CPU binds the kernel delivers: dubbo on a 2-CPU/6GB container drops 27.8-28.6s → 22.3-22.8s (~1.25×). The identified lever for the many-core headline is decoding kernel buffers directly into store rows (skipping per-node JS object materialization); the buffer contract already carries everything. DEFAULT_ROUTED now includes java. Full suite: 2,467 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c8cca9a601 |
feat(kernel): R3 — TS/JS equivalence gate passed, kernel default-on
Gate evidence (docs/design/rust-kernel-migration-plan.md §4b):
- Graph parity, byte-identical (stronger than the §5 ≤0.5% bar): full
codegraph-init dump-diffs kernel-vs-wasm on express (13,712 rows),
excalidraw (89,898), and vscode (2,378,238 rows) — identical bytes.
Python control repo (flask) identical + timing unchanged. The parity
harness is now ORDER-sensitive (emission order drives rowids, which
drive resolution order) and dumps come from the new
scripts/dump-graph.mjs (natural keys, no rowids/timestamps).
- The one real find, caught by the vscode tier: tree-sitter error
RECOVERY is encoding-dependent — byte-identical grammar sources and
the same core (0.25.10) recover erroring files differently under
UTF-8 (native) vs UTF-16 (web-tree-sitter) parsing; proven by
reproducing the wasm tree with a native UTF-16 parse. Policy: the
kernel defers any file whose tree has_error() to the wasm extractor
(silent 'defer:' signal, per file) — parity by construction on
erroring files (incidence 0-0.42% across the gate repos), and the
harness fails if deferrals exceed 10% so a broken kernel can't hide
behind the fallback.
- Retrieval invariants: canonical excalidraw flow (mutateElement →
renderStaticScene) connects end-to-end on the kernel-indexed graph;
synthesized-edge families present. Agent A/B is vacuous under
byte-identical DBs (same justification as #1320-#1322).
- Perf: vscode init 105.4s → 82.1s (1.28×) on an 11-core Mac;
excalidraw on a 2-CPU/6GB Linux container (the CI-runner envelope)
6.2-7.1s → 4.3-4.8s (~1.5×). Linux arm64 in-container build: all 22
kernel tests green under CODEGRAPH_KERNEL_EXPECT=1. Windows VM leg
deferred (VM stopped; prlctl start needs Parallels Pro) — benign: a
missing .node falls back to wasm, and the release matrix builds and
gates the win32 prebuilds.
- Full suite: 2,465 tests pass WITH default-on routing, so the entire
extraction corpus now exercises the kernel for TS/JS wherever a
.node is staged.
DEFAULT_ROUTED = {typescript, tsx, javascript, jsx}. Override:
CODEGRAPH_KERNEL_LANGS (replaces the set) / CODEGRAPH_KERNEL=0 (kill).
Changelog entry added under [Unreleased].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Fable 5
|
9ad5cd7ba2 |
feat(kernel): R2 — full TypeScript/JavaScript extraction port, byte-parity with the wasm path
Replaces the R1 seed .scm emitter with a bespoke Rust walker (codegraph-kernel/src/tsjs/) that mirrors TreeSitterExtractor's TS/JS paths function-for-function: declarations (incl. #808 field/property classification), qualified names, docstrings (#780 wrapper climbs), signatures, imports/re-exports + per-binding refs, calls with receiver-qualified callees (#1230 literal-receiver skip), instantiations, decorators, inheritance, type annotations (#381), type-alias members + tuple contracts (#359/#634), React component recognition (#841 forwardRef/memo/styled), object-of-functions / zustand-through-middleware / RTK Query endpoints + generated hooks / vuex + pinia store shapes, function-as-value capture with the flush gate (#756), and value-reference edges with the shadow prune (#895/#897). The generic query emitter is deleted — extraction parity needs logic .scm can't express; future languages get walkers too (migration plan §4a). Positions and JS string-slice semantics are emitted in UTF-16 code units natively, so kernel output is byte-identical to web-tree-sitter's — no column diff class exists. Parity evidence (macOS): scripts/kernel-parity.mjs (full-object multiset diff per file) — this repo 353/353 files, excalidraw 643/643 (10,650 nodes / 10,726 edges / 68,307 refs), plus torture fixtures checked into __tests__/fixtures/kernel-parity/ and enforced in npm test by kernel-tsjs-parity.test.ts. The strict compare caught one real decoder bug the loose harness missed: refs must NOT carry denormalized filePath/language at the extractFromSource seam (the store fills them). Perf: extraction 2.6× single-thread on excalidraw (487ms vs 1,255ms, identical outputs). Routing stays opt-in (CODEGRAPH_KERNEL_LANGS) until the R3 equivalence gate (large repo, DB dump-diff, retrieval invariants, agent A/B, Linux/Windows) passes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Fable 5
|
c5eebe6beb |
feat(kernel): R1 scaffold — napi-rs extraction kernel, buffer contract, routing + fallback, grammar-parity CI
Phase 0 of the Rust extraction-kernel migration (docs/design/ rust-kernel-migration-plan.md, now checked in with §3a recording the shipped state): - codegraph-kernel/ napi-rs crate: extractFile(path, content, language) → five flat buffers (meta/nodes/edges/refs/arena), one JS boundary crossing per file. Node ids computed Rust-side, byte-identical to generateNodeId (pinned by test vector). Reserved per-node metrics slot for the Arc 3.2 code-metrics work. - Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures, byte-range scope stack → ::-joined qualified names, contains edges, refs attributed to the innermost enclosing symbol). Seed TS/JS queries are smoke-level; R2 replaces them with the full port. - Routing seam in extractFromSource with per-file wasm fallback. DEFAULT_ROUTED is empty — no behavior change until a language passes its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before routing; EDGE_KINDS became a runtime array because kind order is now wire contract. - Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the exact crate revisions (tree-sitter-typescript v0.23.2, tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) — the tree-sitter-wasms builds were 2023-era, which the new kernel-grammar-parity test caught on day one. Production TS/JS parsing gets 2.5 years of grammar fixes; full suite green (2456 tests). - Build/release wiring: scripts/build-kernel.sh + npm run build:kernel; release.yml kernel prebuild matrix (continue-on-error — the kernel is optional everywhere, bundles fall back to the wasm path); bundles stage lib/kernel/codegraph-kernel.node; release job runs the kernel suites with CODEGRAPH_KERNEL_EXPECT=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ad5300a601 |
fix(ui): show synthesis as a 'Linking dynamic dispatch' phase; mute node:sqlite warning spam (#1299)
Two first-run UX bugs surfaced by indexing a real 1,342-file C repo: 1. After 'Resolving refs' hit 100%, the ~40 dynamic-dispatch synthesis passes ran with no progress surface, so the bar sat frozen at 100% long enough to read as a hang (the C fn-pointer pass alone can hold for a while on C-heavy repos). Synthesis now reports per-pass progress through a new 'linking' IndexProgress phase, rendered as 'Linking dynamic dispatch'. The step total is pinned by a test to the synthesizer's actual __mark() count so adding a pass without bumping it fails loudly. 2. node:sqlite's ExperimentalWarning is emitted once per THREAD, so the main process plus every parse worker printed it mid-index, interleaved with the progress UI. All launch paths now pass --disable-warning=ExperimentalWarning: both bundle launchers, the Windows npm-shim invocation, and the CLI self-relaunch (NODE_RUNTIME_FLAGS, deliberately excluded from the re-exec gate so an older installed launcher never triggers a pointless re-exec, and version-gated off nodes older than the flag). Verified end-to-end on the same repo: zero warnings, live linking bar, byte-identical graph (50,520 nodes / 148,232 edges). Full suite green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
2b0b4b587e |
ci(release): publish npm packages with provenance and attest release bundles (#1296)
Every published artifact is now cryptographically verifiable as built by this repo's Release workflow: npm publishes carry npm provenance (OIDC, shows the Provenance badge on npmjs.com), and the GitHub Release bundles + SHA256SUMS get signed build attestations via actions/attest-build-provenance, verifiable with `gh attestation verify <file> -R colbymchenry/codegraph`. pack-npm.sh now writes a repository field into the generated shim and per-platform package.jsons — npm --provenance refuses to publish without one matching the repo — and the root package.json gains the same field. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
c9f8c0ebaf |
fix(mcp): reap the server when its launcher is killed during startup (#1185) (#1199)
An MCP host that kills the launcher chain within the server's first ~100ms while keeping the stdio pipes open (config probe, cancelled request, startup timeout; Rust hosts that kill a child without dropping its stdio handles) left the server orphaned: it booted already reparented to init, so the PPID watchdog's "ppid changed" baseline was captured as 1 and could never fire, and stdin never EOF'd. The process lingered — idle, ~30MB — until the host itself exited, accumulating one per abandoned launch (the pile-up reported in #1185). Reproduced on released 1.2.0/macOS: SIGKILL the launcher at +50ms → permanent orphan; at +150ms the old late baseline had already run and reaped it. Three-part fix: - Capture process.ppid at the earliest line of the CLI entry (early-ppid.ts) and use it as every watchdog baseline, shrinking the blind window to the few ms before our first JS runs. - Thread the real host pid down the bundled path: the npm shim and the standalone sh launcher set CODEGRAPH_HOST_PPID (an outer launcher's value wins), so the watchdog polls the host directly. Previously only the --liftoff-only relaunch set it, leaving the entire npm/standalone install base with hostPpid=null. - Never-initialized backstop (startup-handshake.ts): a serve --mcp that receives no MCP traffic for CODEGRAPH_STARTUP_HANDSHAKE_TIMEOUT_MS (default 15min, 0 disables) shuts down — the catch-all for a kill landing in the residual pre-JS window. Disarmed on the first byte, so a quiet-but-live session is never touched. Also scrub CODEGRAPH_HOST_PPID from the detached daemon's env — it has no host, and a stale pid must not leak into anything it spawns. Validated end-to-end on the built bundle: the +50ms early-kill orphan is now reaped while the host still holds the pipes open, and all six normal lifecycle paths (clean close, SIGTERM/SIGKILL child, host exit/SIGKILL, fd-holding adversarial host) stay clean. New coverage in startup-handshake.test.ts, mcp-startup-orphan.test.ts, and npm-shim.test.ts. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
41620c60fa |
feat(extraction): add COBOL language support (.cbl/.cob/.cpy) (#590, #648) (#1161)
Programs, sections/paragraphs (reconstructed extents over the grammar's flat header stream), PERFORM/THRU/GO TO/CALL call edges, COPY copybook imports incl. standalone .cpy fragments, DATA DIVISION records/fields/ 88-levels with write-site impact references, and CICS flows: EXEC LINK/XCTL program targets (literal + same-file VALUE deref), EXEC SQL INCLUDE, and pseudo-conversational RETURN/START TRANSID hops resolved to the owning program via a CICS framework resolver. Fixed and free source format (free format via a scanner wide-mode sentinel). Grammar: vendored wasm built from a patched yutaro-sakamoto/ tree-sitter-cobol (EXEC blocks as an external-scanner token, copybook fragment entry point, single-quote continuation, COPY REPLACING pseudo-text, NOT=, CALL GIVING, ENTRY, FREE, bitwise ops, abbreviated relations, COBOL-2002 usages, and more). Patch + provenance + upstream PR draft in docs/grammars/. Parse health: AWS CardDemo 43/44 native (upstream: 9/31), 44/44 through preParse; copybooks 28/29; CobolCraft free-format 17/17 (upstream: 0); NIST COBOL85 unchanged at 373/382. Copybook members resolve to files like C includes (basename index, name-matcher short-circuit so compiler-supplied members stay honestly unresolved): CardDemo imports 5 -> 285. Impact proof: ACCT-CURR-BAL (CVACT01Y copybook) surfaces its 4 writer programs cross-file. Also: run-all.sh now neutralizes the ambient prompt-hook in both A/B arms (CODEGRAPH_NO_PROMPT_HOOK=1); COBOL corpus entries for agent-eval. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
6dd5512d8d |
fix(windows): set windowsHide on remaining child spawns to stop console flash (#1092) (#1104)
On Windows, a black console (conhost) window flashed briefly when CodeGraph ran as a background MCP server. Several child spawns were missing `windowsHide: true`, so Windows created a visible console for the child: - scripts/npm-shim.js — launching the bundled runtime (every server start / daemon-idle reconnect) and the self-heal `tar` extraction of a missing platform bundle. - src/reasoning/login.ts — the detached `cmd /c start` browser open. - src/upgrade/index.ts — package-manager spawn (console-attached, so no flash in practice, but set for uniformity: every child spawn now hides). The daemon spawn (#411) and all git execFileSync sites already set it; this closes the remaining gaps. Adds an all-platforms source guard to __tests__/npm-shim.test.ts asserting every spawn in the shim sets windowsHide. Closes #1092 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
31a58070c8 |
fix(install): prune old version bundles instead of piling them up (#1074) (#1075)
install.sh kept each release in its own versions/<v> dir (~50 MB with the vendored Node runtime) and only moved the `current` symlink, so old versions accumulated forever across upgrades. Keep only the just-installed version and delete the rest; `codegraph upgrade` re-runs install.sh, so this covers upgrades too. The npm-shim self-heal cache (~/.codegraph/bundles/) prunes the same way. Windows installs overwrite a single dir in place and were never affected. Validated real-world on macOS, Linux (Docker/dash), and Windows (VM): a v1.1.2 -> v1.1.4 install leaves only the latest behind. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
dfe13b03c8 |
feat(mcp): off-load read-tool dispatch to a worker pool to fix concurrent-call timeouts (#1002)
The shared daemon served every session on one event loop with synchronous node:sqlite. codegraph_explore is CPU-bound work stitched together by microtask awaits, so N concurrent explores keep the microtask queue continuously full and starve the macrotask phases — timers AND socket I/O. The transport freezes: no response can flush until the whole batch drains, so with ~10 subagents on a large repo clients routinely time out (reported via X by @symbolic2020). Move the heavy read-tool dispatch onto a worker-thread pool. Each worker holds its own WAL read connection (verified: a worker reader sees the main writer's committed catch-up/watcher writes); the single watcher/writer, the catch-up gate, codegraph_status, and the staleness/worktree notices stay on the main thread. Concurrent reads now run in true parallel up to core count and the main loop stays free for the MCP transport, so responses flush incrementally instead of all-at-once after the batch drains. Enabled for the shared daemon only; direct (single-stdio-client) mode is unchanged. - crash recovery: respawn + retry-once, with a circuit breaker that falls back to in-process dispatch if workers can't run on this platform - graceful backstop: an overloaded pool returns success-shaped "busy, retry" guidance, never isError (so it can't teach the agent to abandon codegraph) - pending-aware growth + capped concurrent cold-starts avoid a startup thundering herd (N simultaneous module-loads + DB opens could stall the loop) - config: CODEGRAPH_QUERY_POOL_SIZE (default clamp(cores-1, 1, 16); 0 disables → in-process), CODEGRAPH_QUERY_BUSY_TIMEOUT_MS (default 45s) 10 concurrent explores on vscode (10.5k files): 31s → ~9s, staggered flush, 0 timeouts, byte-identical output; scales with cores (≈3.3× on 8, 1.8× on 2). Full suite passes plus 10 new query-pool tests (fake-worker injection so the scheduling logic is covered without spawning threads). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
3e1547bbe1 |
fix(mcp): bold labels instead of ATX headings in tool results (#778) (#951)
MCP tool results used Markdown ATX headings (##/###/####) for section headers — the status summary, each search hit, every file section in an exploration — which Markdown-rendering clients (e.g. the Claude Code VSCode extension) blow up to H1–H4 font size, filling the transcript with oversized lines (worst on search/explore, where the noise scales with result count). Swap them all for bold labels, which render at body size while keeping the same structure. CLI/TTY output (ContextBuilder) is unchanged — the issue notes it's fine. The format is parse-coupled, so kept in sync: - The explore truncation boundary and the offload chunker (reasoning/reasoner.ts) both key off the per-file header, now a unique `**`-prefixed marker emitted via a shared fileSectionHeader() helper. - Updated the offload strip regexes and switched the opt-in report-style prompt off ATX headings (same client, same rendering issue). - Updated test helpers (sectionFor, sourcedFiles, the callers section-boundary scan) that scanned the old markers. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Colby McHenry
|
4f8782cbe5 |
test(agent-eval): add output-style A/B harness, cost/token analyzer, and DISALLOW/REP_START controls
Three additions to tighten the eval loop: - offload-eval-styles.sh: new 4-arm eval (raw/refs/map/src) isolating the Worker's output shape's effect on main-session tokens, latency, and accuracy. Delegation blocked by default (DISALLOW=Agent) so variance from Haiku subagent spawning doesn't contaminate the measurement. - offload-eval-cost.mjs: cost/token analyzer that reads Claude Code's own per-model accounting (modelUsage.costUSD) rather than re-deriving from raw token counts, giving a correct main(Sonnet)/sub(Haiku) split with proper per-tier pricing. - offload-eval-3arm.sh: adds DISALLOW env to block sub-agent delegation across all arms, and REP_START to append reps to an existing run without clobbering earlier jsonls (e.g. REP_START=4 REPS=3 → reps 4,5,6). Also adds CODEGRAPH_OFFLOAD_STYLE forwarding to the managed gateway so the styles eval can drive output shape end-to-end; the field is stripped before the upstream model call and never sent to BYO endpoints. |
||
|
Colby McHenry
|
f82a662ddb | feat(mcp): pare default tool surface to codegraph_explore alone + redux-thunk synthesizer | ||
|
Colby McHenryandClaude Opus 4.8
|
7ddd3fa7eb |
test(agent-eval): persist offload accuracy/adoption eval harness + front-load hook
Reproducible suite measuring the managed CodeGraph AI offload and the front-load UserPromptSubmit hook (approach 1) vs raw codegraph and no-codegraph, across repo sizes, on time / main-session tokens+cost / CodeGraph-AI tokens+cost / accuracy. All agent arms run claude -p sonnet --effort high; eval-only, nothing shipped. - offload-eval-setup.sh: clone + index 4 memory-probe-verified "not-trained-on" repos (mtkruto/postybirb/shapeshift/trezor — small→large) so the no-codegraph baseline is honest. - offload-eval-3arm.sh / -frontload.sh: one repo, the arms (offload/raw/nocg, frontload). - offload-eval-matrix.sh / -frontload-matrix.sh: drive all 4 tiers. - offload-eval-hook.mjs: the front-load hook (self-locates its engine; CG_FRONTLOAD_DEBUG to log). - offload-eval-metrics.mjs / -judge.mjs (Sonnet) / -summarize.mjs: extract, score, aggregate. - offload-eval-ground-truth.json: source-verified canonical flows (the judge's reference). - offload-eval.md: usage + the 2026-06 findings (raw = the win; offload least-accurate; front-load solves adoption but exposes explore's dynamic-dispatch gaps). Scripts are path-portable (self-locating $HERE/$ENGINE; AGENT_EVAL_OUT scratch dir). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |