Merge remote-tracking branch 'origin/main'

# Conflicts:
#	CHANGELOG.md
This commit is contained in:
Colby McHenry
2026-09-09 00:51:38 -05:00
123 changed files with 11510 additions and 1253 deletions
+36
View File
@@ -0,0 +1,36 @@
# AGENTS.md (docs/)
Nested Codex guidance under `docs/`. Loaded with the repo-root `AGENTS.md` when cwd is under `docs/` (Codex walks root to cwd; shared `project_doc_max_bytes` budget).
Root `AGENTS.md` already carries the non-negotiable retrieval principles (adapt-the-tool, explore budgets, end-to-end synthesis). This file holds the longer validation methodology and Excalidraw worked example that were moved out of root to fit the budget.
### Validation methodology (REQUIRED for every new language/framework)
For each **language × framework**, validate on **small, medium, and large** real repos with **≥3 different flow prompts** each:
1. **Pick the canonical flow** for the framework ("how does X reach Y": state→render, request→handler→view, query→SQL, action→reducer→store…).
2. **Deterministic probes** (`scripts/agent-eval/probe-{node,explore}.mjs` against the built `dist/`): `codegraph_explore` with the flow's symbol names connects from→to end-to-end with no break (its Flow section shows the path); **no node explosion** (`select count(*) from nodes` stable before/after re-index); synthesized-edge **precision** spot-check (`select … where provenance='heuristic'`).
3. **Agent A/B** (`scripts/agent-eval/run-all.sh <repo> "<Q>"`): with vs without codegraph, **≥2 runs/arm** (run-to-run variance is large — never conclude from n=1). Record **duration, total tool calls, Read, Grep**. Optional forced-Read-0 sufficiency proof via the block-read hook (`scripts/agent-eval/hook-settings.json`).
- **Every run also reports three feedback metrics** — residual context occupancy, explore sufficiency (what the agent did NEXT after each explore), and allocation efficiency (share of returned bytes the answer cited) — under each run, plus a side-by-side arm table (`compare-arms.mjs`). Entry point: `docs/benchmarks/agent-eval-feedback-metrics.md`. Reading them: `Read a file we returned` is an allocation miss, `Read a file we did NOT return`/`Grep` is recall; allocation efficiency is **relative** (attribution is by citation) so it is only valid between builds on the same question; occupancy *shares* are Claude Code / 200k and don't transfer to another host — the arm ratio does.
- **The `codegraph` CLI is blocked in every arm** (`no-cli-shim.sh`: sanitized PATH + a PreToolUse hook, shared by both harnesses). Without it 14 of 15 without-arm runs in one 7-repo pass reached codegraph through Bash. Check the contamination row before believing any number: `CLI calls that RETURNED output` > 0 invalidates the run (in a new-vs-baseline A/B it silently drops calls from all three metrics, since a CLI explore is not a tool call).
- **Model policy — every A/B arm runs Claude with `--model sonnet --effort high`. Always. Never Opus/Fable.** All `scripts/agent-eval/*.sh` default to this (`MODEL`/`EFFORT` env override exists — don't raise it without an explicit reason from the maintainer). Two reasons, and the second matters more than cost: (a) Sonnet doesn't burn tokens; (b) **Sonnet is the deliberate floor model** — codegraph's real users attach it to whatever agent they already run (Cursor Composer, Gemini, etc.), so we validate on a "dumber" model on purpose: a stronger model's tool-use covers up the salience/sufficiency problems a weaker one exposes. An affordance that lands on Sonnet generalizes up to every host; one that only works on Opus/Fable doesn't generalize down to the agents most users actually have. Both arms always use the same model.
- **MCP attach is a startup-latency issue, not a hard block.** On a multi-step task the agent dives into Read/grep before codegraph finishes its ~2-3s startup (worse when the eval is itself run nested inside a Claude session, under CPU contention), so it runs with no codegraph. Fix: **pre-warm a persistent daemon** for the target (`CODEGRAPH_DAEMON_IDLE_TIMEOUT_MS` high; spawn `serve --mcp --path <target> </dev/null &`; wait for `.codegraph/daemon.sock`) **and skip the startup re-exec** (`CODEGRAPH_WASM_RELAUNCHED=1`) so claude connects before the agent's first turn. Don't trust claude's `init` snapshot — it can read `status:"pending"` / 0 tools even when it then connects; judge by actual codegraph usage in `parse-run.mjs`'s `by type`. To isolate a change — **new-build vs baseline-build, both codegraph-on** (vs run-all.sh's with-vs-without) — use `scripts/agent-eval/ab-new-vs-baseline.sh <indexed-repo> "<task>" [baseline-ref]` (it bakes in the pre-warm).
4. **Pass bar:** a normal flow question reaches **~0 Read/Grep within the repo's explore-call budget**, runs **faster** than without-codegraph, and shows **no regression on a control repo**. Record the numbers in `docs/design/dynamic-dispatch-coverage-playbook.md` (the coverage matrix).
Full playbook + per-mechanism design: `docs/design/dynamic-dispatch-coverage-playbook.md` and `docs/design/callback-edge-synthesis.md`.
### Worked example — Excalidraw (TS/React, medium, 643 files)
The template to replicate per language/framework. Question: *"how does updating an element re-render the canvas on screen?"* (the full flow crosses three React boundaries: observer callback, `setState``render`, and JSX child).
| Stage | duration | Read | Grep | codegraph |
|---|---|---|---|---|
| Without codegraph | 115139s | 910 | 1011 | 0 |
| Broken (explore-budget regression) | 131139s | 510 | 35 | 614 |
| Fixed (budget + msgs + synthesis) | 64112s | 02 | 24 | 3**10** |
| + trace-first steering | **5174s** | **02** | 04 | **34** |
n=4 unhooked runs/stage, same prompt. After steering flow questions to `codegraph_trace` first: **best run 0 Read / 0 Grep / 3 codegraph / 51s**; **2 of 4 fully clean** (0 Read, 0 Grep). Steering eliminated the over-drill variance — call count tightened from 310 to 34, trace adoption went 3/4 → 4/4, and the `search`+`callers` path-reconstruction floundering dropped to 0. Run-to-run variance is still real; report the range, never a single run. **Residual reads/greps are all the nonce data-flow** (`canvasNonce` — a local prop with no graph edges); that's the def-use/data-flow frontier, left deliberately uncovered (tracking every local would explode the graph). Validated: `trace(mutateElement, renderStaticScene)` connects in **6 hops** across all three boundaries (`mutateElement → triggerUpdate → [callback] triggerRender → [react-render] render → [jsx] StaticCanvas → renderStaticScene`), each hop showing inline source + the wiring site; node count stable at 9,289; 1 callback + 46 react-render + 280 jsx-render synthesized edges (no explosion, precision-checked).
Also see: `docs/design/dynamic-dispatch-coverage-playbook.md`, `docs/design/callback-edge-synthesis.md`, `docs/benchmarks/call-sequence-analysis.md`, `docs/benchmarks/agent-eval-feedback-metrics.md`.
+23 -13
View File
@@ -278,10 +278,18 @@ Hooks PRESENT (port each exactly):
but createNode's extractModifiers merge still runs, so `expect val` /
`actual val` DO get decorators. Return true → the dispatcher runs
`scanFnRefSubtree(node, 0)` (capture-only, halts at nested
function/lambda types) and NEVER descends → **property initializers
emit NO calls/instantiates refs anywhere** (`val SHARED = WidgetK(0)`
→ nothing; `by lazy { compute() }` → nothing, the scan halts at the
lambda_literal). Consequences pinned in `extract-torture.txt`.
function/lambda types) and never descends on its own. **The hook itself
then walks the property's RHS under the property's scope** — the named
child after the `=` token plus a `property_delegate` — via
`ctx.visitFunctionBody`, so `val SHARED = WidgetK(0)`, `val cb =
Runnable { hit() }` and `by lazy { compute() }` all emit their calls
FROM the property node (Go's #693 initializer walk, ported). The
declaration's own children — modifiers, `val`/`var`, the name+type, an
extension receiver's type and type parameters, `getter`/`setter` — are
NOT walked: a same-line `val c get() = f()` still emits nothing, a
next-line accessor still attributes to the class, and a
hook-DECLINED destructuring RHS is still invisible.
Consequences pinned in `extract-torture.txt`.
2. **`lambda_literal` after a fun-interface ERROR (:139-143)** and
3. **fun-interface misparse recovery (:145-214)** (ERROR/
function_declaration shapes; `isFunInterfaceNode` :46; Pattern 1 walks
@@ -391,7 +399,7 @@ Hooks ABSENT (the walker must NOT do these): `preParse`, `resolveName`,
| `anonymous_initializer` (`init { }`) | no branch | recursed → its statements' calls → **`calls` refs FROM THE CLASS node**; its `val` locals → hook 'local' → nothing (pinned: `calls "register" from=class:WidgetK`) |
| `secondary_constructor` | no branch | **NO constructor node**; recursed → body calls attribute to the CLASS (`calls "log" from=class:WidgetK`); the `constructor_delegation_call`'s value_arguments still feed fn-ref capture |
| `getter`/`setter` as SIBLINGS (accessor on its own line) | no branch | recursed → accessor-body calls attribute to the CLASS (or file). See §Properties for the sibling/child split |
| `object_literal` (`object : T { … }` initializer) | no branch anywhere | never a node; see §Body walker for the method-leak quirk |
| `object_literal` (`object : T { … }` initializer) | no branch anywhere | never a node itself; inside a PROPERTY initializer the hook's walk reaches its `fun`s, which leak out as FUNCTIONS under the property (see §Body walker for the same method-leak quirk) |
| `file_annotation` (`@file:JvmName("x")`) | no branch | recursed; its value_arguments feed fn-ref capture (string args → nothing). No decorates ref |
| INSTANTIATION_KINDS (354-361) | **no kotlin member** | extractInstantiation:4610 is **UNREACHABLE** for kotlin — constructor calls `Foo()` are call_expressions → plain `calls` refs named `Foo` (capitalized). Kotlin emits **zero `instantiates` refs**, ever |
| `impl_item`:1274 / property_signature:1282 / export_statement / swift property:1121 | never | not kotlin node kinds (the swift `property_declaration` branch at 1121-1193 is gated `language === 'swift'` — kotlin property_declarations never enter it) |
@@ -659,7 +667,8 @@ refs — kotlin emits NO instantiates, §dispatch table); backticked
### Static-member / value-read refs (4750-4808) — kotlin IS in STATIC_MEMBER_LANGS (345-347)
Called ONLY from the body walker (5218) — top-level/class-scope reads emit
nothing (hook-consumed property initializers doubly so).
nothing — EXCEPT a property initializer, which the hook now walks through
visitFunctionBody under the property's own scope (§Properties).
`navigation_expression` ∈ MEMBER_ACCESS_TYPES (326). Mechanics:
- callee-of-call skip (4772-4778): parent ∈ callTypes AND parent.namedChild(0)
@@ -850,12 +859,13 @@ unwrap/ungatedModes/addressOfOnly.
- Capture points: visitNode:990 (top-level/class-scope call args),
visitFunctionBody:5137, scanFnRefSubtree (hook-consumed property
subtrees — `val x = register(::f)` captures via the inner
value_arguments; **the scan halts at `lambda_literal` (610), so refs
inside `by lazy { }`/trailing lambdas under a hook-consumed property are
NOT captured**). **NOT captured anywhere: property/local initializer
callable refs (`val m = ::caller`, `val bound = w::render`) — kotlin's
dispatch has NO property_declaration/varinit key** (unlike SWIFT_SPEC —
do not borrow it). Pinned: torture emits exactly three function_refs —
value_arguments; **the scan halts at `lambda_literal` (610)**, but the
hook's own initializer walk (§Properties) covers the same subtree with the
PROPERTY on the stack, so refs inside `by lazy { }`/trailing lambdas are
captured there — a shallow `::ref` reachable by BOTH is emitted twice, once
from the class and once from the property). **NOT captured anywhere:
local initializer callable refs — kotlin's dispatch has NO
property_declaration/varinit key** (unlike SWIFT_SPEC — do not borrow it). Pinned: torture emits exactly three function_refs —
`topLevel` (definedHere), `OtherClass::handle`, `this.caller`.
- Flush gate (639-728): generated-file skip; `this.`-prefixed +
`::`-containing candidates always flush; bare names need definedHere
@@ -1039,7 +1049,7 @@ unwrap/ungatedModes/addressOfOnly.
`Unit` / nullable / lambda return / `: T` generic leak; `expect fun`
(bodiless + dec) / `actual fun`; tailrec self-call in expression body;
top-level `val`/`var`/`const val`/`by lazy {}` (constant/variable kinds,
NO initializer refs, NO capture inside the delegate lambda) +
initializer + delegate refs attributed TO the property) +
**destructuring (`val (a,b)` → nothing, both scopes)** + next-line-getter
top-level `val` (getter calls → file/namespace); class with primary ctor
(props invisible, defaults not walked), class-body val/var/computed
@@ -140,7 +140,7 @@ undefined; **no isConst means `const_item`/`static_item` extract as kind
|---|---|---|
| `function_item` (top level) | functionTypes, tree-sitter.ts:994 → extractFunction:1517 | not inside class-like at file scope → extractFunction; **first line of extractFunction (1522): if getReceiverType returns a value → extractMethod instead** (this is how impl-block fns become methods — impl_item does NOT push a scope) |
| `function_signature_item` | same | in a trait body (trait pushed, class-like) → extractMethod; no `body` field → no body walk |
| `struct_item` | structTypes:1059 → extractStruct:1869 | `body` field required: **unit structs `struct Unit;` have no body → NO node minted** (1876, `record_declaration` exemption is C#-only). Tuple structs have body `ordered_field_declaration_list` → extracted. `field_declaration` children make NO nodes (rust has no fieldTypes) — visitNode recurses into them and finds nothing |
| `struct_item` | structTypes:1059 → extractStruct:1869 | ~~`body` field required: unit structs `struct Unit;` have no body → NO node minted~~**superseded: Rust now sets `allowBodilessStruct`, so `struct Unit;` mints a node with no members.** Rust has no forward declarations, so the bodiless skip (meant for C/C++) never applied here; the `record_declaration` exemption is the C# form of the same carve-out. Tuple structs have body `ordered_field_declaration_list` → extracted. `field_declaration` children make NO nodes (rust has no fieldTypes) — visitNode recurses into them and finds nothing |
| `enum_item` | enumTypes:1064 → extractEnum:1914 | body `enum_variant_list`; `enum_variant` children → extractEnumMembers:1958 — **`name` field path: one `enum_member` node from `getChildByField(node,'name')`, then return** (variant payload bodies `B(u32)` / `C { x }` are never walked). Non-variant children (e.g. `attribute_item`) → visitNode (no-op) |
| `trait_item` | interfaceTypes:1054 → extractInterface:1834 | kind `'trait'` (interfaceKind); extractInheritance sees the `trait_bounds` child (see below); body `declaration_list` children visited with the trait pushed → fn items become methods with QN `Trait::name` via nodeStack |
| `impl_item` | dedicated branch:1273-1276 → extractRustImplItem:5690 | emits the implements back-reference (below); **skipChildren stays false** → the `declaration_list` is then visited normally by the loop at 1295 (that's how impl members are reached; impl pushes NOTHING on the nodeStack) |
@@ -490,7 +490,7 @@ inner `array_expression`, but `const CB: fn() = handler;` captures nothing
## Gates (per plan §5, no exceptions)
- **Torture fixture `torture.rs`** (+ CRLF variant, derived in-memory), pinning
at minimum: unit struct (NO node) / tuple struct / field struct; enum with
at minimum: unit struct (node, no members) / tuple struct / field struct; enum with
unit+tuple+struct variants; trait with supertraits incl. a SCOPED one
(`fmt::Debug` — dropped) + `function_signature_item` + default method +
associated type/const (no node; const value call attributes to trait);