Commit Graph
186 Commits
Author SHA1 Message Date
9181dd1ef3 fix(extraction): land upstream declaration initializer walks (#1511) (#1802)
Squash danusha2345's PR #1511 at d282f9e8 onto main 8c9c4761,
preserving its nine non-merge commits and main's existing Unreleased notes.
Calls in Kotlin, Java, TS/JS, Scala, Rust and Python declaration initializers
now retain the owner established by the upstream regression expectations.
Include the upstream CFML, dynamic-dispatch summary and viewer follow-ups.

Linux fail-to-pass validation (Node 22.19.0, rebuilt dist and native kernel):
- Before: TS load belonged to file:app.ts; Python/Kotlin/Scala/Rust calls
  vanished; Java lost the field-lambda, anonymous override and eager calls.
- After: all six languages PASS; 12 native/WASM LF/CRLF parity checks PASS.
- Focused initializer regressions: 10 passed with CODEGRAPH_KERNEL=0 and
  10 passed with the kernel enabled; Kotlin's grammar fallback is recorded.
- Related regression suites: 879 passed, 1 skipped across 15 test files.
- Evidence: /workspace/cg-1510-repro/before and /workspace/cg-1510-repro/after
  (combined test output: after/vitest.log).

Fixes #1510
Supersedes #1511

Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
Co-authored-by: danusha2345 <ewidusoc498@gmail.com>
2026-09-08 17:33:45 -05:00
71d049cd28 fix(rust): index unit structs — bodiless is a definition, not a forward decl (#1800)
Land upstream PR #1514 for issue #1513 by cherry-picking
ctype_lab's a94c9dc94eee348bf63c7e2dd567c0b43577678a.

Keep allowBodilessStruct as a Rust-only opt-in, with matching behavior in
the wasm/TypeScript walker and native kernel. Create the node before
checking for a body and walk members only when present. Resolve against
main's shared struct/union walker while retaining its stack guard, kinds,
fields, and existing impl-receiver fixes.

Verified FAIL to PASS on Linux x64 with Node 22.19.0 after rebuilding via
tsc, copy-assets, and build:kernel. The identical fixture on main 7b339373
had two structs and two implements edges; fresh wasm (CODEGRAPH_KERNEL=0)
and native indexes now have three of each. UnitStruct and its Greet
implements edge are recovered; tuple and brace structs remain intact.
The native run loaded the rebuilt kernel and completed without fallback.

Focused extraction.test.ts and kernel-rustlang-parity.test.ts runs:
645 tests passed with the kernel disabled, and 645 with it enabled;
all three parity tests ran in each configuration, with no skips.

(cherry picked from commit a94c9dc94eee348bf63c7e2dd567c0b43577678a)

Co-authored-by: ctype_lab <cksgud1226@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 16:53:08 -05:00
72869f2d9e docs: make AGENTS.md the canonical agent guide for Codex/Astra (#1742)
Codex reads AGENTS.md by default, not CLAUDE.md. Move project guidance to
AGENTS.md (trim/nest long validation notes under docs/AGENTS.md), leave
CLAUDE.md as a thin @AGENTS.md wrapper for Claude Code, and document the
Codex project_doc_max_bytes raise needed for the ~35KiB root file.

Co-authored-by: Colby McHenry <colbymchenry@users.noreply.github.com>
2026-09-07 22:58:05 -05:00
Colby McHenry 3298db1292 feat(steps): render fork decisions as points with per-arm edges and captions
Adds full support for decisions at forks in both the code graph and the UI. Key changes introduce a decision model for forks (innermost guard decisions), propagate decision data through the server and wire layer, and render decisions in the UI as distinct points with labeled arms. New components (ForkPoint and DecisionCaption) visualize the decision and its arms, while utilities (armWords, forkLabel) generate arm captions. The order reading (canvas) now shows decisions as points, and arms are drawn as separate edges (yes/no/case), with labels and captions displayed under the deciding box. Tests, typings, and docs updated to reflect the new decision visualization and behavior, including selection reach and resting-label semantics. This lays the groundwork for clearer visualization of conditional navigation and guarded branches on the order canvas.
2026-08-31 17:10:13 -05:00
Colby McHenry 882ea143e8 feat(steps): lay out screen pictures by region and render region captions
Adds region-based layout support for screens: steps now carry region information, and the server packs regions into dedicated bands with per-region captions. UI changes introduce RegionCaption and region-aware step rendering; StepsModel and related views (StepsView) consume region data, while the region-aware layout keeps anchor and region boundaries intact. Tests and docs updated to reflect region-driven organization and visualization of screen regions. This enables visualizing a screen’s picture as region-based columns rather than a single distance-driven row.
2026-08-31 15:38:13 -05:00
Colby McHenry 6f4887db80 feat(steps): draw all arms of conditional navigations as separate edges
Adds multi-arm navigation support: when a destination is produced by a conditional, every arm is now drawn as its own edge. Introduces helpers (hrefArms, destinationsForHref) and updates framework resolvers and edge creation to emit multiple navigates edges (via alsoTargets) instead of a single one. Also introduces per-app rooted route tables to avoid cross-app crossings, and updates various resolvers (React Router, TanStack Router, Vue Router, SvelteKit, Vue, and SvelteKit’s linker) and the UI to reflect multiple possible destinations. Tests and docs updated to reflect the new behavior, ensuring the Screens tab shows all possible navigation paths from conditional destinations. This makes navigation visualization more accurate for forked destinations.
2026-08-31 11:34:26 -05:00
Colby McHenryandClaude Opus 5 209a07e881 feat(steps): the order reading is the canvas, not a rail
The first cut drew the code's order as a nested document — a column of boxes,
forks as rows of arm columns. Wrong picture: hard to read, and it threw away
the thing that made the tree legible. The ask was the canvas back, with the
timing fixed: the 200 comes after the token is signed, so it should branch out
of it.

So the order reading is now the SAME canvas, the same boxes, the same pills,
hover and panel — only the graph changes. `ui/src/lib/program-model.ts` walks
the server's block tree carrying a set of tails (the steps a next step would
follow) and emits one edge per "and then": proshop's login draws the anchor,
`User.findOne`, then the fork — `jwt.sign` under one arm with the `200` a row
below it, the `401` under the other. A row down is one more thing that has
already happened; an arm that answers, returns or throws has nothing leaving
it; a helper, a loop, `later` and `together` ride on the line into what they
hold. Rows are settled by relaxation, because a step reached twice can make
the graph cyclic.

A line means "and then" here and "leads to" in the tree, so the key says which.
The fork conditions are drawn at rest rather than only for a selected box —
`placeLabels` takes an `atRest` flag — because on this picture they are the
content, and two ways to one step merge as one condition (`WHEN userExists OR
NOT user`), not as two rendered labels stuck together.

`StepsRail.svelte` and `RailBlock.svelte` are gone; `StepBox.svelte` stays as
the box both readings draw.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 14:16:01 -05:00
Colby McHenryandClaude Opus 5 75686502e3 feat(steps): an early exit reads as a guard clause, not as a branch
A fork with nothing on one side is `if (!user) return` — a reader takes it as a
guard, not as a decision with two sides. Drawn as a branch it costs a column
and a step right, and a handler with four guards (every server handler) read as
four nested branches with three-quarters of the width holding the words
"returns here". It is now one line — the condition and where the code leaves —
with everything below it running because it did not, and the rail stays on its
own hairline. next-saas-starter's `signIn` goes from not fitting the screen to
fitting it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 13:53:50 -05:00
Colby McHenryandClaude Opus 5 22f92a828b docs(steps): the in-order reading, and what validating it found
Spec §3.13.1 describes the rail as built: what it is made of (records the same
pass makes as the links), what makes the fold possible (a guard naming the
decision it belongs to), the items, and the words. `CLAUDE.md` names
`api/program.ts` and what the guard reader now returns. `CHANGELOG.md` gets the
user-facing feature and the three fixes under it.

Both plans now say what happened: the 2026-08-29 plan carries a BUILT header
with where the build differs from it (a guard's `branch`, reading a function
once per rail, blocks as one kind carrying facts, loops needing a reading of
their own), the answers to its open questions, and a §8 recording the six
endpoints read against their source — plus the two gaps left open on purpose,
a mongoose `product.save()` the effects table does not know and the nested
`const handleX = async () => …` that is still not a node.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 13:48:04 -05:00
Colby McHenryandClaude Fable 5 fc149b7d74 docs(plan): point the servers plan at the in-order plan
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 12:41:59 -05:00
Colby McHenryandClaude Fable 5 34ef71a1d5 docs(plan): Steps in the code's order — a rail reading of a handler (sequence + forks) derived from the existing walk
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 12:41:40 -05:00
Colby McHenryandClaude Fable 5 783f3954ec feat(steps): rows read in the code's order; a hop written inside another call says so
- branch-guards callSiteInTree: a call's span and the call it is written inside the arguments of (`within`), stopping at a function or block boundary
- steps.ts: each step records the hop that first reached it (position, span, enclosing call — the fold's first hop out of the root, inherited down the fold); a row is ordered by that position, a hop inside another site's arguments before that site, and `WireStep.order` carries it; links carry `within`
- map-model: an `order` option — the row's initial order, sweeps over parents only, tie-broken by it; the Map and Screens tabs pass none and are unchanged
- viewer: rows laid out by `order`; `inside res.json(…)` in the panel rows and the tooltip
- tests: servers fixture (a token signed inside the reply's arguments: `within`, and the row `create · queue · mail · jwt.sign · 201`), model row order; spec §3.13, CHANGELOG

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 12:25:27 -05:00
Colby McHenryandClaude Fable 5 46e3e7aaa0 feat(steps): one reply box per outcome
A reply's identity is its status, not its call: a handler answering 200 or 401 draws two boxes (id per function, response, status), so each line from the handler carries its own condition on the picture — the Screens view's idiom — and the anchor's Leads-to list reads as the contract; replies whose status the code does not spell out share one box labelled by the call. Panel note, spec §3.13, CHANGELOG, plan; servers test asserts the ASP.NET and Spring outcomes per box.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 12:11:40 -05:00
Colby McHenryandClaude Fable 5 435a7fd37a feat(ui): double-click a step to start the picture there; double-click a screen for what happens on it
- StepsView / StepNode: a double-click on any step with a symbol re-anchors the Steps picture on it (the panel's Start here) — an endpoint or another screen drawn as a boundary opens as its own chapter in one gesture; detected in the view's click path (two clicks on one box within 400 ms) since the flow canvas does not reliably pass dblclick on, with ondblclick kept
- ScreensView / ScreenNode: a double-click on a screen (or an origin) opens its Steps picture
- a boundary's panel says it is not entered instead of "nothing leaves this step"; tooltips and the boundary notes mention the gesture
- spec §3.13, CHANGELOG

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-29 11:09:09 -05:00
Colby McHenryandClaude Fable 5 7f461269c4 docs(playbook): first P7 agent A/B numbers (proshop, Sonnet, 2 runs/arm) and the plan's P7 status
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 14:57:43 -05:00
Colby McHenryandClaude Fable 5 bc45e9071d feat(routes): FastAPI prefixes, ASP.NET endpoint groups, wrapped server actions on the Screens tab
- python.ts postExtract: APIRouter(prefix=) and literal include_router(prefix=) composed down the include tree (module import, alias, local); a computed prefix leaves that mount alone; full-stack-fastapi-template 23 routes named by path
- csharp.ts: handler-first MapPost(Handler[, "path"]) under the endpoint-group class, the app's $"/api/{groupName}" head read in postExtract, RoutePrefix honoured; detection covers Endpoints/ files; CleanArchitecture 10 routes
- tier-synthesizer: a type argument between a client call and its parentheses (useSWR<T>(…), ky.get<T>(…)); recorded callee without it
- screens.ts: a file-scope navigation attributed to the value spanning it; a value nothing calls attributed to the functions mentioning it in importing files (request-time source read, bounded); steps.ts lends navigates edges to a value root
- tests: servers fixture (FastAPI prefixed routers, ASP.NET endpoint group end to end), frameworks.test.ts (group form, RoutePrefix), cross-tier (generic useSWR)
- docs: CHANGELOG, plan, playbook rows

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 14:52:40 -05:00
Colby McHenryandClaude Fable 5 9c6bc23b21 feat(screens): Next.js as a Screens app — pages, route handlers, links and redirects
- frameworks/nextjs.ts (split out of react.ts): App Router app/**/page.tsx and Pages Router pages → routes named by path ((group) stripped, [slug] → :slug, [...all] → :all*), bound to the default export; app/**/route.ts exports → METHOD /api/… endpoints referencing their functions; pages/api → ANY; resolve() claims router.push/replace/prefetch, redirect/permanentRedirect and NextResponse.redirect(new URL(…)) into navigates edges via the Expo href readers, against a Next-only route table gated on the app's root
- next-router-synthesizer.ts: <Link href> and internal <a href> → dashed navigates edges from the component (next-link, registeredAt)
- expo-router.ts: href readers exported; matcher accepts :param / :all* segments
- steps.ts: a Next page's own work fires from page load; a Next page makes the project a web app; {status: 201} read off the call site (branch-guards CallSiteText.status) for response rows
- frameworks/package-deps.ts: nested package.json files probed on disk (getAllFiles lists only sources); Express/React/Expo/Nest detectors use it; routing manifest names constant handlers
- tests: nextjs.test.ts (file→route rules, extract, verbs, end to end with Screens and Steps); frameworks.test.ts Next cases moved to the Next resolver
- docs: CHANGELOG, spec §3.12 frameworks paragraph, CLAUDE.md, synthesis doc, plan P4 built, playbook rows for Next / MERN / Nest channels

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 14:40:14 -05:00
Colby McHenryandClaude Fable 5 b1f40c57dd feat(steps): cross-tier channels — a client's fetch onto its own route, queue jobs onto consumers, bus and socket events onto handlers
- resolution/tier-synthesizer.ts: http-client (literal fetch/axios/ky/got/$fetch paths, axios.create baseURL instances, template holes as :params, base-URL holes by a two-segment tail; unique match only), queue-job (BullMQ/Bull add ↔ @Process/@Processor, WorkerHost process, new Worker, queue.process), event-bus (EventEmitter2 emit ↔ @OnEvent with globs; socket emit ↔ @SubscribeMessage / socket.on both ways with tier); channel, tier, callee, registeredAt on every edge; generic transport events never pair; test and generated files never sources; registered before the emitter pass
- steps.ts: crossing() reads tier/channel before languages; an endpoint reached across a tier is a bridge box and a boundary like a screen (through=1 enters it); a channel's call is not also an effect; sites read as written; a Next 'use server' action is a crossing by its directive (when.ts directive); a function-valued constant handler (asyncHandler(...)) is a route root and borrows the file-scope calls and refs within its lines
- express.ts: app.use('/prefix', router) mounts composed onto route names in postExtract (nested, by import or require); chained router.route('/x').get(h).put(h2) extracted, across lines
- frameworks/package-deps.ts: dependencies read from workspace package.json files too (Express, React, Expo Router, NestJS detect)
- routing manifest names constant handlers; e2e/ is a test directory; explore's Flow section labels the new channels
- tests: ui-steps-cross-tier (monorepo fixture: Next client + Express/Nest API), servers test updated for the queue landing
- docs: CHANGELOG, spec §3.13 cross-tier paragraph, CLAUDE.md, callback-edge-synthesis.md, plan P3 built

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 14:31:01 -05:00
Colby McHenryandClaude Fable 5 950686def4 feat(ui): Steps for servers — route roots, server effects, request/decorator triggers, guards for Python/Java/Kotlin/C#/Go/C
- api/route-roots.ts: the symbol a route runs (references-edge handler, exported page component, or the route itself for an inline handler), shared by steps and screens; the bare Steps tab lists an API's endpoints by router file
- api/effects.ts: database / response / queue / email / payments / cache / auth / process / network / storage / device / telemetry, matched on the call as written per language family, with model + read/write and the literal status on a response site
- graph/branch-guards.ts: callSitesForFile (the whole member chain), memberTypesInTree, decoratorsForFile, request/decorator triggers with the middleware/guard chain; guard + argument rules for Python, Java, Kotlin, C#, Go and C
- steps.ts: classify on the chain before trusting a name match, retarget this.x.y() by declared type, skip test doubles after the effect pre-check, project kind on the wire
- viewer: kindWord/kindWords per project kind, endpoint chooser, response boxes labelled by status codes
- python.ts: FastAPI detected from a monorepo sub-directory; is-test-file: samples/examples package paths are not tests
- tests: ui-steps-api-servers, ui-effects, branch-guards-languages; spec §3.13 Servers paragraph, CHANGELOG, plan doc

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 13:45:38 -05:00
Colby McHenry 5e06204deb feat(expo-router): add Expo Router support for Screens and navigations and introduce Steps API
- Introduces trigger metadata for steps and edges to capture what fires a site (JS prop, on* option, or callback) to improve cross-boundary flow analysis.
- Extends parsing/analysis to detect triggers in JSX attributes, on* bindings, and late-bound callbacks; adds utilities (calleeText, lastSegment) to extract trigger sources.
- Ships new trigger structures (WireStepTrigger, trigger on WireStepSite/WireStep) and propagates trigger through built steps; updates step labeling to reflect trigger information.
- Adds triggerWords helper and uses it to render human-readable trigger descriptions in Steps UI, including edge labels and per-site visuals.
- Updates UI (ScreensView, StepsView) to display FIRES FROM information, with styling tweaks to highlight triggers and related elements; enhances tooltips and inline text wrapping for readability.
- Extends tests to cover trigger detection and rendering across various binding patterns (prop, option, callback) and inline RN listeners.
- Updates design/docs and changelog to reflect Expo Router integration, per-site trigger metadata, and the new Steps surface.
2026-08-28 10:40:38 -05:00
Colby McHenry e288d7645b feat(expo-router): add Expo Router support for Screens and navigations and introduce Steps API
- Adds Expo Router integration with a new Screens view and a Steps API to surface screens and their transitions.
- Extends codegraph extraction/resolution to handle namespace objects, React hook bindings for handlers, and Swift RN bridge evidence; introduces per-site guard arguments and trigger metadata, enabling richer flow analysis across JS ↔ native boundaries.
- Introduces UI and data-model changes to represent conditions as words (WHEN/AND/OR/NOT), display per-site call arguments, and show what fires a site (triggers). Adds new utilities (ui/conditions.ts) and updates ScreensView and StepsView to render scenarios with multiple sites and “ways” counts.
- Implements site readers for WHEN/ARGS/TRIGGER, and wiring to expose steps via API endpoints (including /api/steps); enhances tests to cover namespace resolution, useCallback-driven handlers, and inline RN event listeners.
- Updates styling and templates to reflect the new wording, scenario rows, and per-site details, including NOT instead of leading negation strings and multi-way links.
- Documents and reflects changes in changelog and design docs to describe Expo Router integration and the Steps surface.
2026-08-28 10:21:46 -05:00
Colby McHenry 873f133c96 feat(expo-router): add Expo Router support for Screens and navigations and introduce Steps API
Introduce Expo Router integration with a new Screens view and API to surface screens and transitions, plus a new Steps API and UI to depict typed steps from anchors or symbols. Extend codegraph’s extraction and resolution to handle namespace objects (export default NAME, two-statement forms, and default bindings) and React hook bindings for handlers, improving accuracy of flows across JS ↔ native boundaries. Add Swift/React Native bridge receiver evidence (RCT_EXTERN_MODULE, RCT_EXTERN_METHOD) and related resolution logic, with tests covering namespace-object resolution, useCallback-driven handlers, and inline RN event listeners. Update UI to include a Steps tab and associated components (StepsView, StepNode, ScreenEdge) and wire navigation to expose steps-based exploration via /api/steps and UI routes. Documentation and changelog reflect the new Expo Router integration and steps surface capabilities.
2026-08-28 09:46:50 -05:00
Colby McHenry f0eafe31f9 feat(expo-router): add Expo Router support for Screens and navigations
Introduce Expo Router integration by adding a new Screens view and API to surface screens and their transitions. Implement a map-based layout with directional ports, extended layering and port pitch to accommodate edge labels, and a pill-based labeling system for transition conditions. Include tests for the new map/screens models, updates to the UI components, and design/docs changes reflecting the Screens design. Merge CodeGraph UI viewer changes to render and interact with Expo Router-based screen graphs. This enables CodeGraph UI to surface screens and navigations from Expo Router apps.
2026-08-28 00:19:46 -05:00
Colby McHenryandClaude Opus 5 47576b392e feat(ui): saved trails — a walk you named, kept, and still true after a re-index (CG-60)
Save trail on the trail bar writes the walk to .codegraph/ui/trails/ as one
JSON file, listed on the empty screen and on Entry points above the derived
suggestions, reopened at the symbol you left with the whole path restored.

A hop is stored by qualified name, kind and file — never by node id, which
contains a start line and so changes the first time anybody edits above the
symbol. Every hop is re-resolved against the current index on the way out and
each row says what became of it: still here, moved to another file, now
ambiguous, or gone. A hole is never stitched over: the row opens the longest
run of CONSECUTIVE resolved hops and says which ones those are, because the
trail is a path and a skipped hop would draw a call that does not exist.

This is the first write the viewer makes, and the boundary moved with it:
POST/DELETE answer under /api/ only, must carry X-CodeGraph-UI and
application/json (neither of which a cross-origin form can produce without a
preflight this server answers none of), and --read-only refuses both while
still listing what is there. The blanket "read-only" claim is retired from the
banner, the README, the CLI help and the docs site in favour of the narrower
true one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 08:31:48 -05:00
Colby McHenryandClaude Opus 5 56dfdb0655 feat(ui): dead code and islands — what nothing reaches, and everything that could still reach it (CG-59)
A Dead code screen and a mark on the Map, both drawn from one derivation in
src/graph/dead-code.ts so a second surface can never disagree with the first.

The SQL half is four lines — no incoming edge but `contains`. It returns ~2 500
candidates on this repository and the shipped list is 20; everything in between
is the feature. A candidate is dropped the moment there is any reason to believe
something outside the graph reaches it: exported symbols and header
declarations, test and generated files, abstract and interface members, anything
carrying a `decorates` edge, overrides of an ancestor's member, names the
language calls by itself, vendored directories, files nothing in the index
reaches (those are islands, and the Map says so instead), names the resolver
failed to resolve somewhere, and names shared with a symbol that IS referenced —
the mis-resolution that leaves a used method with a self-edge and its twin with
nothing. The last rule is the only one that is not a graph query: before a claim
is made, the declaring file and every file that reaches it are read and the
identifier counted, which is what catches the references the extractor never
recorded (`this.handleMessage.bind(this)`, a call inside an object literal, a
shorthand property).

Every subtraction is counted and printed under the list with the scale it came
from, and the caveat line above it never collapses: the claim is "no static
reference in the index", not "unused".

On the Map a module nothing depends on keeps its stroke and says so in its count
line, and tool-generated files and modules recede to ink-4 there, in the map's
file list, in search results and on the file screen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 07:57:00 -05:00
Colby McHenry 2a0c6dc58f feat(ui): the type hierarchy — what a type is built on, and what dispatches through it (CG-58)
A vertical tree above the members outline for classes, interfaces, structs,
traits, protocols, enums, unions and type aliases: ancestors above (the whole
chain, not just the direct parent), the focus in accent, subtypes below indented
per level. `extends` draws solid, `implements` dashed; a synthesized edge — Go's
implicit interface satisfaction — draws dashed wider and carries the site it was
wired at, so a relation the resolver inferred never reads like one the source
wrote down. For an interface the fan below IS the set of runtime targets a call
can land on, and a type with eight or more implementers leads with that in a
sentence. Members that redeclare an ancestor's are marked in the outline.

The walk lives in `src/graph/type-hierarchy.ts`, following CG-50/CG-51: shared
computation in `src/graph/`, presentation in the caller. Its `countImplementers`
is now also what `ToolHandler.buildPolymorphicBoundaries` counts with, so "N
types implement X" is the same N whether an agent reads it or a person does.
`/api/node` carries the block as `hierarchy` rather than a second endpoint —
it is part of the Symbol view's first paint, and gated to types, so a function
costs one kind test.

Layout is arithmetic (24px rows, 22px indent, orthogonal connectors computed
from the two): no ResizeObserver, same payload → same picture. The header's
`extends X` / `implemented by …` chips are suppressed while the tree is on
screen — two renderings of one relation in one column is how a reader ends up
trusting neither.

`TypeHierarchy` is exported from `@colbymchenry/codegraph-ui` and takes its data
as a prop, so a host holding a `WireSymbolPayload` renders it without a second
read.
2026-08-27 07:14:55 -05:00
Colby McHenry c15413f200 feat(ui): the viewer's screens as @colbymchenry/codegraph-ui, behind one adapter (CG-61)
`ui/src` now builds two ways from one tree: the static app `codegraph ui`
serves, and — via `svelte-package` — a Svelte library the Pro app imports.
A forked component would be a second answer to the same question about the
same graph, so there is no fork.

Everything a screen knows arrives through a `GraphAdapter`: eleven methods
answering the wire shapes verbatim, with `createHttpAdapter()` (the loopback
JSON API) as the default and a host's in-process engine reads as the point.
`lib/api.ts` became a one-line-per-call facade over it, which is why no call
site in the views changed. The payload types moved to `lib/wire.ts` — no
imports, no runtime — so a host can depend on the vocabulary alone.

Two more seams and one guard:

- `lib/navigation.ts` holds the href builders behind a `NavigationDriver`, so
  a host addresses its own URL space. The app's half — the hash parser and the
  live route, which attach window listeners at module scope — stays in
  `router.svelte.ts` and is pruned out of the package: rendering a Symbol view
  must not install a hash router in somebody else's application.
- `lib/theme.css` carries the design tokens and maps Svelte Flow's `--xy-*`
  variables onto them, so a host never sees library defaults. Dark now also
  answers to a bare `[data-theme]`, which is how `<CodegraphUi theme>` themes
  a container rather than the document.
- `scripts/check-ui-package.mjs` prunes the app's shell, resolves the
  extensionless specifiers svelte-package leaves behind, and asserts that
  nothing but `lib/adapter.js` reaches the network.

The search box, its keyboard and its panel are one component now
(`SearchPalette`), because splitting them is what breaks a palette.

`__tests__/ui-package.test.ts` mounts the three screens from the package entry
against a mock adapter in jsdom; it runs as a second vitest project so the
`browser` resolve condition it needs cannot reach the engine's suites.

Versioned with the engine. Prepared, not published: `private: true` is the
guard and `pack-npm.sh` only packs a tarball under CODEGRAPH_PACK_UI=1.
2026-08-27 06:52:57 -05:00
Colby McHenry ad91c8fdd8 feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57)
The viewer ran a second highlighter over source the engine had already parsed
with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in
dist/textmate/. The classification now comes off that tree instead, so a file is
read by exactly the grammar that decided what its symbols are.

The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript
and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and
`npm run build:textmate` are deleted, and check-ui-build.mjs asserts the
tree-sitter grammars in dist/extraction/wasm instead of dist/textmate.

The wire contract is unchanged — `[classId, text]` pairs with the class names
alongside — so the viewer's decoder and code blocks did not have to be rewritten.
Two classes are added to the six: `type` (a named type reference, painted at
plain ink) and `def` (the name a definition declares, weight 600), the latter
taken from the extractors' own definition tables so it cannot drift from what
indexing calls a definition.

Three differences are not cosmetic:

* Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code,
  not as string. The call-site overlay refuses to claim a token classed string,
  so calls written inside interpolated strings now link.
* Built-in type words are emitted whole and classed `type` in every language.
  The grammars disagree about whether `string` is a type_identifier or an
  anonymous token inside a predefined_type, and TextMate scoped them
  inconsistently too.
* 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms.

Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render
plain. .svelte/.vue/.astro are classified through their <script> blocks, the same
delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms
would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap
corruption for every other language in the same process.

Measured parity, per-language before/after screenshots and the reproduction
recipe: docs/design/cg57-highlighting-parity.md.
2026-08-27 06:22:11 -05:00
Colby McHenry 8ac0138940 feat(ui): copy the flow or the map as an image, for a PR comment or a README (CG-55)
"Copy image" and "Download SVG" on the Flow strip's header and in the Map's
side panel. The image is the distribution loop: a flow pasted into a review, a
map pasted into a README, read by somebody with no viewer open.

The exporter serialises the LAYOUT OBJECT rather than scraping the DOM — no
html-to-image, no foreignObject, no new dependency. buildFlowLayout and
buildMapLayout already compute every rectangle, port and curve before a
component renders, so the image and the screen come from one piece of
arithmetic and cannot drift apart, and the whole exporter is a pure function a
test runs with no browser. Output is presentation-only SVG (rect, line, path,
polygon, text, tspan, clipPath) — no script, no external reference, no data:
URL — which is what GitHub's sanitiser accepts in a README.

Light theme is forced whatever the viewer is set to: a dark strip on GitHub's
white comment background reads as a mistake, not a preference. 24px of paper
around the drawing, a caption naming the path or the root at the bottom left,
a CodeGraph mark at the bottom right.

Fonts travel as family stacks, not bytes (spec). An SVG loaded as an image may
not fetch a webfont, so a raster falls back to the platform's own monospace —
every fallback in the stack advances at ~0.6em like IBM Plex Mono, so the code
grid survives and only the letterforms change. Text is truncated
arithmetically with an ellipsis and clipped as well, so a wider fallback
cannot spill a source line out of a card.

`scale` multiplies only the root width/height while the viewBox stays in CSS
pixels, so the raster draws an image whose intrinsic size is already 2x
instead of upscaling a 1x bitmap. The clipboard write uses the ClipboardItem
promise form (Safari discards the gesture across an await) and falls back to
downloading the PNG, saying which happened rather than claiming a copy it did
not make.

Measured on this repo: execute -> rowToFileRecord (8 hops) exports 3690x253
CSS px, 491 kB PNG at 2x / 38 kB SVG; the 16-module map reproduces the canvas
exactly — 16 boxes, 52 links, 9 layer rules, both band labels, and with
src/index.ts selected 15 links and 4 dimmed boxes.
2026-08-27 05:41:34 -05:00
Colby McHenryandClaude Opus 5 94f4e287e6 feat(ui): entry points — routes, executable files and tests as flow starting points (CG-54)
`#/entry` answers "where does anything start" at full length, and turns any row
that names a symbol into a flow.

Server. `/api/entrypoints` gains `frameworks` (from `getDetectedFrameworks`), a
`tests` list, a `routes` limit of its own, and a cache keyed on the index build
— nothing here is read from disk, so unlike `/api/source` a cached answer cannot
be stale about drift. `routes.items` is now a `WireList` like every other list on
the payload.

Routes carry where the URL is REGISTERED as well as where it is served:
`getRoutingManifest` selects the route node's id, file and line, and
`buildRoutes` splits the verb off the name against a fixed list (never "the
first word", which would take the head off a file-routed `/blog/[slug]`). All
four payroll-go routes register in one router file and three are served from
another — group by the handler file and one router becomes two groups plus an
orphan.

`isTestFile` is split into `isTestPath` (test filename and directory
conventions) + the non-production catch-all, byte-identical at every existing
call site. The Tests list uses the narrow half: an example, a benchmark or a
fixture is off-target for ranking but is not a test, and a heading that says
"Tests" must not quietly count them. Tests rank by REACH — distinct other files
touched — because Go, Rust and Java put test work inside functions where a
module-level-calls ranking sees nothing. Two read-only engine queries make that
affordable: `getFileReachCounts` (the mirror of `getFileDependentCounts`, driven
from `nodes` by path so the cost follows the files asked about rather than the
edge table) and `getFileNodes`.

Viewer. `ui/src/lib/entry-model.ts` folds the four lists into file groups —
pure, and `panel.rows` stays exactly the sections it draws. `EntryView` +
`EntrySection` render them with the caller rail's `.filegroup` / `.row` shapes
rather than a second visual language for the same idea. A row that names a
callable symbol carries a `Flow ›` chip; the other end is typed or picked with
`→ here` on another row. File and test rows carry none: `/api/flow` searches by
name, and a file has none the path finder can look up.

A project with fewer than three resolvable routes gets no Routes heading at all,
not an empty one. Typing into the search box now also returns matching entry
points under their own heading below the symbol matches, so a URL comes back
with its handler attached; rows already in the results are dropped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 05:20:15 -05:00
Colby McHenry dc7f1e590e feat(ui): where the graph stops — the Flow strip's dynamic-dispatch end cap (CG-51)
A flow that does not reach what it was asked about now ends in a cap instead
of in silence: the dispatch form that ended it, the line, the static key when
the source spells one out, the candidate runtime targets as clickable rows,
and the name-only matches under 0.6 the search refused to follow. A flow that
does reach its destination never shows one.

The verdict is lifted out of `ToolHandler` into
`src/graph/dynamic-boundary-report.ts` and both callers render it —
`codegraph_explore`'s prose and `/api/flow`'s `WireFlowBoundary` — the same
move `named-symbol-flow.ts` made for the path finder, and for the same reason:
a reader holding the strip and the MCP answer must not be told two different
things. The explore prose is unchanged, byte for byte.

When nothing connects at all and a dispatch site explains why, the strip is
that site: one card opened at the line where the static path ends, plus the
cap. When nothing explains it, no stopping point is invented.
2026-08-27 04:54:37 -05:00
Colby McHenry ecd6e1cd15 feat(ui): live refresh and drift banners — the viewer keeps up with the project (CG-53)
`GET /api/events` is a server-sent-event stream the viewer holds open for the
life of the page. Two signals, two things the browser could not know:

  changed  source files touched on disk, before any sync — the drift banner
  index    the graph moved, naming what the sync re-indexed — the live refresh

The server WATCHES and never syncs: the project tree through the engine's own
FileWatcher with a notify-only syncFn, the index through one non-recursive
fs.watch on the data directory settled at 400 ms. Both start with the first
subscriber and stop with the last, so a viewer nobody has open costs no watch
descriptors. Nothing polls, on either side.

Drift is now parity with codegraph_node (#1474) rather than an absence.
`/api/source?ondrift=current` serves a drifted file's CURRENT bytes flagged
`showing: 'current'`, and the three screens that can say so switch off
everything anchored to the old line numbering — gutter ports, call-site links,
call arcs, the callee rail's anchoring — while keeping the source. The banner is
paper-2 with a hairline rule, never amber: amber belongs to the untested badge.

Also fixes a stale read this exposed. A long-lived reader holds an LRU of nodes
by id that only its own writes invalidate, so `/api/node/<id>` kept answering
with a symbol another process's sync had deleted while `/api/search` beside it
said it was gone. GraphSession now drops the read caches when the database (or
its WAL) has been written, and the Symbol view follows a symbol whose id changed
because an edit above it moved its start line, carrying the trail across.

Measured on a live viewer: banner 360 ms after a save, toast 440 ms after
`codegraph sync` returns, 0 requests in 4 idle seconds, and the client gives up
reconnecting after ~90 s with "Not live" rather than hammering a dead port.
2026-08-27 04:28:56 -05:00
Colby McHenryandClaude Opus 5 bd99c5e99a feat(ui): the whole file — full source with gutter ports and intra-file call arcs (CG-52)
The File view gains a Source tab: the file itself, top to bottom, with the
Symbol view's line grid, gutter ports and call-site links, a line-anchored
callee rail, and — in the left margin — an arc for every call that stays inside
the file, drawn from the calling line to the callee's definition line.

The arcs are the point. Source order is already a layout, chosen by whoever
wrote the file, so a file's internal call structure can be drawn with no
algorithm placing anything. Crabviz's idea, in the one place it is legible.

Everything is arithmetic, not measurement. The Symbol view queries the laid-out
DOM to place a callee row beside its line; a 6 820-line file cannot afford that.
Here a line is exactly 20px at `10 + (n - 1) x 20`, so ~90 line elements exist at
a time and the arcs, ports, rail rows and connectors are all functions of a line
number. `src/mcp/tools.ts` scrolls at a 16.6ms median frame.

- `GET /api/filecode/<path>` — outline, one call group per (caller, callee) PAIR
  with its call-site lines, unresolved references, and the file's length. The
  source is NOT in it: it pages through `/api/source` 800 lines at a time with a
  discarded 150-line lead-in, so a page starting inside a block comment does not
  render prose as code, and so the ports and arcs are complete from the first
  frame while the text fills in behind them.
- `intraFileCalls` is counted over the groups actually returned, so the header
  and the picture under it cannot disagree once a cap bites.
- Above 40 arcs the diagram narrows to the symbol under the pointer (or the one
  the scroll position is inside) and the header states the total. Accent is for
  the pointer only, never for the filter.
- Sticky outline rail at >= 1400px, following the reader down the file.
- `QueryBuilder.getUnresolvedReferencesInFile` — one indexed lookup instead of
  one per symbol; `buildOutlineEntries` lifted out of `/api/file` so both
  readings of a file draw the same rows.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 03:47:42 -05:00
Colby McHenryandClaude Opus 5 58dad12f89 feat(ui): the File view — outline in source order between two dependency rails (CG-46)
Clicking a file path now opens the file itself: what reaches into it, its
symbols in source order, and what it reaches.

The two rails count DEPENDENCIES, not import statements. The prototype drew
`imports` edges; on this repo `src/graph/traversal.ts` imports two files and
depends on four, because it reaches the LRU cache through a call no import
names. A rail headed "Imports 2" would be quietly wrong about what changing the
file would touch, which is the only question the screen answers — so the rails
read `getFileDependencies` / `getFileDependents` and merge the import rows in
for the symbol names. Imports that resolved to nothing indexed keep their own
section rather than vanishing.

The outline is windowed above 250 rows against a fixed 28px row: this repo's
own fixtures hold a 1,681-symbol `.d.ts`, and paging it would hide the one
thing an outline is for. `src/mcp/tools.ts` draws its 135 rows whole.

`/api/file` gains `topLevel.calls` — module-level calls out of the file node —
so a file that RUNS something offers the badge that opens it as a symbol, the
only place code belonging to no symbol can be read.

File results in the search palette and the entry-point list now land here
rather than on the file node's Symbol view.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 01:43:00 -05:00
Colby McHenry 2ad836d935 feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43)
The viewer's code block stops lexing with a hand-rolled dialect table and
reads real TextMate grammars instead, run once in `/api/source`.

Three things make that safe to depend on:

* Highlighting never fails a request. A missing grammar, an oversized
  slice, an ESM import that did not resolve — every one of them answers
  `engine: 'plain'` with a reason and the source still goes out.
* Identifiers survive whatever token boundaries a grammar chose. Every
  code token is split into identifier runs before it goes on the wire, so
  the graph's call-site overlay claims a token the highlighter produced
  rather than re-cutting the line. `assignRefs` now matches on a token's
  text rather than on the class a grammar gave it, so a language that
  scopes type names as `storage.type` still links.
* The theme classifies rather than colours: its foregrounds are sentinels
  the server maps back to class names, and the viewer paints them from
  CSS custom properties — one token stream serves light and dark with no
  refetch, and the ramp lives only in app.css.

Comments move from --ink-3 to a new --code-comment. --ink-3 measures
3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px
text; --code-comment is the smallest step along the same ramp that clears
4.5:1 on every background a code line can have, and stays quieter than
the strings and numbers above it.

Shipping: @shikijs/core and @shikijs/engine-javascript are runtime
dependencies (no wasm, no native module); @shikijs/langs stays a
devDependency and `npm run build:textmate` writes only the closure the
engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for
all 722. check-ui-build.mjs asserts the tree after every build and inside
every release archive.
2026-08-27 01:23:10 -05:00
Colby McHenryandClaude Opus 5 87afc50e76 feat(ui): the search palette, entry points and a trail that survives the URL (CG-45)
Search: `/` or ⌘K focuses the box; results arrive grouped by kind with their
glyph, signature and file:line, ↑/↓/Enter walk them, Esc dismisses. A group
appears where its best result did, so flattening the groups reproduces the
ranking the keyboard walks — the panel's flat item list IS that concatenation.
A flow question ("how does X reach Y", "X -> Y") is recognised and searches
both endpoints with a note, rather than offering a row that would land on the
phase-2 Flow view.

Entry points answer "where do I start" on the empty screen and in the resting
palette, all derived from the graph: routes, files that run something at module
level (the engine records a top-level statement as an edge out of the file node,
which is what makes src/bin/codegraph.ts the root of the CLI flow — ranked by
calls x the files they reach, so a registration table calling into itself does
not outrank the CLI), and the most depended-on symbols. Tests are excluded from
both derived lists.

Trail: hops record the direction they were walked (→ into a call, ← up to a
caller), clicking one truncates back to it, Clear keeps the place instead of
throwing it away, and the whole walk travels in the URL. A shared or reloaded
trail arrives as ids, so hops learn their names back through a new batch
endpoint and a session name cache — without it, walking back across a
truncation redrew earlier hops as raw hashes. "Read as flow" stays hidden until
there is a Flow view to send it to.

New endpoints: /api/entrypoints and /api/nodes. New engine reads:
getTopCallingFiles, getFileDependentCounts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 00:54:18 -05:00
Colby McHenry a72f22a6d3 feat(ui): scaffold the codegraph ui viewer as a Svelte 5 + Vite workspace (CG-40)
Adds `ui/` as an npm workspace (Svelte 5.56 + Vite 7, devDependencies only —
the engine's runtime dependencies are untouched) and chains its build into
`npm run build`, so the browser viewer ships inside `dist/` with everything
else: `build-bundle.sh` already copies `dist` wholesale and `pack-npm.sh`
packs that bundle.

Output is `dist/viewer/`, NOT `dist/ui/`: `src/ui/` is the engine's terminal
ui (shimmer progress + its worker) and tsc compiles it to `dist/ui/`, so
emitting there both deletes those modules — the CLI then dies at startup with
`Cannot find module '../ui/shimmer-progress'` — and would leave the static
server handing out compiled engine internals. The design spec is corrected to
match.

`scripts/check-ui-build.mjs` is the release guard: index.html must exist, be
non-trivial, and every local asset it references must be on disk, and the
compiled engine next door must still be intact. It runs after every UI build,
again in `build-bundle.sh` once the bundle stage has copied `dist`, and again
in `pack-npm.sh` once each archive is unpacked — so a broken viewer fails the
release instead of shipping a CLI that serves a 404.

`vite build` does not override an ambient NODE_ENV, so a shell or runner with
NODE_ENV=development silently shipped dev-mode Svelte (~13 kB of dev-only
runtime checks, warning in the user's console). The config now pins production
for `command === 'build'`; macOS and Windows ARM64 then emit byte-identical
bundle hashes.

The shell itself follows docs/design/codegraph-ui-design-spec.md §2–§3.1:
design tokens as CSS custom properties (light on bare `:root`, dark under both
`prefers-color-scheme` and `[data-theme="dark"]`), square corners, hairline
rules, one oxblood accent; top bar 48px / trail bar 34px / main; a hash router
over `#/s/<id>`, `#/file/<path>`, with `#/map` and `#/flow` reserved for phase
2. Fonts are vendored through @fontsource rather than fetched, so a local
reader works offline and never announces the project to a CDN.

Verified: clean `npm run build` from an empty dist on macOS and on the Windows
ARM64 VM (forward-slash asset URLs, CLI still starts, both assertion failure
modes exit 1); `dist/viewer` present in a real darwin-arm64 bundle and in the
packed npm platform package; shell geometry, tokens, all seven routes, both
themes and font loading checked in headless Chromium with no console errors;
`npm test` unaffected.
2026-08-26 15:55:10 -05:00
Colby MchenryandGitHub 838006c947 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581.

## What was wrong

`codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI
process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's
`clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is
exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the
native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …,
one frame per AST level) overflowed the thread's stack. A native overflow can't be caught
the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so
the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial
index.

Two things made "just give the worker a bigger stack" the wrong fix:

- it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a
  default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the
  8 MiB **main** thread too;
- the walkers are shared by every kernel-routed language (20 of them), and each has
  several recursion points with different frame sizes, so no single stack size is a
  provable bound.

Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its
own `RangeError` per file and stores a partial result with a `parse_error`. The kernel
just needed a way to get there instead of dying.

## What this does

**The kernel guards its own recursion against the calling thread's real stack bounds and
defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for
files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm
path for this file", silently.

- `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per
  thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS
  `pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32
  `GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`).
  `exhausted()` is one thread-local load and one compare: true once the stack pointer is
  within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report
  bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack
  pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on
  the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike.
- `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker
  function — all **150** self-recursive or on-cycle functions across the 15 walker modules,
  found by script (every cycle in the call graph, not just direct self-calls). It returns
  `Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops
  descending; a hook returning `false` sends its caller down the generic child walk, whose
  own guard returns at once.
- `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set
  afterwards the (truncated) result is discarded and replaced by
  `defer: nesting too deep for the native walker — wasm recovery handles it`.
- `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately
  no `resourceLimits.stackSizeMb` bump.
- No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock`
  transitively). No wire/ABI change.

Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as
`function foo` plus a recorded parse warning, and the other 31,607 files index normally.
`CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed.

## Tests

**Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for
C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's
worker default) with 30k-deep nesting and must return `defer:` instead of crashing;
shallow files are untouched; the latch resets between runs; the OS bounds are sane on the
main thread and describe a small thread's own stack.

**`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`,
fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel
suites):
- every default-routed language (all 20) survives a 60k-deep expression on the main thread
  — clean result or the wasm fallback's partial result, never a crash;
- the reporter's exact 16,384-brace C file is indexed (partial) on the main thread;
- 200-deep expressions in every language still take the kernel path clean (the guard never
  trips on normal code);
- inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's
  `deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0,
  and a normal file still extracts natively;
- end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c`
  exits 0 and records both files, with both functions.

**Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`,
`kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard
never fires on the parity fixtures.

**Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build):
`deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0.
Main thread: `deep.c` / `deep100k.c` → wasm partial with
`Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a
3,000-branch `else if` chain stay on the kernel path with clean results.

**Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved
main/new ×3, `codegraph init`, macOS arm64):

| repo | main (median) | guarded (median) | nodes / edges |
|---|---|---|---|
| express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical |
| redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical |

Within run-to-run noise, as expected for one TLS load + compare per recursion entry.

**Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** —
the reporter's platform and the glibc `pthread_getattr_np` bounds path:

```
=== platform ===
Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov  4 21:21:47 UTC 2025 aarch64 GNU/Linux
v22.22.3
-rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node

=== reporter repro (issue #1581): 16,384-brace deep.c, codegraph init ===
│
└  Done

init exit code: 0
  file: deep.c
  file: deep100k.c
  file: ok.c
  function: add
  function: bar
  function: foo

=== worker probe: kernel raw extract in a default 4 MiB worker ===
deep.c: deferred
deep.c: worker exitCode=0
deep100k.c: deferred
deep100k.c: worker exitCode=0
ok.c: kernel nodes=2
ok.c: worker exitCode=0

=== cargo test stack:: (glibc pthread_getattr_np bounds path) ===
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s

=== vitest: kernel-deep-nesting + kernel-scaffold ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms
Test Files  2 passed (2)
Tests  18 passed (18)
```

(The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.)

**Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM,
`GetCurrentThreadStackLimits` path)**:

```
head: cbf8485 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581)
=== cargo build --release (win32-arm64) ===
    Finished `release` profile [optimized] target(s) in 2m 04s
staged: 35086848 bytes
=== cargo test (stack guard unit tests) ===
test stack::tests::normal_files_are_untouched_by_the_guard ... ok
test stack::tests::os_bounds_are_sane_on_this_platform ... ok
test stack::tests::small_stack_reports_its_own_bounds ... ok
test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok
test stack::tests::latch_resets_between_runs ... ok
test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.49s
=== reporter repro: codegraph init on a 16,384-brace deep.c ===
└  Done
init exit code: 0
=== vitest: deep-nesting + scaffold (CODEGRAPH_KERNEL_EXPECT=1) ===
✓ __tests__/kernel-scaffold.test.ts (10 tests) 55ms
✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 67239ms
   ✓ every default-routed language survives a 60k-deep expression on the main thread 52801ms
   ✓ inside a default-sized (4 MiB) parse worker, through dist/ > defers a 60k-deep expression in every default-routed language 13050ms
   ✓ end-to-end: codegraph init on a repo holding the deep file > exits 0 and records deep.c alongside the normal files 936ms
Test Files  2 passed (2)
Tests  18 passed (18)
```

(The end-to-end test is what reads the Windows index back through `node:sqlite` — `files` = `deep.c`, `ok.c`; functions `add`, `foo`.)

Full `npm test` on this branch (macOS arm64, kernel staged): **190 files passed, 3,185 tests passed, 10 skipped, 0 failed.**

Clippy note: `cargo clippy` on the current toolchain (1.92) reports 18 pre-existing lints
(`manual_contains`, `unnecessary_to_owned`, …) in walker code this PR only touched by
inserting guard lines; none are in `stack.rs`/`lib.rs`. Left alone to keep the diff
reviewable.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
2026-08-26 10:38:29 -05:00
Colby MchenryandGitHub 7963672689 fix(rust): resolve self.field.method() on the field's declared type instead of a same-named method (#1585) (#1599)
Fixes #1585. **Stacked on #1596** (the base branch is `fix/1588-rust-impl-type-qualification`; this PR's own diff is the second commit). Merge #1596 first, then retarget/merge this one.

## What was wrong

```rust
impl Outer {
    pub fn run(&mut self) {
        self.inner.run();      // inner: Inner
    }
}
```

produced `Outer::run -> Outer::run` — recursion the source doesn't contain. The extractor collapsed every `self.<field>.<method>()` receiver to the bare method name (`run`), so the resolver only ever saw `run` and exact-matched the nearest same-named method — the calling method itself, or a method of an unrelated type. Nothing marked the edge as a guess, and no row stayed in `unresolved_refs`, so a consumer had no way to tell.

The same happened when the field's type isn't a project type at all (`its: std::vec::IntoIter<_>` → `self.its.next()`, `matcher: Regex` → `self.matcher.is_match()`): the bare `next` / `is_match` attached to whatever local method shared the name. ripgrep had 279 self-edges on `main`; the issue lists three sites, all of this shape.

(The issue's C++ control — "`Outer::run -> Inner::run` resolves correctly" — doesn't actually hold on `main`: `inner.h` is classified as C by the `.h` heuristic, so `Inner::run` never exists and the C++ repro self-edges too. That's #1592, fixed separately.)

## What this does

Rust struct fields are not graph nodes, so the field's type can only come from the struct's declaration text. This follows the Go 2-hop precedent exactly (`matchGoFieldChainCall`, #1276), including its exclusivity rule:

1. **Extraction (TS walker + native kernel, identical, parity-tested):** a call whose receiver is `self.<field>` keeps the owner-field shape — `self.inner.run()` is emitted as `self.inner.run`. Deeper chains (`self.a.b.m()`), call receivers (`self.f().m()`), parenthesized receivers and bare `self` keep the bare name, exactly as before.
2. **Resolution (`matchRustSelfFieldCall`):** owner type = the calling method's qualified-name prefix (`Outer::run` → `Outer`); the field's declared type is read from the owner struct's **own declaration lines** (comment-stripped, line by line — same discipline as the Go helper); the method is resolved **and validated** on that type by `resolveMethodOnType` (confidence 0.85, `instance-method`).
3. **Exclusive:** when the field is declared with an external type, a generic parameter (`T`), a container that doesn't auto-deref (`Option`/`Vec`/`Mutex`/…), or can't be found, the ref **stays unresolved** — it never falls through to the bare-name strategies. That is the safe behaviour the issue asks for, and it is what #1276 already chose for Go.

`rustFieldTypeName` looks through exactly the layers Rust's method-call auto-deref looks through: references (`&`, `&'a mut`) and the owning smart pointers `Box`/`Rc`/`Arc`. `Box<dyn Source>` yields the trait, whose method node the interface-impl synthesizer then fans out to every implementation. `Option<Inner>` is left alone — `self.inner.take()` is Option's method and must not become `Inner::take`.

Why it stacks on #1596: the owner is taken from the method's qualified name, which for a generic/lifetime impl was the trait's name before that fix.

## Measured on ripgrep (110 `.rs` files, #1596 build vs this branch)

| | #1596 | this PR |
|---|---|---|
| nodes | 4029 | 4029 |
| `calls` self-edges | 279 | **146** (none of the `self.<field>` shape remain — 116 bare-receiver, 30 other dotted) |
| `self.<field>.m()` calls resolved through a validated field type | — | **292** (`DecompressionMatcher::command -> GlobSet::matches`, `Parser::find_long -> FlagMap::find`, `Haystack::path -> DirEntry::path`, …) |
| `self.<field>.m()` calls left unresolved | — | **417** — every sampled one is a std/container method: `self.commands.push`, `self.child.wait`, `self.pre.is_some`, `self.colors.clone`, `self.path_terminator.unwrap_or` |
| `calls` edges total | 9150 | 8878 (the 272 removed are the former bare-name guesses for those 417) |

The issue's three sites: `walk.rs:824` now resolves to `IgnoreBuilder::add_custom_ignore_filename` (was a self-edge); `walk.rs:1195` (`self.its.next`, `IntoIter`) and `globset/lib.rs:983` (`self.matcher.is_match`, `Regex`) are parked as unresolved instead of guessed.

The issue's repro gives `Outer::run -> Inner::run` (`instance-method`, confidence 0.85) on both the kernel path and `CODEGRAPH_KERNEL=0`.

## Tests

- `__tests__/extraction.test.ts`: only the single-hop `self.<field>.<method>()` call keeps the prefix; deeper / call / parenthesized / bare-`self` receivers and a local receiver are unchanged.
- `__tests__/resolution.test.ts` (end-to-end, Cargo layout): the issue's repro → `Outer::run -> Inner::run`, no self-edge; an external field type (`std::vec::IntoIter`) with a local `next` decoy → no edge at all; `Box<Inner>` and `&'a mut Inner` resolve, `Option<Inner>` does not (even though `Inner` declares the method); a generic `T` field → no edge; genuine `self.run()` recursion keeps its self-edge; the #1588 repro's `UsesFile::go` / `UsesBuf::go` resolve to `FileSource::read` / `BufSource::read`, and a `Box<dyn Source>` field lands on `Source::read` with the synthesizer fanning out to both impls.
- `__tests__/fixtures/kernel-parity/torture.rs` grows the receiver shapes; all 15 kernel parity suites pass against the rebuilt kernel (147 tests).
- Full `npm test` on this branch: 189 files, 3187 passed, 9 skipped, 0 failed.

Re-index after upgrading.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
2026-08-26 10:37:50 -05:00
Colby MchenryandGitHub 12f7a59f26 fix(rust): qualify generic/lifetime impl methods by the implementing type, not the trait (#1588) (#1596)
Fixes #1588.

## What was wrong

The receiver of an `impl` block — the name that qualifies its methods, owns the `contains` edge, and sources the `implements` edge — was found positionally: the **last bare `type_identifier` child** of the `impl_item`. That works for `impl Source for FileSource`. But once the implementing type carries parameters it parses as a `generic_type`, and the only bare identifier left is the **trait's**:

```rust
impl Source for FileSource         →  FileSource::read   ✓
impl<T> Source for BufSource<T>    →  Source::read       ✗  (should be BufSource::read)
impl<'a> Iterator for Parents<'a>  →  Iterator::next     ✗
impl Trait for &Foo                →  Trait::method      ✗
```

Two consequences, both reproduced on `main`:

- `BufSource::read` did not exist in the graph, so `resolveMethodOnType("BufSource", "read")` and "who calls `BufSource::read`" had no answer, and every generic implementation of a trait collapsed onto the same trait-qualified name.
- Because the impl's method carried the trait's qualified name, the interface-impl synthesizer treated the impl **body** as a second trait declaration and emitted a dispatch edge from it (`Source::read -> FileSource::read`, registered at the generic impl's line — a body of `{ 0 }` containing no call at all).

The native kernel (`rustlang.rs`) mirrored the positional rule deliberately, bug-for-bug, to hold byte-parity with the TS walker — its header said "preserve, never fix via the grammar's trait:/type: fields". So the fix has to land on both sides at once.

## What this does

Both extractors now read the grammar's **named fields** instead of scanning children. One shared rule (`rustImplTypeName` in `languages/rust.ts`, `impl_type_name` in the kernel), applied to `impl_item.type`:

| implementing type | node | receiver |
|---|---|---|
| `Foo` | `type_identifier` | `Foo` |
| `Foo<T>` / `Foo<'a>` | `generic_type` → its `type` field | `Foo` |
| `m::Foo` | `scoped_type_identifier` → its `name` field | `Foo` (was: no receiver) |
| `&Foo` / `&'a mut Foo` | `reference_type` → its `type` field | `Foo` |
| `(A, B)`, `dyn Tr`, `*const T`, `u32`, fn types | anything else | none — extracted as plain functions, exactly as before |

The `implements` back-reference reads `impl_item.trait` (full text, so `fmt::Display` and `From<u32>` keep their spelling) and bails when the field is absent (inherent impl). Everything else — the no-scope impl quirk, the source-order `contains` owner scan, method extraction — is untouched; the `contains` edge simply lands on the implementing type now instead of the trait.

The kernel header comment, the parity test's description, and the two design docs that documented the quirk as "preserve" are updated to say what changed.

## Measured on ripgrep (110 `.rs` files, `main` build vs this branch)

| | main | this PR |
|---|---|---|
| nodes / methods | 4029 / 2202 | 4029 / 2202 |
| impl methods qualified by a **trait** name (node outside that trait's extent) | 61 | **0** |
| `Iterator::*` methods | 2 | 0 |
| duplicate method qualified names | 77 | 42 |
| synthesized `interface-impl` edges originating **outside** any trait declaration (the phantom fan-outs) | 38 | **0** |
| synthesized `interface-impl` edges originating at a real trait declaration | 33 | **52** |
| plain (non-heuristic) `calls` edges | 9098 | 9098 |

So the synthesizer lost every phantom edge and *gained* 19 legitimate fan-outs to implementations it could not previously see as implementations. `contains` edges went 5237 → 5224: the 13 removed were trait→impl-method edges produced by the mis-qualification.

The issue's repro now gives `BufSource::read` at line 12, `BufSource -> Source`, and both synthesized edges registered at the declaration (line 2) — identical on the kernel path and with `CODEGRAPH_KERNEL=0`. (The remaining `UsesFile::go -> BufSource::read` exact-match guess there is the separate `self.field.method()` receiver problem, #1585, which stacks on this.)

## Tests

- `__tests__/extraction.test.ts` (Rust Extraction): method qualified names for generic / lifetime / reference / scoped / generic-trait impls; the trait's qualified name names exactly one node; `implements` refs come from the implementing type for every shape; the `contains` edge lands on the type; tuple / `dyn` impls keep producing plain functions with no `implements` ref.
- `__tests__/resolution.test.ts` (end-to-end): `Source::read` names only the declaration; dispatch fans out to **both** `FileSource::read` and `BufSource::read`, every synthesized edge registered at line 2; neither impl body sprouts a synthesized call.
- `__tests__/fixtures/kernel-parity/torture.rs` grows all the new impl shapes; `kernel-rustlang-parity` (LF + CRLF) passes against the rebuilt kernel.
- `CODEGRAPH_KERNEL_EXPECT=1 npx vitest run __tests__/kernel-*.test.ts` — all 15 suites, 147 tests pass.
- Full `npm test`: 3180 passed, 9 skipped, 1 failed — `mcp-daemon.test.ts > daemon idle-times-out after the last client disconnects`, a 30 s timing test that passed on re-run in isolation (the machine was running four parallel suites and kernel builds at the time); unrelated to extraction.

Re-index after upgrading to pick up the corrected names.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
2026-08-26 10:36:53 -05:00
Colby McHenry d289bf84d3 Merge main into fix/union-declarations-not-indexed
Resolves the CHANGELOG conflict — main and this branch each prepended a
bullet to [Unreleased] > Fixes; both are kept. Everything else auto-merged,
including src/mcp/tools.ts, which main reworked heavily for the explore
allocation/displacement work (CG-28/31/36/38) while this branch added the
`union` kind to its container sets.

Verified on the merged tree with the native kernel built: 3070 passed,
9 skipped, 0 failed.
2026-08-07 21:26:53 -05:00
Colby MchenryandGitHub 99f2ebf0d1 Merge pull request #1527 from colbymchenry/bugfix/CG-38
CG-38: guarantee an agent-named symbol renders, wherever it sits
2026-08-07 13:19:25 -05:00
Colby McHenry 2c708caf7c Merge branch 'main' into feature/CG-35 2026-08-06 21:17:56 -05:00
Colby McHenry 89c53ddf24 fix(explore): guarantee an agent-named symbol renders, wherever it sits (CG-38)
`codegraph_explore` never returned `queueMessage` (L1087) or
`flushQueuedMessages` (L1102) from a 1,414-line file, on a symbol bag or a
prose question, even with that file at rank #1 holding 67% of the envelope —
the agent got a same-stem `QueuedMessage` interface at L70 and had to Read the
file for the functions it had named. Pre-existing at every build including
pre-epic (controlled bisect, index held fixed).

Two independent causes:

1. `buildFlowFromNamedSymbols` returns the Flow prose AND the set of node ids
   the agent named — and the latter is the whole guarantee, since it injects a
   named def into its file's cluster ranges at importance 9. Its bail-outs
   returned EMPTY, zeroing the identity whenever there was nothing to PRINT.
   Two sibling closures that never call each other produce no chain, no synth
   hop and no boundary, so both defs lost importance 9 and the file rendered
   from its head. `identityOnly()` now separates the two, gated on
   shape-precise tokens so a prose word that exact-matches a callable cannot
   promote itself.

2. The ceiling trim filled in SOURCE order, so an over-ceiling render always
   dropped the END of a large file first. The shrink HAD kept both symbols
   (1022-1121); the trim cut back to 839. `windowToCeiling` now takes the
   spine call site plus every importance>=9 member as focus lines, tries the
   full ceiling first, and splits the held-back reserve evenly with
   carry-forward — greedy-in-source-order reproduced the bug one level down.

The shrink's loose size estimate is left alone deliberately, and the comment
now says why: making it exact was built and measured WORSE (it stops at the
last member that fits whole and the released bytes carry forward to
lower-ranked files, costing payroll-go's `s.store.Upsert`). `bound()` clamps to
the ceiling anyway, so the slack costs no bytes; it just must not pick the
survivors, which is what the trim now handles.

The measurement gap this closes: every existing probe is aggregate — envelope
share, per-file spend, source totals, file counts — and all are green on a
response that returns 25K from the right file and omits the named function.
`probe-named-symbol.mjs` checks the definition LINE against the response's
rendered lines, per symbol.

Suite envelope byte-identical to main on all six repos; probe-allocation 4/4,
no starvation flags; 180 files / 2,997 tests green. Fixture: 7/7 fail on main,
7/7 pass here, deterministic over 4 runs per arm.
2026-08-06 21:11:51 -05:00
Colby McHenryandClaude Opus 5 07338ff12e docs(benchmarks): record CG-38 as open, and correct the regression claim
The epic record said nothing was open. CG-38 is: agent-named symbols in the
tail of a large file never render, which the epic's probes cannot see because
none of them measures whether the named symbol appeared.

Also corrects a wrong claim made while investigating it. The epic was said to
have regressed its own motivating query; that comparison varied the index as
well as the engine. A controlled bisect holding the index fixed shows the
pre-epic engine rendering 12 lines and CG-36 rendering 463 — the epic strictly
improves the case, and the symbols render at neither.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 20:29:14 -05:00
Colby McHenry 10f1ac601a docs(benchmarks): record the CG-36 cluster-starvation measurement
The issue blamed the density tiebreak; both real cases lost on maxImportance,
so ranking was left alone. Full before/after table, the one cost (okhttp's
rank-6 file, squeezed out by reservations that were already structurally
over-subscribed), and what ships to keep it measurable.
2026-08-06 15:13:35 -05:00
Colby McHenryandClaude Opus 5 76ab1fe130 docs(benchmarks): record the CG-24 epic resolution
Four shipped fixes, one open defect (CG-36), and five issues closed because
measurement contradicted them. The headline is that the reported symptom was
not an explore bug at all — it was a degraded index (CG-33), and the reported
query answers correctly on a clean rebuild with no explore change.

Records the two traps that cost real time and are now guarded in tooling: the
nonexistent .codegraph/graph.db path that sqlite3 silently creates, and
ab-new-vs-baseline.sh swapping src/ mid-run so a commit captures baseline
sources.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:46:11 -05:00
Colby McHenryandClaude Opus 5 9efae0f8f2 fix(explore): damp ambient declaration files on flow queries (CG-28)
A file that declares nothing but types and that nothing in the index depends
on — a hand-written ambient `.d.ts` of global shims, vendored typings, module
augmentation — cannot answer a flow question: no bodies, no call edges, no
behaviour, nothing typed by it. But the identifiers it declares are exactly the
generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`,
`ReadableStream`), so on term overlap it out-scored the implementation. Measured
on the new fixture: rank #1 and 51% of delivered source, with the flow's own
entry file pushed out of the response entirely.

Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that
opened this is already handled by CG-25's banner detection, worth 15-46 points
of envelope share across four flow queries. CG-25 credited; only the un-bannered
case needed anything.

`rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken
as the STRONGER of it and the generated penalty rather than multiplied — one
property two signals see must not be charged twice. Detection is structural, not
by extension, and four conditions deep. Two of them were forced by measurement:
requiring every symbol to be type-level takes the corpus flag rate from 1-18%
(which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's
locale tables) down to 0-4%; requiring that nothing depends on the file
separates an ambient shim from a working types module, and without it the rule
demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate.

A query that NAMES a declared type is exempt, so a question about a type still
reaches its declaration at full weight. Precise tokens only, so "…the file
body…" cannot exempt a `Body` interface it never meant to name; this needs its
own set because `namedSeedIds` is callable-only and a type never becomes one.

Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md:
6-repo envelope sweep byte-identical against a clean baseline build, zero
ambient files reach the candidate set on VS Code across five queries, corpus
flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 14:35:56 -05:00
Colby McHenry 91cb5b4317 measure(explore): the factory-closure envelope premise does not hold (CG-27)
CG-27 asked whether the >50%-of-file envelope drop should cover `function` /
`method`, so a `createFoo()` factory returning an object of closures stops
merging every closure inside it into one cluster. Measured on a hermetic
fixture, it should not, and the issue is closed as obsolete with CG-30 credited.

Two mechanisms already absorb the shape. shrinkCluster orders members by
(importance desc, size ASC) and refuses any member that overruns the cap once
something is kept, so a file-spanning member is only selected when it is the
sole member of the top importance tier — eight of nine query shapes never
selected it at all. When it IS selected, CG-30 windows it on whole lines, so
the file still delivers bounded, readable source (6 of 9 closure definitions
in that configuration).

Dropping the range instead SPLITS the file, and only the first-chosen cluster
may be shrunk: a trivial 7-line cluster won the density tiebreak and the
answer-bearing cluster was dropped whole — rank-#1 file 7,539 chars and 7 of 11
closures to 397 and none. Reaching the same intent more carefully (defer the
envelope MEMBER inside shrinkCluster, leaving clustering untouched) is noise:
69 vs 68 closure definitions across nine query shapes. Nothing shipped.

Adds the fixture, the probe, a standing gate on the outcome, and the record —
including a real defect the measurement exposed on the epic tip: django's
query.py leaves 8,212 of 10,135 unspent and drops a score-290 cluster to keep a
score-14 one. Filed separately.

No behaviour change, so no CHANGELOG entry.
2026-08-06 14:07:22 -05:00
Colby McHenryandClaude Opus 5 03893b0ab9 CG-33: converge incremental sync with a full rebuild
A live, auto-synced index did not converge to a clean rebuild of the same
tree — 4.3% of distinct edges wrong in both directions on this repo's own
index, overwhelmingly `calls`, which is what flow queries traverse and what
explore's file ranking weights. Silent: nothing warned, and the symptom read
as "codegraph isn't very good" rather than "this index needs rebuilding."

Two causes, and the fix needed both. Resolution binds a reference to one of
the same-named definitions PROJECT-WIDE, so a definition appearing or
vanishing changes the correct answer for references in files the sync never
touches — and those references resolved successfully once, which deletes
their unresolved_refs row, leaving nothing to revisit them with (#1240's
retry only revisits refs parked as failed). Separately, when nothing
disambiguated the candidates the winner came down to rowid, i.e. the order
files happened to be WRITTEN, which differs between a scan-order full index
and a sync that appends each file as it changes. That second one is why
re-resolution alone could not converge: re-resolving against the identical
graph still picked a different candidate.

So getNodesByName now orders by (file_path, start_line) — a property of the
code, not of the write order — and sync computes a definitionDelta and
re-opens the resolution edges whose answer it may have invalidated,
re-inserting each as the reference that created it for the orphan sweep to
bind against the post-sync graph.

The delta compares `file\0name` pairs per file rather than one name set over
the batch: a commit that adds `collect` to a new file while an unrelated
changed file already defines `collect` cancels out of a batch-wide set, and
that miss was the largest residual class in the first measurement.

Conservative where the failure modes are asymmetric — a wrong deletion is a
permanent edge loss, a missed rebind is only residual drift. Edges without a
refName stamp are never touched (nothing to restore them from), sources the
sync already re-extracted are skipped, and a per-name ceiling declines the
generic names. Edges are deleted before the sweep re-inserts, since
INSERT OR IGNORE against idx_edges_identity would otherwise keep both rows
when a reference rebinds elsewhere.

Replaying real commits of this repo through sync, then diffing against a
rebuild: 16 commits 48 -> 0; 80 commits 1,634 -> 361, with the actively
misleading direction (stale edges the index keeps asserting) 671 -> 2.
Index and sync wall-clock are unchanged; the ORDER BY costs 18% per uncached
name lookup, which never reaches wall-clock because the resolver memoizes it.

The 357-edge residual at 80 commits is one pre-existing class: refs to
generic names (`push`, `join`) parked above #1240's per-name retry ceiling,
which a rebuild resolves into cross-language garbage — a TS test file
"calling" an R method. Converging there would mean manufacturing wrong edges,
so it is left alone. And no drift metric in `codegraph status`: it cannot be
computed without the rebuild it would be recommending, and a proxy would fire
on that residual and train users to ignore it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 04:32:40 -05:00