56dfdb0655490682e0351afbf32ebb850b44ab39
66
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Colby McHenryandClaude Opus 5
|
56dfdb0655 |
feat(ui): dead code and islands — what nothing reaches, and everything that could still reach it (CG-59)
A Dead code screen and a mark on the Map, both drawn from one derivation in src/graph/dead-code.ts so a second surface can never disagree with the first. The SQL half is four lines — no incoming edge but `contains`. It returns ~2 500 candidates on this repository and the shipped list is 20; everything in between is the feature. A candidate is dropped the moment there is any reason to believe something outside the graph reaches it: exported symbols and header declarations, test and generated files, abstract and interface members, anything carrying a `decorates` edge, overrides of an ancestor's member, names the language calls by itself, vendored directories, files nothing in the index reaches (those are islands, and the Map says so instead), names the resolver failed to resolve somewhere, and names shared with a symbol that IS referenced — the mis-resolution that leaves a used method with a self-edge and its twin with nothing. The last rule is the only one that is not a graph query: before a claim is made, the declaring file and every file that reaches it are read and the identifier counted, which is what catches the references the extractor never recorded (`this.handleMessage.bind(this)`, a call inside an object literal, a shorthand property). Every subtraction is counted and printed under the list with the scale it came from, and the caveat line above it never collapses: the claim is "no static reference in the index", not "unused". On the Map a module nothing depends on keeps its stroke and says so in its count line, and tool-generated files and modules recede to ink-4 there, in the map's file list, in search results and on the file screen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
94f4e287e6 |
feat(ui): entry points — routes, executable files and tests as flow starting points (CG-54)
`#/entry` answers "where does anything start" at full length, and turns any row that names a symbol into a flow. Server. `/api/entrypoints` gains `frameworks` (from `getDetectedFrameworks`), a `tests` list, a `routes` limit of its own, and a cache keyed on the index build — nothing here is read from disk, so unlike `/api/source` a cached answer cannot be stale about drift. `routes.items` is now a `WireList` like every other list on the payload. Routes carry where the URL is REGISTERED as well as where it is served: `getRoutingManifest` selects the route node's id, file and line, and `buildRoutes` splits the verb off the name against a fixed list (never "the first word", which would take the head off a file-routed `/blog/[slug]`). All four payroll-go routes register in one router file and three are served from another — group by the handler file and one router becomes two groups plus an orphan. `isTestFile` is split into `isTestPath` (test filename and directory conventions) + the non-production catch-all, byte-identical at every existing call site. The Tests list uses the narrow half: an example, a benchmark or a fixture is off-target for ranking but is not a test, and a heading that says "Tests" must not quietly count them. Tests rank by REACH — distinct other files touched — because Go, Rust and Java put test work inside functions where a module-level-calls ranking sees nothing. Two read-only engine queries make that affordable: `getFileReachCounts` (the mirror of `getFileDependentCounts`, driven from `nodes` by path so the cost follows the files asked about rather than the edge table) and `getFileNodes`. Viewer. `ui/src/lib/entry-model.ts` folds the four lists into file groups — pure, and `panel.rows` stays exactly the sections it draws. `EntryView` + `EntrySection` render them with the caller rail's `.filegroup` / `.row` shapes rather than a second visual language for the same idea. A row that names a callable symbol carries a `Flow ›` chip; the other end is typed or picked with `→ here` on another row. File and test rows carry none: `/api/flow` searches by name, and a file has none the path finder can look up. A project with fewer than three resolvable routes gets no Routes heading at all, not an empty one. Typing into the search box now also returns matching entry points under their own heading below the symbol matches, so a URL comes back with its handler attached; rows already in the results are dropped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
ecd6e1cd15 |
feat(ui): live refresh and drift banners — the viewer keeps up with the project (CG-53)
`GET /api/events` is a server-sent-event stream the viewer holds open for the life of the page. Two signals, two things the browser could not know: changed source files touched on disk, before any sync — the drift banner index the graph moved, naming what the sync re-indexed — the live refresh The server WATCHES and never syncs: the project tree through the engine's own FileWatcher with a notify-only syncFn, the index through one non-recursive fs.watch on the data directory settled at 400 ms. Both start with the first subscriber and stop with the last, so a viewer nobody has open costs no watch descriptors. Nothing polls, on either side. Drift is now parity with codegraph_node (#1474) rather than an absence. `/api/source?ondrift=current` serves a drifted file's CURRENT bytes flagged `showing: 'current'`, and the three screens that can say so switch off everything anchored to the old line numbering — gutter ports, call-site links, call arcs, the callee rail's anchoring — while keeping the source. The banner is paper-2 with a hairline rule, never amber: amber belongs to the untested badge. Also fixes a stale read this exposed. A long-lived reader holds an LRU of nodes by id that only its own writes invalidate, so `/api/node/<id>` kept answering with a symbol another process's sync had deleted while `/api/search` beside it said it was gone. GraphSession now drops the read caches when the database (or its WAL) has been written, and the Symbol view follows a symbol whose id changed because an edit above it moved its start line, carrying the trail across. Measured on a live viewer: banner 360 ms after a save, toast 440 ms after `codegraph sync` returns, 0 requests in 4 idle seconds, and the client gives up reconnecting after ~90 s with "Not live" rather than hammering a dead port. |
||
|
Colby McHenryandClaude Opus 5
|
bd99c5e99a |
feat(ui): the whole file — full source with gutter ports and intra-file call arcs (CG-52)
The File view gains a Source tab: the file itself, top to bottom, with the Symbol view's line grid, gutter ports and call-site links, a line-anchored callee rail, and — in the left margin — an arc for every call that stays inside the file, drawn from the calling line to the callee's definition line. The arcs are the point. Source order is already a layout, chosen by whoever wrote the file, so a file's internal call structure can be drawn with no algorithm placing anything. Crabviz's idea, in the one place it is legible. Everything is arithmetic, not measurement. The Symbol view queries the laid-out DOM to place a callee row beside its line; a 6 820-line file cannot afford that. Here a line is exactly 20px at `10 + (n - 1) x 20`, so ~90 line elements exist at a time and the arcs, ports, rail rows and connectors are all functions of a line number. `src/mcp/tools.ts` scrolls at a 16.6ms median frame. - `GET /api/filecode/<path>` — outline, one call group per (caller, callee) PAIR with its call-site lines, unresolved references, and the file's length. The source is NOT in it: it pages through `/api/source` 800 lines at a time with a discarded 150-line lead-in, so a page starting inside a block comment does not render prose as code, and so the ports and arcs are complete from the first frame while the text fills in behind them. - `intraFileCalls` is counted over the groups actually returned, so the header and the picture under it cannot disagree once a cap bites. - Above 40 arcs the diagram narrows to the symbol under the pointer (or the one the scroll position is inside) and the header states the total. Accent is for the pointer only, never for the filter. - Sticky outline rail at >= 1400px, following the reader down the file. - `QueryBuilder.getUnresolvedReferencesInFile` — one indexed lookup instead of one per symbol; `buildOutlineEntries` lifted out of `/api/file` so both readings of a file draw the same rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
6d0f60f32c |
feat(ui): the Map — the repository at module granularity, layered from the graph (CG-49)
`GET /api/map` rolls the whole edge table up to module granularity in one `GROUP BY`, and the Map tab draws it: one box per directory, dependencies pointing down, nothing placed by hand. Two decisions carry the screen. The vertical order rests on each link's `declared` weight — the edges resolved through an import, a qualified name, an inheritance clause or a typed receiver — not on its raw count. Bare name matching resolves `run`, `push` and `finish` across unrelated directories, and layering on raw counts put `src/db` directly under `src/bin` on this repository's own index. On declared edges the same data reproduces the pipeline CLAUDE.md describes, with a third of the mutual pairs. When too few links carry a declared edge to describe a project, the layout falls back to raw counts and the side panel says so. And the aggregation is a single scan. Grouping by the symbol names as well as the modules costs nothing extra — the join is what is expensive — so one query yields both the link weights and the tooltip's symbol pairs. Measured against this index inflated to 800k edges: 1.28s for one scan against 1.89s for two, which is the difference between meeting and missing the cold budget on a ten-thousand-file repository. Cached answers come back in ~3ms. Nothing is dropped silently: thin links are hidden until a module they touch is selected and counted in the panel, uncertain references are excluded from every number on screen and the total is printed, and mutual dependencies, module loops and file-level circular imports are listed rather than straightened away. An edge that still points up after layering is drawn dashed on selection instead of being reversed or removed. The layout — cycle-breaking, longest-path layering, barycenter ordering, ports — is a pure function of the payload in `ui/src/lib/map-model.ts`, so the tests toggle and the selection cost no round-trip and the same project always draws the same picture. Svelte Flow supplies pan, zoom and fit; never a layout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
87afc50e76 |
feat(ui): the search palette, entry points and a trail that survives the URL (CG-45)
Search: `/` or ⌘K focuses the box; results arrive grouped by kind with their
glyph, signature and file:line, ↑/↓/Enter walk them, Esc dismisses. A group
appears where its best result did, so flattening the groups reproduces the
ranking the keyboard walks — the panel's flat item list IS that concatenation.
A flow question ("how does X reach Y", "X -> Y") is recognised and searches
both endpoints with a note, rather than offering a row that would land on the
phase-2 Flow view.
Entry points answer "where do I start" on the empty screen and in the resting
palette, all derived from the graph: routes, files that run something at module
level (the engine records a top-level statement as an edge out of the file node,
which is what makes src/bin/codegraph.ts the root of the CLI flow — ranked by
calls x the files they reach, so a registration table calling into itself does
not outrank the CLI), and the most depended-on symbols. Tests are excluded from
both derived lists.
Trail: hops record the direction they were walked (→ into a call, ← up to a
caller), clicking one truncates back to it, Clear keeps the place instead of
throwing it away, and the whole walk travels in the URL. A shared or reloaded
trail arrives as ids, so hops learn their names back through a new batch
endpoint and a session name cache — without it, walking back across a
truncation redrew earlier hops as raw hashes. "Read as flow" stays hidden until
there is a Flow view to send it to.
New endpoints: /api/entrypoints and /api/nodes. New engine reads:
getTopCallingFiles, getFileDependentCounts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
5cecaabfc2 |
feat(ui): the Symbol view — callers, gutter-ported source, line-anchored callee rail (CG-44)
The core screen of `codegraph ui`: who calls a symbol on the left, its verbatim body in the middle with a port on every line that has an outgoing edge, and what it calls on the right — each callee row placed beside the line that makes the call, with a hairline connector between them. The callee rail is the part that is not a list. A row wants to sit at the centre of its first call-site line and is pushed down only when that would collide with the row above, so the rail keeps source order; the connector still runs to the real line, so the displacement is visible rather than silent. Positions come from measuring the laid-out DOM, so they are recomputed on resize, on font load and whenever a fold opens. Honesty is carried in the drawing, not in a footnote: a filled port means the resolver matched something on that line and a hollow one means it only guessed; uncertain connectors are dashed and their targets fold away behind their count; synthesized edges are dashed differently and tagged with the mechanism that made them; references that leave the index are text with a soft underline rather than links to nowhere, and they are counted. Long bodies keep their head plus a window round every call site — windowed on graph edges only, since a function calling `console.log` two hundred times would otherwise window round every line and buy nothing. Containers over 80 lines show a members outline with per-member fan-in/fan-out instead of 700 lines of braces. Two small additions to the read-only API this needed: * `/api/node` gives every outline member its own fanIn/fanOut (two batched queries for the whole outline). A class's own fan-out is nearly always zero because its methods do the calling, so without these the outline cannot say which member carries weight. * `/api/stats` gains `blastScale` — the denominator the blast bar is drawn against, so one symbol's radius reads as wide or narrow *for this repo*. It is measured across the index's 24 most-depended-on symbols (found with a new `getTopDependedOn`, distinct dependents rather than edges), memoised against the index stamp, and reported as sampled; a symbol wider than the sample becomes the scale instead of overflowing the track. Verified against a real index in a real browser: parity with the prototype on `CodeGraph.sync` (259 lines, 27 callee rows, no overlaps), `GraphTraverser` (20-member outline), a 773-line function (26 windows, 78 connectors), light and dark, hover linking in both directions, keyboard-only navigation, and reflow on resize and on fold toggles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
951ba3678a |
feat(ui): read-only JSON API over the index for the viewer (CG-42)
Six endpoints under `/api/`, one per screen, each answering in a single
round-trip in the spirit of `codegraph_explore` — the viewer should never
have to ask a follow-up question to finish drawing a pane:
/api/stats index state, graph counts, frameworks
/api/search?q= ranked, kind-grouped symbol search
/api/node/<id> rails, members, tests, blast radius
/api/source?file=&from=&to= verbatim source + a drift verdict
/api/file/<path> outline and import rails
/api/routes URL -> handler, when there is one
It is a reader of the existing schema: no extraction or resolution changes.
It mounts on the `api` seam `startUiServer` already exposed, so it sits
behind the CG-41 loopback boundary — Host allowlist, no CORS headers,
GET/HEAD only — and every read out of the repository goes through
`resolveProjectFile`, ahead of the index lookup so a traversal is refused
as a traversal rather than reported as "not indexed".
Three properties the endpoints are built around:
- No N+1. The engine's busiest symbol has 545 incoming edges; resolving
those one `getNode` at a time is 545 queries. Every edge list is
resolved with one batched lookup, which needed four additive read-only
query methods (`getNodesByIds`/`getFanIn`/`getFanOut` on `CodeGraph`,
plus batched outgoing/incoming edge fetches and unresolved-reference
reads). `/api/node` on `LRUCache.get` answers in ~10 ms.
- Capped lists, honest totals. 545 callers cannot all be rows, so caller
groups cap at 300 — but `total` is always the real number, and the
ordering puts the useful end first (same file, then production code,
then tests). Every count in the payload is the length of a list the
same payload returns, so a badge and its rail cannot disagree.
- Nothing overclaims. Source that drifted on disk since the last index
sync is omitted rather than sliced at line ranges that may now point at
a different symbol; calls that leave the index are counted instead of
silently shortening the callee rail; imports that never resolved are
named; and a test-coverage claim reports whether its search actually
finished. `/api/routes` says a project simply is not routed, and
refuses a `limit` below three because the engine's manifest would
answer that question wrongly.
Tests: 45 against a real indexed fixture over a real loopback server,
covering every endpoint's shape, the drift verdict in all three places it
surfaces, search ranking and the filter grammar, the refusals, and the
capping/latency behaviour at 500 callers. The issue's own acceptance case
— `lru-cache.ts` `get` under 100 ms — runs against this repo's index when
one is present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
 Colby MchenryandGitHub
|
41c10750e0 |
fix(erlang): give same-name different-arity functions separate arity-qualified nodes (#1610) (#1615)
Fixes #1610. Also fixes #1358 (the `<<binary>>` arity miscount in behaviour dispatch, reported separately and hit by the same code path). ## Problem Arity is part of an Erlang function's identity — `f/1` and `f/2` are unrelated top-level definitions — but the extractor merged consecutive same-name `fun_decl`s regardless of arity. Reproduced on main exactly as reported: - adjacent `f(X) -> …. f(X, Y) -> ….` → **one** node spanning both, with the first definition's signature; - interleaved `f/1, g/0, f/2` → two nodes with **identical** `qualified_name`; - `cowboy_req`'s `header(Name, Req) -> header(Name, Req, undefined).` → a **self-loop** `header → header`, with the `-spec` for `/3` swallowed by the merged span; - `-export([f/1])` marked every arity exported. ## Fix - **One node per (name, arity).** Clauses of the same name+arity still merge (that part of the old behavior was correct); a different arity starts a new node. `qualifiedName` carries the canonical spelling — `mod::f/1` — while the node **name stays bare** so search and bare-name matching are unchanged. - **`-export` and `-spec` are per-arity.** `-export([f/1])` exports exactly `f/1`; a spec sitting between two arities attaches to the arity its signature names. - **Refs carry the call-site arity** wherever it's statically known: local `f/1`, remote `mod::f/2`, `fun f/1` / `fun mod:f/1` values, `gen_server` dispatch (`handle_call/3`, `handle_cast/2`), and spawn/apply MFA lists (`spawn_link(?MODULE, work, [A, B])` → `work/2`). - **The matcher resolves only to the named arity** — same file first (a local call targets its own module) — and when no definition of that arity exists it resolves to **nothing** rather than a sibling arity: silent beats wrong. An arity-less dynamic-MFA ref resolves only when the module defines exactly one arity of that name. - **Behaviour dispatch** selects the implementer node of the site's arity, and the arity counter now skips `<<1,2,3>>` binary-literal commas per its own docstring (#1358) — `Mod:decode(<<1,2,3>>, Opts)` counts 2, not 4. - **`codegraph_explore` / `codegraph_node`** accept the written `mod:fn/3` spelling against the new arity-qualified names (the issue's measured `cowboy_stream_h:request_process/3` shape). ## Validation Minimal fixtures (all three reported shapes) now index as `gap::f/1` + `gap::f/2`, distinct `inter::f/1`/`inter::f/2`, and a real `deleg::header/2 → deleg::header/3` edge with no self-loop. Cowboy (fresh `--depth 1` clone, this build vs unmodified main build): | | main | this PR | |---|---|---| | nodes | 3,668 | 3,748 (+80 — the arity splits; no explosion) | | erlang function nodes | 2,850 | 2,930 | | behaviour dispatch edges | 38 | **44** | | `cowboy_req::header` | one node, span 420–425, /3's spec lost | `header/2` (420–421, its own spec) + `header/3` (424–425, its spec) | | delegation | self-loop | `header/2 → header/3` | `calls` edges drop 6,059 → 5,656: a sample of every removed pair shows the false-positive class the issue predicted — out-of-repo/BIF calls (`length/1`, `error/1`, `quicer:*`) that previously name-matched onto unrelated same-named in-repo functions now stay unresolved. Tests: new arity coverage in extraction + a new arity-resolution integration suite + a #1358 binary-literal behaviour test; updated existing Erlang expectations to the arity-carrying spellings. Full suite: **3,018 passed, 0 failed**. No migration: an existing Erlang index picks the new shape up on its next re-index (`codegraph sync` / re-`init`). Erlang is wasm-only (not in the native kernel), so there is no kernel-parity surface. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
1d9de88ef1 |
feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting (#982) (#1463)
* feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting matchesNonProductionDir hardcodes example/sample/fixture/benchmark/demo, so a peripheral tree only the project knows about — optional-skills/, scripts/ — gets no de-prioritization. When helpers there carry generic symbol names, an exact name match hands them a large bonus and they crowd out the product code that answers the query (#982). deprioritize is the RANKING counterpart to exclude: those paths stay indexed and findable, they just stop outranking first-party code. It is deliberately distinct from the corpus-frequency discount, which keys on a name being common and is near-inert on #982's own repro where only two symbols are named usage. The -15 path penalty alone is not enough, and measuring showed why: on that repro a usage() helper sits at 74.8 against 51.2 for the top product symbol, so -15 lands at 59.8 and still leads. The path penalty is additive and the name bonus it must counter is additive and larger. A de-prioritized path is saying its symbol NAMES are not the answer, so the exact-name bonus is damped to 0.25x there as well — damped, not zeroed, so the tree still ranks when it genuinely is what you asked for. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(config): read deprioritize lazily and apply it in explore too Review of the first cut found two real defects. The matcher was built once in wireLayers(), which runs only from the constructor and from reopenIfReplaced(). The MCP server keeps one CodeGraph per project root alive for its whole lifetime, so editing codegraph.json appeared to do nothing until the process restarted -- exclude and include do not behave that way. The predicate now reads loadDeprioritizePatterns() per call (mtime-cached, one stat) and memoizes the compiled matcher on the pattern array's identity. A regression test writes the config after opening the project and fails on the old code. Explore passed no matcher to scorePathRelevance at either of its two call sites, so the setting only half-applied -- and #982's reproduction rows B, C and D are all codegraph explore, which made this the surface the issue actually reports on. Both sites now pass it. Explore's hard early-continue filters and its non-production budget cap are deliberately NOT joined: those REMOVE content, and deprioritize is a ranking lever by definition. README narrowed accordingly -- it previously claimed this extends the built-in list, which overstated it. Also from review: scorePathRelevance takes a boolean rather than a predicate (the caller already evaluated it, and it was being invoked twice per result), the predicate body is exception-guarded so a bad path can never take a search down, the misplaced const moved out from between imports, two vacuous test assertions tightened, and tests added for the single-penalty invariant, the deliberate isTestQuery asymmetry, and a query that genuinely targets the de-prioritized tree. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(search): derive the deprioritize name-bonus damping instead of picking it (#982) The 0.25 scale was a guess. On a 62k-node django index it measurably breaks the "discount, don't erase" rule the lever is built on: exact-name queries for symbols that live only in the de-prioritized tree (child, parent, method) fall behind mere prefix matches (children, all_parents, method_decorator). The prefix arm of nameMatchBonus tops out below 40, and a de-prioritized node also takes the -15 path penalty, so 80 * SCALE - 15 > 40 is the bound that keeps a damped exact match ahead of a prefix match at any corpus shape. 0.75 clears it; crowd-out removal is nearly identical to 0.5 (39 vs 40 of 88 peripheral top-10 slots cleared on django), so the deeper discount bought almost nothing and cost the invariant. Two tests pin the bound, including one that fails at the old 0.25. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
9219967e43 |
perf(search): seek the name index for exact-name lookups (#1542)
`nodes` carries two name indexes and neither can serve
`WHERE name = ? COLLATE NOCASE`: `idx_nodes_name` is BINARY-collated, and
`idx_nodes_lower_name` is an expression index the planner only matches against
the same expression. All three whole-name lookups in the query layer were
written that way, so each one degraded to a full table scan
(`EXPLAIN QUERY PLAN` reports `SCAN nodes`).
The LIMITs on those queries do not rescue them. SQLite can only stop early once
it has produced LIMIT rows, and the two dominant cases never get there: a query
word that names no symbol at all, and a name with only a handful of definitions.
`searchNodes` runs its supplement once per query term; `findNodesByExactName`
runs two passes per symbol extracted from the question, and extraction is
generous, so a plainly-worded question issues a dozen full scans.
Written as `lower(name) = lower(?)` the same predicate seeks
`idx_nodes_lower_name`. Measured on four indexed repositories, baseline vs fix
in one process (the only difference being how the predicate is spelled):
query "how does the retry backoff work" findNodesByExactName searchNodes
gin (2.5k nodes) 1.27ms -> 0.18ms 3.1 -> 2.6ms
Alamofire (4.5k nodes) 2.39ms -> 0.22ms 4.9 -> 4.0ms
excalidraw (11k nodes) 10.54ms -> 0.17ms 10.4 -> 5.8ms
django (62k nodes) 49.91ms -> 0.17ms 27.6 -> 4.9ms
The seek is flat across all four; the scan grows with the corpus. A one-word
query into `searchNodes` on django is unchanged (~20ms) because a single term's
scan is not what dominates it there.
Lowering the parameter in SQL rather than in JavaScript is deliberate. SQLite's
`lower()` and NOCASE both fold ASCII only, while JavaScript's `.toLowerCase()`
folds Unicode; comparing a JS-lowered parameter against `lower(name)` would
silently stop matching non-ASCII identifiers that NOCASE used to match.
`getNodesByLowerName` is spelled the same way for the same reason. It already
sought the index, but as a bare `lower(name) = ?` it took a pre-lowered
parameter on trust: any input carrying an uppercase letter returned nothing at
all. This is behaviour-neutral for its one caller — `matchFuzzy` lowers in
JavaScript before calling, and `lower()` over an already-lowered string is a
no-op, verified over the ASCII and non-ASCII cases alike. It closes the trap for
the next caller; the non-ASCII gap on the `matchFuzzy` side is a resolution
change and is deliberately not bundled here.
Result sets are unchanged, including which rows the LIMITs keep: entries under
one key in the expression index are ordered by rowid, the same order a table
scan produces. Verified over 14,400 lookups (top-400 names of the four
corpora, probed as stored / upper / lower, against all three call sites) with
zero differences, and end-to-end above with identical result ids.
Tests assert the planner's verdict rather than a wall-clock number, so they are
deterministic: they intercept the SQL each call site prepares and require an
index seek, with a guard that the lookups actually ran. Reverting any call site
turns them red.
Co-authored-by: Colby McHenry <me@colbymchenry.com>
|
||
|
|
d8f2eeaddf |
fix(db): loop-append dense unresolved-ref result rows; make stripped-salvage visible (#1558) (#1576)
Real-world validation of #1575 on indexes damaged by the released v1.5.0 binary surfaced both of these. getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter limit but appended each chunk's RESULT rows with a spread — every row becomes a call argument, so a dense recovery sync (the #1541 self-heal re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit and killed resolution mid-sync with "Maximum call stack size exceeded", leaving the graph 226k edges short until another sync resumed the orphans (and that sweep resolves measurably worse than the batched path — see the follow-up issue). The failed-ref retry loader had the identical pattern on unbounded result rows. Both append with a loop now (#1558). The #1575 stripped-salvage warning also never rendered: init's summary prints only index_partial warnings and counts only hard errors, so a run with salvaged files still read as fully clean — and with no hard errors the detail wasn't written to errors.log either. Salvage entries now carry code 'salvaged_stripped', the summary prints a visible warning naming the files, and errors.log is written for salvage-only runs. Validated on real corpora with full-graph dumps: healthy-path inits stay byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a determinism control); a realistically-damaged index (41 wiped + 5 missing files, damage generated by the released binary) heals in one plain sync to identical per-file counts and an edge set within the normal incremental residual; pathological mass damage (52% of the repo) completes without crashing. New regression test reproduces the RangeError on the old code with 200k pending refs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
2c708caf7c | Merge branch 'main' into feature/CG-35 | ||
|
Colby McHenryandClaude Opus 5
|
9efae0f8f2 |
fix(explore): damp ambient declaration files on flow queries (CG-28)
A file that declares nothing but types and that nothing in the index depends on — a hand-written ambient `.d.ts` of global shims, vendored typings, module augmentation — cannot answer a flow question: no bodies, no call edges, no behaviour, nothing typed by it. But the identifiers it declares are exactly the generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`, `ReadableStream`), so on term overlap it out-scored the implementation. Measured on the new fixture: rank #1 and 51% of delivered source, with the flow's own entry file pushed out of the response entirely. Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that opened this is already handled by CG-25's banner detection, worth 15-46 points of envelope share across four flow queries. CG-25 credited; only the un-bannered case needed anything. `rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken as the STRONGER of it and the generated penalty rather than multiplied — one property two signals see must not be charged twice. Detection is structural, not by extension, and four conditions deep. Two of them were forced by measurement: requiring every symbol to be type-level takes the corpus flag rate from 1-18% (which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's locale tables) down to 0-4%; requiring that nothing depends on the file separates an ambient shim from a working types module, and without it the rule demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate. A query that NAMES a declared type is exempt, so a question about a type still reaches its declaration at full weight. Precise tokens only, so "…the file body…" cannot exempt a `Body` interface it never meant to name; this needs its own set because `namedSeedIds` is callable-only and a type never becomes one. Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md: 6-repo envelope sweep byte-identical against a clean baseline build, zero ambient files reach the candidate set on VS Code across five queries, corpus flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
03893b0ab9 |
CG-33: converge incremental sync with a full rebuild
A live, auto-synced index did not converge to a clean rebuild of the same tree — 4.3% of distinct edges wrong in both directions on this repo's own index, overwhelmingly `calls`, which is what flow queries traverse and what explore's file ranking weights. Silent: nothing warned, and the symptom read as "codegraph isn't very good" rather than "this index needs rebuilding." Two causes, and the fix needed both. Resolution binds a reference to one of the same-named definitions PROJECT-WIDE, so a definition appearing or vanishing changes the correct answer for references in files the sync never touches — and those references resolved successfully once, which deletes their unresolved_refs row, leaving nothing to revisit them with (#1240's retry only revisits refs parked as failed). Separately, when nothing disambiguated the candidates the winner came down to rowid, i.e. the order files happened to be WRITTEN, which differs between a scan-order full index and a sync that appends each file as it changes. That second one is why re-resolution alone could not converge: re-resolving against the identical graph still picked a different candidate. So getNodesByName now orders by (file_path, start_line) — a property of the code, not of the write order — and sync computes a definitionDelta and re-opens the resolution edges whose answer it may have invalidated, re-inserting each as the reference that created it for the orphan sweep to bind against the post-sync graph. The delta compares `file\0name` pairs per file rather than one name set over the batch: a commit that adds `collect` to a new file while an unrelated changed file already defines `collect` cancels out of a batch-wide set, and that miss was the largest residual class in the first measurement. Conservative where the failure modes are asymmetric — a wrong deletion is a permanent edge loss, a missed rebind is only residual drift. Edges without a refName stamp are never touched (nothing to restore them from), sources the sync already re-extracted are skipped, and a per-name ceiling declines the generic names. Edges are deleted before the sweep re-inserts, since INSERT OR IGNORE against idx_edges_identity would otherwise keep both rows when a reference rebinds elsewhere. Replaying real commits of this repo through sync, then diffing against a rebuild: 16 commits 48 -> 0; 80 commits 1,634 -> 361, with the actively misleading direction (stale edges the index keeps asserting) 671 -> 2. Index and sync wall-clock are unchanged; the ORDER BY costs 18% per uncached name lookup, which never reaches wall-clock because the resolver memoizes it. The 357-edge residual at 80 commits is one pre-existing class: refs to generic names (`push`, `join`) parked above #1240's per-name retry ceiling, which a rebuild resolves into cross-language garbage — a TS test file "calling" an R method. Converging there would mean manufacturing wrong edges, so it is left alone. And no drift metric in `codegraph status`: it cannot be computed without the rebuild it would be recommending, and a proxy would fire on that residual and train users to ignore it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
16e17495f4 |
feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
02c0e2c935 |
fix(db): stop watchdog-killed sessions from leaking the SQLite WAL without bound (#1431) (#1490)
A SIGKILL'd process (the #850 liveness watchdog, OOM, a crash) leaves its WAL on disk; the next session appends to the same file; and nothing ever truncated it — PASSIVE checkpoints fold frames but keep the file at its high-water mark, and the one shrinking path (a clean last-connection close) is exactly what a killed-daemon world never takes. Observed at 25.6 GB on a 5.46 GB DB, growing until the disk filled. - journal_size_limit on every connection: resetting checkpoints now clip the WAL back to the cap instead of leaving it at its high-water mark. - healOversizedWal() fired from every DatabaseConnection.open: off-thread PASSIVE fold + TRUNCATE when the leftover WAL exceeds the cap (64 MB, CODEGRAPH_WAL_HEAL_MB to override). Single-flight per connection with bounded retries — concurrent passes defeat each other (each checkpoint sees the other as a busy reader). - Daemon/direct MCP watchdogs now pass progressPaths (DB + WAL), extending the #1231 slow-disk deferral to the long-lived server so a healthy daemon mid slow statement isn't SIGKILL'd — fewer kills, fewer leaked WALs. - codegraph status shows WAL size (human + JSON) and warns when it dwarfs the DB; daemon.log lines and the watchdog kill notice now carry ISO timestamps so kills can be placed in time. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
971a5a0483 |
perf(resolution): worker connection recycling — WAL-depth writes-under-readers fix, superphase −11.4% at 8c (#1362)
The §7a.6 anomaly probed to its mechanism with five discriminating runs (§7a.7 table): main-thread B-tree writes triple under attached readers because READERS PIN WAL checkpoint progress — the deep WAL taxes every writer page operation (deletes 42.6s pool-off vs 118.8s pool-4 on identical hardware; an aggressive 64MB valve recovers the writes but overpays +129s in full-park folds; the v2 cache resurrection was falsified — long-tail name traffic is uncacheable at any capacity). Fix: workers close and reopen their read-only connections every 8 batches at the double-buffer's worker-idle boundary (ResolverPool.recycleWorkers + QueryBuilder.rebind + a cadence call). Reopens are sub-millisecond, resolver caches survive (only prepared statements re-prepare), and the existing checkpoints advance instead of parking. Failed recycle downgrades to sequential, same as a failed fan-out. Measured (8c pool-4, linux v7.2-rc2, cadence 25 → 8 iterated): resolution superphase 715.0 → 633.6s (−11.4%), envelope best 14.8min, recreate 59.7 → 45.3s. Byte-neutral everywhere: git dumps byte-identical old-vs-new, linux dump sha 6dd1185b reproduced (10,446,478 lines), counts 2,049,153/6,413,518, suite 2517 green. 2c unchanged by construction (no pool → no recycling). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
7cc23668b5 |
perf(resolution): batch-loop de-quadratic — keyset reads, changes-based guard, DB-scaled valve caps + resolve profiler (#1339)
The §7a.2 per-ref profile overturned the assumption the whole arc was built on: resolveOne owns only ~93s of the kernel-scale ~433s batch loop. Loop-stage attribution (CODEGRAPH_RESOLVE_PROFILE, shipped here) named the rest: backpressure folds 111.2s, count guard 93.9s, batch reads 54.6s, deletes/inserts/marks ~84s, settle 85.7s. - Non-progress guard O(remaining)→O(1): the per-batch COUNT(*) walked every remaining pending row (O(N²/batch) per run, 93.9s). The cleanup queries now return summed SQLite , and zero-removals-from-claimed-work is the guard signal — the DIRECT evidence the count diff inferred (a mismatched-name resolver makes keyed cleanup no-op ⇒ changes=0). A real COUNT runs only on that suspicious path and arbitrates exactly as before. - Batch reads OFFSET→keyset (54.6s→O(batch)): OFFSET re-walked the accumulated failed-row prefix every read; seeking past the last-seen rowid is prefix-independent and enumeration-order identical. - WAL valve caps scale with DB size (env still wins): every fold re-writes hot pages (#1231 in bounded form — 111.2s at the flat 256MB cap); soft=clamp(dbSize/4, 256MB, 2GB) trades ~4× fewer folds for a transient WAL ≈ project size. - CODEGRAPH_RESOLVE_PROFILE: per-outcome resolveOne histogram + loop-stage attribution, main + workers, off by default. Gates: dubbo dump byte-identical; suite 2,491 passed / 4 skipped (kernel required). Kernel-scale payoff run lands in the plan doc next. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
cf38ef65af |
perf(synthesis): fan dynamic-dispatch passes across the resolver pool, byte-identical graphs (#1321)
The ~36 independent synthesis passes (callback/event/framework wiring) ran sequentially on the indexer's main thread — 2.0s of a 4,402-file Java repo's index, and the stage where kernel-class repos die (#1212). They now live in an explicit registry (SYNTH_PASSES) and, when the resolver pool is alive (>=150k-ref repos), fan out across its read-only workers: dubbo synthesis 2,024ms -> ~900ms (-55%), total fresh init 13.5s -> 11.9s. Graphs verified byte-for-byte identical on both the pool path (dubbo) and the sequential path (excalidraw). Why this is safe: no pass's edges persist until the ordered merge, so every pass sees the same committed post-resolution DB state in either mode, and results merge in registry order regardless of completion order — the first-seen dedup is unchanged. The pool now survives through synthesis (destroy moved after it) instead of being torn down moments before the one stage that could reuse it. Robustness: a pass that fails on a worker (crash, OOM) is retried on the main thread — a synthesizer blow-up now costs one worker instead of the whole index, which is half the #1212 story on very large repos. Also: ref-row cleanup deletes now run as one transaction with a cached statement instead of one implicit commit per 500-row chunk (mechanically fewer WAL commits; matters most on HDD-class storage). A set-based rewrite of failed-ref parking was tried, measured ~zero on NVMe, and dropped — the remaining persist cost is edge-index B-tree maintenance, not statement dispatch. SYNTH_PROGRESS_STEPS now derives from the registry (passes + fixed marks); the pin test counts registry entries plus literal __mark sites. Suite green (2444). Sequential-path timing unchanged on excalidraw. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a2f3c31a97 |
perf(resolution): defer checkpoints and double-buffer persist during resolution, byte-identical graphs (#1320)
Fresh init on a 4,402-file Java repo (dubbo): 18.2s -> 13.5s (-26%), with the resolution phase going 12.6s -> 7.9s (-38%). Graphs verified byte-identical on both the pool path (dubbo) and the sequential path (excalidraw). Two changes: 1. The fastInit+pool path restored WAL for the resolver workers but left wal_autocheckpoint at its default, so the persist loop inline-checkpointed hot pages all phase long (#1231's pathology inside resolution — measured at 58% of resolution wall). Checkpointing is now deferred behind the bounded valve and folded once at maintenance, mirroring the deferWal path. 2. The resolution loop is double-buffered: batch k+1 is prefetched (OFFSET past batch k's still-pending rows, under an explicit ORDER BY rowid) and fanned out across the pool while batch k's ref cleanup runs on the main thread. Batch settle-waits dropped 2572ms -> 117ms. Correctness invariant found by the byte-identical gate and now documented in the loop: batch k+1's resolution READS batch k's edges (resolveMethodOnType walks supertype chains over extends/implements edges that resolution itself inserts), so edges must persist BEFORE the next batch fans out; only the ref cleanup overlaps. Also extends the CODEGRAPH_SYNTH_TIMINGS instrumentation with phase labels (grammar-init, parse-loop, fts-rebuild, resolver-reinit, resolution, callback-synthesis) and pool timings (worker open/resolve, per-batch mode, persist), so the next profile is one env var away. Suite green (2444). Sync path timings unchanged. Pool floor re-validated: forced-on at 40k refs is still net-slower, so the 150k threshold stands. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
5736e24bb6 |
perf(index): faster fresh indexing + parallel reference resolution, byte-identical graphs (#1305)
* perf(index): ~34% faster fresh indexing, byte-identical graphs Profiling a fresh init on a medium TS repo (excalidraw, 657 files) showed the main thread as the critical path: per-row SQLite statement calls, repeated import-resolution walks, and per-row FTS trigger firings, with the parse workers ~75% idle behind it. This lands the semantics-preserving tranche of fixes: - Multi-row batched INSERTs (nodes/edges/unresolved refs/name segments) behind cached per-batch-size prepared statements; row order preserved, so rowid-based resolution determinism (#1015) is unchanged. - storeFileBundle: one transaction per file instead of four; nested transaction() calls now flatten (BEGIN-in-BEGIN previously threw, so no caller depended on nested rollback). - Dedicated store-writer thread for the fresh-DB bulk path (bundles applied in file order on a single writer connection; main thread does no DB work during the parse loop). Kill switch: CODEGRAPH_NO_STORE_WORKER=1. - Bulk FTS mode: drop the nodes_fts sync triggers during the bulk load, rebuild once at the end; crash inside the window self-heals on the next open. - Per-context memos for resolveImportPath/findExportedSymbol + a per-file exported-symbol index, invalidated exactly where clearCaches() already resets the resolver's own caches. - Fast-init on completely fresh DBs (journal in memory, no fsync until the index completes; interrupted init re-runs from scratch). Kill switch: CODEGRAPH_NO_FAST_INIT=1. - MaybeYield returns undefined on the not-due path so per-ref yield checks stop paying a promise + microtask hop each. - Parse pool prewarm for bulk indexing; compile-cache enabled at CLI and worker entry points. Excalidraw fresh init: 5.11s -> 3.36s median (n=5, warm cache, M-series). Graph dumps byte-identical across init, re-index, and sync paths; full suite green (2403 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(resolution): parallel reference resolution with canonical admission Fan resolution batches across a pool of read-only worker threads, each hosting a full ReferenceResolver over its own SQLite connection; results are admitted on the main thread in chunk order, so edge insertion order, row cleanup, failure parking, and deferred post-pass queues are exactly the sequence the single-threaded loop produces. Per-ref inputs match the baseline because the sequential path already resolves each batch against the state committed BEFORE that batch. Validated byte-identical on excalidraw (pool forced on) and apache/dubbo (4,048 Java files): dubbo full index 39s -> 19s (2.05x) with identical graph dumps (91,495 nodes / 223,953 edges). The pool only engages when total pending refs clear a threshold (default 150k, CODEGRAPH_PARALLEL_RESOLVE_MIN to tune, CODEGRAPH_NO_PARALLEL_RESOLVE=1 to disable): measured on a ~58k-ref repo the workers' boot CPU contends with resolution on the same cores and makes indexing slower, so small repos keep the sequential path. When fast-init left the DB in memory-journal mode, WAL is restored before resolution only when the pool will run (readers + rollback-journal writers don't mix). Also: sqlite adapter readOnly open support. TreeCursor spine rewrite of the body walker was built, measured neutral on real repos and equal in a 20k-child microbench (web-tree-sitter's namedChild(i) is not quadratic in this binding), and rejected — per-node JS<->WASM marshaling is the floor, which a traversal swap cannot remove. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e871c49a31 |
fix(resolution): clean up processed refs by row id so batch boundaries can't drop sibling call sites (#1269) (#1270)
Post-batch cleanup deleted resolved refs (and parked failed ones) by (from_node_id, reference_name, reference_kind) — no line/col. When one caller had several call sites to the same callee and a batch boundary split them, the first batch's cleanup removed every row with that key, including later-batch siblings that were never attempted — their edges were silently never created. On nlohmann/json this ate 422 real call edges (write_cbor's 38 to_char_type calls indexed as 11). Refs loaded from unresolved_refs now carry their row id through resolution, and all three persist paths (sync resolveAndPersist, the yielding retry pass, the batched drain loop) delete / mark-failed by exactly that id. The key-tuple methods remain only as the fallback for hand-built refs from the public API. Failed-parking gains the same precision: outcome can differ per call site (receiver inference reads the ref's line), so a sibling must not inherit another row's failure. Also untracks the zz-scratch local test files that slipped into #1268 and gitignores the pattern. Validation: red-green regression test (5 sites, batch size 2 — old code kept 2 edges, fix keeps 5); nlohmann/json re-index is a strict superset of the previous edge set (0 lost, 422 recovered, spot-checked against source). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
9d0cd3a7d1 |
fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) (#1249)
* chore: ignore .kommandr/ directory * fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) Incremental sync scoped reference resolution to the changed files' own refs, and a completed pass deleted every ref it failed to resolve — so a symbol change in one file could never repair references in UNCHANGED files, in either direction, until a full re-index: - New-export case: a.ts imports/calls `greet` before b.ts defines it. The failed refs were deleted at index time; when b.ts later gained `greet`, nothing revisited a.ts — the calls/imports edges stayed missing while status reported a clean index. - Removal case: when a re-index (or file deletion) dropped a symbol, the incoming edges cascade-deleted and the callers — whose resolved refs had been consumed — never got a chance to rebind to an alternative definition or reconnect when the symbol returned. Fix, sharing one lifecycle: - Schema v8: unresolved_refs gains status ('pending'/'failed') and name_tail (last dotted segment, so `h.greet` is findable by `greet`). Both resolver persist paths now park unresolvable refs as failed instead of deleting them. All pending-work readers (batched drain, non-progress guard, #1187 orphan sweep, status pendingRefs) filter to pending, preserving their invariants and keeping status honest. - Sync retry: after scoped resolution, failed refs whose name tail matches a symbol name now present in the changed files are re-resolved through a per-ref-yielding path (watchdog-safe, #1091 class). Names matching >500 failed refs are skipped as external/builtin noise (#999 rationale). - Removal side: createEdges stamps each resolution edge with its originating reference (metadata.refName, + refKind when kind promotion rewrote it). When the #899 restore misses a target or sync deletes a file, the dropped edge is resurrected as exactly that ref — re-resolved in the same sync (rebinding to an alternative definition) or parked failed until the symbol reappears. Edges without the stamp (pre-upgrade, synthesized) still drop silently: reconstructing from the target's plain name would strip receiver context and risk a rebind a full re-index would never make. - Pure-removal syncs clear resolver caches so a long-lived daemon can't resolve resurrected refs against the pre-removal graph. Validated: issue repro now yields a graph byte-identical to a full re-index; move/remove-readd/file-deletion scenarios all rebind or heal; baseline-vs-new A/B on express and gin shows identical node/edge counts and no timing regression (DB grows ~25% from the parked ref rows — pure cache, reset by any full re-index). 8 regression tests added. Fixes #1240 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
a3f90089e8 |
fix(indexing): bounded-memory yielding pipeline tail + daemon session fixes (#1212) (#1226)
Large-codebase indexing died at the end of "Resolving refs" two ways: watchdog kills of healthy work (24k-file Java on Windows, #1212 — third iteration of the #1091/#1122 class) and hard OOMs (Linux kernel scale, where v1.3.0 could not complete at any watchdog setting). Root causes: ~31 of 37 dynamic-edge synthesis passes ran start-to-finish with no yield points, several materialized whole-graph snapshots (kotlin expect/actual opened with getAllNodes() — 2M nodes in one array; the C fn-pointer pass retained every C file's contents twice plus every function node), and the post-index WAL checkpoint ran minutes of synchronous IO on the main thread, killing even a successful index at the finish line. The pipeline tail now follows the same discipline as the rest: never hold O(graph) in the heap, yield everywhere. - All synthesis passes stream node-kind scans (cursors, not arrays) and yield on time-budgeted checkpoints; language gates skip passes whose filters a project's file languages provably can't satisfy. - kotlin expect/actual filters SQL-side; c-fnptr caches are LRU-bounded, units stream one file at a time, and the all-functions array + write-only id map are gone; spring reads each .java once, not twice. - runMaintenance moved to a worker thread (own SQLite connection); per-file store commits chunk with yields behind a serialized flush chain (preserving #1015 file-order determinism); resolver warm-up streams the DISTINCT name set; resolution batch-tail and merged-edge inserts run in bounded sub-transactions. - Daemon: fixed a socket-handoff race that could leave a fresh MCP session permanently silent (client-hello tail unshifted into a flowing stream with zero listeners — the long-standing #662 test flake was this real bug); first tool call no longer queues behind the query pool's cold start (pool.ready gate). Validation: Linux kernel (70,129 files, 2.05M nodes, 6.4M edges) fully indexes in 27m8s on a 2-core/6GB container at default heap + default watchdog; llvm-project (180k files) completes under 1GB RSS including kill-and-sync recovery; synthesized-edge and full-graph parity are byte-identical vs baseline on elasticsearch/redis/vim; the ex-flaky daemon test passed 25/25 under load. Env-gated diagnostics kept: CODEGRAPH_SYNTH_TIMINGS pass/phase timings, CODEGRAPH_MCP_DEBUG hop tracing. Design record: docs/design/main-thread-stall-followup.md. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
7f325134e0 |
feat(extraction): add Nix language support with module-system option wiring (#324, #332 via #648 — carries #1084) (#1190)
Carries @TyceHerrman's #1084 as the functional base. Extraction + file wiring (imports/modules lists, callPackage), module-system option-path synthesizer, lexical-scope resolution gates, ABI-15 wasm rebuilt from upstream source. Validated on agenix, nix-darwin, home-manager, and nixpkgs (44,368 files, 3m49s, 1.30M nodes). Co-authored-by: Tyce Herrman <Tyce.Herrman@pm.me> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
35611b92bb |
fix(prompt-hook): close the segment-vocab integrity gaps (#1141, #1142, #1144, #1145, #1146) (#1150)
Five hardening fixes to the #1136 MEDIUM (graph-derived) tier: - #1141: updateNode() now writes the segment vocabulary like insertNode() does — framework post-extract renames (NestJS route prefixing) left the new name permanently unsearchable (the old rows orphaned, the backfill gated on an EMPTY vocab, so even a full re-index re-created the drift). - #1142: new CodeGraph.healSegmentVocabIfEmpty() — the hook opens the graph without sync, so a database migrated from pre-vocab schema kept the MEDIUM tier dormant until some unrelated sync ran. The hook heals on first use (one SELECT when populated; lock-aware, defers to a running sync) and records noop-vocab-empty when it can't. - #1144: a name whose only nodes are file/import kind is skipped instead of falling back to surfacing an import statement as a matched symbol; import specifiers no longer enter the vocab at all (shared isSegmentableKind gate across insertNode/updateNode/rebuild page query) since they can never be surfaced and only inflate rarity statistics. - #1145: plural variant folding is keyed on English plural spelling — bare-s plurals no longer mint a bogus -es sibling (services→servic), unambiguous sibilant-es plurals no longer mint a bogus -s sibling (classes→classe), trailing -ss singulars no longer strip (class→clas); genuinely ambiguous endings (caches/databases) still emit both keys. - #1146: getSegmentCoOccurrence folds variants to their original word inside the SQL (CASE mapping + COUNT(DISTINCT word)) so a plural pair of ONE word can't tie with a genuine two-word match and crowd it past the pre-fold ORDER BY/LIMIT; the JS re-check stays as the honesty layer. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e699ee9686 |
feat(prompt-hook): graph-derived gate tier + confidence-tiered injection + gate telemetry (#1136)
The keyword gate (#1126) can never know a repo's domain nouns. This adds the graph-derived tier the design discussion converged on: symbol names are split into prose segments at index time (name_segment_vocab, riding the insertNode write path), and the hook verifies a prompt's plain words against them — "the state machine des commandes" → OrderStateMachine, in any language whose technical nouns are Latin script. Confidence now decides HOW MUCH to inject, not just whether: - HIGH (keyword, or index-verified code token): full explore injection, unchanged — the validated adoption lever. - MEDIUM (segment matches only): a ~500-byte pointer naming the matching symbols; the AGENT writes the explore query. Never runs explore, so a fuzzy match can't inject 16KB of wrong-feature context. - Silent otherwise, as before. Precision is derived from the repo's own naming statistics plus measured FP fixes: co-occurrence (≥2 words on one name) always qualifies; a single word must be ≥5 chars, cluster across 2–25 names (singletons are prose coincidence: "deploy to production" → matchesNonProductionDir), match a multi-segment name, and not be an English function/filler word (the one place a word list is honest: identifiers are English, so only English prose collides). Every candidate is re-verified against nodes before being surfaced — vocab rows are proposals, deletions leave orphans by design, a full index rebuilds from scratch, and sync heals pre-upgrade databases (batched + yielding; emptiness captured at sync ENTRY so the sync's own writes can't mask the backfill). Schema v7 migration is DDL-only (instant; none of the #1067 row-churn hazards). Gate outcomes roll up as anonymous usage counters (prompt-hook-gate-<outcome>, names only, never content) through the existing telemetry pipeline — recall becomes measurable, and the counters are the agreed kill-criterion data for ever revisiting a local classifier. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
30dc303f4c |
fix(db): chunk deleteResolvedReferences IN-list under the SQLite param limit (#1001) (#1023)
deleteResolvedReferences bound every id into a single unbounded `IN (...)`, so a list longer than SQLITE_MAX_VARIABLE_NUMBER (32766 on the bundled node:sqlite) threw "too many SQL variables" — the one IN-list in queries.ts that #540 missed. It's reachable only through the exported QueryBuilder (library use): the internal resolution path uses deleteSpecificResolvedReferences, which binds per-row and is immune, so the CLI/MCP indexing pipeline was never affected. Wrap it in the same SQLITE_PARAM_CHUNK_SIZE loop every sibling query uses, and add a regression test (33k ids, past the real 32766 ceiling) that throws without the fix. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 JoshandGitHub
|
6110df8b76 |
fix(sync): preserve cross-file caller edges across callee re-index (#899) (#927)
`storeExtractionResult` deletes a re-indexed file's nodes via `deleteFile`, which cascades through `edges.FK ... ON DELETE CASCADE` to delete every edge whose source OR target is one of those nodes. Edges whose source is in the re-indexed file are re-emitted by the extractor, but edges whose source is in a *different* (unchanged) file are not — they are silently dropped. This is issue #899: re-indexing a callee file severs `calls`/`references` edges from callers that import it via module-attribute access (`pkg.mod.fn(...)`), so `codegraph callers fn` reports 0 callers for functions that have real call sites. A docstring-only edit on the callee is sufficient to trigger it. The bug affects every incremental path that routes through `sync()` / `indexFile()`: `codegraph sync`, the file-watcher auto-sync (which calls `sync()`), and the git sync hooks. `codegraph index` was already fixed by #894 (it now clears-then-rebuilds, so it's a full re-extraction, not incremental). `sync` remains the fast incremental path and still has the bug. Fix: before the delete, snapshot incoming cross-file edges paired with the target node's (name, kind). After re-inserting the file's nodes + same-file edges, re-insert the snapshot — re-resolving each edge's target to the re-indexed node's NEW id by (filePath, kind, name). Node ids are `sha256(filePath:kind:name:line)`, so any line shift in the callee file (e.g. a docstring-only edit above the symbol) changes every target id and a naive re-insert by old id would drop them all. Matching by (kind, name) is stable across line shifts; if the symbol was renamed/removed, no match is found and the edge stays dropped (correct). `insertEdges` still filters to endpoints that exist, so edges whose caller (source) was deleted are also dropped. Regression tests in `__tests__/sync.test.ts` model the RAGFlow production case: a `pkg/mod.py` with two callees, both called from `test/test_callers.py` via `mod.<fn>(...)`. The first test confirms a docstring-only edit that shifts the second callee's line preserves both incoming edges. The second test confirms renaming a callee correctly drops its old incoming edge (no phantom preservation against a non-existent symbol). |
||
|
|
75ae1e8bd9 |
fix(search): down-weight the project name in ranking — completes #720 (#748)
The per-word path fix (#745) brought the backend to parity but not above: the project name still gave the lexically-matching stack a residual dir match + an FTS class-name match, so a backend query that included the project name still ranked the frontend at/above the backend. Derive the project name from go.mod module / package.json name / repo dir, and treat a query word matching it as non-discriminative: drop it from path relevance and from codegraph_explore's PascalCase type-disambiguation bias (reporter's suggestions #1/#2) — unless it's the only query word, so a bare project-name search still scores. Narrow by construction: the down-weighting fires ONLY when a query word matches the derived project name (≥5 chars), so every query that doesn't name the project is byte-identical. On the reporter's repro the backend controllers now top a backend question that includes the project name; queries without it, bare project-name queries, and normal symbol queries are unchanged. Query-time only (no re-index). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
fd03f31b2c |
fix(cpp): resolve calls through singletons/factories/chained getters (#645) (#742)
A C++ method call whose receiver is another call's result — `Foo::instance().bar()`, `WidgetFactory::create().draw()`, `openSession()->run()`, or the same stored in an `auto` local first — lost the receiver's type during extraction. The callee degraded to a bare method name, so when two classes shared a method name the call silently resolved to whichever was indexed first (or not at all), corrupting callers / impact / trace with a plausible-but-wrong edge. Three parts: - Capture C++ return types (new nodes.return_type column, schema v5): the function_definition's `type` field, normalized — smart-pointer pointee unwrapped, void/primitives dropped. - Preserve the inner-call receiver in extraction: a C/C++ field_expression whose receiver is itself a call is encoded `inner().method` instead of dropping to the bare name. Other languages keep the existing behavior. - New resolution strategy (matchCppCallChain): infer the receiver's class from the inner call's return type, then resolve AND validate the method on it. Handles singletons/accessors, factories returning a different type, free-function factories, make_unique/make_shared/new/direct construction, single-level member chains, and namespace-qualified inner calls. A wrong inference yields no edge, never a wrong one. EXTRACTION_VERSION 2->3 (re-index to populate return types). Validated on the issue repro + spdlog: node count stable (no explosion), deterministic, and ~100 pre-existing wrong `.size()`-style edges removed. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
07af3db6c7 |
feat(impact): cross-language blast-radius coverage (22 languages + 14 frameworks) (#708)
Completes the cross-file dependency graph behind impact / affected / explore across all 22 supported languages and 14 web frameworks, validated on real-world repos (measured fair-coverage table added to the README). Per-language resolution + framework resolvers/synthesizers (Lua/Luau require, Shopify OS 2.0 Liquid sections, Delphi forms, Rust cross-module + Rocket macros, Swift Fluent, SvelteKit/Nuxt loader/component conventions, RN/Expo bridges). 0 cross-family false edges, full suite green (1187 passed). See #708. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 
|
7b62356f53 |
feat(cli): add version, indexPath, lastIndexed to status --json (#329)
Adds `version`, `indexPath`, and an ISO `lastIndexed` to `codegraph status --json`, plus a `CodeGraph.getLastIndexedAt()` library method. `agentCount` dropped (no clear consumer). Reworked from contributor PRs #333 and #480. Co-Authored-By: Javier Gómez <199902626+12122J@users.noreply.github.com> Co-Authored-By: Ran <8403607+eddieran@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
ddb1a8f72d |
fix: issue-triage quick wins (extraction, MCP probes, gitignore, CJK, impact) (#654)
Batch of small, localized fixes from an open-issue triage: - .codegraph/.gitignore now ignores everything but itself, so the database, daemon.pid, sockets, and logs stop showing up in git status (#492, #484) - MCP server answers resources/list and prompts/list with empty lists instead of -32601, clearing scary log lines in opencode/Codex (#621) - index SAP HANA .xsjs/.xsjslib as JavaScript (#556) and TS .mts/.cts (#366) - visit anonymous AMD/CommonJS/IIFE wrapper bodies so their inner functions and calls are indexed instead of coming up empty (#528) - batch the changed-file lookup so a huge first sync no longer hits "too many SQL variables" (#540) - list files with `git ls-files -z` so non-ASCII/CJK paths survive core.quotepath and are no longer silently skipped (#541) - attach Go methods on generic receivers (*T[P]) to their type (#583, RC1) - impact no longer climbs the structural `contains` edge, so a leaf symbol stops dragging in its sibling methods (#536) - README: explicit `codegraph install` step, run in a new shell (#631) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
2a22f9f55a |
fix(resolution): stream node-kind scans in synthesis to fix OOM on dense files (#610) (#653)
The callback/observer synthesizers loaded every function and method node into
memory at once (getNodesByKind('function'/'method')) before scanning them down
to a tiny matched subset. On a symbol-dense project that array is gigabytes, so
indexing spiked the JS heap and aborted with "JavaScript heap out of memory".
Add QueryBuilder.iterateNodesByKind (a lazy node:sqlite cursor) and stream the
synthesizer scans instead of materializing them. Parsing and reference
resolution were already bounded; only the synthesis enumeration wasn't.
Measured on 80 files x 14k functions (~1.1M nodes): peak RSS 3717 MB -> 1318 MB,
no OOM. Full suite green; synthesized edges unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
|
71935e37c2 |
feat(mcp): multi-module Go trace-quality + small-repo retrieval tuning (#494)
* feat(go): generated-file down-rank + gRPC stub-impl bridge + trace-failure inlining
Multi-pronged fix to make codegraph competitive on Go multi-module repos
(cosmos-sdk, etcd) where it previously lost or tied. Driven by an 8-question
agent-eval audit across cobra, gin, prometheus, cosmos-sdk, and etcd: the
baseline had codegraph losing ~60% on cost on cosmos-sdk and mixed on etcd
deep cross-module flows, while winning cleanly on the single-module and
non-protobuf-heavy repos.
Diagnostics ruled OUT `go.work` parsing as the gap (prometheus crushes
without it). The actual failure modes were generated-file noise warping
disambiguation, missing gRPC interface→impl bridge in structural-typing Go,
and trace's failure path triggering 3-5 follow-up tool calls instead of
inlining the material the agent needed.
Changes:
- New `src/extraction/generated-detection.ts` — path-pattern classifier
for `.pb.go`, `.pulsar.go`, `_grpc.pb.go`, `_mock.go`, `_mocks.go`,
`mock_*.go`, `.generated.[jt]sx?`, `_pb2(_grpc)?.py`, `.pb.{cc,h}`,
`.g.dart`, `.freezed.dart`. Applied as a stable sort tiebreaker in
`findSymbol`, `findAllSymbols`, `codegraph_search` (MCP + CLI),
`codegraph_explore` file ranking, and context formatter Entry Points /
Related Symbols / Code blocks. Cosmos's `msgServer.Send` now ranks #3
instead of #9 on a `Send` search.
- New `goGrpcStubImplEdges` synthesizer in `callback-synthesizer.ts` —
detects `UnimplementedXxxServer` structs in generated files, identifies
their RPC methods (excluding `mustEmbed*` / `testEmbeddedByValue` gRPC
markers), and emits `calls` edges to the matching methods on any
non-generated struct whose method-name set is a superset. Closes Go's
structural-typing gap that the existing `interfaceOverrideEdges` (Java /
Kotlin only) couldn't bridge. 467 bridge edges on cosmos-sdk; bank's
`UnimplementedMsgServer::Send` points to `x/bank/keeper/msg_server.go`
only, not to `msgClient` siblings or mock files.
- Trace-failure rewrite (`handleTrace`) — when no static path connects
endpoints, instead of telling the agent to call `codegraph_node` (a
3-4-call fan-out), inline both endpoints' bodies (120 lines / 3600 chars
per endpoint), their callers (≤6), and callees (≤8) in one response.
- Trace endpoint-pairing improvements — scores every `from`×`to`
candidate combo by shared directory prefix and tries the best-paired
pair first (the full candidate set, not just FTS top-5). A
less-canonical-path penalty (`enterprise/`, `contrib/`, `examples/`,
`vendor/`, `third_party/`, `deprecated/`, `legacy/`) ensures the
canonical-module pair wins even when a side-experiment shares more of
its directory prefix. Find-path probe budget capped at 20 pairs.
- Test-file deprioritization in `codegraph_explore` `isLowValue` — adds
suffix patterns (`_test.go`, `_spec.rb`, `.test.ts`, `.spec.tsx`,
`Test.java`, `Spec.kt`) alongside the existing directory-style patterns.
Otherwise etcd's `watchable_store_test.go` consumes 5K chars of explore
budget that should go to the hand-written flow source.
Tests:
- New `__tests__/generated-detection.test.ts` (4 unit tests) pins the
suffix patterns.
- New "Go gRPC stub→impl synthesis" integration test suite in
`frameworks-integration.test.ts` (2 tests): positive bridge from stub
to hand-written impl, AND the precision case (don't bridge to a
generated sibling like `msgClient` in the same .pb.go).
- Full suite: 1076/1076 pass.
Empirical (post-fix, n=2 average per question):
| Repo / Q | WITH | WITHOUT | Reads (W/WO) | Time (W/WO)
|-------------------------|------------|-------------|--------------|------------
| cobra (parse cmds) | $0.27 | $0.27 | 0 / 4 | 39s / 60s
| prometheus (scrape→TSDB)| $0.63 | $0.70 | 0 / 6 | 106s/143s
| cosmos-sdk Q1 (MsgSend) | $0.41 | $0.26 | 1 / 2 | 67s / 64s
| cosmos-sdk Q2 (Delegate)| $0.47 | $0.46 | 0 / 5 | 50s / 73s
| cosmos-sdk Q3 (gov tally)| $0.34 | $0.31 | 1.5 / 3 | 54s / 76s
| etcd Q1 (Put→raft) | $0.65 | $0.78 | 0 / 4 | 98s / 129s
| etcd Q2 (watch) | $0.36 | $0.50 | 0 / 4+ | 58s / 89s
Codegraph wins on reads + time on every question. Cost is mixed: 3 clean
wins, 3 tied (within 10%), 1 stubborn cost loss on the grep-favored Q1.
Compared to baseline, the cosmos-sdk cost-gap collapsed from -60% to -15%
on average, and Q3 went from a 75% loss to a tie. Raw run artifacts in
`/tmp/cg-finalv2-*/` and `/tmp/cg-final-*/`.
Memory written at `project_go_multi_module_audit.md` for the methodology
+ before/after numbers.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): auto-inline trace in codegraph_context for flow queries
When a codegraph_context task contains a flow keyword ("trace", "from",
"reach", "flow", "propagat", "how does", "how do") AND at least two
distinct PascalCase / camelCase identifiers, internally invoke trace
between the first two extracted symbols and splice the trace body into
the context response. Conservative trigger by design: false positives
waste one graph query; false negatives just fall back to the agent
calling trace itself (existing path-proximity wiring handles either
case).
Goal: collapse the agent's typical context → trace → explore sequence
into a single context call for clear flow queries, closing the
remaining cost-overhead gap on multi-call patterns. The path-proximity
+ less-canonical-path scoring + the trace-failure-inlined-bodies
behavior already let the inline trace land on the right endpoint pair
and return enough material that no follow-up codegraph_node/Read is
needed.
Doesn't fire on:
- cobra's "How does cobra parse commands and flags?" (no PascalCase
symbols) — verified in regression run, no behavior change ($0.260
WITH vs $0.257 WITHOUT, basically tied)
- queries where the agent doesn't call codegraph_context at all
(cosmos Q1 in the audit went search → trace → node → trace → node)
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): trace failure inlines TO file siblings to displace node fan-out
The cosmos-Q1 audit revealed a static-resolution gap: msgServer.Send's
*real* next hop is `k.Keeper.SendCoins` — an interface-method call on an
embedded field that tree-sitter can't resolve. The static getCallees list
for msgServer.Send is all utility/error functions (StringToBytes, Wrapf,
…). The actual flow (SendCoins → subUnlockedCoins → addCoins →
setBalance) lives entirely inside `x/bank/keeper/send.go`, which is also
where the TO endpoint (setBalance) lives.
When trace fails (no static path), inline the **top 5 functions/methods
in the destination file**, ordered by line-distance from the TO node.
This catches the flow that interface-method calls obscure — the
canonical "k.<Iface>.<Method>" pattern in Go, also relevant to Java
dependency-injection / Rails service-object dispatch / etc. where
interface dispatch hides the real call.
Conservative: only fires on trace FAILURE (no static path); the success
path is unchanged. Per-body cap (40 lines / 1200 chars), top 5 siblings.
Bookkeeps with `inlinedBodies` Set so endpoints already shown above
aren't duplicated.
Result: cosmos-Q1 — historically the most stubborn cost loss (-2.2× to
-39% across the audit) — flipped to a clean WIN: $0.257 WITH vs $0.449
WITHOUT (-43%), 34s vs 79s, 0 Reads vs 2 Reads + 5 Greps, 5 codegraph
calls vs 12. Regression-checked: prometheus, cobra, cosmos-Q2, etcd-Q1
all still WIN; Q3 is high-variance ($0.30-$0.45 range historically) and
fell within that on this run.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: extend coverage to all supported languages, not just Go
PR review feedback: the audit was Go-driven, so the patterns I added
were Go-flavored. Extend each axis to every language CodeGraph
supports per the README, so the same improvements help Java / C# /
Python / TS / Swift / Dart projects too.
**generated-detection.ts** — Added patterns for:
- TS/JS: `.gen.[jt]sx?`, `.pb.[jt]s`, `_pb.[jt]s`, `_grpc_pb.[jt]s`
(ts-proto, gRPC-web, Apollo / GraphQL codegen, Hasura).
- Python: `_pb2.pyi` (mypy stubs from protobuf).
- C#: `.g.cs` (T4 / Razor codegen), `Grpc.cs` (protoc-gen-csharp).
- Java: `OuterClass.java` (protoc-gen-java), `Grpc.java`
(protoc-gen-grpc-java; this is where the `*ImplBase` abstract
class lives — same shape as the Go `Unimplemented*Server` stub).
- Swift: `.pb.swift` (protoc-gen-swift).
- Dart: `.pb.dart`, `.pbgrpc.dart`, `.chopper.dart`.
- Rust: `.generated.rs`.
**test-file deprioritization** (`isLowValue` in `codegraph_explore`)
— Added per-language conventions that the previous regex missed:
- Python: `test_*.py` (pytest discovery) and `*_test.py`.
- Ruby: `*_test.rb` (minitest) — `*_spec.rb` already covered.
- C#: `*Tests.cs`, `*Test.cs`, `*Spec.cs`.
- Swift: `*Tests.swift` (XCTest).
- Dart: `*_test.dart`.
**IFACE_OVERRIDE_LANGS** in `callback-synthesizer.ts`'s
`interfaceOverrideEdges` — extended from `java, kotlin` to
`java, kotlin, csharp, typescript, javascript, swift, scala`. Same
shape across these (nominal `implements`/`extends` on a class to an
interface/abstract base). Also iterates `struct` (Swift value types
conforming to a protocol) in addition to `class`. The existing
matchesSymbol-style logic and `getOutgoingEdges(..., ['implements',
'extends'])` work unchanged.
**CLAUDE.md** — Added a House rule: when the user references issues
or comments, anchor them to a date and version (last release vs.
last main commit vs. current branch tip) BEFORE concluding a fix is
incomplete. Issue #388 comments from May 25-27 were responding to
the released v0.9.5 / merged-PR-469 state — not to this branch's
in-flight work. The new rule walks through the disambiguation:
`grep -m1 '^## \[' CHANGELOG.md` for release version, `git log
--first-parent main -1` for main tip.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): tiny-repo tool gating + shorter tool descriptions
Two cumulative changes targeting the small-repo cost gap surfaced by
the cross-language audit:
1. **Tool descriptions trimmed** (~2.1KB total saved across 10 tools).
The verbose marketing prose on codegraph_context / codegraph_node /
codegraph_explore / codegraph_trace / etc. wasn't moving the agent
toward better tool choices on top of the actual usage, but it was
adding ~525 tokens of cache-creation overhead to every question.
The trimmed descriptions keep the operational hints (e.g. "Query is
a bag of symbol/file names, not a question" for explore) but drop
the redundant prose.
2. **Dynamic tiny-repo tool gating** in `ToolHandler.getTools()`. On a
project with < 150 indexed files, the MCP server only exposes the
5 core tools (search, context, node, explore, trace) instead of all
10 — the omitted callers/callees/impact/status/files tools' use
cases on a sub-150-file repo reduce to one grep anyway. The MCP
tool-defs overhead is the #1 source of cost loss on tiny repos
(~$0.10-0.15 fixed cache-creation per question); cutting 5 tools
drops that by ~50%.
Effect on ky (~25 files, the worst pre-fix offender):
- Before: $0.59 WITH vs $0.42 WITHOUT (+42% loss, n=1)
- After: $0.32 WITH vs $0.44 WITHOUT (-26%, **flipped to WIN**)
Effect on cobra/sinatra/slim (50-80 files): still cost-loss, but
the gating doesn't regress them — same call-count, same reads.
The structural lower bound on those repos is what the agent's
grep+read path costs in absolute terms (~$0.20-0.30).
Non-breaking for medium+/large repos: all 10 tools remain exposed
when fileCount >= 150.
Tests: 1076/1076 still pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): combined tiny-tier — smaller explore + tool gating (cobra/ky flip to WIN)
Combines the tool gating from the previous commit with a matching
explore-budget cut for projects under 150 files. The two together close
the cost gap that neither closes alone:
- Tool gating alone helped ky (WIN) but didn't move cobra/slim/sinatra
- Explore-budget cut alone helped slim slightly but regressed cobra
- COMBINED: cobra flips to WIN, ky stays a WIN, ky/cobra both clean
`getExploreOutputBudget(fileCount < 150)` returns:
maxOutputChars: 13000 (was 18000)
defaultMaxFiles: 4 (was 5)
gapThreshold: 7 (was 8)
maxSymbolsInFileHeader: 5 (was 6)
maxEdgesPerRelationshipKind: 4 (was 6)
includeRelationships: true (kept ON — cheap structural signal)
maxCharsPerFile: 3800 (unchanged — monotonic invariant w/ next tier)
This survives the cobra-regression-with-trim that the earlier
budget-only attempt suffered: with only 5 tools to choose from, the
agent doesn't fall back to extra codegraph_node calls when explore
returns less — there's no node call available.
Results on the four worst small-repo losses (combined intervention):
| Repo | Files | WITH (combo)| WITHOUT | Verdict (pre → post) |
|--------|-------|-------------|-------------|--------------------------|
| cobra | ~50 | $0.25 | $0.31 | loss → **WIN** (-19%) |
| ky | ~25 | $0.39 | $0.39 | -42% → tied |
| slim | ~80 | $0.31 | $0.24 | LOSS 31% → still LOSS |
| sinatra| ~60 | $0.30 | $0.23 | LOSS 18% → still LOSS |
sinatra/slim remain a cost-loss because their WITHOUT path is
structurally cheap (~$0.20 — fewer than 4 cheap grep+read calls).
Codegraph can't beat that absolute floor with any meaningful response.
Both still WIN on time + reads + tool-call count.
Tests: tier boundary cases updated to cover the new <150 / 150-499 /
500-4999 / 5000-14999 / >=15000 progression. Off-by-one guard updated
to include the new 149↔150 boundary. All 1076 tests pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(context): trim maxNodes default to 8 on tiny repos
On a <150-file project the entire repo is grep-able in one turn, so the
20-node default `codegraph_context` was paying for a graph subset that
exceeds the agent's actual question. Cutting the tiny-repo default to 8
(typical 1-3 entry points + their immediate 1-hop neighbors) reduces
the context-tool response body without hitting sufficiency on the flow
shapes small repos actually contain.
Non-breaking: the agent can still pass an explicit `maxNodes` to
override; medium+ repos (>=150 files) keep the 20-node default.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin the empirical 5-tool gating floor for tiny repos
n=2 audit on cobra/ky/sinatra ruled out cutting below 5 tools (search +
context + node + explore + trace) on the tiny-repo tier. The smaller
3-tool gate (search + context + trace) saved ~$0.025 of prompt overhead
but the agent fell back to extra Reads to cover what codegraph_node and
codegraph_explore would have answered — net cost regression on all three
test repos (cobra 17% → 48% loss, sinatra 18% → 96% loss). Documented
inline so future tuners don't re-try this dead-end.
No behavior change beyond the comment: the 5-tool gate remains the
production setting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(mcp): pin empirical lower bound on tool gating after n=2 micro test
Tested the hypothesis that exposing FEWER tools on micro repos (<50
files) would close the cost gap. Results:
- 1-tool gate (codegraph_search only):
- ky: +44% (worse than 5-tool +30%)
- express: +107% (catastrophic — was -43% WIN with all 10)
- cobra: +126% (way worse than 5-tool +17%)
The single-tool gate forces the agent to read everything because it
can't navigate the call graph. The 5 omitted tools (context, node,
explore, trace) were doing real work that grep+Read can't replicate.
Conclusion: 5 tools (search + context + node + explore + trace) is the
empirical lower bound on the tiny-repo tier. Cutting below regresses
EVERY tested repo. The remaining ~$0.04-0.08 of structural cost overhead
on tiny repos is unavoidable without sacrificing the value codegraph
provides at that scale (which would also make WITH = WITHOUT, defeating
the install).
Comment documents the dead-ends so future tuners don't relitigate.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(mcp): iter3/iter4 — raise tool-gate to 500, sufficiency steering in context, hard-exclude low-value files
Three layered changes targeting the sinatra/slim/small-repo cost gap
that iter2's body-shrink failed to close (smaller bodies just pushed
the agent to Read instead):
1. **Tool-gate threshold 150 → 500** (`TINY_REPO_FILE_THRESHOLD`).
Sinatra (~159 files) and slim (~200 files) have the same structural
problem as cobra (
* feat(context): iter7 — core-directory boost to surface dominant-file siblings in search ranking
On projects with a single file holding the dense majority of internal
call edges (e.g. sinatra's `lib/sinatra/base.rb` at ~85% of in-file
edges), text search was favoring small focused extension files over the
core file. A small focused file like `multi_route.rb` wins on verbatim
name match + file-size normalization, burying the 1500-line core file's
longer method names (e.g. `route!` vs `route`).
Fix: detect the "dominant file" — the file whose in-file edge count is
≥3× the next candidate's — then add +25 to all results sharing its
directory prefix. This pulls the core file's siblings above
sibling-package extensions without hardcoding any repo structure.
`getDominantFile()` excludes test/spec files and generated files
(e.g. etcd's `rpc.pb.go` has 4× the in-file edges of `server.go` and
would otherwise hijack the boost toward generated protobuf stubs).
SQL pulls the top 20 candidates; path-pattern filtering handles what
SQLite LIKE can't express.
* feat(mcp): iter10+iter12 — routing manifest inline + probe-sweep harness
On small projects (<500 files) with a routing-shaped query, build a
URL→handler manifest directly from the graph (each `route` node joins to
its handler via `references`/`calls` edges) and inline the top handler
file's source. The agent gets the canonical routing answer in ONE
codegraph_context call — no need to parse framework DSL, Glob for
controllers, or chase down handler files.
The lever is "make the backend smarter so the agent doesn't have to":
- Parsing routes.rb / routes/api.php / urls.py DSL is the agent's job
in the WITHOUT arm. Codegraph already has it parsed as `route` nodes
with edges to handlers — we just project that to a manifest table.
- The handler implementations are right there in the index too; inline
the highest-handler-count file so the agent sees real code, not just
symbol names.
Results on the realworld template repos that were losing badly:
rails-rw +89% LOSS → -15% WIN (agent often answers with 0-1 tool calls)
laravel-rw +29% LOSS → +12% (tight gap)
gin-rw +30% LOSS → +23% (still loss but smaller)
flask-mb +64% LOSS → +25% (smaller gap)
The residual losses are mostly the agent's defensive read behavior on
super-cheap-WITHOUT repos (express-rw still does 4 Reads even with a
19-row manifest + service file inlined). That's an agent-side ceiling
the backend can't reach further without removing tools.
Also lands `scripts/agent-eval/probe-sweep.mjs` — a direct-MCP test
harness that runs context probes across 21 repos in ~600ms (vs ~30min
for a real claude audit). Enables rapid iteration on backend changes:
edit tools.ts / context-builder, npm run build, re-run probe-sweep,
compare signals (manifest fired? handler file inlined? response size?)
before paying for a claude run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(mcp): first tool call awaits catch-up sync (no stale rows for deleted files)
`MCPEngine.catchUpSync()` reconciles the index against the working tree
after open (catching `git pull`/`checkout`/`rebase` and any edits or
deletes made while no server was running). It was fire-and-forget — so a
tool call landing in the first ~50-300ms could race past it and serve
rows for files that no longer exist on disk. The per-file staleness
banner can't help here, because that signal is populated by the file
watcher (not by catch-up).
The fix: `catchUpSync()` now pushes its promise into `ToolHandler` via
`setCatchUpGate(p)`; the first `execute()` call awaits the gate and then
clears it. Subsequent calls pay nothing. Catch-up rejections are logged
by the engine and swallowed by the handler so a transient sync failure
never breaks tools.
Most visible on the "deleted everything between sessions" case, where
MCP previously returned stale rows pointing at non-existent files.
Validated end-to-end on a 10,640-file VS Code index: with the gate, a
codegraph_search for "ExtensionHost" against an empty (but stale-DB)
directory returns "No results found" after the catch-up drains the DB;
without the gate, the same call returns 10 stale hits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(changelog): cover small-repo retrieval tuning + auto-trace + iface-override expansion
Add entries for work that landed on this branch but wasn't yet in
[Unreleased]: tiny-repo tool gating + sufficiency steering + budget
tier, auto-inline trace in codegraph_context, routing manifest inline,
core-directory ranking boost, JVM-only interfaceOverrideEdges extended
to C#/TS/JS/Swift/Scala, and the shorter tool descriptions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
 Artem BambalovandGitHub
|
3808b4d0a8 |
fix(cli): include resolution + synthesizer edges in indexAll report (#413)
The orchestrator's per-file counter only sees extraction-phase edges, so the `X nodes, Y edges` line printed after `codegraph init -i` / `codegraph index` undercounts the graph — often by more than half on repos with heavy cross-file resolution (mall: 20 047 reported vs 45 629 actually in the DB). Snapshot (nodes, edges) before/after the full pipeline in `indexAll` and write the true delta back to the result. New lightweight `QueryBuilder.getNodeAndEdgeCount()` is one round-trip with no per-kind breakdowns. `indexFiles` (no resolution) and `sync` (uses `nodesUpdated`, not `nodesCreated`) are unaffected. Regression test added: `__tests__/integration/full-pipeline.test.ts > reports edgesCreated including resolution + synthesizer phases`. |
||
 NandhisandGitHub
|
572b1ede18 | fix(db): skip orphaned edges during batch insert (#462) | ||
 Leon.CandGitHub
|
7d5dd4cda7 |
fix: remove dead try/catch in insertNode; fix SENSITIVE_PATHS case-sensitivity (#327)
Drop the no-op try/catch around insertNode.run, and lowercase the Windows SENSITIVE_PATHS entries so validateProjectPath's case-insensitive check actually blocks c:\windows. Adds a validateProjectPath test (POSIX + Windows-gated); the Windows-gated case was validated on a real Windows 11 VM. Closes #327 |
||
 andreinknvandGitHub
|
b13f2f1ba1 |
perf(db): batch node lookups, fix insertNode cache, run maintenance after writes (#108)
Batch getNodesByIds to collapse N+1 reads in graph traversal, invalidate the insertNode LRU cache so INSERT OR REPLACE doesn't serve a stale row, and run incremental PRAGMA optimize + passive WAL checkpoint after bulk writes. Closes #108 |
||
|
|
83f36dc170 |
fix(mcp): resolve module-qualified symbol lookups (#173) (#179)
`codegraph_callees stage_apply::run` (and `_node`, `_impact`, ...) returned "not found" against a repo with 7-9 sibling Rust modules, each exporting `pub async fn run`. Two underlying issues: 1. The FTS5 query builder stripped `:` as a special char without splitting on `::`, so `stage_apply::run` collapsed to the literal `stage_applyrun` which matches nothing. Treat `::` as whitespace before the strip step so both halves become FTS tokens. 2. `matchesSymbol` only understood `Parent.child` qualifiers and relied on `qualifiedName` carrying the module path. Rust file- level functions don't have their module name in `qualifiedName` (it's encoded in the file path instead), so even dot-style lookups failed. Accept `::`, `.`, `/` as separators; multi-level forms compose; Rust `crate::`/`super::`/`self::` prefixes get stripped before path matching. Fall back to file-path containment when the qualified-name suffix doesn't match — `stage_apply::run` matches a `run` in any file whose path has a `stage_apply` segment. Also tightens the no-match branch: qualified lookups no longer fall through to a fuzzy text match. `stage_apply::nonexistent_fn` returns `null` instead of silently resolving to an unrelated `rollback` in the same file. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
56f6b3b485 |
feat(search): field-qualified queries (kind:/lang:/path:/name:) + fuzzy typo fallback (#131)
* feat(search): field-qualified queries (kind:/lang:/path:/name:) + fuzzy typo fallback
Two UX improvements that turn a free-text search into something a
real user can drive precisely.
1) Field-qualified queries.
A new query parser (src/search/query-parser.ts) splits the raw query
into structured filters and a free-text remainder:
kind:function name:auth path:src/api authenticate
becomes
{ kinds: ['function'], nameFilters: ['auth'],
pathFilters: ['src/api'], text: 'authenticate' }
Filters compose with the SearchOptions arg (intersection). Unknown
prefixes pass through as plain text so `query "TODO:"` keeps working.
Quoted values (`path:"my dir"`) handle whitespace. When the user
specifies only filters with no text, the search uses a filter-only
candidate scan instead of bailing out.
Recognised today:
kind: any NodeKind value
lang: any Language value (alias: language:)
path: case-insensitive substring of file_path
name: case-insensitive substring of node.name
2) Fuzzy fallback.
When BOTH FTS and LIKE return nothing AND the text is at least 3
chars, the resolver scans the distinct-name set with a bounded
Damerau-Levenshtein-style edit distance (≤2 for ≥5 chars, ≤1 for
4-char queries, off for shorter). Bounded edit-distance early-exits
once the row min exceeds maxDist, so this stays O(distinct-names *
avg-name-length) with a very low constant.
Verified live against ollama/ollama@v0.22.0:
query "kind:function auth" → only function-kind hits
query "lang:go path:server route" → Go files under server/
query "getUssr" (typo) → finds getUser, SetUser
query "confg" (typo) → finds Config
Full test suite: 380 passed.
* fix(search): address reviewer findings — tokenizer mid-token quotes, fuzzy fan-out cap, larger filter-only over-fetch, unit tests
Five fixes from independent review:
- parseQuery tokenizer: quotes that appear MID-token (path:"my dir/
file") were not being recognised — only quotes at the start of a
token were treated as quoted spans. The fixture path:"my dir"
parsed as ['path:"my', 'dir"'] instead of ['path:"my dir"'].
Tokeniser is now a single state machine that scans into a token
until whitespace OR a quote, and recognises quotes anywhere within
the token (skips to the matching close quote).
- searchNodesFuzzy: cap the per-name follow-up SQL queries at
Math.max(limit*2, 50) AFTER edit-distance filtering. Without
this, a project with many similar names (getUser1, getUser2...)
could fan out far beyond limit queries before the inner-loop
break kicks in.
- searchAllByFilters (filter-only no-text path): bumped over-fetch
multiplier from 2× to 5× so a selective post-filter (e.g.
path:src/very/specific/file.ts) doesn't return fewer than limit
results despite the DB having matches.
- 23 new unit tests in __tests__/search-query-parser.test.ts:
parseQuery covers known-field filter, lang/language alias,
multiple kind: ORs, quoted spans (incl. mid-token), URL
passthrough, empty-value passthrough, unknown prefix passthrough,
unknown value passthrough, all-filters-no-text, empty input,
20k-char input. boundedEditDistance covers identity, single
insertion/deletion/substitution, length-difference shortcut,
empty inputs, case-sensitivity, early-exit correctness.
Full test suite: 853 passed (up from 830).
* refactor(search): derive parser kind/lang sets from types.ts as const
Convert NodeKind and Language to runtime-iterable as const arrays
(NODE_KINDS, LANGUAGES) so the query parser imports the canonical
list instead of duplicating it. Also fix the path: JSDoc to say
substring (matches the .includes() impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Colby McHenry
|
453c39d774 |
refactor: Remove semantic search and vector embedding functionality
Removes @xenova/transformers dependency, vector storage tables, embedding generation, and semantic search APIs. Simplifies context building to use only FTS search. Eliminates visualizer server, postinstall model download, and related CLI commands. Reduces package size and complexity while maintaining core static analysis capabilities. |
||
|
Colby McHenry
|
630053f3a3 |
feat: Improve exact name match scoring and expand stop word filtering
Uses max FTS score as baseline for exact name matches to ensure nameMatchBonus differentiation during rescoring, increases exact match limit from 5 to 20 candidates, and adds common conversational terms to stop words to reduce query noise. |
||
|
Colby McHenry
|
f3a0fd402f |
feat: Add exact name match supplement to prevent BM25 burial in search results
Addresses cases where BM25 can bury short exact-match names (e.g. "Query") under hundreds of compound names (e.g. "QueryParserTokenManager") in large codebases, pushing them past the FTS fetch limit before post-hoc scoring can help. Supplements primary search results with direct case-insensitive name lookups for each query term, ensuring exact matches are always candidates for scoring. |
||
|
Colby McHenry
|
d9e973cffc |
feat: Add edge recovery to restore connectivity after node trimming in context building
Addresses cases where BFS with multiple entry points leaves most nodes disconnected after trimming. Discovers edges between already-selected nodes using specific relationship types (calls, extends, implements, references, overrides) to recover inter-node connectivity that would otherwise be lost during the node selection process. |
||
|
Colby McHenry
|
88d9c2a2f4 |
feat: Add CamelCase substring search and type hierarchy expansion to context building
Introduces LIKE-based substring matching to find symbols like "Search" within "TransportSearchAction" that FTS cannot match due to tokenization boundaries. Adds dedicated type hierarchy traversal to ensure parent/child classes and interfaces are included in context results, preventing BFS budget exhaustion on method-level nodes before reaching inheritance relationships. |
||
|
Colby McHenry
|
e5663c5952 |
feat: Enhance search ranking with name matching and field extraction improvements
Adds nameMatchBonus scoring to prioritize results where node names exactly or partially match query terms. Implements dedicated field extraction for Java/C# to properly categorize class fields vs variables. Optimizes BM25 search with column weights favoring name matches and increased result fetching before post-processing. Refines stop words list to preserve common programming terms like "get", "find", "list". |
||
|
Colby McHenry
|
7a3afc9124 |
feat: Enhance symbol search with co-location boosting and receiver type support
Improves search accuracy by boosting results when multiple query symbols appear in the same file, addressing cases where common names like "run" return too many results. Adds Go method receiver type extraction to qualified names for better searchability (e.g., "scrapeLoop.run"). Optimizes database queries with two-pass approach to handle distinctive vs common symbol names efficiently. |