0d17dfd6a84290ea731d1b931beec0c9690af708
895
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Colby MchenryandGitHub
|
0d17dfd6a8 |
feat(cli): install --init and init --yes for a one-shot, non-interactive bootstrap (#1578) (#1595)
Fixes #1578. ## What was wrong Bootstrapping CodeGraph in a fresh environment — the issue's case is a throwaway container per AI session — took two commands, `codegraph install --yes` and then `codegraph init`, and the second one could still stop on a prompt (the gitignored-child-repos offer, the watch-fallback offer on WSL/`/mnt`). There was no way to wire agents and build the project's index in one non-interactive line. The installer's "never index implicitly" rule is deliberate (a surprise index of `$HOME` is exactly what `init` refuses), so the gap is an explicit opt-in, not a change in default behavior. ## What this does - **`codegraph install -i, --init`** — after wiring the agents, runs the `init` flow in the current directory. It also runs when nothing was wired (`--target none`, no agents detected), since the installer returns normally in that case. Every `init` guard applies: a home directory / filesystem root / parent of home is **refused with exit code 1** (no implied `--force`), and an already-initialized project just reports that and exits 0. `--print-config` and `--refresh` return before the install, so `--init` is a no-op with them. - **`codegraph init -y, --yes`** — non-interactive: the ignored-repos offer prints its one-line `includeIgnored` opt-in snippet instead of prompting (the existing non-TTY behavior), and the watch-fallback offer takes its `yes` default. `install --init` passes `--yes` through, so `codegraph install --yes --init` is a fully unattended bootstrap. - The `init` action body becomes `runInit()`, shared by both commands. The plain `init` path is behavior-identical (same refusal, already-initialized notice, supervised index, telemetry, offers, outro). - The post-install "Next: index a project" note gains one line mentioning `--init`; README gets the flag row and a `--yes --init` example. On the reporter's other observation — `install --yes` skipping the "install the CLI on your PATH" step: that's by design for scripted use (it assumes the CLI is already present), and the `bunx @colbymchenry/codegraph serve --mcp` MCP entry they found is the self-contained alternative. Not changed here. ## Tests `__tests__/cli-install-init.test.ts` — end-to-end against the built binary with stdin closed (a blocking prompt would fail), always `--target none` so the suite never touches an agent config on the host: - `install --yes --target none --init` → exit 0, installer reports nothing to wire, `Initialized in <tmp>`, `.codegraph/codegraph.db` exists; - the same on an already-initialized project → `Already initialized`, exit 0; - the same at the filesystem root → exit 1, `Refusing to initialize`, nothing written; - `init --yes` with stdin closed → exit 0, index built; - `init --help` lists `-y, --yes`, `install --help` lists `-i, --init`. `npx vitest run __tests__/installer-targets.test.ts __tests__/upgrade.test.ts` → 283 passed, 3 skipped. Full `npm test` → see the checks on this PR / below. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
278a8edc35 |
fix(resolution): resolve calls to object-literal namespace members (#1573) (#1597)
Fixes #1573. Thanks @IAliceBobI — the report had the root cause exactly right, and the fix sits one layer up from the suggested spot (resolution rather than the container-kind set), for the reason below. ## What was wrong Methods of an exported object-literal constant — `export const api = { call() {…}, get: () => {…} }` used as a module's API surface — never received a call edge from `api.call()`, same-file or through an import. The members are extracted as plain functions with **bare** qualified names (`call`, not `api::call`) sitting inside the constant's source extent, so: - the `Container::member` lookup the class-shaped kinds use (#825) bails on kind `constant`, and even with `constant` added to that set there is no `api::call` to find; - the declared-type inference for imported singleton instances (#1292) finds no type in a literal and falls back to the constant edge; - the same-file strategies only consider classes and `method` kinds, so the call resolved to nothing at all. Net effect: `callers` / impact reported zero for methods called from everywhere, with no boundary warning because nothing about `obj.method()` looks dynamic. ## What this does Adds one helper that resolves a member **by containment** — a node named `member` whose source range lies inside the value's range, in the value's own file — and uses it from both halves: - **Import path**: when the imported value is a constant/variable, the literal member is tried right after the `Container::member` lookup and before the #1292 instance inference, so the cross-file edge lands on the method instead of the constant. - **Same-file path**: a same-file constant/variable receiver (TS/JS family only) is checked before the class-name strategies. Precision rules, all tested: calls accept callable kinds only; a declaration nested inside another member's body is not a member; nothing outside the value's range can donate a match — a same-named top-level function, or a method returned by a factory the value merely holds — so those cases keep today's behavior rather than guessing. Class statics (`C.s()`) and non-literal values are untouched. Extraction and qualified names are deliberately left alone: changing how literal members are named would have to be mirrored in the native kernel byte-for-byte, and the resolver-side lookup is contained and language-gated. ## Tests - The issue's repro end-to-end: `sameFileCallers` and `crossFileCaller` are both callers of `m`; a decoy `m` in a third file gets none; the `C.s()` static control resolves exactly as before; `crossFileCaller` no longer has a `calls` edge to the constant. - Arrow-property and method members both resolve; a `function call()` nested inside `get`'s body is never taken for `api.call()`. - A value holding a factory's result (`const obj = makeObj()`) with a same-named top-level `m` in the file: no false attribution, existing behavior kept. - The two positive tests fail on `main`; the control passes both ways, as a guard should. - Full suite: 189 files, 3181 passed / 9 skipped. With the built CLI on the issue's `a.ts`/`b.ts`: `codegraph callers m` → 2 callers (`sameFileCallers`, `crossFileCaller`); `callers s` unchanged; edges `sameFileCallers -> m` (0.85) and `crossFileCaller -> m` (import, 0.9), none to `obj`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
7963672689 |
fix(rust): resolve self.field.method() on the field's declared type instead of a same-named method (#1585) (#1599)
Fixes #1585. **Stacked on #1596** (the base branch is `fix/1588-rust-impl-type-qualification`; this PR's own diff is the second commit). Merge #1596 first, then retarget/merge this one. ## What was wrong ```rust impl Outer { pub fn run(&mut self) { self.inner.run(); // inner: Inner } } ``` produced `Outer::run -> Outer::run` — recursion the source doesn't contain. The extractor collapsed every `self.<field>.<method>()` receiver to the bare method name (`run`), so the resolver only ever saw `run` and exact-matched the nearest same-named method — the calling method itself, or a method of an unrelated type. Nothing marked the edge as a guess, and no row stayed in `unresolved_refs`, so a consumer had no way to tell. The same happened when the field's type isn't a project type at all (`its: std::vec::IntoIter<_>` → `self.its.next()`, `matcher: Regex` → `self.matcher.is_match()`): the bare `next` / `is_match` attached to whatever local method shared the name. ripgrep had 279 self-edges on `main`; the issue lists three sites, all of this shape. (The issue's C++ control — "`Outer::run -> Inner::run` resolves correctly" — doesn't actually hold on `main`: `inner.h` is classified as C by the `.h` heuristic, so `Inner::run` never exists and the C++ repro self-edges too. That's #1592, fixed separately.) ## What this does Rust struct fields are not graph nodes, so the field's type can only come from the struct's declaration text. This follows the Go 2-hop precedent exactly (`matchGoFieldChainCall`, #1276), including its exclusivity rule: 1. **Extraction (TS walker + native kernel, identical, parity-tested):** a call whose receiver is `self.<field>` keeps the owner-field shape — `self.inner.run()` is emitted as `self.inner.run`. Deeper chains (`self.a.b.m()`), call receivers (`self.f().m()`), parenthesized receivers and bare `self` keep the bare name, exactly as before. 2. **Resolution (`matchRustSelfFieldCall`):** owner type = the calling method's qualified-name prefix (`Outer::run` → `Outer`); the field's declared type is read from the owner struct's **own declaration lines** (comment-stripped, line by line — same discipline as the Go helper); the method is resolved **and validated** on that type by `resolveMethodOnType` (confidence 0.85, `instance-method`). 3. **Exclusive:** when the field is declared with an external type, a generic parameter (`T`), a container that doesn't auto-deref (`Option`/`Vec`/`Mutex`/…), or can't be found, the ref **stays unresolved** — it never falls through to the bare-name strategies. That is the safe behaviour the issue asks for, and it is what #1276 already chose for Go. `rustFieldTypeName` looks through exactly the layers Rust's method-call auto-deref looks through: references (`&`, `&'a mut`) and the owning smart pointers `Box`/`Rc`/`Arc`. `Box<dyn Source>` yields the trait, whose method node the interface-impl synthesizer then fans out to every implementation. `Option<Inner>` is left alone — `self.inner.take()` is Option's method and must not become `Inner::take`. Why it stacks on #1596: the owner is taken from the method's qualified name, which for a generic/lifetime impl was the trait's name before that fix. ## Measured on ripgrep (110 `.rs` files, #1596 build vs this branch) | | #1596 | this PR | |---|---|---| | nodes | 4029 | 4029 | | `calls` self-edges | 279 | **146** (none of the `self.<field>` shape remain — 116 bare-receiver, 30 other dotted) | | `self.<field>.m()` calls resolved through a validated field type | — | **292** (`DecompressionMatcher::command -> GlobSet::matches`, `Parser::find_long -> FlagMap::find`, `Haystack::path -> DirEntry::path`, …) | | `self.<field>.m()` calls left unresolved | — | **417** — every sampled one is a std/container method: `self.commands.push`, `self.child.wait`, `self.pre.is_some`, `self.colors.clone`, `self.path_terminator.unwrap_or` | | `calls` edges total | 9150 | 8878 (the 272 removed are the former bare-name guesses for those 417) | The issue's three sites: `walk.rs:824` now resolves to `IgnoreBuilder::add_custom_ignore_filename` (was a self-edge); `walk.rs:1195` (`self.its.next`, `IntoIter`) and `globset/lib.rs:983` (`self.matcher.is_match`, `Regex`) are parked as unresolved instead of guessed. The issue's repro gives `Outer::run -> Inner::run` (`instance-method`, confidence 0.85) on both the kernel path and `CODEGRAPH_KERNEL=0`. ## Tests - `__tests__/extraction.test.ts`: only the single-hop `self.<field>.<method>()` call keeps the prefix; deeper / call / parenthesized / bare-`self` receivers and a local receiver are unchanged. - `__tests__/resolution.test.ts` (end-to-end, Cargo layout): the issue's repro → `Outer::run -> Inner::run`, no self-edge; an external field type (`std::vec::IntoIter`) with a local `next` decoy → no edge at all; `Box<Inner>` and `&'a mut Inner` resolve, `Option<Inner>` does not (even though `Inner` declares the method); a generic `T` field → no edge; genuine `self.run()` recursion keeps its self-edge; the #1588 repro's `UsesFile::go` / `UsesBuf::go` resolve to `FileSource::read` / `BufSource::read`, and a `Box<dyn Source>` field lands on `Source::read` with the synthesizer fanning out to both impls. - `__tests__/fixtures/kernel-parity/torture.rs` grows the receiver shapes; all 15 kernel parity suites pass against the rebuilt kernel (147 tests). - Full `npm test` on this branch: 189 files, 3187 passed, 9 skipped, 0 failed. Re-index after upgrading. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
12f7a59f26 |
fix(rust): qualify generic/lifetime impl methods by the implementing type, not the trait (#1588) (#1596)
Fixes #1588. ## What was wrong The receiver of an `impl` block — the name that qualifies its methods, owns the `contains` edge, and sources the `implements` edge — was found positionally: the **last bare `type_identifier` child** of the `impl_item`. That works for `impl Source for FileSource`. But once the implementing type carries parameters it parses as a `generic_type`, and the only bare identifier left is the **trait's**: ```rust impl Source for FileSource → FileSource::read ✓ impl<T> Source for BufSource<T> → Source::read ✗ (should be BufSource::read) impl<'a> Iterator for Parents<'a> → Iterator::next ✗ impl Trait for &Foo → Trait::method ✗ ``` Two consequences, both reproduced on `main`: - `BufSource::read` did not exist in the graph, so `resolveMethodOnType("BufSource", "read")` and "who calls `BufSource::read`" had no answer, and every generic implementation of a trait collapsed onto the same trait-qualified name. - Because the impl's method carried the trait's qualified name, the interface-impl synthesizer treated the impl **body** as a second trait declaration and emitted a dispatch edge from it (`Source::read -> FileSource::read`, registered at the generic impl's line — a body of `{ 0 }` containing no call at all). The native kernel (`rustlang.rs`) mirrored the positional rule deliberately, bug-for-bug, to hold byte-parity with the TS walker — its header said "preserve, never fix via the grammar's trait:/type: fields". So the fix has to land on both sides at once. ## What this does Both extractors now read the grammar's **named fields** instead of scanning children. One shared rule (`rustImplTypeName` in `languages/rust.ts`, `impl_type_name` in the kernel), applied to `impl_item.type`: | implementing type | node | receiver | |---|---|---| | `Foo` | `type_identifier` | `Foo` | | `Foo<T>` / `Foo<'a>` | `generic_type` → its `type` field | `Foo` | | `m::Foo` | `scoped_type_identifier` → its `name` field | `Foo` (was: no receiver) | | `&Foo` / `&'a mut Foo` | `reference_type` → its `type` field | `Foo` | | `(A, B)`, `dyn Tr`, `*const T`, `u32`, fn types | anything else | none — extracted as plain functions, exactly as before | The `implements` back-reference reads `impl_item.trait` (full text, so `fmt::Display` and `From<u32>` keep their spelling) and bails when the field is absent (inherent impl). Everything else — the no-scope impl quirk, the source-order `contains` owner scan, method extraction — is untouched; the `contains` edge simply lands on the implementing type now instead of the trait. The kernel header comment, the parity test's description, and the two design docs that documented the quirk as "preserve" are updated to say what changed. ## Measured on ripgrep (110 `.rs` files, `main` build vs this branch) | | main | this PR | |---|---|---| | nodes / methods | 4029 / 2202 | 4029 / 2202 | | impl methods qualified by a **trait** name (node outside that trait's extent) | 61 | **0** | | `Iterator::*` methods | 2 | 0 | | duplicate method qualified names | 77 | 42 | | synthesized `interface-impl` edges originating **outside** any trait declaration (the phantom fan-outs) | 38 | **0** | | synthesized `interface-impl` edges originating at a real trait declaration | 33 | **52** | | plain (non-heuristic) `calls` edges | 9098 | 9098 | So the synthesizer lost every phantom edge and *gained* 19 legitimate fan-outs to implementations it could not previously see as implementations. `contains` edges went 5237 → 5224: the 13 removed were trait→impl-method edges produced by the mis-qualification. The issue's repro now gives `BufSource::read` at line 12, `BufSource -> Source`, and both synthesized edges registered at the declaration (line 2) — identical on the kernel path and with `CODEGRAPH_KERNEL=0`. (The remaining `UsesFile::go -> BufSource::read` exact-match guess there is the separate `self.field.method()` receiver problem, #1585, which stacks on this.) ## Tests - `__tests__/extraction.test.ts` (Rust Extraction): method qualified names for generic / lifetime / reference / scoped / generic-trait impls; the trait's qualified name names exactly one node; `implements` refs come from the implementing type for every shape; the `contains` edge lands on the type; tuple / `dyn` impls keep producing plain functions with no `implements` ref. - `__tests__/resolution.test.ts` (end-to-end): `Source::read` names only the declaration; dispatch fans out to **both** `FileSource::read` and `BufSource::read`, every synthesized edge registered at line 2; neither impl body sprouts a synthesized call. - `__tests__/fixtures/kernel-parity/torture.rs` grows all the new impl shapes; `kernel-rustlang-parity` (LF + CRLF) passes against the rebuilt kernel. - `CODEGRAPH_KERNEL_EXPECT=1 npx vitest run __tests__/kernel-*.test.ts` — all 15 suites, 147 tests pass. - Full `npm test`: 3180 passed, 9 skipped, 1 failed — `mcp-daemon.test.ts > daemon idle-times-out after the last client disconnects`, a 30 s timing test that passed on re-run in isolation (the machine was running four parallel suites and kernel builds at the time); unrelated to extraction. Re-index after upgrading to pick up the corrected names. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
 Colby McHenryandClaude Fable 5
|
44e1812d3b |
changelog: cover the merged contributor batch (#1547, #1215, #594, #1463)
Write the missing [Unreleased] entries for the Vapor route hang fix, the untracked-directory status gap (described for its current status-only symptom — sync itself reconciles off the filesystem), and the new deprioritize config key; move the .xsjs/.xsjslib resolution entry out of the released 1.0.0 block, where a stale rebase had left it; credit @maxmilian across the batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
1d9de88ef1 |
feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting (#982) (#1463)
* feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting matchesNonProductionDir hardcodes example/sample/fixture/benchmark/demo, so a peripheral tree only the project knows about — optional-skills/, scripts/ — gets no de-prioritization. When helpers there carry generic symbol names, an exact name match hands them a large bonus and they crowd out the product code that answers the query (#982). deprioritize is the RANKING counterpart to exclude: those paths stay indexed and findable, they just stop outranking first-party code. It is deliberately distinct from the corpus-frequency discount, which keys on a name being common and is near-inert on #982's own repro where only two symbols are named usage. The -15 path penalty alone is not enough, and measuring showed why: on that repro a usage() helper sits at 74.8 against 51.2 for the top product symbol, so -15 lands at 59.8 and still leads. The path penalty is additive and the name bonus it must counter is additive and larger. A de-prioritized path is saying its symbol NAMES are not the answer, so the exact-name bonus is damped to 0.25x there as well — damped, not zeroed, so the tree still ranks when it genuinely is what you asked for. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(config): read deprioritize lazily and apply it in explore too Review of the first cut found two real defects. The matcher was built once in wireLayers(), which runs only from the constructor and from reopenIfReplaced(). The MCP server keeps one CodeGraph per project root alive for its whole lifetime, so editing codegraph.json appeared to do nothing until the process restarted -- exclude and include do not behave that way. The predicate now reads loadDeprioritizePatterns() per call (mtime-cached, one stat) and memoizes the compiled matcher on the pattern array's identity. A regression test writes the config after opening the project and fails on the old code. Explore passed no matcher to scorePathRelevance at either of its two call sites, so the setting only half-applied -- and #982's reproduction rows B, C and D are all codegraph explore, which made this the surface the issue actually reports on. Both sites now pass it. Explore's hard early-continue filters and its non-production budget cap are deliberately NOT joined: those REMOVE content, and deprioritize is a ranking lever by definition. README narrowed accordingly -- it previously claimed this extends the built-in list, which overstated it. Also from review: scorePathRelevance takes a boolean rather than a predicate (the caller already evaluated it, and it was being invoked twice per result), the predicate body is exception-guarded so a bad path can never take a search down, the misplaced const moved out from between imports, two vacuous test assertions tightened, and tests added for the single-penalty invariant, the deliberate isTestQuery asymmetry, and a query that genuinely targets the de-prioritized tree. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(search): derive the deprioritize name-bonus damping instead of picking it (#982) The 0.25 scale was a guess. On a 62k-node django index it measurably breaks the "discount, don't erase" rule the lever is built on: exact-name queries for symbols that live only in the de-prioritized tree (child, parent, method) fall behind mere prefix matches (children, all_parents, method_decorator). The prefix arm of nameMatchBonus tops out below 40, and a de-prioritized node also takes the -15 path penalty, so 80 * SCALE - 15 > 40 is the bound that keeps a damped exact match ahead of a prefix match at any corpus shape. 0.75 clears it; crowd-out removal is nearly identical to 0.5 (39 vs 40 of 88 peripheral top-10 slots cleared on django), so the deeper discount bought almost nothing and cost the invariant. Two tests pin the bound, including one that fails at the old 0.25. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Max HsuandGitHub
|
bc894802ff |
feat(installer): support project-local Codex installs (#1531) (#1551)
Codex CLI has a first-class project config layer — `.codex/config.toml` is layer 4 of the loader stack, above the user config at layer 6 (`codex-rs/config/src/loader/README.md` in openai/codex), and it landed in openai/codex#8354 on 2025-12-22. The CodexTarget's "Codex has no project-local config concept" note was therefore never accurate, and `supportsLocation('local') === false` made Codex the one agent that forces a machine-wide MCP install. `mcp_servers` is not on the project layer's denylist (which strips base URLs, model providers, `notify`, profiles and otel — settings repo contents shouldn't choose), so a project-scoped `[mcp_servers.codegraph]` is honored. - Path helpers take a `Location`: global keeps `~/.codex/config.toml` + `~/.codex/AGENTS.md`; local writes `<cwd>/.codex/config.toml` and the project-root `<cwd>/AGENTS.md` — the same split the gemini and opencode targets already use for their local layout. - Drops the five `loc !== 'global'` early returns from detect, install, uninstall, printConfig and describePaths. - Local install returns a note that Codex only applies a project layer in a project marked trusted; untrusted projects load the layer but leave it disabled, so a silent success would be misleading. - Refreshes the two doc comments that used Codex as the example of a global-only target (now the Copilot CLI). Tests: two new cases covering the local write layout, the trust note, global config staying untouched, and local uninstall leaving the global entry intact. Both fail against the previous implementation. The generic per-target contract suite now also exercises codex at location=local. |
||
|
|
474f051d3c |
fix(resolution): load path aliases through tsconfig extends and base configs (#1534) (#1548)
`loadProjectAliases()` read only the root `tsconfig.json` / `jsconfig.json` own `compilerOptions`, so an Nx-style monorepo — every alias declared in a `tsconfig.base.json` — got `null` back and every cross-package import fell through to name-based matching. Silently: no unresolved-import warning, and the results still look precise. Two things were missing, and either one alone leaves a common Nx layout broken: Fold the `extends` chain into the effective options before building the alias map. Relative and `node_modules` package specifiers both resolve, the nearest config wins (tsc replaces `paths` rather than merging), and a config already on the current chain is not re-entered, so `a extends b extends a` terminates instead of recursing forever. `paths` are anchored at `baseUrl` when one is declared — itself relative to the config that declared it — and otherwise at the directory of the config that declared the `paths`, which is what tsc does and what keeps an inherited `src/*` from being read as root-relative. Read `tsconfig.base.json` as a last candidate. A root `tsconfig.json` is still authoritative when it exists and reaches the base through `extends`; the fallback covers the layouts where that never happens — a solution-style root config (`references`, no `extends`, no `paths`, which is what nx's own repository ships) or no root `tsconfig.json` at all. A candidate that contributes no aliases no longer shadows a later one that does. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Colby McHenry <me@colbymchenry.com> |
||
|
|
9219967e43 |
perf(search): seek the name index for exact-name lookups (#1542)
`nodes` carries two name indexes and neither can serve
`WHERE name = ? COLLATE NOCASE`: `idx_nodes_name` is BINARY-collated, and
`idx_nodes_lower_name` is an expression index the planner only matches against
the same expression. All three whole-name lookups in the query layer were
written that way, so each one degraded to a full table scan
(`EXPLAIN QUERY PLAN` reports `SCAN nodes`).
The LIMITs on those queries do not rescue them. SQLite can only stop early once
it has produced LIMIT rows, and the two dominant cases never get there: a query
word that names no symbol at all, and a name with only a handful of definitions.
`searchNodes` runs its supplement once per query term; `findNodesByExactName`
runs two passes per symbol extracted from the question, and extraction is
generous, so a plainly-worded question issues a dozen full scans.
Written as `lower(name) = lower(?)` the same predicate seeks
`idx_nodes_lower_name`. Measured on four indexed repositories, baseline vs fix
in one process (the only difference being how the predicate is spelled):
query "how does the retry backoff work" findNodesByExactName searchNodes
gin (2.5k nodes) 1.27ms -> 0.18ms 3.1 -> 2.6ms
Alamofire (4.5k nodes) 2.39ms -> 0.22ms 4.9 -> 4.0ms
excalidraw (11k nodes) 10.54ms -> 0.17ms 10.4 -> 5.8ms
django (62k nodes) 49.91ms -> 0.17ms 27.6 -> 4.9ms
The seek is flat across all four; the scan grows with the corpus. A one-word
query into `searchNodes` on django is unchanged (~20ms) because a single term's
scan is not what dominates it there.
Lowering the parameter in SQL rather than in JavaScript is deliberate. SQLite's
`lower()` and NOCASE both fold ASCII only, while JavaScript's `.toLowerCase()`
folds Unicode; comparing a JS-lowered parameter against `lower(name)` would
silently stop matching non-ASCII identifiers that NOCASE used to match.
`getNodesByLowerName` is spelled the same way for the same reason. It already
sought the index, but as a bare `lower(name) = ?` it took a pre-lowered
parameter on trust: any input carrying an uppercase letter returned nothing at
all. This is behaviour-neutral for its one caller — `matchFuzzy` lowers in
JavaScript before calling, and `lower()` over an already-lowered string is a
no-op, verified over the ASCII and non-ASCII cases alike. It closes the trap for
the next caller; the non-ASCII gap on the `matchFuzzy` side is a resolution
change and is deliberately not bundled here.
Result sets are unchanged, including which rows the LIMITs keep: entries under
one key in the expression index are ordered by rowid, the same order a table
scan produces. Verified over 14,400 lookups (top-400 names of the four
corpora, probed as stored / upper / lower, against all three call sites) with
zero differences, and end-to-end above with identical result ids.
Tests assert the planner's verdict rather than a wall-clock number, so they are
deterministic: they intercept the SQL each call site prepares and require an
index seek, with a guard that the lookups actually ran. Reverting any call site
turns them red.
Co-authored-by: Colby McHenry <me@colbymchenry.com>
|
||
|
Max HsuandGitHub
|
a74029105a |
fix(resolution): resolve ES imports targeting .xsjs/.xsjslib files (#556) (#594)
The extraction half of #556 — indexing `.xsjs` / `.xsjslib` as JavaScript — already landed on main via #654. This PR is now scoped to the remaining resolution gap: the JS import-resolution list did not include the SAP HANA extensions, so an extensionless `import { x } from './helpers'` in a `.xsjs` file resolved to nothing and the cross-file call edge was dropped. Add `.xsjs` / `.xsjslib` to the `javascript` entry in EXTENSION_RESOLUTION so those imports resolve to their target file and `codegraph_callers` / `codegraph_impact` see the edge. One resolution test covers the .xsjs -> .xsjslib import; the now-redundant extraction/detection tests were dropped (covered by #654). |
||
|
Max HsuandGitHub
|
cc9ce09256 |
fix(extraction): detect untracked files inside untracked directories (#1213) (#1215)
git status --porcelain collapses an entirely-untracked directory into a single '?? dir/' entry. collectGitStatus only recurses into such dirs to find embedded git repos, so source files in a plain untracked directory were never surfaced to sync — 'codegraph sync' reported 'Already up to date' and the watcher missed them too. Add -uall so git lists individual untracked files. Nested untracked git repos still collapse to '?? repo/' even with -uall (git never crosses a repo boundary), so the embedded-repo recursion is unaffected. Export getGitChangedFiles and add regression tests for both the plain untracked-directory case and the embedded-repo recursion (no -uall regression). Root-cause analysis and fix suggested by the reporter in #1213. |
||
|
Max HsuandGitHub
|
340d4b033e |
fix(swift): remove catastrophic backtracking in Vapor route regex (#1547)
The arg-list group `(?:[^,()]+,\s*)*` was ambiguous: the trailing `\s*` and the next iteration's `[^,()]+` could both claim the same run of spaces, so a `.METHOD(...)` call with many comma-separated args that never reaches `use:` forced an exponential search. Measured on `app.get(arg0: value0, ...)`: 40ms at 20 args, 647ms at 24, 41.7s at 30, and no result after 120s at 60. Anchoring each repetition at a comma (`(?:[^,()]+,)*\s*`) makes the split unique — `,` is outside the char class, so there is nothing to re-partition. Same input is now 0.09ms at 1000 args. Match behaviour is unchanged: all four capture groups are identical on 18 hand-written Vapor route shapes (no args, single/multi path segments, `X.parameter`, multi-line calls, Environment.get non-matches) and on 200k fuzzed inputs. Fixes #1544 |
||
|
Colby McHenry
|
ccb0295259 |
fix(explore): reliably pin extension-less kebab-case file basenames in queries
Previously, naming a kebab-case file without its extension (e.g., `background-image-table` vs. `background-image-table.tsx`) in a `codegraph_explore` query would shred the name into fragments (`background`, `image`, `table`), admitting irrelevant sibling files and crowding out the intended target. This change introduces a new resolution pass in `extractQueryPaths` specifically for extension-less kebab basenames. Queries now accurately identify and pin these files. Unresolved hyphenated prose (e.g., `cross-call`) is left in the query for FTS without being flagged as an unknown path. Resolution prioritizes explicit slashed/dotted paths and respects an ambiguity budget for common stems to prevent over-pinning. |
||
 
|
81e1f4a92f |
fix: harden daemon and large-index recovery paths (#1562)
* fix: harden indexing recovery and daemon liveness * test: cover daemon and recovery review gaps * test: pin that a failure marker never blocks a later successful parse (#1557 retry-discard guard) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: danusha2345 <ewidusoc498@gmail.com> Co-authored-by: Colby McHenry <me@colbymchenry.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d8f2eeaddf |
fix(db): loop-append dense unresolved-ref result rows; make stripped-salvage visible (#1558) (#1576)
Real-world validation of #1575 on indexes damaged by the released v1.5.0 binary surfaced both of these. getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter limit but appended each chunk's RESULT rows with a spread — every row becomes a call argument, so a dense recovery sync (the #1541 self-heal re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit and killed resolution mid-sync with "Maximum call stack size exceeded", leaving the graph 226k edges short until another sync resumed the orphans (and that sweep resolves measurably worse than the batched path — see the follow-up issue). The failed-ref retry loader had the identical pattern on unbounded result rows. Both append with a loop now (#1558). The #1575 stripped-salvage warning also never rendered: init's summary prints only index_partial warnings and counts only hard errors, so a run with salvaged files still read as fully clean — and with no hard errors the detail wasn't written to errors.log either. Salvage entries now carry code 'salvaged_stripped', the summary prints a visible warning naming the files, and errors.log is written for salvage-only runs. Validated on real corpora with full-graph dumps: healthy-path inits stay byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a determinism control); a realistically-damaged index (41 wiped + 5 missing files, damage generated by the released binary) heals in one plain sync to identical per-file counts and an edge set within the normal incremental residual; pathological mass damage (52% of the repo) completes without crashing. New regression test reproduces the RangeError on the old code with 200k pending refs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
26045b3159 |
fix(extraction): decode kernel results in indexAll retry passes; self-heal wiped rows (#1541) (#1575)
The parse-pool workers return kernel-language extractions as an undecoded buffer transport (nodes/edges EMPTY, tables in kernelBuffers). indexAll's main loop decodes them (or hands the buffers to the store worker), but its two retry passes — plain retry and the comments-stripped last resort — stored the transport as-is: the storage gate passed via errors.length === 0, zero nodes were inserted, and the files row was written with node_count = 0 while the original error was spliced out of the summary. Any worker crash/timeout whose in-flight file was a kernel-routed language permanently recorded that file as "(0 symbols)" — silently, and immune to later syncs because the stored hash matches the on-disk bytes (#1541; v1.4.1 predates the kernel path, which is why it was unaffected). - Both retry passes now materialize kernel results before the gate, store, counters, and log lines. - storeExtractionResult materializes at entry as defense-in-depth, so no storage path can persist an undecoded transport again. - Zero-node rows on symbol-bearing languages (only the wipe produces these — every real extraction stores at least the file node) are dropped during full-reconcile sync and indexAll so already-affected files re-index automatically after upgrading. Scoped watcher syncs leave rows outside their scope untouched. - The comments-stripped salvage now downgrades the failure to a visible warning instead of erasing it: the recovered result can be incomplete, and reporting clean success made a fresh index quietly disagree with a later per-file re-parse of the same bytes (#1565's init-vs-sync divergence). Repro (released 1.5.0): CODEGRAPH_PARSE_TIMEOUT_MS=1 codegraph init on any Python project → "Retry OK: <file> (0 nodes)" and permanent "(python, 0 symbols)" rows. Fixed build stores real symbols under the same forcing, and heals rows wiped by prior runs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
d806317897 |
fix(explore): improve query accuracy for file paths, camelCase, and variables
`codegraph_explore` previously struggled with accurately interpreting user queries. Explicitly named file paths were shredded, making it hard to target specific files; natural language queries often missed camelCase identifiers; and state held in variables was overlooked as starting symbols. This commit introduces several improvements: - **Reliable File Path Resolution:** Naming a file by its path in a `codegraph_explore` query now works reliably. The path is resolved against the index, and that file is guaranteed a place at the top of the answer. Previously, paths were broken into fragments — bracketed route segments like SvelteKit's `[id]` made this worst — and pieces like `page` or `runs` matched every sibling file, so the file you actually named could be crowded out. A path that doesn't match any indexed file is now called out instead of silently ignored. - **CamelCase Matching for Queries:** Plainly-worded `codegraph_explore` questions now find camelCase code. A query like "auto-scroll to bottom" can reach a function named `scrollFeedToBottom`, because query words are matched against the words inside identifiers, not just whole names. - **Variable and Constant Seeding:** Variables and constants now count when `codegraph_explore` picks its starting symbols, so state held in plain variables — `$state`-style variables in Svelte, for example — no longer gets overlooked. |
||
|
Colby McHenry
|
238dbc5cec |
fix(explore): accurately resolve query file paths and find camelCase symbols
Previously, `codegraph_explore` queries explicitly naming files by path (e.g., `src/routes/m/projects/[id]/runs/[runId]/+page.svelte`) were shredded. Bracketed path segments exploded into "named symbol" seeds, and FTS on fragments like `page` or `runs` admitted every sibling file, starving the user's intended target. This change introduces: - **Query path pinning:** File paths named in a query are now resolved against the index, "pinned," and stripped from the query. Pinned files are guaranteed inclusion, top ranking, and fair allocation. Unresolvable path-like spans are reported. - **Segment vocabulary supplement:** Natural language query terms (e.g., "auto-scroll to bottom") can now reach camelCase identifiers (e.g., `pinFeedIfNearBottom`, `feedAtBottom`) by matching against their constituent segments. - **Variable seeding:** `variable` and `constant` node kinds are now included in identifier seeding, improving recall for `$state`-style variables common in frameworks like Svelte. |
||
|
Colby MchenryandGitHub
|
c6aaa20358 |
Merge pull request #1516 from ctype-lab/fix/union-declarations-not-indexed
fix(c,cpp,objc,rust): index union declarations as a first-class `union` node kind (#1515) |
||
|
Colby McHenry
|
d289bf84d3 |
Merge main into fix/union-declarations-not-indexed
Resolves the CHANGELOG conflict — main and this branch each prepended a bullet to [Unreleased] > Fixes; both are kept. Everything else auto-merged, including src/mcp/tools.ts, which main reworked heavily for the explore allocation/displacement work (CG-28/31/36/38) while this branch added the `union` kind to its container sets. Verified on the merged tree with the native kernel built: 3070 passed, 9 skipped, 0 failed. |
||
|
Colby McHenryandClaude Opus 5
|
5b0c4b8b93 |
fix(resolution): trait dispatch reaches union implementors (#1515)
Making unions first-class nodes leaves the third loss in #1515 open: interfaceOverrideEdges enumerates its concrete side as ['class','struct'], so a union implementor is skipped even though it now has a real node and a real `implements` edge. "Who implements this trait" then answers wrongly rather than incompletely — the struct beside it bridges and the union does not. Add 'union' to that tuple, plus a regression test that pins the Rust trait -> union-impl hop (the struct implementor is the control proving the synthesizer ran). Verified the test fails on the union assertion alone before this change. No EXTRACTION_VERSION bump: main is already at 25 against v1.5.0's 24, so existing indexes are flagged stale for the next release regardless, and over-bumping is what turns the re-index hint into noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby MchenryandGitHub
|
222f82b9a5 |
Merge pull request #1440 from colbymchenry/feat/copilot-installer-targets
feat(installer): GitHub Copilot targets — VS Code, Copilot CLI, JetBrains |
||
|
Colby McHenry
|
493d4210f1 |
Merge branch 'main' into feat/copilot-installer-targets
# Conflicts: # CHANGELOG.md |
||
|
Colby MchenryandGitHub
|
c84ce55855 |
Merge pull request #1528 from colbymchenry/issue/671-mcp-supported-languages
feat(mcp): surface supported languages in MCP server instructions (recut of #678) |
||
|
Colby McHenryandClaude Fable 5
|
2962e7e1f4 |
feat(mcp): surface supported languages in MCP server instructions (#671)
Recut of #678 against the current instructions — the original predated the explore-first rewrite and conflicted in both files it touched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby MchenryandGitHub
|
1b36132a89 |
Merge pull request #1498 from colbymchenry/bugfix/CG-16
fix(telemetry-dashboard): accept Origin: null on login — no-referrer policy locked Chromium out (CG-16) |
||
|
Colby MchenryandGitHub
|
99f2ebf0d1 |
Merge pull request #1527 from colbymchenry/bugfix/CG-38
CG-38: guarantee an agent-named symbol renders, wherever it sits |
||
|
Colby MchenryandGitHub
|
fa8a3d7226 |
Merge pull request #1526 from colbymchenry/feature/CG-35
CG-33/CG-35: converge incremental sync with a full rebuild |
||
|
Colby McHenry
|
2c708caf7c | Merge branch 'main' into feature/CG-35 | ||
|
Colby McHenry
|
89c53ddf24 |
fix(explore): guarantee an agent-named symbol renders, wherever it sits (CG-38)
`codegraph_explore` never returned `queueMessage` (L1087) or `flushQueuedMessages` (L1102) from a 1,414-line file, on a symbol bag or a prose question, even with that file at rank #1 holding 67% of the envelope — the agent got a same-stem `QueuedMessage` interface at L70 and had to Read the file for the functions it had named. Pre-existing at every build including pre-epic (controlled bisect, index held fixed). Two independent causes: 1. `buildFlowFromNamedSymbols` returns the Flow prose AND the set of node ids the agent named — and the latter is the whole guarantee, since it injects a named def into its file's cluster ranges at importance 9. Its bail-outs returned EMPTY, zeroing the identity whenever there was nothing to PRINT. Two sibling closures that never call each other produce no chain, no synth hop and no boundary, so both defs lost importance 9 and the file rendered from its head. `identityOnly()` now separates the two, gated on shape-precise tokens so a prose word that exact-matches a callable cannot promote itself. 2. The ceiling trim filled in SOURCE order, so an over-ceiling render always dropped the END of a large file first. The shrink HAD kept both symbols (1022-1121); the trim cut back to 839. `windowToCeiling` now takes the spine call site plus every importance>=9 member as focus lines, tries the full ceiling first, and splits the held-back reserve evenly with carry-forward — greedy-in-source-order reproduced the bug one level down. The shrink's loose size estimate is left alone deliberately, and the comment now says why: making it exact was built and measured WORSE (it stops at the last member that fits whole and the released bytes carry forward to lower-ranked files, costing payroll-go's `s.store.Upsert`). `bound()` clamps to the ceiling anyway, so the slack costs no bytes; it just must not pick the survivors, which is what the trim now handles. The measurement gap this closes: every existing probe is aggregate — envelope share, per-file spend, source totals, file counts — and all are green on a response that returns 25K from the right file and omits the named function. `probe-named-symbol.mjs` checks the definition LINE against the response's rendered lines, per symbol. Suite envelope byte-identical to main on all six repos; probe-allocation 4/4, no starvation flags; 180 files / 2,997 tests green. Fixture: 7/7 fail on main, 7/7 pass here, deterministic over 4 runs per arm. |
||
|
Colby MchenryandGitHub
|
969ea1ec37 |
Merge pull request #1525 from colbymchenry/feature/CG-24
CG-24: explore response noise — allocation fixes, generated-file detection, and index-drift convergence |
||
|
Colby McHenryandClaude Opus 5
|
07338ff12e |
docs(benchmarks): record CG-38 as open, and correct the regression claim
The epic record said nothing was open. CG-38 is: agent-named symbols in the tail of a large file never render, which the epic's probes cannot see because none of them measures whether the named symbol appeared. Also corrects a wrong claim made while investigating it. The epic was said to have regressed its own motivating query; that comparison varied the index as well as the engine. A controlled bisect holding the index fixed shows the pre-epic engine rendering 12 lines and CG-36 rendering 463 — the epic strictly improves the case, and the symbols render at neither. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
8a4623463d |
merge: shrink a later cluster into the remainder instead of dropping it (CG-36)
A file whose top-ranked cluster was trivial kept it, dropped the cluster carrying the answer WHOLE, and left most of its reservation unspent — because only the first-chosen cluster could be shrunk. CG-31's carry-forward then correctly handed that slack down the rank order, so the budget was not merely unspent but REDIRECTED to weaker files. django's sql/query.py (score 83, reserved 7,947) went from 1,923 delivered chars to 10,082, and its envelope share from 7.7% to 40.4%; contrib/admin/ filters.py (score 18) went from 8,057 at 355% of its reservation down to 2,198 at 97%. All 8 starvation flags across the suite clear. Net +1,012 source chars. The issue named the wrong fix point and the measurement said so: both real cases lost on maxImportance, NOT on the density tiebreak the issue and its duplicate (CG-37) suspected. Cluster ranking was left untouched, so the Session.swift case density-first exists for still works — now pinned by a dense-header fixture. Accepted cost: okhttp trades its rank-6 file (score 21, reserved 1,999) for +7,196 chars in the two files that answer the question, taking it from 6 delivered files to 5. django -159, okhttp -219 and tokio -25 source chars against the epic tip; gin +1,176, alamofire +187, excalidraw +52. django also stops cutting its epilogue. Ships probe-file-spend.mjs, a standing suite-wide probe for reservation vs spend, so this stays measurable — the original evidence came from ad-hoc instrumentation that no longer existed and had to be re-derived by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
10f1ac601a |
docs(benchmarks): record the CG-36 cluster-starvation measurement
The issue blamed the density tiebreak; both real cases lost on maxImportance, so ranking was left alone. Full before/after table, the one cost (okhttp's rank-6 file, squeezed out by reservations that were already structurally over-subscribed), and what ships to keep it measurable. |
||
|
Colby McHenry
|
eed16447c3 |
fix(explore): shrink a later cluster into the remainder instead of dropping it (CG-36)
A file's ranked clusters were all-or-nothing past the first one: the top-ranked cluster was taken (shrunk to fit when it had to be) and every cluster below it was rendered whole, then either fit the remainder or was dropped entirely. On a file whose top-ranked cluster is TRIVIAL that discards the answer — django's `db/models/sql/query.py` kept a 22-line glue cluster and dropped the 624-line `Query` body, spending 1,923 of a 7,947 reservation; okhttp's `RealInterceptorChain.kt` did the same behind its import header. The response stayed full, which is why this was invisible: the unspent reservation carried forward exactly as designed and a file scoring a fifth as much took the bytes. Two sites, the same rule — hold the remainder while it is still worth a section (CG-26's between-FILES lesson, applied between CLUSTERS): - selection now shrinks a later cluster into what is left of the file's budget, by the same whole-member rule the first cluster already used; - the ceiling trim re-renders the weakest cluster into the room that remains before dropping it. On excalidraw's `typeChecks.ts` the section-cost estimate missed by 13 chars and a 1,512-char cluster — the file's highest-SCORING one — was thrown away to pay for it. Cluster RANKING is untouched: measured, both real cases lost on `maxImportance`, not on the density tiebreak the issue suspected, and density-first is what keeps Alamofire's `Session.swift` from burying its methods under the property list. Suite (6 repos, clean-rebuilt indexes): all 8 starvation flags cleared, +1,012 source chars net. django's `sql/query.py` 1,923 -> 10,082 of 7,947, okhttp's `RealInterceptorChain.kt` 1,474 -> 6,038 of 6,058, gin's `routergroup.go` 3,273 -> 5,632. okhttp trades its rank-6 file (score 21) for +7,196 chars in the two files that answer the question. Ships two fixtures pulling in opposite directions (`starved-cluster-ts` and `dense-header-ts`), a `spendShareAtLeast` gate in probe-allocation, and probe-file-spend.mjs — a standing per-file reservation-vs-delivered sweep. |
||
|
Colby McHenryandClaude Opus 5
|
76ab1fe130 |
docs(benchmarks): record the CG-24 epic resolution
Four shipped fixes, one open defect (CG-36), and five issues closed because measurement contradicted them. The headline is that the reported symptom was not an explore bug at all — it was a degraded index (CG-33), and the reported query answers correctly on a clean rebuild with no explore change. Records the two traps that cost real time and are now guarded in tooling: the nonexistent .codegraph/graph.db path that sqlite3 silently creates, and ab-new-vs-baseline.sh swapping src/ mid-run so a commit captures baseline sources. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
ed6c56f3fa |
merge: damp undepended-on ambient declaration files on flow queries (CG-28)
Both halves of the issue were measured on a hermetic fixture of four declaration-shaped files varying on banner and depended-on-ness. CG-25 already handles the motivating file: the Wrangler worker-configuration.d.ts that opened this issue is demoted by the generated penalty alone, worth 15-46 points of envelope share across four flow queries. No new mechanism for it. The narrower gap is real. A declaration file with NO banner carried pen 1.00, took rank #1 and 51% of delivered source on a prose flow query, and displaced the flow's own entry file out of the response entirely. The rule is deliberately narrow, and both conditions were derived by survey rather than guessed. 'Declares no callable and calls nothing' flags 1.1-18.0% of files across the corpus and catches real source — okhttp's SocketPolicy.kt, tokio/src/runtime/mod.rs, Alamofire's umbrella file, django's locale format tables. Requiring every symbol to be type-level drops that to 0-4%. The 'nothing depends on it' condition was added after the broader version demoted a pure-interface file with 13 inbound imports and broke the CG-31 displacement gate — a different invariant entirely. Does NOT stack with the generated penalty: rankPenalty takes Math.min of the two, so a file that is both takes the stronger, never the product. A query that NAMES a declaration symbol exempts its file entirely, so asking about a type still reaches it at full weight. Six-repo envelope is byte-identical to the pre-change tip — the rule does not fire on any benchmark repo, consistent with the 0-4% survey. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
9efae0f8f2 |
fix(explore): damp ambient declaration files on flow queries (CG-28)
A file that declares nothing but types and that nothing in the index depends on — a hand-written ambient `.d.ts` of global shims, vendored typings, module augmentation — cannot answer a flow question: no bodies, no call edges, no behaviour, nothing typed by it. But the identifiers it declares are exactly the generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`, `ReadableStream`), so on term overlap it out-scored the implementation. Measured on the new fixture: rank #1 and 51% of delivered source, with the flow's own entry file pushed out of the response entirely. Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that opened this is already handled by CG-25's banner detection, worth 15-46 points of envelope share across four flow queries. CG-25 credited; only the un-bannered case needed anything. `rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken as the STRONGER of it and the generated penalty rather than multiplied — one property two signals see must not be charged twice. Detection is structural, not by extension, and four conditions deep. Two of them were forced by measurement: requiring every symbol to be type-level takes the corpus flag rate from 1-18% (which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's locale tables) down to 0-4%; requiring that nothing depends on the file separates an ambient shim from a working types module, and without it the rule demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate. A query that NAMES a declared type is exempt, so a question about a type still reaches its declaration at full weight. Precise tokens only, so "…the file body…" cannot exempt a `Body` interface it never meant to name; this needs its own set because `namedSeedIds` is callable-only and a type never becomes one. Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md: 6-repo envelope sweep byte-identical against a clean baseline build, zero ambient files reach the candidate set on VS Code across five queries, corpus flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
463f6e7844 |
merge: factory-closure envelope premise measured and rejected (CG-27)
CG-27 proposed adding function/method to ENVELOPE_KINDS so a factory closure spanning most of its file stops merging every inner symbol into one cluster. The issue required the ranking claim be measured before any fix. It was, on a hermetic fixture built to make the pattern maximally visible, and it does not hold — nothing shipped to src/. The literal change is a large regression: dropping the enclosing range SPLITS the file into a trivial cluster (a type alias plus a helper, span 7) and the answer-bearing one (every closure, span 359). Cluster ranking breaks the equal maxImportance tie on density, so the trivial cluster wins, is taken first, and is the only one that may be shrunk; the answer-bearing cluster then does not fit and is dropped whole. Rank #1 fell from 7,539 delivered chars to 397, and from 7 of 11 inner closures to 0. The enclosing range was holding the file together as one cluster, inside which shrinkCluster already did the per-symbol ranking the issue asked for. A better mechanism reaching the same intent — deferring the envelope member inside shrinkCluster, leaving clustering untouched — is noise: 69 vs 68 inner definitions across nine query shapes, one better, one worse, seven unchanged. The one configuration where the envelope IS selected (the factory as sole top-tier member) is already absorbed by CG-30, which windows it on whole lines: a contiguous readable head carrying 6 of 9 closures, bounded and never empty. Kept: the fixture, the deterministic probe, and the measurement record. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
91cb5b4317 |
measure(explore): the factory-closure envelope premise does not hold (CG-27)
CG-27 asked whether the >50%-of-file envelope drop should cover `function` / `method`, so a `createFoo()` factory returning an object of closures stops merging every closure inside it into one cluster. Measured on a hermetic fixture, it should not, and the issue is closed as obsolete with CG-30 credited. Two mechanisms already absorb the shape. shrinkCluster orders members by (importance desc, size ASC) and refuses any member that overruns the cap once something is kept, so a file-spanning member is only selected when it is the sole member of the top importance tier — eight of nine query shapes never selected it at all. When it IS selected, CG-30 windows it on whole lines, so the file still delivers bounded, readable source (6 of 9 closure definitions in that configuration). Dropping the range instead SPLITS the file, and only the first-chosen cluster may be shrunk: a trivial 7-line cluster won the density tiebreak and the answer-bearing cluster was dropped whole — rank-#1 file 7,539 chars and 7 of 11 closures to 397 and none. Reaching the same intent more carefully (defer the envelope MEMBER inside shrinkCluster, leaving clustering untouched) is noise: 69 vs 68 closure definitions across nine query shapes. Nothing shipped. Adds the fixture, the probe, a standing gate on the outcome, and the record — including a real defect the measurement exposed on the epic tip: django's query.py leaves 8,212 of 10,135 unspent and drops a score-290 cluster to keep a score-14 one. Filed separately. No behaviour change, so no CHANGELOG entry. |
||
|
Colby McHenry
|
d49265043c |
test(explore): add the factory-closure fixture and its selection probe (CG-27)
A file whose top-level symbol spans almost all of it — createFoo() returning
an object of closures — is how Svelte 5 rune stores, React custom-hook modules,
IIFE module-pattern JS and Zustand's create((set,get)=>({…})) are all written.
probe-factory-closure.mjs measures what such a file DELIVERS from within: which
inner symbols' definitions reach the agent, not how many bytes did.
|
||
|
Colby McHenryandClaude Opus 5
|
dc4fd755ef |
merge: recognize Wrangler-style generated banners (CG-25)
A generated Cloudflare Wrangler ambient-types file was not flagged generated, so
it ranked with no penalty and competed with hand-written source on generic token
overlap. The banner shape it uses — "Generated by <tool> by running <command>" —
matched none of the existing content patterns, all of which require DO NOT EDIT,
a standalone @generated, or the "auto(matically) generated by" phrasings.
Precision is held by requiring TWO 'by' clauses: the banner must name a tool and
then say 'by running'. Ordinary prose ("the report is generated by running the
nightly job") has only one and does not match.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
8bb0f53bca |
merge: explore allocation — bounded overshoot, displacement guard, exact budget (CG-30, CG-31, CG-26)
Lands the three-branch allocation stack. Every admitted file now receives at least its reservation before any file draws on carry-forward slack, on every render path — cluster, whole-file GRACE, and whole-file BUY. CG-30 bounded how far an oversize cluster member may overshoot (windowed on whole lines past 1.5x rather than emitted whole or dropped). CG-31 gave the cluster path the `owedBelow` displacement guard the BUY arm always had, holding back only the prefix of what is owed below that the response can actually pay. CG-26 closed the three remaining holes: the whole-file arms had no displacement guard at all, section overhead was charged at a flat 200 against a real 300-500, and `owedPayableBelow` held all-or-nothing where it should hold partially. Deterministic across the 6-repo suite, clean-rebuilt indexes, both builds: no repo truncates, no repo loses a file, okhttp gains one, and every repo lands at or under the 25,000 hard ceiling. Accepted trade (maintainer decision): excalidraw -552 and okhttp -164 source chars against the CG-31 tip, in exchange for the trailing pointer list surviving instead of being discarded whole. Those bytes existed at the CG-31 tip only because it over-filled a ceiling it mis-measured and then dropped the entire epilogue; a pointer the agent can act on beats a few hundred chars on the last-ranked file. Two issues opened during this work were closed as invalid rather than fixed: CG-32 (named-file ordering) and CG-34 (allocator over-reservation). Both were filed on diagnoses that did not survive measurement — CG-32's symptom was index drift (CG-33), and CG-34's premise was overturned by CG-31's own results. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
57e0854213 |
fix(explore): recognize Wrangler-style "generated by … by running" banners (CG-25)
Cloudflare Wrangler's `worker-configuration.d.ts` (~12k lines of ambient types) carried no banner any GENERATED_CONTENT_PATTERNS entry matched: every existing marker requires `DO NOT EDIT`, a standalone `@generated`, `<auto-generated>`, or the literal `automatically/auto-generated by` phrasings. Wrangler emits a bare `Generated by Wrangler by running `wrangler types``, so the file ranked with pen 1.00 and won 79.4% of an explore envelope on generic token overlap alone (CG-24). The discriminator is the reproduction instruction, not the word "generated": the banner must name a tool AND then say `by running`, i.e. two separate "by" clauses. That keeps prose out — "the nightly summary is generated by running the ETL job" has only one — while catching every CLI-driven emitter that tells you how to regenerate. Precision swept over 441,856 files across the whole local source tree: 5 hits, all genuine Wrangler output, no false positives. Isolated before/after on the CG-24 repro (same query, same index, only the `files.generated` flag differing): before pen 1.00 score 115.0 share 79.4% 3 files rendered after pen 0.30 score 35.4 share 21.1% 4 files rendered The new pattern stays in the existing table position, below the header window the detector scans, so the module still does not classify itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
02ee151e46 |
CG-35: give the sync-convergence suite teeth against the rebind pass
The suite passed unchanged with `CODEGRAPH_NO_REBIND=1`, so the larger half of CG-33 — the rebind pass — had no coverage at all. The cause was the ground truth, not the cases: `rebuildEdgeSet` called `indexAll()` on the live handle. That is not a rebuild. Every file hashes identical, so the store writes nothing (`nodesCreated: 0`), no reference is re-created, and every edge survives — the comparison read the synced index against itself and could never fail. It now goes through `CodeGraph.recreate`, which deletes the database file the way the CLI's `index` command does. With a real rebuild, three existing cases fail under the kill switch. Adds two more for the rules that carry the risk: - an edge with no `refName` stamp (older engine) and a synthesized (`provenance='heuristic'`) edge are never deleted — both planted directly, and each verified load-bearing by mutation; - a name over the 500-edge ceiling is declined losslessly rather than rebound in part, with a rare name in the same sync as the control that proves the pass ran. The per-file-vs-batch-wide delta rule is likewise confirmed by mutation: a batch-wide name set fails its case. CODEGRAPH_NO_REBIND=1 now fails 4 cases; unset is green; full suite green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
03893b0ab9 |
CG-33: converge incremental sync with a full rebuild
A live, auto-synced index did not converge to a clean rebuild of the same tree — 4.3% of distinct edges wrong in both directions on this repo's own index, overwhelmingly `calls`, which is what flow queries traverse and what explore's file ranking weights. Silent: nothing warned, and the symptom read as "codegraph isn't very good" rather than "this index needs rebuilding." Two causes, and the fix needed both. Resolution binds a reference to one of the same-named definitions PROJECT-WIDE, so a definition appearing or vanishing changes the correct answer for references in files the sync never touches — and those references resolved successfully once, which deletes their unresolved_refs row, leaving nothing to revisit them with (#1240's retry only revisits refs parked as failed). Separately, when nothing disambiguated the candidates the winner came down to rowid, i.e. the order files happened to be WRITTEN, which differs between a scan-order full index and a sync that appends each file as it changes. That second one is why re-resolution alone could not converge: re-resolving against the identical graph still picked a different candidate. So getNodesByName now orders by (file_path, start_line) — a property of the code, not of the write order — and sync computes a definitionDelta and re-opens the resolution edges whose answer it may have invalidated, re-inserting each as the reference that created it for the orphan sweep to bind against the post-sync graph. The delta compares `file\0name` pairs per file rather than one name set over the batch: a commit that adds `collect` to a new file while an unrelated changed file already defines `collect` cancels out of a batch-wide set, and that miss was the largest residual class in the first measurement. Conservative where the failure modes are asymmetric — a wrong deletion is a permanent edge loss, a missed rebind is only residual drift. Edges without a refName stamp are never touched (nothing to restore them from), sources the sync already re-extracted are skipped, and a per-name ceiling declines the generic names. Edges are deleted before the sweep re-inserts, since INSERT OR IGNORE against idx_edges_identity would otherwise keep both rows when a reference rebinds elsewhere. Replaying real commits of this repo through sync, then diffing against a rebuild: 16 commits 48 -> 0; 80 commits 1,634 -> 361, with the actively misleading direction (stale edges the index keeps asserting) 671 -> 2. Index and sync wall-clock are unchanged; the ORDER BY costs 18% per uncached name lookup, which never reaches wall-clock because the resolver memoizes it. The 357-edge residual at 80 commits is one pre-existing class: refs to generic names (`push`, `join`) parked above #1240's per-name retry ceiling, which a rebuild resolves into cross-language garbage — a TS test file "calling" an R method. Converging there would mean manufacturing wrong edges, so it is left alone. And no drift metric in `codegraph status`: it cannot be computed without the rebuild it would be recommending, and a proxy would fire on that residual and train users to ignore it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
5f32478b57 |
docs(benchmarks): record the CG-26 A/B — the invariant holds on every path
Deterministic 6-repo table, the three agent A/Bs (django, excalidraw, okhttp, 2 runs/arm, Read 0 in all 12 runs), and an honest read of the two repos that deliver a few hundred fewer source chars: at the CG-31 tip both were over-filled by the flat-200 section overhead and paid for it by discarding their epilogue whole. Also: CHANGELOG entries for the two user-visible changes, and the memory note now carries the fourth accounting gap plus the two lessons — hold the REMAINDER when a full reservation no longer fits, and never skip a file over an accounting difference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
 ctype_lab
|
e2195940fb | fix(union): complete downstream container handling | ||
|
Colby McHenryandClaude Opus 5
|
7cbde95ce2 |
fix(explore): pay every admitted file on every render path (CG-26)
The invariant this closes: every admitted file receives at least its reservation before any file draws on carry-forward slack. CG-30 bounded an oversize cluster member and CG-31 gave the cluster path a displacement guard; three holes were left, and each one starved a file that had been admitted, reserved and — in the worst case — rendered. 1. The whole-file arms had no displacement guard. BUY's fit test read `renderCeiling - totalChars` (everyone's room) while its source-space sibling refused the same trade, and GRACE was not fit-tested at all. okhttp's CallServerInterceptor.kt shipped 8,499 chars on a 5,964 funded ceiling and the rank-6 file below it delivered nothing. Both arms now test the render they actually produce against `fundedHeadroom`, and a whole render that does not fit falls through to clustering instead of skipping the file. 2. Every section was charged a flat 200 chars while a real header runs 300-500. The loop believed it had room it did not have — okhttp allocated 26,601 against a 24,400 ceiling — so the final truncation threw a fully-rendered section away. Sections are charged their real cost now, the owed-below arithmetic uses a per-file overhead estimated from the file's own symbols, and a marginal overrun trims the weakest cluster (or windows the last one into the room that is left) rather than skipping the file over a rounding difference. 3. `owedPayableBelow` held all-or-nothing. When the last admitted file's FULL reservation no longer fit, nothing was held for it: on the precise-query fixture the rank-5 file took 4,134 chars against a 2,948 reservation while rank 6 — admitted, reserved 2,539 — was left 4 chars and skipped. It now holds the remainder while that remainder is still worth a section (MIN_CHARS). And the epilogue is budgeted instead of discarded. The flat 600-char margin was neither the epilogue's size (1,064 gin, 1,788 django, 2,231 excalidraw) nor a bound on it, so four of six suite repos shipped with no pointer list and no reminders at all. The loop now reserves the epilogue's FLOOR — the one line that says an uncovered area exists, plus a pointer for every file whose bytes were deliberately withheld (CG-12) — and the rest is fitted to the room that actually remains, in priority order, entry by entry. Sized from the real strings; no constant was swept against the suite. Deterministic, same clean-rebuilt indexes, baseline = CG-31 tip: repo base source new source files ceiling django 20,791 20,878 6 -> 6 was discarding its epilogue tokio 21,521 21,607 5 -> 5 was discarding its epilogue okhttp 19,034 18,870 5 -> 6 +1 file delivered excalidraw 20,204 19,652 8 -> 8 keeps its pointer list gin 10,776 10,776 4 -> 4 byte-identical alamofire 11,662 11,662 2 -> 2 byte-identical No repo truncates any more and none loses a file. okhttp and excalidraw trade 164 and 552 source chars on their LAST-ranked file for the pointer list naming what the response could not cover — bytes the CG-31 tip only had because it over-filled a ceiling it mis-measured and then discarded the epilogue whole. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
c54e0080c2 |
fix(explore): keep the drift warning out of the cuttable epilogue (CG-31)
The '⚠ changed on disk after the last index sync' banner is an honesty claim about source we DID render — line refs elsewhere in the response may be shifted — not a note about the response. Drawing the epilogue boundary after it means the size cut can never be what silences it. Suite numbers unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |