bcfe52efa5d23eeb5bede2df2d4fa27a0e389ee1
403
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Colby McHenryandClaude Fable 5
|
6a056ec5db |
docs: say what the WAL fix bounds — the log's resting size, never the index (#1431)
The 1.6.0 notes said the write-ahead log is "capped", which reads as a limit on how much can be indexed. It bounds only the log's resting size (64 MB default, CODEGRAPH_WAL_HEAL_MB) and folds a killed session's leftover back into the index; a large repository's log still grows in proportion to its index while it is built. Say so in both entries, and document the two knobs in the README's troubleshooting section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MC52FSFLtKtDCLqT81tZYG |
||
 github-actions[bot]
|
dfccdf6254 |
docs(changelog): promote [Unreleased] into [1.6.0]
[skip ci] Auto-generated by Release workflow. |
||
|
Colby McHenryandClaude Fable 5
|
60f920a66d |
docs(changelog): open [Unreleased] with a Highlights block and group the fixes
The [Unreleased] section had 58 long entries in two flat lists — fine as a record, unreadable as an update. It now opens with a short Highlights list (nine plain-language bullets plus the re-index note) that a non-engineer can read in a minute, the seven features are ordered by what users notice first, and the 51 fixes are grouped under four sub-headings. Every entry is preserved verbatim; only order and headings changed. CLAUDE.md gains the matching rule so the block is refreshed at each release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MC52FSFLtKtDCLqT81tZYG |
||
 Colby MchenryandGitHub
|
41c10750e0 |
fix(erlang): give same-name different-arity functions separate arity-qualified nodes (#1610) (#1615)
Fixes #1610. Also fixes #1358 (the `<<binary>>` arity miscount in behaviour dispatch, reported separately and hit by the same code path). ## Problem Arity is part of an Erlang function's identity — `f/1` and `f/2` are unrelated top-level definitions — but the extractor merged consecutive same-name `fun_decl`s regardless of arity. Reproduced on main exactly as reported: - adjacent `f(X) -> …. f(X, Y) -> ….` → **one** node spanning both, with the first definition's signature; - interleaved `f/1, g/0, f/2` → two nodes with **identical** `qualified_name`; - `cowboy_req`'s `header(Name, Req) -> header(Name, Req, undefined).` → a **self-loop** `header → header`, with the `-spec` for `/3` swallowed by the merged span; - `-export([f/1])` marked every arity exported. ## Fix - **One node per (name, arity).** Clauses of the same name+arity still merge (that part of the old behavior was correct); a different arity starts a new node. `qualifiedName` carries the canonical spelling — `mod::f/1` — while the node **name stays bare** so search and bare-name matching are unchanged. - **`-export` and `-spec` are per-arity.** `-export([f/1])` exports exactly `f/1`; a spec sitting between two arities attaches to the arity its signature names. - **Refs carry the call-site arity** wherever it's statically known: local `f/1`, remote `mod::f/2`, `fun f/1` / `fun mod:f/1` values, `gen_server` dispatch (`handle_call/3`, `handle_cast/2`), and spawn/apply MFA lists (`spawn_link(?MODULE, work, [A, B])` → `work/2`). - **The matcher resolves only to the named arity** — same file first (a local call targets its own module) — and when no definition of that arity exists it resolves to **nothing** rather than a sibling arity: silent beats wrong. An arity-less dynamic-MFA ref resolves only when the module defines exactly one arity of that name. - **Behaviour dispatch** selects the implementer node of the site's arity, and the arity counter now skips `<<1,2,3>>` binary-literal commas per its own docstring (#1358) — `Mod:decode(<<1,2,3>>, Opts)` counts 2, not 4. - **`codegraph_explore` / `codegraph_node`** accept the written `mod:fn/3` spelling against the new arity-qualified names (the issue's measured `cowboy_stream_h:request_process/3` shape). ## Validation Minimal fixtures (all three reported shapes) now index as `gap::f/1` + `gap::f/2`, distinct `inter::f/1`/`inter::f/2`, and a real `deleg::header/2 → deleg::header/3` edge with no self-loop. Cowboy (fresh `--depth 1` clone, this build vs unmodified main build): | | main | this PR | |---|---|---| | nodes | 3,668 | 3,748 (+80 — the arity splits; no explosion) | | erlang function nodes | 2,850 | 2,930 | | behaviour dispatch edges | 38 | **44** | | `cowboy_req::header` | one node, span 420–425, /3's spec lost | `header/2` (420–421, its own spec) + `header/3` (424–425, its spec) | | delegation | self-loop | `header/2 → header/3` | `calls` edges drop 6,059 → 5,656: a sample of every removed pair shows the false-positive class the issue predicted — out-of-repo/BIF calls (`length/1`, `error/1`, `quicer:*`) that previously name-matched onto unrelated same-named in-repo functions now stay unresolved. Tests: new arity coverage in extraction + a new arity-resolution integration suite + a #1358 binary-literal behaviour test; updated existing Erlang expectations to the arity-carrying spellings. Full suite: **3,018 passed, 0 failed**. No migration: an existing Erlang index picks the new shape up on its next re-index (`codegraph sync` / re-`init`). Erlang is wasm-only (not in the native kernel), so there is no kernel-parity surface. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
c382225461 |
fix(cli): register the documented context command (#1611) (#1613)
Fixes #1611. ## What `codegraph context <task>` has been advertised in the CLI usage header since the first commit, and the ContextBuilder behind the public `buildContext` API has always shipped in the package — but the command was never registered with commander (verified via `git log -S`: this is drift present from day one, not a removal). Invoking it errored with `unknown command 'context'`, which broke external integrations built against the documented contract — Memorix 1.8.1 invokes `codegraph context --path <project-root> --format json --max-nodes 8 --no-code <task>` and silently falls back to its own heuristic index when the command is missing. ## How Registers `context <task...>` next to the other read commands (`query`/`explore` pattern), mapping flags 1:1 onto `BuildContextOptions`: - `-p, --path <path>` — resolved exactly like every sibling command (nearest initialized project) - `-f, --format <format>` — `markdown` (default) or `json`, unknown values rejected with exit 1 - `-n, --max-nodes <number>` — positive integer, validated - `--no-code` — structure only (`includeCode: false`) JSON output is clean, machine-parseable stdout — `error()` and warnings go to stderr — and the uninitialized-project path matches the sibling commands' error text and exit code. The usage-header line needed no change; the registered syntax matches what it has always advertised. ## Tested New `__tests__/cli-context-command.test.ts` (modeled on `cli-query-command.test.ts`, spawning the built binary against a temp fixture): JSON parseability + shape, `--max-nodes` bounding, the exact Memorix invocation shape (`--format json --max-nodes 8 --no-code`), markdown default, uninitialized-project failure, unknown-format rejection. `npx vitest run __tests__/cli-context-command.test.ts __tests__/context.test.ts __tests__/context-ranking.test.ts __tests__/cli-query-command.test.ts` → 4 files, 39 tests, all green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
a5c2709e6d |
fix(mcp): adopt a single indexed sub-project below the server root + say when no project resolves (#1606, #1607) (#1614)
Fixes #1606 and #1607 together — the MCP server's root resolution never got the sub-project down-scan `planFrontload` gained in #964, and the resulting no-default state was completely silent. ## What changed **Adoption (#1606).** A new `resolveServerRoot()` in `src/directory.ts` is the single resolution every server entry point now uses: up-walk first (`findNearestCodeGraphRoot`, the common case, unchanged), and when that misses, the existing bounded down-scan (`findIndexedSubprojectRoots` — depth 4, max 64, heavy dirs skipped). **Exactly one** indexed sub-project is unambiguous and is adopted as the default project — `open` → `startWatching` → `catchUpSync` → query pool, the full normal path. Zero or several candidates → no default, never a guess. Wired into: - `MCPEngine.doInitialize()` and `retryInitializeSync()` — the retry path also picks up a child indexed *after* the server started (its down-scan is throttled to once per 5s so the persistent no-default state doesn't pay a directory walk per tool call; the up-walk still runs every time). - `resolveDaemonRoot()` — the adopted root gets the shared daemon (one watcher, one writer, socket keyed on the child) instead of a direct-mode server per host, exactly as the issue suggested. - `MCPSession.handleInitialize()` — the instructions variant is picked with the same resolution, so a workspace whose single child becomes the default gets the full single-project playbook. Race-free by construction: handshake and engine compute it independently, no ordering assumed. **Workspace-root gate (the open question in #1606).** Decided deliberately: the down-scan runs only when the base has a workspace manifest (`looksLikeProjectRoot`, unchanged list) **or a `.git` entry** — the exact container shape that motivated the report — and never when the base is `$HOME` or the filesystem root. The gate lives in the new helper only; `planFrontload` and the prompt-hook are untouched, so #1454's surface is not widened. **Diagnostics (#1607).** The no-root branch is no longer silent: ``` [CodeGraph MCP] No .codegraph/ at or above <searchFrom>: no default project, live sync disabled. [CodeGraph MCP] Indexed sub-projects found: service-a, service-b. Pass `projectPath` per call, or launch with --path. ``` (second line only when the scan found candidates), plus one line naming the adopted child when adoption happens. The same fact is protocol-reachable: the "No CodeGraph project is loaded" tool response now lists the discovered sub-projects with `projectPath` guidance. The list is engine-maintained (initial resolve + throttled retry) — tool calls never scan — and the response stays SUCCESS-shaped (`NotIndexedError` → `textResult`, never `isError`). ## Tested - New `__tests__/mcp-subproject-adoption.test.ts` (real spawned server over stdio, same harness as `mcp-roots.test.ts`): single child → tool call answers from it, full instructions, adoption stderr; two children → no default, both listed in the tool response and stderr, per-project instructions; no manifest/no `.git` → gate holds, no scan, plain one-line message. - `mcp-subproject-adoption` + `mcp-roots` + `mcp-initialize` + `daemon-bind-failure`: **13/13 pass**. - End-to-end repro harness against the built `dist/` on an unmodified-main build first (confirmed: empty stderr, no adoption, NO_ROOT instructions even with one adoptable child), then on this branch (all three shapes behave as above; catch-up sync runs on the adopted child). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
d618d94144 |
fix(extraction): detect a plain struct Derived : Base base clause in .h headers as C++ (#1592) (#1593)
Fixes #1592. ## What was wrong A `.h` header whose only C++ construct is a plain derived type — ```cpp struct Base {}; struct Derived : Base {}; ``` — was classified as C. The `.h` language check (`looksLikeCpp`) recognizes `class`, `namespace`, `template`, access sections, `virtual`, `using`, and — since #1159/#1207 — the export-macro form `struct ENGINE_API Derived : Base`. The plain form has none of those signals. Routed through the C extractor, `Derived` vanished from the index and the base clause was read as a K&R-style declaration, minting a phantom `function Base` with `returnType=Derived` (the exact output in the issue). A second, independent miss the reporter called out: the check only read the first 8192 characters, so a large header with a long C-compatible preamble (include guards, `#define`s, plain typedefs) hid the signal even when it was there. ## What this does `looksLikeCpp()` now runs two passes: 1. The existing 8 KB sample regex, unchanged. 2. A scan of the **whole file** (comments stripped) for a class/struct **base clause**: `class`/`struct` + tag + optional `final` + `:` + optional `public`/`protected`/`private`/`virtual` + a base name (scoped, optionally templated) followed by the body's `{` or a `,` introducing the next base. That shape has no valid C reading, so widening it to the whole file can't drag a C header over to C++: - a bit-field's `:` follows a member *name* inside the body (`unsigned a : 3;`), not the tag; - a ternary's `:` is separated from the tag by `)` / `*` / a declarator (`sizeof(struct foo) : 0`); - a label or identifier like `struct_end:` has no whitespace after `struct`; - comments are removed before the scan, so doc-comment prose (`/* struct timeval: seconds, microseconds */`) can't match; and the `{`/`,` terminator keeps a string literal's prose from matching too. Detection only — the C++ extractor already handles the header correctly once it's routed there (renaming to `.hpp`, as the issue notes, already worked). ## Tests `__tests__/extraction.test.ts`: - plain / `: public Base` / `: ns::Base` / `: Base<int, Foo<T>>` / `final : Base` / multi-base with `{` on the next line / `: virtual Base` → `cpp`; - a base clause placed **after** 8192 characters of C-compatible preamble → `cpp`; - controls that must stay `c`: a bit-field struct, `sizeof(struct foo) : 0` + a cast ternary, a `struct_end:` label and `struct_a` identifiers, doc-comment prose shaped like a base clause, and the two pre-existing C controls; - end-to-end `extractFromSource('src/min.h', …)` on the issue's header: a `struct` node `Derived` (language `cpp`), exactly one `Base` node and it is a `struct` — no phantom function. Issue repro re-run against this build: `codegraph init` → `query Derived` returns the `cpp` struct; `query Base` returns only the struct; the files table records `src/min.h` as `cpp`. Full suite: `npm test` → 174 files passed, 3010 tests passed, 179 skipped. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
cf1b0e341a |
fix(sync): refresh the watcher's scope when codegraph.json or a .gitignore changes (#1590) (#1594)
Fixes #1590. ## What was wrong The live file watcher built its scope matcher — built-in defaults + `.gitignore` + the `codegraph.json` `exclude`/`include` rules — once in `start()` and kept it for the watcher's lifetime. The MCP server is long-lived, so a `codegraph.json` created or edited after it started was invisible to the watcher, while `codegraph sync` (a fresh process with a fresh matcher) honoured it immediately. From the user's side: the CLI removed a newly excluded file, and the daemon re-indexed it a few seconds later, which reads as "`exclude` doesn't work". As the report points out, `extensions` on the very same config file *was* read live (its loader is mtime-cached), so two fields of one file behaved differently. There was a second half to it. The watcher's scoped fast path hands the exact edited paths to sync, and that path stat'ed and re-parsed them without consulting the scope matcher at all — so the stale view of scope leaked straight into the index. ## What this does **Watcher — rebuild on a scope change, then reconcile in full.** An event for the root `codegraph.json` or `.gitignore` rebuilds the matcher, marks the next sync as a full reconcile, and schedules it. A scope change has no per-file events: newly excluded files must be *removed* from the index and newly included ones *added*, and only the scan-diff (which builds its own fresh matcher) knows which those are. Two ordering details are deliberate: - the two root files are checked *before* the matcher is consulted, so a user pattern that happens to cover them (`*.json`, `.*`) can't hide their own edits; - a nested `.gitignore` (an embedded child repo's own rules, or a subdirectory rule the git-backed scan honours) is checked *after* the matcher, so the thousands of package-local `.gitignore`s an `npm install` writes under an ignored `node_modules/` can never trigger a rebuild storm. Rebuilding runs embedded-repo discovery (one `git ls-files`), which is fine per config edit and never happens per event. Replacing the field serves both watch strategies: the recursive handler and the per-directory `shouldIgnoreDir` walk read it on every call. **Scoped sync — re-check the paths it was handed.** The orchestrator now runs scoped paths through the same scope matcher and source-extension gate the full walk applies. An out-of-scope path is treated as absent: removed if tracked, never parsed on trust. The matcher is memoized on the mtimes of the two root files it derives from (two `stat`s per sync while nothing changed), so the scoped path keeps skipping O(repo) work — paying embedded-repo discovery per sync would defeat its whole point. ## Tests - `watcher.test.ts` — a `codegraph.json` edit schedules a full sync, after which an edit inside the newly excluded tree is dropped by the live matcher (not pending, no sync) while an in-scope edit still syncs scoped; a root `.gitignore` edit behaves the same; a nested `.gitignore` forces a full sync; a `.gitignore` under `node_modules/` schedules nothing; dropping the exclude again readmits the tree. - `sync.test.ts` — end-to-end through `CodeGraph`: a scoped sync of a path that `codegraph.json` now excludes removes it (`filesRemoved: 1`, nothing parsed — the symbol added to the file never appears), stays out on a repeat, and is re-added through the same scoped path once the exclude is dropped. - All five new tests fail on `main`; the `node_modules` guard passes both ways as expected. - Full suite: 189 files, 3184 passed / 9 skipped. - CLI half of the issue's repro (init with `exclude`, edit the config + the file, `codegraph sync`): the newly excluded file is removed and its new symbol never enters the index. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
838006c947 |
fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) (#1600)
Fixes #1581. ## What was wrong `codegraph init` / `codegraph index` died with `Segmentation fault` — the whole CLI process, not a parse worker — on a C/C++ file with very deep brace nesting (llvm's `clang/test/Parser/parser_overflow.c`, 16,384 nested `{`). The reporter's diagnosis is exactly right: tree-sitter's parser is iterative, so the file parses fine, and then the native kernel's **recursive walker** (`visit_node` → `visit_for_calls_and_structure` → …, one frame per AST level) overflowed the thread's stack. A native overflow can't be caught the way a wasm abort can, and a parse worker is a thread of the `codegraph` process, so the SIGSEGV took the entire indexer down — no message, no per-file fallback, no partial index. Two things made "just give the worker a bigger stack" the wrong fix: - it only moves the cliff — reproduced here: the reporter's 16,384-deep file kills a default 4 MiB worker (rc=132 on macOS / 139 on Linux), and a 100k-deep file kills the 8 MiB **main** thread too; - the walkers are shared by every kernel-routed language (20 of them), and each has several recursion points with different frame sizes, so no single stack size is a provable bound. Meanwhile the wasm path already handles this shape gracefully: its JS walker catches its own `RangeError` per file and stores a partial result with a `parse_error`. The kernel just needed a way to get there instead of dying. ## What this does **The kernel guards its own recursion against the calling thread's real stack bounds and defers a too-deep file to wasm** — the same `defer:` routing signal it already uses for files with parse errors, which `src/extraction/kernel/index.ts` treats as "take the wasm path for this file", silently. - `codegraph-kernel/src/stack.rs`: per-thread stack bounds from the OS, computed once per thread and cached — glibc/musl `pthread_getattr_np` + `pthread_attr_getstack`, macOS `pthread_get_stackaddr_np` + `pthread_get_stacksize_np`, Win32 `GetCurrentThreadStackLimits` (a hand-declared `kernel32` extern; no `windows-sys`). `exhausted()` is one thread-local load and one compare: true once the stack pointer is within a 256 KiB red zone of the limit, and it latches a flag. Where the OS can't report bounds it falls back to a fixed 1 MiB descent budget measured from the entry stack pointer — safe on anything from Node's 4 MiB worker default up. So the guard is exact on the 4 MiB worker, the 8 MiB main thread, and any `resourceLimits.stackSizeMb` alike. - `stack_guard!()` (defined in `lib.rs`) is the first statement of every recursive walker function — all **150** self-recursive or on-cycle functions across the 15 walker modules, found by script (every cycle in the call graph, not just direct self-calls). It returns `Default::default()` (`()`, `false`, `None`, `""`) so an exhausted walk simply stops descending; a hook returning `false` sends its caller down the generic child walk, whose own guard returns at once. - `extract_file` runs the whole walk under `stack::run_guarded`: if the flag is set afterwards the (truncated) result is discarded and replaced by `defer: nesting too deep for the native walker — wasm recovery handles it`. - `parse-pool.ts`: a comment at `new Worker(scriptPath)` records why there is deliberately no `resourceLimits.stackSizeMb` bump. - No new crates beyond `libc` as a direct unix dependency (already in `Cargo.lock` transitively). No wire/ABI change. Net effect for the reporter's repo: `deep.c` goes to the wasm path, lands as `function foo` plus a recorded parse warning, and the other 31,607 files index normally. `CODEGRAPH_KERNEL=0` and the `exclude` workaround are no longer needed. ## Tests **Rust unit tests** (`cargo test`, 21 passed — 7 new in `stack.rs`): the walkers for C, C++, Rust, TypeScript and Python are driven on a **1 MiB** thread (a quarter of Node's worker default) with 30k-deep nesting and must return `defer:` instead of crashing; shallow files are untouched; the latch resets between runs; the OS bounds are sane on the main thread and describe a small thread's own stack. **`__tests__/kernel-deep-nesting.test.ts`** (new, 8 tests — skips without a staged `.node`, fails under `CODEGRAPH_KERNEL_EXPECT=1` if the kernel is missing, like the other kernel suites): - every default-routed language (all 20) survives a 60k-deep expression on the main thread — clean result or the wasm fallback's partial result, never a crash; - the reporter's exact 16,384-brace C file is indexed (partial) on the main thread; - 200-deep expressions in every language still take the kernel path clean (the guard never trips on normal code); - inside a **default-sized 4 MiB `worker_threads` Worker** through `dist/`: the reporter's `deep.c` and a 60k-deep expression in every language come back `deferred` with exit 0, and a normal file still extracts natively; - end-to-end through the built CLI: `codegraph init` on a repo holding `deep.c` + `ok.c` exits 0 and records both files, with both functions. **Existing kernel suites**: all 15 (`kernel-*-parity`, `kernel-scaffold`, `kernel-retry-materialize`, `kernel-grammar-parity`) pass unchanged, 147 tests — the guard never fires on the parity fixtures. **Reporter's probes** (`one.js` from the issue, default 4 MiB worker, this build): `deep.c` → `deferred`, exitCode=0 (was rc=132/139); `deep100k.c` → `deferred`, exitCode=0. Main thread: `deep.c` / `deep100k.c` → wasm partial with `Parse error: Maximum call stack size exceeded`; a 6,000-term binary expression and a 3,000-branch `else if` chain stay on the kernel path with clean results. **Perf** (same `dist/`, only the `.node` swapped via `CODEGRAPH_KERNEL_PATH`; interleaved main/new ×3, `codegraph init`, macOS arm64): | repo | main (median) | guarded (median) | nodes / edges | |---|---|---|---| | express (141 files) | 0.60 s (0.58–0.65) | 0.61 s (0.58–0.61) | 1,084 / identical | | redis (786 C/H files) | 4.44 s (4.39–4.66) | 4.49 s (4.41–4.70) | 19,942 / 76,446 identical | Within run-to-run noise, as expected for one TLS load + compare per recursion entry. **Linux (Docker, `node:22-bookworm`, kernel built in-container, `docker run --rm --init`)** — the reporter's platform and the glibc `pthread_getattr_np` bounds path: ``` === platform === Linux efe3cc86947b 6.12.54-linuxkit #1 SMP Tue Nov 4 21:21:47 UTC 2025 aarch64 GNU/Linux v22.22.3 -rwxr-xr-x 1 root root 35332288 Aug 22 18:02 codegraph-kernel/prebuilds/linux-arm64/codegraph-kernel.node === reporter repro (issue #1581): 16,384-brace deep.c, codegraph init === │ └ Done init exit code: 0 file: deep.c file: deep100k.c file: ok.c function: add function: bar function: foo === worker probe: kernel raw extract in a default 4 MiB worker === deep.c: deferred deep.c: worker exitCode=0 deep100k.c: deferred deep100k.c: worker exitCode=0 ok.c: kernel nodes=2 ok.c: worker exitCode=0 === cargo test stack:: (glibc pthread_getattr_np bounds path) === test stack::tests::os_bounds_are_sane_on_this_platform ... ok test stack::tests::small_stack_reports_its_own_bounds ... ok test stack::tests::normal_files_are_untouched_by_the_guard ... ok test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok test stack::tests::latch_resets_between_runs ... ok test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.23s === vitest: kernel-deep-nesting + kernel-scaffold === ✓ __tests__/kernel-scaffold.test.ts (10 tests) 30ms ✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 36989ms Test Files 2 passed (2) Tests 18 passed (18) ``` (The pre-fix crash was reproduced on macOS — rc=132 in a default worker, rc=139 on the main thread at 100k depth — not re-run inside this container; the reporter's Linux x86_64 trace is the SIGSEGV form of the same overflow.) **Windows (Parallels ARM64 VM, MSVC 14.44, `cargo 1.97`, kernel built on the VM, `GetCurrentThreadStackLimits` path)**: ``` head: cbf8485 fix(kernel): guard the native walkers against stack overflow and defer deep files to wasm (#1581) === cargo build --release (win32-arm64) === Finished `release` profile [optimized] target(s) in 2m 04s staged: 35086848 bytes === cargo test (stack guard unit tests) === test stack::tests::normal_files_are_untouched_by_the_guard ... ok test stack::tests::os_bounds_are_sane_on_this_platform ... ok test stack::tests::small_stack_reports_its_own_bounds ... ok test stack::tests::deep_braces_c_defer_instead_of_crashing ... ok test stack::tests::latch_resets_between_runs ... ok test stack::tests::deep_parens_cpp_rust_ts_python_defer_instead_of_crashing ... ok test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 15 filtered out; finished in 0.49s === reporter repro: codegraph init on a 16,384-brace deep.c === └ Done init exit code: 0 === vitest: deep-nesting + scaffold (CODEGRAPH_KERNEL_EXPECT=1) === ✓ __tests__/kernel-scaffold.test.ts (10 tests) 55ms ✓ __tests__/kernel-deep-nesting.test.ts (8 tests) 67239ms ✓ every default-routed language survives a 60k-deep expression on the main thread 52801ms ✓ inside a default-sized (4 MiB) parse worker, through dist/ > defers a 60k-deep expression in every default-routed language 13050ms ✓ end-to-end: codegraph init on a repo holding the deep file > exits 0 and records deep.c alongside the normal files 936ms Test Files 2 passed (2) Tests 18 passed (18) ``` (The end-to-end test is what reads the Windows index back through `node:sqlite` — `files` = `deep.c`, `ok.c`; functions `add`, `foo`.) Full `npm test` on this branch (macOS arm64, kernel staged): **190 files passed, 3,185 tests passed, 10 skipped, 0 failed.** Clippy note: `cargo clippy` on the current toolchain (1.92) reports 18 pre-existing lints (`manual_contains`, `unnecessary_to_owned`, …) in walker code this PR only touched by inserting guard lines; none are in `stack.rs`/`lib.rs`. Left alone to keep the diff reviewable. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
0d17dfd6a8 |
feat(cli): install --init and init --yes for a one-shot, non-interactive bootstrap (#1578) (#1595)
Fixes #1578. ## What was wrong Bootstrapping CodeGraph in a fresh environment — the issue's case is a throwaway container per AI session — took two commands, `codegraph install --yes` and then `codegraph init`, and the second one could still stop on a prompt (the gitignored-child-repos offer, the watch-fallback offer on WSL/`/mnt`). There was no way to wire agents and build the project's index in one non-interactive line. The installer's "never index implicitly" rule is deliberate (a surprise index of `$HOME` is exactly what `init` refuses), so the gap is an explicit opt-in, not a change in default behavior. ## What this does - **`codegraph install -i, --init`** — after wiring the agents, runs the `init` flow in the current directory. It also runs when nothing was wired (`--target none`, no agents detected), since the installer returns normally in that case. Every `init` guard applies: a home directory / filesystem root / parent of home is **refused with exit code 1** (no implied `--force`), and an already-initialized project just reports that and exits 0. `--print-config` and `--refresh` return before the install, so `--init` is a no-op with them. - **`codegraph init -y, --yes`** — non-interactive: the ignored-repos offer prints its one-line `includeIgnored` opt-in snippet instead of prompting (the existing non-TTY behavior), and the watch-fallback offer takes its `yes` default. `install --init` passes `--yes` through, so `codegraph install --yes --init` is a fully unattended bootstrap. - The `init` action body becomes `runInit()`, shared by both commands. The plain `init` path is behavior-identical (same refusal, already-initialized notice, supervised index, telemetry, offers, outro). - The post-install "Next: index a project" note gains one line mentioning `--init`; README gets the flag row and a `--yes --init` example. On the reporter's other observation — `install --yes` skipping the "install the CLI on your PATH" step: that's by design for scripted use (it assumes the CLI is already present), and the `bunx @colbymchenry/codegraph serve --mcp` MCP entry they found is the self-contained alternative. Not changed here. ## Tests `__tests__/cli-install-init.test.ts` — end-to-end against the built binary with stdin closed (a blocking prompt would fail), always `--target none` so the suite never touches an agent config on the host: - `install --yes --target none --init` → exit 0, installer reports nothing to wire, `Initialized in <tmp>`, `.codegraph/codegraph.db` exists; - the same on an already-initialized project → `Already initialized`, exit 0; - the same at the filesystem root → exit 1, `Refusing to initialize`, nothing written; - `init --yes` with stdin closed → exit 0, index built; - `init --help` lists `-y, --yes`, `install --help` lists `-i, --init`. `npx vitest run __tests__/installer-targets.test.ts __tests__/upgrade.test.ts` → 283 passed, 3 skipped. Full `npm test` → see the checks on this PR / below. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
278a8edc35 |
fix(resolution): resolve calls to object-literal namespace members (#1573) (#1597)
Fixes #1573. Thanks @IAliceBobI — the report had the root cause exactly right, and the fix sits one layer up from the suggested spot (resolution rather than the container-kind set), for the reason below. ## What was wrong Methods of an exported object-literal constant — `export const api = { call() {…}, get: () => {…} }` used as a module's API surface — never received a call edge from `api.call()`, same-file or through an import. The members are extracted as plain functions with **bare** qualified names (`call`, not `api::call`) sitting inside the constant's source extent, so: - the `Container::member` lookup the class-shaped kinds use (#825) bails on kind `constant`, and even with `constant` added to that set there is no `api::call` to find; - the declared-type inference for imported singleton instances (#1292) finds no type in a literal and falls back to the constant edge; - the same-file strategies only consider classes and `method` kinds, so the call resolved to nothing at all. Net effect: `callers` / impact reported zero for methods called from everywhere, with no boundary warning because nothing about `obj.method()` looks dynamic. ## What this does Adds one helper that resolves a member **by containment** — a node named `member` whose source range lies inside the value's range, in the value's own file — and uses it from both halves: - **Import path**: when the imported value is a constant/variable, the literal member is tried right after the `Container::member` lookup and before the #1292 instance inference, so the cross-file edge lands on the method instead of the constant. - **Same-file path**: a same-file constant/variable receiver (TS/JS family only) is checked before the class-name strategies. Precision rules, all tested: calls accept callable kinds only; a declaration nested inside another member's body is not a member; nothing outside the value's range can donate a match — a same-named top-level function, or a method returned by a factory the value merely holds — so those cases keep today's behavior rather than guessing. Class statics (`C.s()`) and non-literal values are untouched. Extraction and qualified names are deliberately left alone: changing how literal members are named would have to be mirrored in the native kernel byte-for-byte, and the resolver-side lookup is contained and language-gated. ## Tests - The issue's repro end-to-end: `sameFileCallers` and `crossFileCaller` are both callers of `m`; a decoy `m` in a third file gets none; the `C.s()` static control resolves exactly as before; `crossFileCaller` no longer has a `calls` edge to the constant. - Arrow-property and method members both resolve; a `function call()` nested inside `get`'s body is never taken for `api.call()`. - A value holding a factory's result (`const obj = makeObj()`) with a same-named top-level `m` in the file: no false attribution, existing behavior kept. - The two positive tests fail on `main`; the control passes both ways, as a guard should. - Full suite: 189 files, 3181 passed / 9 skipped. With the built CLI on the issue's `a.ts`/`b.ts`: `codegraph callers m` → 2 callers (`sameFileCallers`, `crossFileCaller`); `callers s` unchanged; edges `sameFileCallers -> m` (0.85) and `crossFileCaller -> m` (import, 0.9), none to `obj`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
7963672689 |
fix(rust): resolve self.field.method() on the field's declared type instead of a same-named method (#1585) (#1599)
Fixes #1585. **Stacked on #1596** (the base branch is `fix/1588-rust-impl-type-qualification`; this PR's own diff is the second commit). Merge #1596 first, then retarget/merge this one. ## What was wrong ```rust impl Outer { pub fn run(&mut self) { self.inner.run(); // inner: Inner } } ``` produced `Outer::run -> Outer::run` — recursion the source doesn't contain. The extractor collapsed every `self.<field>.<method>()` receiver to the bare method name (`run`), so the resolver only ever saw `run` and exact-matched the nearest same-named method — the calling method itself, or a method of an unrelated type. Nothing marked the edge as a guess, and no row stayed in `unresolved_refs`, so a consumer had no way to tell. The same happened when the field's type isn't a project type at all (`its: std::vec::IntoIter<_>` → `self.its.next()`, `matcher: Regex` → `self.matcher.is_match()`): the bare `next` / `is_match` attached to whatever local method shared the name. ripgrep had 279 self-edges on `main`; the issue lists three sites, all of this shape. (The issue's C++ control — "`Outer::run -> Inner::run` resolves correctly" — doesn't actually hold on `main`: `inner.h` is classified as C by the `.h` heuristic, so `Inner::run` never exists and the C++ repro self-edges too. That's #1592, fixed separately.) ## What this does Rust struct fields are not graph nodes, so the field's type can only come from the struct's declaration text. This follows the Go 2-hop precedent exactly (`matchGoFieldChainCall`, #1276), including its exclusivity rule: 1. **Extraction (TS walker + native kernel, identical, parity-tested):** a call whose receiver is `self.<field>` keeps the owner-field shape — `self.inner.run()` is emitted as `self.inner.run`. Deeper chains (`self.a.b.m()`), call receivers (`self.f().m()`), parenthesized receivers and bare `self` keep the bare name, exactly as before. 2. **Resolution (`matchRustSelfFieldCall`):** owner type = the calling method's qualified-name prefix (`Outer::run` → `Outer`); the field's declared type is read from the owner struct's **own declaration lines** (comment-stripped, line by line — same discipline as the Go helper); the method is resolved **and validated** on that type by `resolveMethodOnType` (confidence 0.85, `instance-method`). 3. **Exclusive:** when the field is declared with an external type, a generic parameter (`T`), a container that doesn't auto-deref (`Option`/`Vec`/`Mutex`/…), or can't be found, the ref **stays unresolved** — it never falls through to the bare-name strategies. That is the safe behaviour the issue asks for, and it is what #1276 already chose for Go. `rustFieldTypeName` looks through exactly the layers Rust's method-call auto-deref looks through: references (`&`, `&'a mut`) and the owning smart pointers `Box`/`Rc`/`Arc`. `Box<dyn Source>` yields the trait, whose method node the interface-impl synthesizer then fans out to every implementation. `Option<Inner>` is left alone — `self.inner.take()` is Option's method and must not become `Inner::take`. Why it stacks on #1596: the owner is taken from the method's qualified name, which for a generic/lifetime impl was the trait's name before that fix. ## Measured on ripgrep (110 `.rs` files, #1596 build vs this branch) | | #1596 | this PR | |---|---|---| | nodes | 4029 | 4029 | | `calls` self-edges | 279 | **146** (none of the `self.<field>` shape remain — 116 bare-receiver, 30 other dotted) | | `self.<field>.m()` calls resolved through a validated field type | — | **292** (`DecompressionMatcher::command -> GlobSet::matches`, `Parser::find_long -> FlagMap::find`, `Haystack::path -> DirEntry::path`, …) | | `self.<field>.m()` calls left unresolved | — | **417** — every sampled one is a std/container method: `self.commands.push`, `self.child.wait`, `self.pre.is_some`, `self.colors.clone`, `self.path_terminator.unwrap_or` | | `calls` edges total | 9150 | 8878 (the 272 removed are the former bare-name guesses for those 417) | The issue's three sites: `walk.rs:824` now resolves to `IgnoreBuilder::add_custom_ignore_filename` (was a self-edge); `walk.rs:1195` (`self.its.next`, `IntoIter`) and `globset/lib.rs:983` (`self.matcher.is_match`, `Regex`) are parked as unresolved instead of guessed. The issue's repro gives `Outer::run -> Inner::run` (`instance-method`, confidence 0.85) on both the kernel path and `CODEGRAPH_KERNEL=0`. ## Tests - `__tests__/extraction.test.ts`: only the single-hop `self.<field>.<method>()` call keeps the prefix; deeper / call / parenthesized / bare-`self` receivers and a local receiver are unchanged. - `__tests__/resolution.test.ts` (end-to-end, Cargo layout): the issue's repro → `Outer::run -> Inner::run`, no self-edge; an external field type (`std::vec::IntoIter`) with a local `next` decoy → no edge at all; `Box<Inner>` and `&'a mut Inner` resolve, `Option<Inner>` does not (even though `Inner` declares the method); a generic `T` field → no edge; genuine `self.run()` recursion keeps its self-edge; the #1588 repro's `UsesFile::go` / `UsesBuf::go` resolve to `FileSource::read` / `BufSource::read`, and a `Box<dyn Source>` field lands on `Source::read` with the synthesizer fanning out to both impls. - `__tests__/fixtures/kernel-parity/torture.rs` grows the receiver shapes; all 15 kernel parity suites pass against the rebuilt kernel (147 tests). - Full `npm test` on this branch: 189 files, 3187 passed, 9 skipped, 0 failed. Re-index after upgrading. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby MchenryandGitHub
|
12f7a59f26 |
fix(rust): qualify generic/lifetime impl methods by the implementing type, not the trait (#1588) (#1596)
Fixes #1588. ## What was wrong The receiver of an `impl` block — the name that qualifies its methods, owns the `contains` edge, and sources the `implements` edge — was found positionally: the **last bare `type_identifier` child** of the `impl_item`. That works for `impl Source for FileSource`. But once the implementing type carries parameters it parses as a `generic_type`, and the only bare identifier left is the **trait's**: ```rust impl Source for FileSource → FileSource::read ✓ impl<T> Source for BufSource<T> → Source::read ✗ (should be BufSource::read) impl<'a> Iterator for Parents<'a> → Iterator::next ✗ impl Trait for &Foo → Trait::method ✗ ``` Two consequences, both reproduced on `main`: - `BufSource::read` did not exist in the graph, so `resolveMethodOnType("BufSource", "read")` and "who calls `BufSource::read`" had no answer, and every generic implementation of a trait collapsed onto the same trait-qualified name. - Because the impl's method carried the trait's qualified name, the interface-impl synthesizer treated the impl **body** as a second trait declaration and emitted a dispatch edge from it (`Source::read -> FileSource::read`, registered at the generic impl's line — a body of `{ 0 }` containing no call at all). The native kernel (`rustlang.rs`) mirrored the positional rule deliberately, bug-for-bug, to hold byte-parity with the TS walker — its header said "preserve, never fix via the grammar's trait:/type: fields". So the fix has to land on both sides at once. ## What this does Both extractors now read the grammar's **named fields** instead of scanning children. One shared rule (`rustImplTypeName` in `languages/rust.ts`, `impl_type_name` in the kernel), applied to `impl_item.type`: | implementing type | node | receiver | |---|---|---| | `Foo` | `type_identifier` | `Foo` | | `Foo<T>` / `Foo<'a>` | `generic_type` → its `type` field | `Foo` | | `m::Foo` | `scoped_type_identifier` → its `name` field | `Foo` (was: no receiver) | | `&Foo` / `&'a mut Foo` | `reference_type` → its `type` field | `Foo` | | `(A, B)`, `dyn Tr`, `*const T`, `u32`, fn types | anything else | none — extracted as plain functions, exactly as before | The `implements` back-reference reads `impl_item.trait` (full text, so `fmt::Display` and `From<u32>` keep their spelling) and bails when the field is absent (inherent impl). Everything else — the no-scope impl quirk, the source-order `contains` owner scan, method extraction — is untouched; the `contains` edge simply lands on the implementing type now instead of the trait. The kernel header comment, the parity test's description, and the two design docs that documented the quirk as "preserve" are updated to say what changed. ## Measured on ripgrep (110 `.rs` files, `main` build vs this branch) | | main | this PR | |---|---|---| | nodes / methods | 4029 / 2202 | 4029 / 2202 | | impl methods qualified by a **trait** name (node outside that trait's extent) | 61 | **0** | | `Iterator::*` methods | 2 | 0 | | duplicate method qualified names | 77 | 42 | | synthesized `interface-impl` edges originating **outside** any trait declaration (the phantom fan-outs) | 38 | **0** | | synthesized `interface-impl` edges originating at a real trait declaration | 33 | **52** | | plain (non-heuristic) `calls` edges | 9098 | 9098 | So the synthesizer lost every phantom edge and *gained* 19 legitimate fan-outs to implementations it could not previously see as implementations. `contains` edges went 5237 → 5224: the 13 removed were trait→impl-method edges produced by the mis-qualification. The issue's repro now gives `BufSource::read` at line 12, `BufSource -> Source`, and both synthesized edges registered at the declaration (line 2) — identical on the kernel path and with `CODEGRAPH_KERNEL=0`. (The remaining `UsesFile::go -> BufSource::read` exact-match guess there is the separate `self.field.method()` receiver problem, #1585, which stacks on this.) ## Tests - `__tests__/extraction.test.ts` (Rust Extraction): method qualified names for generic / lifetime / reference / scoped / generic-trait impls; the trait's qualified name names exactly one node; `implements` refs come from the implementing type for every shape; the `contains` edge lands on the type; tuple / `dyn` impls keep producing plain functions with no `implements` ref. - `__tests__/resolution.test.ts` (end-to-end): `Source::read` names only the declaration; dispatch fans out to **both** `FileSource::read` and `BufSource::read`, every synthesized edge registered at line 2; neither impl body sprouts a synthesized call. - `__tests__/fixtures/kernel-parity/torture.rs` grows all the new impl shapes; `kernel-rustlang-parity` (LF + CRLF) passes against the rebuilt kernel. - `CODEGRAPH_KERNEL_EXPECT=1 npx vitest run __tests__/kernel-*.test.ts` — all 15 suites, 147 tests pass. - Full `npm test`: 3180 passed, 9 skipped, 1 failed — `mcp-daemon.test.ts > daemon idle-times-out after the last client disconnects`, a 30 s timing test that passed on re-run in isolation (the machine was running four parallel suites and kernel builds at the time); unrelated to extraction. Re-index after upgrading to pick up the corrected names. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK |
||
|
Colby McHenryandClaude Fable 5
|
44e1812d3b |
changelog: cover the merged contributor batch (#1547, #1215, #594, #1463)
Write the missing [Unreleased] entries for the Vapor route hang fix, the untracked-directory status gap (described for its current status-only symptom — sync itself reconciles off the filesystem), and the new deprioritize config key; move the .xsjs/.xsjslib resolution entry out of the released 1.0.0 block, where a stale rebase had left it; credit @maxmilian across the batch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
 Max HsuandGitHub
|
bc894802ff |
feat(installer): support project-local Codex installs (#1531) (#1551)
Codex CLI has a first-class project config layer — `.codex/config.toml` is layer 4 of the loader stack, above the user config at layer 6 (`codex-rs/config/src/loader/README.md` in openai/codex), and it landed in openai/codex#8354 on 2025-12-22. The CodexTarget's "Codex has no project-local config concept" note was therefore never accurate, and `supportsLocation('local') === false` made Codex the one agent that forces a machine-wide MCP install. `mcp_servers` is not on the project layer's denylist (which strips base URLs, model providers, `notify`, profiles and otel — settings repo contents shouldn't choose), so a project-scoped `[mcp_servers.codegraph]` is honored. - Path helpers take a `Location`: global keeps `~/.codex/config.toml` + `~/.codex/AGENTS.md`; local writes `<cwd>/.codex/config.toml` and the project-root `<cwd>/AGENTS.md` — the same split the gemini and opencode targets already use for their local layout. - Drops the five `loc !== 'global'` early returns from detect, install, uninstall, printConfig and describePaths. - Local install returns a note that Codex only applies a project layer in a project marked trusted; untrusted projects load the layer but leave it disabled, so a silent success would be misleading. - Refreshes the two doc comments that used Codex as the example of a global-only target (now the Copilot CLI). Tests: two new cases covering the local write layout, the trust note, global config staying untouched, and local uninstall leaving the global entry intact. Both fail against the previous implementation. The generic per-target contract suite now also exercises codex at location=local. |
||
|
|
474f051d3c |
fix(resolution): load path aliases through tsconfig extends and base configs (#1534) (#1548)
`loadProjectAliases()` read only the root `tsconfig.json` / `jsconfig.json` own `compilerOptions`, so an Nx-style monorepo — every alias declared in a `tsconfig.base.json` — got `null` back and every cross-package import fell through to name-based matching. Silently: no unresolved-import warning, and the results still look precise. Two things were missing, and either one alone leaves a common Nx layout broken: Fold the `extends` chain into the effective options before building the alias map. Relative and `node_modules` package specifiers both resolve, the nearest config wins (tsc replaces `paths` rather than merging), and a config already on the current chain is not re-entered, so `a extends b extends a` terminates instead of recursing forever. `paths` are anchored at `baseUrl` when one is declared — itself relative to the config that declared it — and otherwise at the directory of the config that declared the `paths`, which is what tsc does and what keeps an inherited `src/*` from being read as root-relative. Read `tsconfig.base.json` as a last candidate. A root `tsconfig.json` is still authoritative when it exists and reaches the base through `extends`; the fallback covers the layouts where that never happens — a solution-style root config (`references`, no `extends`, no `paths`, which is what nx's own repository ships) or no root `tsconfig.json` at all. A candidate that contributes no aliases no longer shadows a later one that does. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Colby McHenry <me@colbymchenry.com> |
||
|
|
9219967e43 |
perf(search): seek the name index for exact-name lookups (#1542)
`nodes` carries two name indexes and neither can serve
`WHERE name = ? COLLATE NOCASE`: `idx_nodes_name` is BINARY-collated, and
`idx_nodes_lower_name` is an expression index the planner only matches against
the same expression. All three whole-name lookups in the query layer were
written that way, so each one degraded to a full table scan
(`EXPLAIN QUERY PLAN` reports `SCAN nodes`).
The LIMITs on those queries do not rescue them. SQLite can only stop early once
it has produced LIMIT rows, and the two dominant cases never get there: a query
word that names no symbol at all, and a name with only a handful of definitions.
`searchNodes` runs its supplement once per query term; `findNodesByExactName`
runs two passes per symbol extracted from the question, and extraction is
generous, so a plainly-worded question issues a dozen full scans.
Written as `lower(name) = lower(?)` the same predicate seeks
`idx_nodes_lower_name`. Measured on four indexed repositories, baseline vs fix
in one process (the only difference being how the predicate is spelled):
query "how does the retry backoff work" findNodesByExactName searchNodes
gin (2.5k nodes) 1.27ms -> 0.18ms 3.1 -> 2.6ms
Alamofire (4.5k nodes) 2.39ms -> 0.22ms 4.9 -> 4.0ms
excalidraw (11k nodes) 10.54ms -> 0.17ms 10.4 -> 5.8ms
django (62k nodes) 49.91ms -> 0.17ms 27.6 -> 4.9ms
The seek is flat across all four; the scan grows with the corpus. A one-word
query into `searchNodes` on django is unchanged (~20ms) because a single term's
scan is not what dominates it there.
Lowering the parameter in SQL rather than in JavaScript is deliberate. SQLite's
`lower()` and NOCASE both fold ASCII only, while JavaScript's `.toLowerCase()`
folds Unicode; comparing a JS-lowered parameter against `lower(name)` would
silently stop matching non-ASCII identifiers that NOCASE used to match.
`getNodesByLowerName` is spelled the same way for the same reason. It already
sought the index, but as a bare `lower(name) = ?` it took a pre-lowered
parameter on trust: any input carrying an uppercase letter returned nothing at
all. This is behaviour-neutral for its one caller — `matchFuzzy` lowers in
JavaScript before calling, and `lower()` over an already-lowered string is a
no-op, verified over the ASCII and non-ASCII cases alike. It closes the trap for
the next caller; the non-ASCII gap on the `matchFuzzy` side is a resolution
change and is deliberately not bundled here.
Result sets are unchanged, including which rows the LIMITs keep: entries under
one key in the expression index are ordered by rowid, the same order a table
scan produces. Verified over 14,400 lookups (top-400 names of the four
corpora, probed as stored / upper / lower, against all three call sites) with
zero differences, and end-to-end above with identical result ids.
Tests assert the planner's verdict rather than a wall-clock number, so they are
deterministic: they intercept the SQL each call site prepares and require an
index seek, with a guard that the lookups actually ran. Reverting any call site
turns them red.
Co-authored-by: Colby McHenry <me@colbymchenry.com>
|
||
|
Max HsuandGitHub
|
a74029105a |
fix(resolution): resolve ES imports targeting .xsjs/.xsjslib files (#556) (#594)
The extraction half of #556 — indexing `.xsjs` / `.xsjslib` as JavaScript — already landed on main via #654. This PR is now scoped to the remaining resolution gap: the JS import-resolution list did not include the SAP HANA extensions, so an extensionless `import { x } from './helpers'` in a `.xsjs` file resolved to nothing and the cross-file call edge was dropped. Add `.xsjs` / `.xsjslib` to the `javascript` entry in EXTENSION_RESOLUTION so those imports resolve to their target file and `codegraph_callers` / `codegraph_impact` see the edge. One resolution test covers the .xsjs -> .xsjslib import; the now-redundant extraction/detection tests were dropped (covered by #654). |
||
|
Colby McHenry
|
ccb0295259 |
fix(explore): reliably pin extension-less kebab-case file basenames in queries
Previously, naming a kebab-case file without its extension (e.g., `background-image-table` vs. `background-image-table.tsx`) in a `codegraph_explore` query would shred the name into fragments (`background`, `image`, `table`), admitting irrelevant sibling files and crowding out the intended target. This change introduces a new resolution pass in `extractQueryPaths` specifically for extension-less kebab basenames. Queries now accurately identify and pin these files. Unresolved hyphenated prose (e.g., `cross-call`) is left in the query for FTS without being flagged as an unknown path. Resolution prioritizes explicit slashed/dotted paths and respects an ambiguity budget for common stems to prevent over-pinning. |
||
 
|
81e1f4a92f |
fix: harden daemon and large-index recovery paths (#1562)
* fix: harden indexing recovery and daemon liveness * test: cover daemon and recovery review gaps * test: pin that a failure marker never blocks a later successful parse (#1557 retry-discard guard) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: danusha2345 <ewidusoc498@gmail.com> Co-authored-by: Colby McHenry <me@colbymchenry.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
d8f2eeaddf |
fix(db): loop-append dense unresolved-ref result rows; make stripped-salvage visible (#1558) (#1576)
Real-world validation of #1575 on indexes damaged by the released v1.5.0 binary surfaced both of these. getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter limit but appended each chunk's RESULT rows with a spread — every row becomes a call argument, so a dense recovery sync (the #1541 self-heal re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit and killed resolution mid-sync with "Maximum call stack size exceeded", leaving the graph 226k edges short until another sync resumed the orphans (and that sweep resolves measurably worse than the batched path — see the follow-up issue). The failed-ref retry loader had the identical pattern on unbounded result rows. Both append with a loop now (#1558). The #1575 stripped-salvage warning also never rendered: init's summary prints only index_partial warnings and counts only hard errors, so a run with salvaged files still read as fully clean — and with no hard errors the detail wasn't written to errors.log either. Salvage entries now carry code 'salvaged_stripped', the summary prints a visible warning naming the files, and errors.log is written for salvage-only runs. Validated on real corpora with full-graph dumps: healthy-path inits stay byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a determinism control); a realistically-damaged index (41 wiped + 5 missing files, damage generated by the released binary) heals in one plain sync to identical per-file counts and an edge set within the normal incremental residual; pathological mass damage (52% of the repo) completes without crashing. New regression test reproduces the RangeError on the old code with 200k pending refs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
26045b3159 |
fix(extraction): decode kernel results in indexAll retry passes; self-heal wiped rows (#1541) (#1575)
The parse-pool workers return kernel-language extractions as an undecoded buffer transport (nodes/edges EMPTY, tables in kernelBuffers). indexAll's main loop decodes them (or hands the buffers to the store worker), but its two retry passes — plain retry and the comments-stripped last resort — stored the transport as-is: the storage gate passed via errors.length === 0, zero nodes were inserted, and the files row was written with node_count = 0 while the original error was spliced out of the summary. Any worker crash/timeout whose in-flight file was a kernel-routed language permanently recorded that file as "(0 symbols)" — silently, and immune to later syncs because the stored hash matches the on-disk bytes (#1541; v1.4.1 predates the kernel path, which is why it was unaffected). - Both retry passes now materialize kernel results before the gate, store, counters, and log lines. - storeExtractionResult materializes at entry as defense-in-depth, so no storage path can persist an undecoded transport again. - Zero-node rows on symbol-bearing languages (only the wipe produces these — every real extraction stores at least the file node) are dropped during full-reconcile sync and indexAll so already-affected files re-index automatically after upgrading. Scoped watcher syncs leave rows outside their scope untouched. - The comments-stripped salvage now downgrades the failure to a visible warning instead of erasing it: the recovered result can be incomplete, and reporting clean success made a fresh index quietly disagree with a later per-file re-parse of the same bytes (#1565's init-vs-sync divergence). Repro (released 1.5.0): CODEGRAPH_PARSE_TIMEOUT_MS=1 codegraph init on any Python project → "Retry OK: <file> (0 nodes)" and permanent "(python, 0 symbols)" rows. Fixed build stores real symbols under the same forcing, and heals rows wiped by prior runs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
d806317897 |
fix(explore): improve query accuracy for file paths, camelCase, and variables
`codegraph_explore` previously struggled with accurately interpreting user queries. Explicitly named file paths were shredded, making it hard to target specific files; natural language queries often missed camelCase identifiers; and state held in variables was overlooked as starting symbols. This commit introduces several improvements: - **Reliable File Path Resolution:** Naming a file by its path in a `codegraph_explore` query now works reliably. The path is resolved against the index, and that file is guaranteed a place at the top of the answer. Previously, paths were broken into fragments — bracketed route segments like SvelteKit's `[id]` made this worst — and pieces like `page` or `runs` matched every sibling file, so the file you actually named could be crowded out. A path that doesn't match any indexed file is now called out instead of silently ignored. - **CamelCase Matching for Queries:** Plainly-worded `codegraph_explore` questions now find camelCase code. A query like "auto-scroll to bottom" can reach a function named `scrollFeedToBottom`, because query words are matched against the words inside identifiers, not just whole names. - **Variable and Constant Seeding:** Variables and constants now count when `codegraph_explore` picks its starting symbols, so state held in plain variables — `$state`-style variables in Svelte, for example — no longer gets overlooked. |
||
|
Colby McHenry
|
d289bf84d3 |
Merge main into fix/union-declarations-not-indexed
Resolves the CHANGELOG conflict — main and this branch each prepended a bullet to [Unreleased] > Fixes; both are kept. Everything else auto-merged, including src/mcp/tools.ts, which main reworked heavily for the explore allocation/displacement work (CG-28/31/36/38) while this branch added the `union` kind to its container sets. Verified on the merged tree with the native kernel built: 3070 passed, 9 skipped, 0 failed. |
||
|
Colby McHenryandClaude Opus 5
|
5b0c4b8b93 |
fix(resolution): trait dispatch reaches union implementors (#1515)
Making unions first-class nodes leaves the third loss in #1515 open: interfaceOverrideEdges enumerates its concrete side as ['class','struct'], so a union implementor is skipped even though it now has a real node and a real `implements` edge. "Who implements this trait" then answers wrongly rather than incompletely — the struct beside it bridges and the union does not. Add 'union' to that tuple, plus a regression test that pins the Rust trait -> union-impl hop (the struct implementor is the control proving the synthesizer ran). Verified the test fails on the union assertion alone before this change. No EXTRACTION_VERSION bump: main is already at 25 against v1.5.0's 24, so existing indexes are flagged stale for the next release regardless, and over-bumping is what turns the re-index hint into noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
493d4210f1 |
Merge branch 'main' into feat/copilot-installer-targets
# Conflicts: # CHANGELOG.md |
||
|
Colby McHenryandClaude Fable 5
|
2962e7e1f4 |
feat(mcp): surface supported languages in MCP server instructions (#671)
Recut of #678 against the current instructions — the original predated the explore-first rewrite and conflicted in both files it touched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby MchenryandGitHub
|
99f2ebf0d1 |
Merge pull request #1527 from colbymchenry/bugfix/CG-38
CG-38: guarantee an agent-named symbol renders, wherever it sits |
||
|
Colby McHenry
|
2c708caf7c | Merge branch 'main' into feature/CG-35 | ||
|
Colby McHenry
|
89c53ddf24 |
fix(explore): guarantee an agent-named symbol renders, wherever it sits (CG-38)
`codegraph_explore` never returned `queueMessage` (L1087) or `flushQueuedMessages` (L1102) from a 1,414-line file, on a symbol bag or a prose question, even with that file at rank #1 holding 67% of the envelope — the agent got a same-stem `QueuedMessage` interface at L70 and had to Read the file for the functions it had named. Pre-existing at every build including pre-epic (controlled bisect, index held fixed). Two independent causes: 1. `buildFlowFromNamedSymbols` returns the Flow prose AND the set of node ids the agent named — and the latter is the whole guarantee, since it injects a named def into its file's cluster ranges at importance 9. Its bail-outs returned EMPTY, zeroing the identity whenever there was nothing to PRINT. Two sibling closures that never call each other produce no chain, no synth hop and no boundary, so both defs lost importance 9 and the file rendered from its head. `identityOnly()` now separates the two, gated on shape-precise tokens so a prose word that exact-matches a callable cannot promote itself. 2. The ceiling trim filled in SOURCE order, so an over-ceiling render always dropped the END of a large file first. The shrink HAD kept both symbols (1022-1121); the trim cut back to 839. `windowToCeiling` now takes the spine call site plus every importance>=9 member as focus lines, tries the full ceiling first, and splits the held-back reserve evenly with carry-forward — greedy-in-source-order reproduced the bug one level down. The shrink's loose size estimate is left alone deliberately, and the comment now says why: making it exact was built and measured WORSE (it stops at the last member that fits whole and the released bytes carry forward to lower-ranked files, costing payroll-go's `s.store.Upsert`). `bound()` clamps to the ceiling anyway, so the slack costs no bytes; it just must not pick the survivors, which is what the trim now handles. The measurement gap this closes: every existing probe is aggregate — envelope share, per-file spend, source totals, file counts — and all are green on a response that returns 25K from the right file and omits the named function. `probe-named-symbol.mjs` checks the definition LINE against the response's rendered lines, per symbol. Suite envelope byte-identical to main on all six repos; probe-allocation 4/4, no starvation flags; 180 files / 2,997 tests green. Fixture: 7/7 fail on main, 7/7 pass here, deterministic over 4 runs per arm. |
||
|
Colby McHenry
|
10f1ac601a |
docs(benchmarks): record the CG-36 cluster-starvation measurement
The issue blamed the density tiebreak; both real cases lost on maxImportance, so ranking was left alone. Full before/after table, the one cost (okhttp's rank-6 file, squeezed out by reservations that were already structurally over-subscribed), and what ships to keep it measurable. |
||
|
Colby McHenryandClaude Opus 5
|
9efae0f8f2 |
fix(explore): damp ambient declaration files on flow queries (CG-28)
A file that declares nothing but types and that nothing in the index depends on — a hand-written ambient `.d.ts` of global shims, vendored typings, module augmentation — cannot answer a flow question: no bodies, no call edges, no behaviour, nothing typed by it. But the identifiers it declares are exactly the generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`, `ReadableStream`), so on term overlap it out-scored the implementation. Measured on the new fixture: rank #1 and 51% of delivered source, with the flow's own entry file pushed out of the response entirely. Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that opened this is already handled by CG-25's banner detection, worth 15-46 points of envelope share across four flow queries. CG-25 credited; only the un-bannered case needed anything. `rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken as the STRONGER of it and the generated penalty rather than multiplied — one property two signals see must not be charged twice. Detection is structural, not by extension, and four conditions deep. Two of them were forced by measurement: requiring every symbol to be type-level takes the corpus flag rate from 1-18% (which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's locale tables) down to 0-4%; requiring that nothing depends on the file separates an ambient shim from a working types module, and without it the rule demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate. A query that NAMES a declared type is exempt, so a question about a type still reaches its declaration at full weight. Precise tokens only, so "…the file body…" cannot exempt a `Body` interface it never meant to name; this needs its own set because `namedSeedIds` is callable-only and a type never becomes one. Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md: 6-repo envelope sweep byte-identical against a clean baseline build, zero ambient files reach the candidate set on VS Code across five queries, corpus flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
dc4fd755ef |
merge: recognize Wrangler-style generated banners (CG-25)
A generated Cloudflare Wrangler ambient-types file was not flagged generated, so
it ranked with no penalty and competed with hand-written source on generic token
overlap. The banner shape it uses — "Generated by <tool> by running <command>" —
matched none of the existing content patterns, all of which require DO NOT EDIT,
a standalone @generated, or the "auto(matically) generated by" phrasings.
Precision is held by requiring TWO 'by' clauses: the banner must name a tool and
then say 'by running'. Ordinary prose ("the report is generated by running the
nightly job") has only one and does not match.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
57e0854213 |
fix(explore): recognize Wrangler-style "generated by … by running" banners (CG-25)
Cloudflare Wrangler's `worker-configuration.d.ts` (~12k lines of ambient types) carried no banner any GENERATED_CONTENT_PATTERNS entry matched: every existing marker requires `DO NOT EDIT`, a standalone `@generated`, `<auto-generated>`, or the literal `automatically/auto-generated by` phrasings. Wrangler emits a bare `Generated by Wrangler by running `wrangler types``, so the file ranked with pen 1.00 and won 79.4% of an explore envelope on generic token overlap alone (CG-24). The discriminator is the reproduction instruction, not the word "generated": the banner must name a tool AND then say `by running`, i.e. two separate "by" clauses. That keeps prose out — "the nightly summary is generated by running the ETL job" has only one — while catching every CLI-driven emitter that tells you how to regenerate. Precision swept over 441,856 files across the whole local source tree: 5 hits, all genuine Wrangler output, no false positives. Isolated before/after on the CG-24 repro (same query, same index, only the `files.generated` flag differing): before pen 1.00 score 115.0 share 79.4% 3 files rendered after pen 0.30 score 35.4 share 21.1% 4 files rendered The new pattern stays in the existing table position, below the header window the detector scans, so the module still does not classify itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
03893b0ab9 |
CG-33: converge incremental sync with a full rebuild
A live, auto-synced index did not converge to a clean rebuild of the same tree — 4.3% of distinct edges wrong in both directions on this repo's own index, overwhelmingly `calls`, which is what flow queries traverse and what explore's file ranking weights. Silent: nothing warned, and the symptom read as "codegraph isn't very good" rather than "this index needs rebuilding." Two causes, and the fix needed both. Resolution binds a reference to one of the same-named definitions PROJECT-WIDE, so a definition appearing or vanishing changes the correct answer for references in files the sync never touches — and those references resolved successfully once, which deletes their unresolved_refs row, leaving nothing to revisit them with (#1240's retry only revisits refs parked as failed). Separately, when nothing disambiguated the candidates the winner came down to rowid, i.e. the order files happened to be WRITTEN, which differs between a scan-order full index and a sync that appends each file as it changes. That second one is why re-resolution alone could not converge: re-resolving against the identical graph still picked a different candidate. So getNodesByName now orders by (file_path, start_line) — a property of the code, not of the write order — and sync computes a definitionDelta and re-opens the resolution edges whose answer it may have invalidated, re-inserting each as the reference that created it for the orphan sweep to bind against the post-sync graph. The delta compares `file\0name` pairs per file rather than one name set over the batch: a commit that adds `collect` to a new file while an unrelated changed file already defines `collect` cancels out of a batch-wide set, and that miss was the largest residual class in the first measurement. Conservative where the failure modes are asymmetric — a wrong deletion is a permanent edge loss, a missed rebind is only residual drift. Edges without a refName stamp are never touched (nothing to restore them from), sources the sync already re-extracted are skipped, and a per-name ceiling declines the generic names. Edges are deleted before the sweep re-inserts, since INSERT OR IGNORE against idx_edges_identity would otherwise keep both rows when a reference rebinds elsewhere. Replaying real commits of this repo through sync, then diffing against a rebuild: 16 commits 48 -> 0; 80 commits 1,634 -> 361, with the actively misleading direction (stale edges the index keeps asserting) 671 -> 2. Index and sync wall-clock are unchanged; the ORDER BY costs 18% per uncached name lookup, which never reaches wall-clock because the resolver memoizes it. The 357-edge residual at 80 commits is one pre-existing class: refs to generic names (`push`, `join`) parked above #1240's per-name retry ceiling, which a rebuild resolves into cross-language garbage — a TS test file "calling" an R method. Converging there would mean manufacturing wrong edges, so it is left alone. And no drift metric in `codegraph status`: it cannot be computed without the rebuild it would be recommending, and a proxy would fire on that residual and train users to ignore it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
5f32478b57 |
docs(benchmarks): record the CG-26 A/B — the invariant holds on every path
Deterministic 6-repo table, the three agent A/Bs (django, excalidraw, okhttp, 2 runs/arm, Read 0 in all 12 runs), and an honest read of the two repos that deliver a few hundred fewer source chars: at the CG-31 tip both were over-filled by the flat-200 section overhead and paid for it by discarding their epilogue whole. Also: CHANGELOG entries for the two user-visible changes, and the memory note now carries the fourth accounting gap plus the two lessons — hold the REMAINDER when a full reservation no longer fits, and never skip a file over an accounting difference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
be7c968439 |
docs(benchmarks): record the CG-31 A/B — no regression, four repos stop truncating
Deterministic (6 repos, clean rebuilds, both builds): four deliver more source and one more file each, two are byte-identical, none deliver less. Agent A/B (django n=3, okhttp n=2, gin n=2, sonnet/effort high, both arms codegraph-on, 0 contamination): the new arm is faster on all three, Read at or below baseline, occupancy lower. Also records the two corrections the suite forced on the first cut of the guard, and the two residuals CG-26 inherits — the render loop's 600-char epilogue margin (a sweep was run and deliberately NOT shipped) and the BUY arm's source-space-only guard. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
 ctype_lab
|
ba58365c6f | docs(union): describe first-class union nodes | ||
|
Colby McHenryandClaude Opus 5
|
089dcc276f |
fix(explore): hold back what is still owed below a clustered render (CG-31)
Carry-forward slack let a file spend what the files ABOVE it left on the
table. Nothing held back what was promised BELOW it. The whole-file BUY arm
has always refused that trade (`owedBelow`); the cluster path read `headroom`
— what is left before the hard ceiling — instead of what is still owed, so
`fileBudget` and `SPINE_CEILING` could pay a 1.5x overshoot out of another
file's reservation.
`fundedHeadroom` is the same inequality in the units the cluster path spends
in: source PLUS the per-section overhead each unreached file will charge.
Floored at the file's own reservation — a kept promise is not a displacement —
and it is <= `headroom` by construction, so it is the only bound the three
render sites need. The skeleton path's `bodyCap` takes it too.
Measured on `__tests__/fixtures/displacement-ts` (a 4-stage pipeline padded
past 500 files, where the 24K envelope genuinely saturates the 24.4K render
ceiling):
before ingest.ts emitted 9,301 on a 6,289 spendable, then lost the whole
section to the final ceiling — 0 delivered. types.ts and sink.ts
skipped `budget-whole-file`. 3 of 6 admitted files delivered.
after ingest.ts bounded to the 4,913 actually free. 6 of 6 delivered,
envelope 14,908 -> 22,066.
The self-query allocation fixture flips back to PASS with it, on a clean full
rebuild of this repo's index (CG-33). Its `afterCG30` verdict blamed an
over-RESERVED incidental file; the reservation was identical in both arms —
the file was over-SPENDING. Recorded honestly in `afterCG31`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
ctype_labandClaude Opus 5
|
86854cd0d7 |
docs(union): changelog entry + port-checklist annotations
The two kernel port checklists record the extractor configs as surveyed at porting time; their structTypes lines are marked superseded rather than rewritten, so the surveys stay readable as history. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
d652c148f6 |
docs(cg-30): changelog entry + record the self-query probe flip honestly
The self-query allocation probe fixture's delivered-share gates now fail. The cause is not the new bound: allocation is unchanged between arms (parse-run.mjs 32.3% vs 33.9% on main) and tools.ts delivers the same 8,282 chars in both. What changed is that the incidental file now DELIVERS — on main its whole section was cut by the hard-ceiling truncation, so the fixture passed on truncation luck. Every file on this repo obeys the new bound (max 1.40x of spendable). Recorded as `afterCG30` with that reasoning rather than tuning the bound to restore the pass. The over-reservation it exposes is epic CG-24's subject. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenry
|
7a7ea30cbd |
docs: cross-call dedup — its gates, where the bytes go, and the all-pointer guard (CG-18)
Extends the session-state design doc with the layer built on it: what gates a withheld span (session, content fingerprint, two size floors, kill switch), why the fingerprint and not #1474's drift flag, the two channels the reclaimed bytes leave by, and why "no duplicate ranges across calls" holds for every call that had something new to say rather than universally. |
||
|
Colby McHenryandClaude Opus 5
|
c65d56ceba |
docs: CG-22 — the epic's gate, re-run at CG-15's exact setup (#1500)
CG-21 fixed the unspent-reservation defect and re-ran the A/B itself. CG-22 is
the gate proper: CG-15's setup, unchanged, measured independently of the task
that wrote the fix. RUNS=3, both arms codegraph-on, sonnet/high,
CODEGRAPH_NO_PROMPT_HOOK=1 on both, baseline pinned to
|
||
|
Colby McHenry
|
51cd053d85 |
docs: CG-21 — spending the reservation (design record + CHANGELOG precision)
Records both levers, the funding-pool design (and the per-file version that dropped payslip_builder.go), the resolved memory-budget.ts exception, and the two hermetic fixtures with their mutation matrix. The CHANGELOG clause 'no longer trimmed while a smaller, weakly-related one is included whole' was imprecise after CG-21: the smaller file often IS still included whole now, when its share nearly covers it. Reworded to say what the fix actually guarantees. |
||
|
Colby McHenryandClaude Opus 5
|
8077b83eb3 |
docs: consolidate the #1500 CHANGELOG entries into user-facing shape (CG-16)
Three separate engineer-shaped entries (CG-5 generated detection, CG-10 scoring, CG-12 allocation) become two user-facing bullets under Fixes, in the shape #1500 actually reported: explore concentrates on the code that answers the question, and a generated CRUD/protobuf layer no longer crowds out the hand-written code beside it. Per the CHANGELOG rules: strips the benchmark counts and percentages (the client-go 2,001-file count, the quarter-to-four-fifths envelope shift) and the internal symbol names, keeps the re-index note, and credits the reporter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Colby McHenryandClaude Opus 5
|
5f7f5f59df |
feat(mcp): score-proportional byte allocation for explore, with a relative cliff (CG-12, #1500)
The explore envelope used to follow FILE SIZE, not relevance. Every admitted
file was capped at the same flat `maxCharsPerFile`, while the whole-file rule
handed anything under `maxCharsPerFile * 3` its entire contents — a 3x swing
decided by how big a file happened to be:
- self-query: `memory-budget.ts` (score 18) shipped whole and took 51.2% of
the response; `src/mcp/tools.ts` (score 41, 4x the graph mass, 3x the term
hits — it holds the allocator itself) was clipped at 3,800 and got 32.9%.
- #1500 Go fixture: two generated CRUD files shipped whole at ~4.5K each AND
consumed two of the tier's four file slots, so `BuildPayslip` — the
hand-written "calculate" half of the question — ranked #6 and never
rendered at all.
`allocateExploreBudget` now reserves each ranked file a share of the envelope
before anything renders, so the render loop spends a reservation instead of
racing for whatever the files above it left:
- weight = score x worth x (spine ? 2 : 1), where `worth` is `rankPenalty`
applied a SECOND time — ranking answers "is this file about the query",
allocation answers "will these bytes teach the agent anything", and
generated CRUD can legitimately rank while its bytes stay boilerplate;
- a relative cliff at 15% of the top weight (capped at SCORE_FLOOR_MAX, so a
god-file can't silence peers the score floor just admitted) gives a file
ZERO source — path, symbols and line numbers only — and crucially frees its
`maxFiles` slot for a file that earns its bytes;
- every admitted file gets MIN_CHARS, then the remainder splits by weight:
the floor keeps a diffuse survey question returning a spread, the remainder
concentrates a precise one;
- the flat per-file cap is retired as the primary guard, leaving a 70%-of-
envelope safety valve.
Two changes were needed to make the reservation bite: an oversize cluster now
shrinks by whole MEMBER symbol ranges (a single-cluster god-file previously
took ~40% more than allotted, and the file below it was dropped for lack of
room), and the arrival-order budget stops are gone — they cut files by the
order they were reached rather than by merit.
Measured: payroll-go answer group 25.6% -> 78.7%, generated 57.4% -> 0%, and
`func (s *Service) BuildPayslip` now delivered; self-query `tools.ts` 18.5% ->
60.6%, past the epic's >50% bar. Controls hold: cobra/gin diffuse survey
queries keep their file spread (3->3, 3->4), express's middleware query is
byte-identical, and gin's flow query moves its top file from the thin `ginS`
singleton wrapper to `routergroup.go`.
One documented exception to "no previously-unclipped file becomes clipped":
`memory-budget.ts` was unclipped-whole at 5,672 and now clusters within its
3.1K reservation. That is the epic's own diagnosis of the bug — it scored 18
against 58 and was taking the larger slice purely for being small.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Colby McHenryandClaude Opus 5
|
a3898cdc70 |
feat(mcp): relevance scoring overhaul for explore — kill incidental name-collision matches (CG-10, #1500)
Explore's per-file relevance awarded +50/+10/+3/+1 by match class and admitted
anything scoring >= 3. Neither half held up: the tier said HOW a symbol reached
us, never whether the match was evidence, and an absolute floor admits noise on
any repo where the top file scores 50+. Three scripts/agent-eval/*.mjs harnesses
took 63% of this repo's own "how does explore allocate its output budget" answer
on nothing but an unused `const explore` and a `const BUDGET`.
Four levers:
- KIND WEIGHT (RELEVANCE_KIND_WEIGHT): callables and types 1.0, members ~0.5,
variable/constant/parameter 0.15-0.35. A weak-kind symbol with no usage edge
anywhere in the graph (`contains` excluded — nesting is not usage) drops to
0.08. Only weak kinds in the top two tiers pay for the DB probe; the subgraph's
own edges answer most cases free. No measurable latency change (210 vs 211
ms/call, n=12 interleaved).
- PERIPHERAL CAP: nodes >=2 hops from any match accumulate into a bucket capped
at 5. Uncapped they added a flat +1 each, so a file grew more relevant by being
bigger — parse-session.mjs reached 22 off one constant plus twelve unrelated
symbols.
- RANK PENALTY: generated files x0.3, low-value x0.5, applied to the score AND
the graph mass. Score alone would not have fixed #1500 — the generated CRUD
carries MORE graph mass than the hand-written use-case, and graph mass outranks
score in the comparator. Self-normalizing, never a hard exclusion.
- RELATIVE FLOOR: clamp(topScore * 0.2, 1, 10). Capped at one full-strength
direct match so concentration elsewhere can never exclude one (without it a
named-seed-heavy file pushed the floor to 21 and dropped a file the agent had
named by class name). Backfills to 3 candidates when it would leave fewer, and
drops the evidence requirement rather than return nothing at all.
excludeLowValueFiles was dead config — declared per tier, read nowhere; the
test/spec exclusion has been unconditional for a while. Removed. The real gap was
the detector: `isLowValue` anchored on a leading `/`, so a repo-ROOT `test/` dir
(express, cobra, most of npm and Go) never matched — express's routing question
spent 59% of its envelope on three test files. Anchored at `^` too, and the
filter now runs before the floor and judges "are there other candidates?" on the
whole gather.
Measured before/after on the same indexes (baseline
|
||
|
Colby McHenryandClaude Opus 5
|
16e17495f4 |
feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
49c11fc2e0 |
Self-hosted telemetry on Cloudflare D1 + password-gated admin dashboard (CG-7) (#1497)
* feat(telemetry): D1 schema + migrations for raw events and daily rollups First step of replacing PostHog with self-hosted telemetry on Cloudflare D1. Creates the codegraph-telemetry database binding and the initial migration; no worker code paths change yet (the ingest write path and the nightly rollup cron land next). Schema is raw events plus daily rollups: `events` holds one row per sanitized event with the envelope broken out into columns and event-specific props as JSON; `daily_machines`, `daily_event_counts` and `daily_dim_counts` are the nightly rollups the dashboard reads; `machine_first_seen` and `machine_days` carry the retention cohorts and are never purged. One generic dimension table covers every bar and pie, so a new breakdown is a cron change rather than a migration. The migration is commented as an audit surface, like the rest of this worker — every column, and which dashboard chart each rollup table serves. Three judgment calls worth flagging, all documented in the file: - `events` gets `(day, event)` instead of the separate `(day)` and `(event, day)` indexes. D1 bills a row write per index touched, so a third index on the hot table costs ~97k writes/day, and `(day, event)` is a covering index for plain day-range scans anyway (verified with EXPLAIN QUERY PLAN). - `daily_event_counts` and `daily_dim_counts` carry a `machines` column, and `machine_days` a `prod` flag. The "users by ..." panels and the production-user count are distinct-machine numbers, not event counts, and they are unrecoverable once raw events are purged. - No CHECK constraint on `event`: the worker's allowlist is the source of truth and the write path is fail-silent, so a rejected INSERT would lose data quietly instead of erroring loudly. Volume note in the migration footer: ~30M row writes/month against the 50M included on Workers Paid. Storage is the tighter constraint — raw events grow ~74 MB/day, so retention should start at 90 days (~6.7 GB) rather than 180, which would exceed D1's 10 GB per-database cap. * feat(telemetry): admin dashboard worker — scaffold + shared-password auth New Cloudflare Worker at telemetry-dashboard/, sibling of telemetry-worker/ and bound read-only to the same D1 database. Serves a static frontend plus a JSON API behind a shared password, on stats.getcodegraph.com. Auth is the simplest thing that is actually safe for exactly two users: one password in a secret, compared in constant time over SHA-256 digests, and an HMAC-signed cookie (HttpOnly; Secure; SameSite=Lax; Path=/) with a one-year expiry so you sign in once per browser. The cookie is a signed assertion, not a lookup key — no session store. Its payload carries a fingerprint of the password it was minted against, so rotating ADMIN_PASSWORD signs everyone out. Login attempts are capped at 5/min per IP via a ratelimit binding. Everything is deny-by-default: assets.run_worker_first routes every request through the worker before the static-asset server sees it, so the dashboard HTML, its JS, its CSS and the chart library are all behind the session check. The login page is rendered inline by the worker rather than served from public/, which leaves no "is this file public?" judgement calls in the asset directory. Unauthenticated pages 302 to /login, unauthenticated /api/* gets 401. A missing secret fails closed rather than opening the dashboard. scripts/smoke-auth.sh is the regression net — 54 assertions against a throwaway `wrangler dev` covering the gate, cookie flags and persistence, forged/flipped/ truncated cookies, open-redirect refusal, brute-force capping, and password rotation invalidating live sessions. Refs CG-11. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(telemetry-dashboard): simplify the chart-library probe in the shell Refs CG-11. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): nightly rollup cron + raw-event retention purge (CG-10) Adds a scheduled() handler to the ingest worker that recomputes daily_event_counts / daily_dim_counts / daily_machines for the just-completed UTC day plus a 2-day overlap (late-arriving offline buffers), then purges raw events past the retention window. Rollup writes are idempotent upserts, so a re-run never double-counts. Also adds an ADMIN_TOKEN-guarded POST /admin/rollup?day=YYYY-MM-DD for backfill/repair, and drops the PostHog forwarding path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): dashboard charts — SQL API over D1 + the Chart.js views (CG-12, CG-13) Replaces the scaffold page with the dashboard proper: 19 panels covering every view of the PostHog dashboard this retires, driven by one filter row. src/api.ts is the read API CG-12 specified: /api/{meta,summary,timeseries, breakdown,activation,retention}, all range-scoped, all parameterized against a closed set of dims and metrics, all shaped labels[] + datasets[] so the frontend does no arithmetic. Rollups answer everything except the activation funnel, which needs raw events and says where they start. The frontend splits into a DOM-free panel registry (public/panels.js) and the page that mounts it (public/app.js), so the render check can drive the same registry the browser rendered from. Panels fail alone, refetch dims rather than flashing, and every chart carries a table twin. Two numbers are labelled rather than rounded off: range-wide "users" per dimension is machine-days (the rollups cannot give distinct machines, and per-day counts are taken as the largest single-event count so one machine's install + index + usage is not counted three times), and recent activation and retention cohorts are marked as still-converting instead of drawn as a cliff. Both colour scales were run through the data-viz validator against the panel surface, not picked by eye; the results are recorded in public/theme.js. Verification, all against the committed fixture (12 machines over 10 days, every expected number worked out by hand from the events, not recorded from a run): scripts/smoke-api.sh 98 assertions scripts/render-check.mjs 79 assertions — real Chromium over CDP, no new deps scripts/smoke-auth.sh 54 assertions (unchanged, still green) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): cutover runbook + the end-to-end gate that de-risks it (CG-14) The account-level steps of the PostHog cutover are the maintainer's to run, so this lands the runbook they follow and the check that has to pass first. The runbook (telemetry-worker/README.md) walks the six steps in the order that keeps them reversible: Workers Paid → migrate → deploy → watch 24h → verify the first rollup and the dashboard → only then delete POSTHOG_KEY and cancel the subscription. Step 3 records the outgoing version id because `wrangler rollback` is the escape hatch for the whole verification window, and that window is precisely why the PostHog key is deleted last rather than first. The new gate (scripts/smoke-cutover.sh, `npm run smoke:cutover`) covers the one seam nothing else did. Both workers declare the same D1 database_id, so pointing them at a single --persist-to directory runs the real chain: a client batch → the ingest worker → D1 → the nightly rollup → the dashboard API reading the numbers back. Every other suite stops at one link — smoke-ingest at the events table, smoke-rollup at hand-checked SQL, smoke-api at a hand-written fixture that the cron never touched. That left the dimension names the rollup WRITES versus the ones the dashboard READS agreeing by convention across two branches, where a mismatch is silent: no error, no failed request, just a panel reading zero forever. 61 assertions, all 13 dimensions, and three deliberate traps — a ci machine that is active but not a production user, usage_rollup counts that must be summed rather than tallied, and an uninstall's `targets` that must not leak into the install-scoped breakdown. Writing it caught that the activation funnel's denominator is first-seen machines, not install events (deliberate — a reinstall must not re-enter the funnel), so the suite now pins that distinction rather than assuming it. Also rewords the last PostHog reference in dashboard code: a comment justifying the 14-day retention curve by pointing at a dashboard step 6 deletes. The reasoning now stands on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(telemetry): tell the truth about where events are stored (CG-15) The telemetry docs are a privacy contract, and they still described a managed analytics store that no longer receives anything. Replace that with what actually happens now — events land in our own D1 database on Cloudflare, the endpoint makes no outbound requests, raw events are purged after 90 days and only anonymous daily rollups outlive them. This strengthens the guarantee rather than restating it: there is no second party to share with. - TELEMETRY.md: new "Where it is stored" section; the never-collected IP bullet no longer leans on a vendor-side setting to hold. - docs/design/telemetry.md: ingest section rewritten around D1 + the nightly rollup/retention cron; volume math redone on Workers Paid and the D1 quota (storage, not writes, is what sets the 90-day window); new section documenting the dashboard worker and cross-linking it. - Fixed three drifts from the worker allowlist the sweep surfaced: schema_version was still 1, client_name/client_version was still marked "plumbing to add" though session.ts passes it today, and the legacy sqlite_backend field the worker still accepts was undocumented. - telemetry-worker/README.md: step 6 claimed a repo-wide grep came back clean, which this runbook itself falsifies. Added step 7 — deleting the runbook is what makes that grep true, and is the completion check. - smoke-cutover.sh: the vendor guarantee is now asserted by class (no analytics-ingest endpoint referenced) rather than by one vendor's name, so it keeps working once the name is gone. Verified it still catches a planted forwarding URL. 61/61 pass. Retention is documented as 90 days, not the 180 in the task notes: 180 days of raw events exceeds D1's 10 GB per-database cap, and the code purges at 90. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: untrack local Kommandr issue DB and ignore its sqlite artifacts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
f6ac7b36e6 |
fix(mcp): blast radius follows caller chains before claiming no test coverage (#1475) (#1494)
The "no covering tests found" flag only inspected a symbol's direct callers, so helpers exercised transitively by tests (logDebug runs 1,471x under npm test) were reported untested — wrong for ~40% of flagged symbols per the issue's measurement. The check now BFSes up the caller graph (3 hops, 64-lookup budget per entry) and reports indirect coverage as "tested via callers: <files>". When nothing is found it claims only what was measured — "no tests found within 3 caller hops", or the weaker "no test calls this directly" if the budget ran out — and drops the warning glyph. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |