41c10750e04573aed9e03fa726083064dfaf1c13
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Colby McHenryandClaude Opus 5
|
16e17495f4 |
feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
9d0cd3a7d1 |
fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) (#1249)
* chore: ignore .kommandr/ directory * fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) Incremental sync scoped reference resolution to the changed files' own refs, and a completed pass deleted every ref it failed to resolve — so a symbol change in one file could never repair references in UNCHANGED files, in either direction, until a full re-index: - New-export case: a.ts imports/calls `greet` before b.ts defines it. The failed refs were deleted at index time; when b.ts later gained `greet`, nothing revisited a.ts — the calls/imports edges stayed missing while status reported a clean index. - Removal case: when a re-index (or file deletion) dropped a symbol, the incoming edges cascade-deleted and the callers — whose resolved refs had been consumed — never got a chance to rebind to an alternative definition or reconnect when the symbol returned. Fix, sharing one lifecycle: - Schema v8: unresolved_refs gains status ('pending'/'failed') and name_tail (last dotted segment, so `h.greet` is findable by `greet`). Both resolver persist paths now park unresolvable refs as failed instead of deleting them. All pending-work readers (batched drain, non-progress guard, #1187 orphan sweep, status pendingRefs) filter to pending, preserving their invariants and keeping status honest. - Sync retry: after scoped resolution, failed refs whose name tail matches a symbol name now present in the changed files are re-resolved through a per-ref-yielding path (watchdog-safe, #1091 class). Names matching >500 failed refs are skipped as external/builtin noise (#999 rationale). - Removal side: createEdges stamps each resolution edge with its originating reference (metadata.refName, + refKind when kind promotion rewrote it). When the #899 restore misses a target or sync deletes a file, the dropped edge is resurrected as exactly that ref — re-resolved in the same sync (rebinding to an alternative definition) or parked failed until the symbol reappears. Edges without the stamp (pre-upgrade, synthesized) still drop silently: reconstructing from the target's plain name would strip receiver context and risk a rebind a full re-index would never make. - Pure-removal syncs clear resolver caches so a long-lived daemon can't resolve resurrected refs against the pre-removal graph. Validated: issue repro now yields a graph byte-identical to a full re-index; move/remove-readd/file-deletion scenarios all rebind or heal; baseline-vs-new A/B on express and gin shows identical node/edge counts and no timing regression (DB grows ~25% from the parked ref rows — pure cache, reset by any full re-index). 8 regression tests added. Fixes #1240 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e699ee9686 |
feat(prompt-hook): graph-derived gate tier + confidence-tiered injection + gate telemetry (#1136)
The keyword gate (#1126) can never know a repo's domain nouns. This adds the graph-derived tier the design discussion converged on: symbol names are split into prose segments at index time (name_segment_vocab, riding the insertNode write path), and the hook verifies a prompt's plain words against them — "the state machine des commandes" → OrderStateMachine, in any language whose technical nouns are Latin script. Confidence now decides HOW MUCH to inject, not just whether: - HIGH (keyword, or index-verified code token): full explore injection, unchanged — the validated adoption lever. - MEDIUM (segment matches only): a ~500-byte pointer naming the matching symbols; the AGENT writes the explore query. Never runs explore, so a fuzzy match can't inject 16KB of wrong-feature context. - Silent otherwise, as before. Precision is derived from the repo's own naming statistics plus measured FP fixes: co-occurrence (≥2 words on one name) always qualifies; a single word must be ≥5 chars, cluster across 2–25 names (singletons are prose coincidence: "deploy to production" → matchesNonProductionDir), match a multi-segment name, and not be an English function/filler word (the one place a word list is honest: identifiers are English, so only English prose collides). Every candidate is re-verified against nodes before being surfaced — vocab rows are proposals, deletions leave orphans by design, a full index rebuilds from scratch, and sync heals pre-upgrade databases (batched + yielding; emptiness captured at sync ENTRY so the sync's own writes can't mask the backfill). Schema v7 migration is DDL-only (instant; none of the #1067 row-churn hazards). Gate outcomes roll up as anonymous usage counters (prompt-hook-gate-<outcome>, names only, never content) through the existing telemetry pipeline — recall becomes measurable, and the counters are the agreed kill-criterion data for ever revisiting a local classifier. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
0da2dcec8e |
fix(db): dedup edges with a UNIQUE identity index so INSERT OR IGNORE works (#1034) (#1050)
`insertEdge` has always used `INSERT OR IGNORE`, but the edges table carried no UNIQUE constraint — only an autoincrement PK and non-unique indexes — so `OR IGNORE` had nothing to conflict on and behaved like a plain INSERT. Whenever two extraction/resolution passes emitted the same edge (e.g. a return type captured by both a type-reference and a value-reference pass), the graph stored byte-identical duplicate rows: ~527 on this repo, inflating edge counts and letting callers/impact list the same relationship twice. Add a UNIQUE identity index on (source, target, kind, IFNULL(line,-1), IFNULL(col,-1)) — in schema.sql for fresh databases and migration v6 (dedup existing rows, then create the index) for existing ones. IFNULL folds the nullable line/col so coordinate-less edges (synthesized / file-level) dedup too; SQLite otherwise treats each NULL as distinct. Distinct call sites (same source/target/kind, different line/col) are preserved — only byte-identical structural duplicates collapse. This is the storage-layer invariant the reporter identified: it makes OR IGNORE keep its promise and catches every double-emit, present and future, rather than chasing each emitting pass. Migration v6 is deterministic (keeps the lowest id per identity group) and idempotent (IF NOT EXISTS index; no-op DELETE once unique). The DELETE's GROUP BY matches the index expression exactly so creation can't fail on a leftover pair. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
fd03f31b2c |
fix(cpp): resolve calls through singletons/factories/chained getters (#645) (#742)
A C++ method call whose receiver is another call's result — `Foo::instance().bar()`, `WidgetFactory::create().draw()`, `openSession()->run()`, or the same stored in an `auto` local first — lost the receiver's type during extraction. The callee degraded to a bare method name, so when two classes shared a method name the call silently resolved to whichever was indexed first (or not at all), corrupting callers / impact / trace with a plausible-but-wrong edge. Three parts: - Capture C++ return types (new nodes.return_type column, schema v5): the function_definition's `type` field, normalized — smart-pointer pointee unwrapped, void/primitives dropped. - Preserve the inner-call receiver in extraction: a C/C++ field_expression whose receiver is itself a call is encoded `inner().method` instead of dropping to the bare name. Other languages keep the existing behavior. - New resolution strategy (matchCppCallChain): infer the receiver's class from the inner call's return type, then resolve AND validate the method on it. Handles singletons/accessors, factories returning a different type, free-function factories, make_unique/make_shared/new/direct construction, single-level member chains, and namespace-qualified inner calls. A wrong inference yields no edge, never a wrong one. EXTRACTION_VERSION 2->3 (re-index to populate return types). Validated on the issue repro + spdlog: node count stable (no explosion), deterministic, and ~100 pre-existing wrong `.size()`-style edges removed. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Colby MchenryandGitHub
|
68eaf0dbd8 |
feat(mcp): codegraph_explore as the sole primary tool + store coverage + overload disambiguation (#647)
## Summary
Completes the explore-overhaul arc: `codegraph_explore` becomes the single primary tool an agent reaches for, and its coverage + output shape are tuned so flow/architecture questions resolve with near-zero Read/Grep.
### What changed
- **explore is the sole primary tool** — removed `codegraph_context` (the fuzzy-input Read-trigger) and `codegraph_trace` (under-picked by agents); explore already surfaces the call flow among the symbols you name. A plain natural-language question now works as the query.
- **Store/handler coverage** — functions defined inside object literals (Zustand `create((set, get) => ({ … }))`, Redux/Pinia/MobX, exported handler/route maps) are indexed as real symbols, including calls through `useStore.getState().fn()` and destructured `const { fn } = useStore.getState()`. A general AST rule, not a per-lib hack.
- **Overload disambiguation** — explore leads with the *right* definition when a method name is overloaded across types (a PascalCase type token in the query biases to that type's own def); `codegraph_node` returns *every* overload's body in one call, with an optional `file`/`line` selector to pin one.
- **Method-atomic render** — explore never returns half a method; at the size budget it drops whole methods/files (and lists what it dropped) instead of truncating a body mid-method.
- **Native-read-shaped output** — per-call output is capped to ~24K with a 25K hard ceiling and concentrated into ~150–250-line flow windows, mirroring how the agent natively reads; repo size scales the *call* budget, not the per-call size (a larger response just gets externalized to a file the host Reads back).
- **Blast radius** folded into explore (dependents + covering tests, locations only).
### Benchmark (refreshed on this build)
Re-validated the 7-repo A/B on 2026-06-02 (Opus 4.8, effort=high, median of 4). WITH arm re-measured on this build, WITHOUT reused:
**~16% cheaper · 47% fewer tokens · 22% faster · 58% fewer tool calls** — 0 file reads on 6 of 7 repos (Gin ~1).
The arc trades larger, cache-heavy explore responses for guaranteed near-zero reads, so cost/token margins soften vs the prior build (Excalidraw and Tokio land at cost break-even) while time and tool-calls stay clear wins everywhere — consistent with the project's stated optimization target (latency + tool-calls, not token cost).
### Validation
- Full suite green: **1112 passed, 2 skipped**.
- 28/28 plain WITH runs across the 7 README repos completed clean; reads median 0 on 6/7.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
|
||
|
|
ac52fd76c0 |
Self-contained distribution: bundle Node + node:sqlite, drop better-sqlite3/wasm (closes #238) (#282)
* fix(db): eliminate concurrent-read "database is locked"; add node:sqlite backend (#238) WAL + busy_timeout were already enabled, so the issue's suggested fix was a no-op. The real causes, addressed here: - busy_timeout is now set first (before journal_mode) and lowered 120s -> 5s, so open-time pragmas wait out a lock instead of hanging for two minutes. - getCodeGraph no longer opens a second connection to the default project when a tool passes its own projectPath (the in-process lock amplifier). - The wasm fallback (no WAL) gets a bounded read-retry on SQLITE_BUSY. - New: node:sqlite backend, preferred over wasm, so installs whose native better-sqlite3 build fails land on a real-WAL backend instead of no-WAL wasm. - codegraph status / codegraph_status now report the effective journal mode, so a lock report is triageable (wal vs delete). - CLI hard-blocks Node < 20 to actually enforce the engines floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db)!: node:sqlite is the sole backend; drop better-sqlite3 + wasm Now that distribution will bundle a Node 24 runtime, node:sqlite (real SQLite with WAL + FTS5) is always available. Collapse the three-backend adapter to node:sqlite only and remove the machinery the other two needed: - Remove better-sqlite3 (optionalDependency) and node-sqlite3-wasm (dependency). - Remove WasmDatabaseAdapter, the named->positional param translation, the SQLITE_BUSY read-retry, the wasm fallback banner, the backend env override, and the native/node-sqlite/wasm selection chain. - createDatabase now opens node:sqlite directly, with a clear error pointing at the bundled release / Node 22.5+ when the module is absent. - NodeSqliteAdapter.close() is idempotent and pragma() supports { simple }, to match the better-sqlite3 behavior callers relied on. - status (CLI + MCP) reports the single node:sqlite backend; journal-mode diagnostics and the getCodeGraph single-connection fix are retained. - Tests repointed off better-sqlite3 onto node:sqlite. Net -1044 lines. Running from source now requires Node 22.5+. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): self-contained bundle prototype (vendored Node + install channels) Phase 3 of the node:sqlite migration: ship a vendored Node runtime so CodeGraph runs with no system Node and no native build (node:sqlite is built in). - scripts/build-bundle.sh: build a per-platform archive (official Node + dist + prod deps + launcher). Same recipe per platform; pins Node v24.16.0. - install.sh: curl|sh installer (no Node required) — detects os/arch, pulls the archive from Releases, symlinks onto PATH; re-run to upgrade, --uninstall to remove. The VPS/SSH path. - scripts/npm-shim.js: thin launcher for the npm channel — resolves the per-platform optionalDependency bundle and execs it, so `npm i -g` keeps working and the real work runs on the bundled Node regardless of the user's. - BUNDLING.md: distribution design + release-pipeline TODO (CI matrix, platform packages, code signing, brew, retiring the Node-version gate). Validated end-to-end: darwin-arm64 and linux-x64 bundles both run init + index + status (Backend: node:sqlite, Journal: wal) + FTS query with NO system Node — linux-x64 verified in a clean ubuntu:24.04 amd64 container. Release archives are gitignored; CI will produce and upload them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): add Windows PowerShell installer (install.ps1) The `irm … | iex` one-liner for Windows, mirroring install.sh: detect arch, pull the matching bundle from Releases, extract to %LOCALAPPDATA%\codegraph, add it to user PATH. Re-run to upgrade. (Windows bundle production in build-bundle.sh is still TODO.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): release workflow + npm packaging; README/CHANGELOG for bundled distro - .github/workflows/release.yml: manually-triggered (workflow_dispatch) release matrix. Builds a self-contained bundle per platform on its own runner (darwin-arm64/x64, linux-x64/arm64), publishes a GitHub Release with all archives, and publishes the npm thin-installer (shim + per-platform packages). Windows targets are TODO (build-bundle.sh is unix-only). - scripts/pack-npm.sh: assemble the npm packages from built bundles — per-platform packages tagged os/cpu + the main shim package with them as optionalDependencies (esbuild pattern). Proven locally: npm-install the tarballs, run via the shim, resolves the bundle and runs on the bundled Node 24 (node:sqlite / WAL). - README: install section now leads with the no-Node one-liners (curl|sh, irm|iex) then npm/npx; "bundled · none required" badge. - CHANGELOG: standout headline for the self-contained release, plus Added/Changed/ Removed for the install channels, node:sqlite backend, and dropped deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): Windows bundles + single-trigger release workflow - build-bundle.sh: add win32-x64 / win32-arm64 targets — download Node's Windows zip, bundle node.exe + a .cmd launcher, output a .zip. Verified structurally (PE32+ node.exe, CRLF .cmd, portable node_modules). Since there are no native addons, any target builds on any OS, so the whole matrix builds on one runner. - pack-npm.sh: handle .zip bundles and win32 packages (os: win32, node.exe). - release.yml: simplified to your spec — manual trigger reads the version from package.json, builds all platform bundles, creates the GitHub Release with notes pulled from CHANGELOG.md, and publishes the npm shim + platform packages. - BUNDLING.md: Windows + build-anywhere notes; release pipeline documented. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
a460b856c2 |
perf(db): drop redundant idx_edges_source / idx_edges_target (#142)
Both narrow indexes are fully covered by the existing (source, kind) and (target, kind) composites via SQLite's left-prefix scan, so they're dead weight on every write. Empirical measurements (from the spike script in PR #122 on a 50K-node / 250K-edge synthetic DB): - DB size: 34.7 MB → 27.0 MB (-22.2%) - Bulk insert (250K edges): 590ms → 431ms (1.37× faster) - source/target lookup latency: no regression Adds migration v4 to drop both on existing databases; fresh-DB schema no longer creates them. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Colby McHenry
|
e4908e1270 |
feat: Add database schema v3 with optimized node lookups and improved error handling
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases. |
||
|
Colby McHenry
|
15b5e56322 |
fix: Lazy grammar loading and quantized embeddings to prevent V8 WASM OOM
Fixes #54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone" on large codebases because all 16 tree-sitter WASM grammar modules were compiled upfront by V8, exhausting the WASM Zone allocator. Changes: - initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()), no longer eagerly loads all grammar files - New loadGrammarsForLanguages() loads only grammars for languages actually present in the project (e.g. a Dart project loads ~2-3 grammars instead of 16) - Orchestrator detects needed languages after file scan, before parsing begins - Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce WASM memory pressure when embeddings are enabled |
||
|
Colby McHenry
|
8346440592 |
Add WASM fallbacks for tree-sitter and SQLite, fix installer
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for universal cross-platform support. Add node-sqlite3-wasm as a fallback when better-sqlite3 native bindings aren't available. Move better-sqlite3 and sqlite-vss to optionalDependencies so installs never fail. Fix installer to use npx fallback when global npm install fails, so MCP config, hooks, and quick-start instructions all work without the bare codegraph command in PATH. Fix tests: update schema version expectation, fix db test paths and method names, extract MAX_OUTPUT_LENGTH as module constant, normalize Windows path separators in import resolver. |
||
|
Colby McHenryandClaude Opus 4.6
|
c8c7785626 |
Add tests for PR #19 improvements (Phases 3-5)
Covers arrow function extraction, best-candidate resolution, graph traversal direction fix, MCP symbol disambiguation, output truncation, CLI uninit command, and more. Tests requiring better-sqlite3 native bindings are conditionally skipped. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |