`isGeneratedFile` was path-only, but Go's own convention is a CONTENT
marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one.
A Go monorepo with generated CRUD in ordinarily-named files sitting beside
hand-written use-cases was therefore invisible to every generated-file
down-rank in the codebase — that is #1500.
Measured on kubernetes/client-go (2,453 Go files): the canonical banner
appears in 2,001 of them, the path check flags 0, the new content check
flags exactly those 2,001 — no false positives, no misses.
Design: decide at INDEX time (content is already in memory for parsing),
persist on `files.generated`, read from the DB. Explore never reads file
headers per request.
- `hasGeneratedHeader(content)` recognizes the standard banners — Go's,
protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator,
FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an
8KB/60-line header window, a comment-line requirement (leader or open
block comment), and markers tight enough that prose can't trip them. A
generator's own source, holding the banner as a string constant in its
body, is not flagged; neither is this module itself (pinned by test).
- `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback.
- Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no
backfill: the flag derives from content the migration cannot see, so
rows stay 0 until a re-index and every reader unions the flag with the
path check — an un-migrated index keeps pre-#1500 behavior rather than
regressing. Re-index required; noted in the CHANGELOG.
- `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1)
lookups. Bounded, not cached: no invalidation, so a ranking call can
never serve a verdict the last sync already replaced. Wired into explore
ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the
context formatter, and the dominant-file/route-file hygiene filters.
Cost (acceptance bar was no measurable index-time regression): a single
unanchored `/generat/i` test over the header rejects ~every hand-written
file before any line splitting. 4.6 µs/file on client-go (worst case —
82% generated). End-to-end `codegraph init` on client-go, n=3 alternating
arms: 5.73s median with detection vs 5.76s path-only baseline; the arms
cross over between runs, so the difference is inside run-to-run noise.
Scope note: generated status remains a stable TIEBREAK at equal score,
exactly where it was. Making it a strong negative signal is CG-10, which
this unblocks by making the signal correct and available.
Two pre-existing tests hard-coded schema version 8; both now track
CURRENT_SCHEMA_VERSION (or the migration table) so future migrations
don't require editing them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: ignore .kommandr/ directory
* fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240)
Incremental sync scoped reference resolution to the changed files' own
refs, and a completed pass deleted every ref it failed to resolve — so
a symbol change in one file could never repair references in UNCHANGED
files, in either direction, until a full re-index:
- New-export case: a.ts imports/calls `greet` before b.ts defines it.
The failed refs were deleted at index time; when b.ts later gained
`greet`, nothing revisited a.ts — the calls/imports edges stayed
missing while status reported a clean index.
- Removal case: when a re-index (or file deletion) dropped a symbol,
the incoming edges cascade-deleted and the callers — whose resolved
refs had been consumed — never got a chance to rebind to an
alternative definition or reconnect when the symbol returned.
Fix, sharing one lifecycle:
- Schema v8: unresolved_refs gains status ('pending'/'failed') and
name_tail (last dotted segment, so `h.greet` is findable by `greet`).
Both resolver persist paths now park unresolvable refs as failed
instead of deleting them. All pending-work readers (batched drain,
non-progress guard, #1187 orphan sweep, status pendingRefs) filter to
pending, preserving their invariants and keeping status honest.
- Sync retry: after scoped resolution, failed refs whose name tail
matches a symbol name now present in the changed files are re-resolved
through a per-ref-yielding path (watchdog-safe, #1091 class). Names
matching >500 failed refs are skipped as external/builtin noise (#999
rationale).
- Removal side: createEdges stamps each resolution edge with its
originating reference (metadata.refName, + refKind when kind promotion
rewrote it). When the #899 restore misses a target or sync deletes a
file, the dropped edge is resurrected as exactly that ref — re-resolved
in the same sync (rebinding to an alternative definition) or parked
failed until the symbol reappears. Edges without the stamp (pre-upgrade,
synthesized) still drop silently: reconstructing from the target's plain
name would strip receiver context and risk a rebind a full re-index
would never make.
- Pure-removal syncs clear resolver caches so a long-lived daemon can't
resolve resurrected refs against the pre-removal graph.
Validated: issue repro now yields a graph byte-identical to a full
re-index; move/remove-readd/file-deletion scenarios all rebind or heal;
baseline-vs-new A/B on express and gin shows identical node/edge counts
and no timing regression (DB grows ~25% from the parked ref rows — pure
cache, reset by any full re-index). 8 regression tests added.
Fixes#1240
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The keyword gate (#1126) can never know a repo's domain nouns. This adds
the graph-derived tier the design discussion converged on: symbol names
are split into prose segments at index time (name_segment_vocab, riding
the insertNode write path), and the hook verifies a prompt's plain words
against them — "the state machine des commandes" → OrderStateMachine, in
any language whose technical nouns are Latin script.
Confidence now decides HOW MUCH to inject, not just whether:
- HIGH (keyword, or index-verified code token): full explore injection,
unchanged — the validated adoption lever.
- MEDIUM (segment matches only): a ~500-byte pointer naming the matching
symbols; the AGENT writes the explore query. Never runs explore, so a
fuzzy match can't inject 16KB of wrong-feature context.
- Silent otherwise, as before.
Precision is derived from the repo's own naming statistics plus measured
FP fixes: co-occurrence (≥2 words on one name) always qualifies; a single
word must be ≥5 chars, cluster across 2–25 names (singletons are prose
coincidence: "deploy to production" → matchesNonProductionDir), match a
multi-segment name, and not be an English function/filler word (the one
place a word list is honest: identifiers are English, so only English
prose collides). Every candidate is re-verified against nodes before
being surfaced — vocab rows are proposals, deletions leave orphans by
design, a full index rebuilds from scratch, and sync heals pre-upgrade
databases (batched + yielding; emptiness captured at sync ENTRY so the
sync's own writes can't mask the backfill).
Schema v7 migration is DDL-only (instant; none of the #1067 row-churn
hazards). Gate outcomes roll up as anonymous usage counters
(prompt-hook-gate-<outcome>, names only, never content) through the
existing telemetry pipeline — recall becomes measurable, and the counters
are the agreed kill-criterion data for ever revisiting a local classifier.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Follow-up to #1065/#1066. Those stopped a *new* index from scanning an
ignored gitlink corpus, but a project that had already built the multi-GB
graph before upgrading still couldn't recover: `codegraph index` printed
only "Indexing project" and was then SIGKILLed (137) by the #850 watchdog
~60s later, before scanning even started.
Root cause is not the scanner. `index` cleared the old graph with a
synchronous `DELETE FROM nodes/edges/files`. `nodes` carries an FTS5
`AFTER DELETE` trigger, so deleting ~1.6M rows fires ~1.6M FTS
delete-markers — O(rows), and it grows the WAL further before it can
finish. A deterministic probe puts the DELETE-clear at 20.4s on 1.5M
synthetic nodes (WAL 1.16->2.14GB); at the report's denser ~2.6KB/node WAL
that crosses the 60s main-thread watchdog. `open()` was never the wedge.
A full re-index is documented as "same result as a fresh init", so make it
one: discard the database files and re-initialize, instead of opening the
old DB and DELETE-ing every row.
- db: add removeDatabaseFiles(dbPath) — unlinks codegraph.db + its
-wal/-shm sidecars (O(1) regardless of size; sidecars best-effort).
- index: add CodeGraph.recreate(projectRoot) — discards the files and
returns a fresh, empty instance. Never opens or migrates the poisoned
DB. POSIX unlinks an open file fine (a live daemon heals via
reopenIfReplaced, #925); a Windows file lock becomes an actionable
"stop the daemon / remove .codegraph" error.
- cli: `codegraph index` now calls recreate() instead of open()+clear();
both clear() calls dropped. The public clear() API is unchanged.
This also reclaims the disk the bloated db/-wal were holding.
Validated: deterministic probe (DELETE O(rows) vs recreate O(1)); an
end-to-end run through the built binary recovering a real 800K-node /
419MB poisoned DB in 0.3s with no wedge and the correct small graph; new
unit + CLI regression tests; existing #874 index tests still green.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`insertEdge` has always used `INSERT OR IGNORE`, but the edges table carried
no UNIQUE constraint — only an autoincrement PK and non-unique indexes — so
`OR IGNORE` had nothing to conflict on and behaved like a plain INSERT.
Whenever two extraction/resolution passes emitted the same edge (e.g. a
return type captured by both a type-reference and a value-reference pass),
the graph stored byte-identical duplicate rows: ~527 on this repo, inflating
edge counts and letting callers/impact list the same relationship twice.
Add a UNIQUE identity index on (source, target, kind, IFNULL(line,-1),
IFNULL(col,-1)) — in schema.sql for fresh databases and migration v6 (dedup
existing rows, then create the index) for existing ones. IFNULL folds the
nullable line/col so coordinate-less edges (synthesized / file-level) dedup
too; SQLite otherwise treats each NULL as distinct. Distinct call sites
(same source/target/kind, different line/col) are preserved — only
byte-identical structural duplicates collapse. This is the storage-layer
invariant the reporter identified: it makes OR IGNORE keep its promise and
catches every double-emit, present and future, rather than chasing each
emitting pass.
Migration v6 is deterministic (keeps the lowest id per identity group) and
idempotent (IF NOT EXISTS index; no-op DELETE once unique). The DELETE's
GROUP BY matches the index expression exactly so creation can't fail on a
leftover pair.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Versions <= 0.9.9 wrote an explicit-allowlist .codegraph/.gitignore
(*.db, cache/, .dirty, ...) that never listed daemon.pid or the socket,
so the daemon's runtime pidfile got committed. The wildcard rewrite in
#654/#492/#484 fixed new inits, but the file is only written when
absent, so existing installs kept their stale file forever — the fix
never reached the people hitting it.
Make the gitignore self-heal: ensureGitignore() writes the file if
absent and upgrades a stale CodeGraph-generated default in place,
leaving a user-authored file untouched. A "stale default" is one that
carries our `# CodeGraph data files` header but predates the wildcard
ignore (no bare `*` line) — a header match heals every historical
variant (v0.7.x..0.9.9, all verified to share it) and is idempotent.
validateDirectory() runs on every open()/openSync(), so existing repos
heal on the next codegraph command after upgrading. The duplicated
template (previously inlined in two formats) is consolidated into one
GITIGNORE_CONTENT constant.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A C++ method call whose receiver is another call's result — `Foo::instance().bar()`,
`WidgetFactory::create().draw()`, `openSession()->run()`, or the same stored in an
`auto` local first — lost the receiver's type during extraction. The callee degraded
to a bare method name, so when two classes shared a method name the call silently
resolved to whichever was indexed first (or not at all), corrupting callers / impact /
trace with a plausible-but-wrong edge.
Three parts:
- Capture C++ return types (new nodes.return_type column, schema v5): the
function_definition's `type` field, normalized — smart-pointer pointee unwrapped,
void/primitives dropped.
- Preserve the inner-call receiver in extraction: a C/C++ field_expression whose
receiver is itself a call is encoded `inner().method` instead of dropping to the
bare name. Other languages keep the existing behavior.
- New resolution strategy (matchCppCallChain): infer the receiver's class from the
inner call's return type, then resolve AND validate the method on it. Handles
singletons/accessors, factories returning a different type, free-function
factories, make_unique/make_shared/new/direct construction, single-level member
chains, and namespace-qualified inner calls. A wrong inference yields no edge,
never a wrong one.
EXTRACTION_VERSION 2->3 (re-index to populate return types).
Validated on the issue repro + spdlog: node count stable (no explosion),
deterministic, and ~100 pre-existing wrong `.size()`-style edges removed.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two environments that share one working tree — most concretely Windows
and WSL — can't safely share a single `.codegraph/`: the daemon lockfile
records a platform-specific pid + socket (named pipe vs Unix socket), and
SQLite locking across the WSL2/Windows filesystem boundary is unreliable,
so two daemons over one index risks corruption.
Add a `CODEGRAPH_DIR` env var (default `.codegraph`) that overrides the
per-project data directory name, so each environment keeps its own index
in the same tree (e.g. `CODEGRAPH_DIR=.codegraph-win` on Windows). The
name is resolved live and validated (rejects separators / `..` / absolute,
falling back to the default with a one-time stderr warning). Indexing and
file-watching now skip ANY `.codegraph-*` sibling so neither side trips
over the other's data.
Routes the previously-hardcoded `.codegraph` literals (db path, lockfile,
error log, watcher ignore, file-scan skip, installer) through the
resolver. No extraction-version bump — index content is unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Batch of small, localized fixes from an open-issue triage:
- .codegraph/.gitignore now ignores everything but itself, so the database,
daemon.pid, sockets, and logs stop showing up in git status (#492, #484)
- MCP server answers resources/list and prompts/list with empty lists instead
of -32601, clearing scary log lines in opencode/Codex (#621)
- index SAP HANA .xsjs/.xsjslib as JavaScript (#556) and TS .mts/.cts (#366)
- visit anonymous AMD/CommonJS/IIFE wrapper bodies so their inner functions and
calls are indexed instead of coming up empty (#528)
- batch the changed-file lookup so a huge first sync no longer hits
"too many SQL variables" (#540)
- list files with `git ls-files -z` so non-ASCII/CJK paths survive
core.quotepath and are no longer silently skipped (#541)
- attach Go methods on generic receivers (*T[P]) to their type (#583, RC1)
- impact no longer climbs the structural `contains` edge, so a leaf symbol
stops dragging in its sibling methods (#536)
- README: explicit `codegraph install` step, run in a new shell (#631)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove .codegraph/config.json and the entire config surface. CodeGraph now
indexes every file whose extension maps to a supported language and respects
.gitignore everywhere — git repos via git itself, non-git projects via the
`ignore` library (root + nested .gitignore files, the same way git does).
- Remove CodeGraphConfig/DEFAULT_CONFIG, src/config.ts, and the public config
API (the `config` option on init, getConfig/updateConfig/getConfigPath).
- Derive the source-file allowlist from EXTENSION_MAP (isSourceFile); maxFileSize
is now a constant. Drop the .codegraphignore marker.
- Behavior change: committed, non-gitignored dirs (vendor/, a committed dist/)
are now indexed — .gitignore is the single source of truth.
Earlier inert fields (languages, frameworks, extractDocstrings, trackCallSites,
customPatterns) and their dead helpers are removed as part of this.
Resolves#283.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both narrow indexes are fully covered by the existing (source, kind)
and (target, kind) composites via SQLite's left-prefix scan, so
they're dead weight on every write. Empirical measurements (from the
spike script in PR #122 on a 50K-node / 250K-edge synthetic DB):
- DB size: 34.7 MB → 27.0 MB (-22.2%)
- Bulk insert (250K edges): 590ms → 431ms (1.37× faster)
- source/target lookup latency: no regression
Adds migration v4 to drop both on existing databases; fresh-DB schema
no longer creates them.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes @xenova/transformers dependency, vector storage tables, embedding generation, and semantic search APIs. Simplifies context building to use only FTS search. Eliminates visualizer server, postinstall model download, and related CLI commands. Reduces package size and complexity while maintaining core static analysis capabilities.
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.
Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.
Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
Colby McHenryandClaude Opus 5
