Closes the visibility gap behind issues #138 (WASM-on-macOS) and #139
(MCP "database is locked"). `better-sqlite3` is in optionalDependencies,
so when the native build fails npm install still succeeds and the
runtime silently falls back to node-sqlite3-wasm — 5-10x slower and
without WAL, so writers block readers (which is what makes the MCP
server appear to "lock the DB" in #139). The only existing signal was
a one-line `console.warn` to stderr that MCP transports typically
swallow.
This patch does NOT change install behavior — better-sqlite3 stays in
optionalDependencies so cross-platform installs keep working. It just
makes the substitution observable + recoverable.
## Visibility (4 surfaces)
- CLI `codegraph status`: new `Backend:` line under Index Statistics.
`native` rendered green; `wasm` rendered yellow with an inline
`npm rebuild better-sqlite3` nudge. Also exposed in `--json` as
`backend: 'native' | 'wasm'`.
- MCP `codegraph_status`: new `**Backend:**` line. Native form reads
`native (better-sqlite3)`; wasm form prepends a warning glyph and
includes the full fix recipe.
- Stderr banner on fallback (`buildWasmFallbackBanner`): replaces the
bare one-line `console.warn` with a multi-line bordered banner
covering macOS + Linux fix steps and optionally appending the
native load error.
- README troubleshooting: new "Indexing is slow / MCP database is
locked / WASM fallback active" entry that walks users to the
`Backend:` line and the fix.
## Per-instance backend tracking
`createDatabase` previously set a module-level `activeBackend` global.
MCP can open multiple project DBs in one process via the
`getCodeGraph()` cache, so the global would race / overwrite. Refactor:
`createDatabase` now returns `{db, backend}`, `DatabaseConnection`
carries `private backend` and exposes `getBackend()`, and
`CodeGraph.getBackend()` is the public surface. The CLI and MCP both
call `cg.getBackend()`.
## What this does NOT fix
The root cause of users landing on WASM is environment-specific (Mac
without Xcode CLT, Node version mismatch, etc.) and not fixable in
code without changing the optionalDependencies design. The README
entry tells users what to run; `Backend: native` after rebuild is the
confirmation signal.
## Tests
New `__tests__/sqlite-backend.test.ts` (6 tests) pins the banner
recipe content (so future edits can't strip the recovery commands),
the `WASM_FALLBACK_FIX_RECIPE` constant, and per-instance
`DatabaseConnection.getBackend()` / `CodeGraph.getBackend()` reporting.
Suite: 503 → 509, all passing.
Credit to @andreinknv whose analysis on #138 (and patches on his fork
at 6d0e7a2 + 69f7001) framed the visibility approach.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both narrow indexes are fully covered by the existing (source, kind)
and (target, kind) composites via SQLite's left-prefix scan, so
they're dead weight on every write. Empirical measurements (from the
spike script in PR #122 on a 50K-node / 250K-edge synthetic DB):
- DB size: 34.7 MB → 27.0 MB (-22.2%)
- Bulk insert (250K edges): 590ms → 431ms (1.37× faster)
- source/target lookup latency: no regression
Adds migration v4 to drop both on existing databases; fresh-DB schema
no longer creates them.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(search): field-qualified queries (kind:/lang:/path:/name:) + fuzzy typo fallback
Two UX improvements that turn a free-text search into something a
real user can drive precisely.
1) Field-qualified queries.
A new query parser (src/search/query-parser.ts) splits the raw query
into structured filters and a free-text remainder:
kind:function name:auth path:src/api authenticate
becomes
{ kinds: ['function'], nameFilters: ['auth'],
pathFilters: ['src/api'], text: 'authenticate' }
Filters compose with the SearchOptions arg (intersection). Unknown
prefixes pass through as plain text so `query "TODO:"` keeps working.
Quoted values (`path:"my dir"`) handle whitespace. When the user
specifies only filters with no text, the search uses a filter-only
candidate scan instead of bailing out.
Recognised today:
kind: any NodeKind value
lang: any Language value (alias: language:)
path: case-insensitive substring of file_path
name: case-insensitive substring of node.name
2) Fuzzy fallback.
When BOTH FTS and LIKE return nothing AND the text is at least 3
chars, the resolver scans the distinct-name set with a bounded
Damerau-Levenshtein-style edit distance (≤2 for ≥5 chars, ≤1 for
4-char queries, off for shorter). Bounded edit-distance early-exits
once the row min exceeds maxDist, so this stays O(distinct-names *
avg-name-length) with a very low constant.
Verified live against ollama/ollama@v0.22.0:
query "kind:function auth" → only function-kind hits
query "lang:go path:server route" → Go files under server/
query "getUssr" (typo) → finds getUser, SetUser
query "confg" (typo) → finds Config
Full test suite: 380 passed.
* fix(search): address reviewer findings — tokenizer mid-token quotes, fuzzy fan-out cap, larger filter-only over-fetch, unit tests
Five fixes from independent review:
- parseQuery tokenizer: quotes that appear MID-token (path:"my dir/
file") were not being recognised — only quotes at the start of a
token were treated as quoted spans. The fixture path:"my dir"
parsed as ['path:"my', 'dir"'] instead of ['path:"my dir"'].
Tokeniser is now a single state machine that scans into a token
until whitespace OR a quote, and recognises quotes anywhere within
the token (skips to the matching close quote).
- searchNodesFuzzy: cap the per-name follow-up SQL queries at
Math.max(limit*2, 50) AFTER edit-distance filtering. Without
this, a project with many similar names (getUser1, getUser2...)
could fan out far beyond limit queries before the inner-loop
break kicks in.
- searchAllByFilters (filter-only no-text path): bumped over-fetch
multiplier from 2× to 5× so a selective post-filter (e.g.
path:src/very/specific/file.ts) doesn't return fewer than limit
results despite the DB having matches.
- 23 new unit tests in __tests__/search-query-parser.test.ts:
parseQuery covers known-field filter, lang/language alias,
multiple kind: ORs, quoted spans (incl. mid-token), URL
passthrough, empty-value passthrough, unknown prefix passthrough,
unknown value passthrough, all-filters-no-text, empty input,
20k-char input. boundedEditDistance covers identity, single
insertion/deletion/substitution, length-difference shortcut,
empty inputs, case-sensitivity, early-exit correctness.
Full test suite: 853 passed (up from 830).
* refactor(search): derive parser kind/lang sets from types.ts as const
Convert NodeKind and Language to runtime-iterable as const arrays
(NODE_KINDS, LANGUAGES) so the query parser imports the canonical
list instead of duplicating it. Also fix the path: JSDoc to say
substring (matches the .includes() impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Removes @xenova/transformers dependency, vector storage tables, embedding generation, and semantic search APIs. Simplifies context building to use only FTS search. Eliminates visualizer server, postinstall model download, and related CLI commands. Reduces package size and complexity while maintaining core static analysis capabilities.
Uses max FTS score as baseline for exact name matches to ensure nameMatchBonus differentiation during rescoring, increases exact match limit from 5 to 20 candidates, and adds common conversational terms to stop words to reduce query noise.
Addresses cases where BM25 can bury short exact-match names (e.g. "Query") under hundreds of compound names (e.g. "QueryParserTokenManager") in large codebases, pushing them past the FTS fetch limit before post-hoc scoring can help. Supplements primary search results with direct case-insensitive name lookups for each query term, ensuring exact matches are always candidates for scoring.
Addresses cases where BFS with multiple entry points leaves most nodes disconnected after trimming. Discovers edges between already-selected nodes using specific relationship types (calls, extends, implements, references, overrides) to recover inter-node connectivity that would otherwise be lost during the node selection process.
Introduces LIKE-based substring matching to find symbols like "Search" within "TransportSearchAction" that FTS cannot match due to tokenization boundaries. Adds dedicated type hierarchy traversal to ensure parent/child classes and interfaces are included in context results, preventing BFS budget exhaustion on method-level nodes before reaching inheritance relationships.
Adds nameMatchBonus scoring to prioritize results where node names exactly or partially match query terms. Implements dedicated field extraction for Java/C# to properly categorize class fields vs variables. Optimizes BM25 search with column weights favoring name matches and increased result fetching before post-processing. Refines stop words list to preserve common programming terms like "get", "find", "list".
Improves search accuracy by boosting results when multiple query symbols appear in the same file, addressing cases where common names like "run" return too many results. Adds Go method receiver type extraction to qualified names for better searchability (e.g., "scrapeLoop.run"). Optimizes database queries with two-pass approach to handle distinctive vs common symbol names efficiently.
Replaces O(n) file scanning with O(log n) indexed database lookups by adding getAllNodeNames query and caching node lookups by name/qualified name. Pre-filters references against known symbol names to skip expensive resolution for non-existent symbols. Consolidates Go resolver helper functions into a unified resolveByNameAndKind function and moves built-in symbol sets to module-level constants for better performance.
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
Java extraction:
- Handle Java method_invocation AST (receiver.method pattern)
- Support Java extends_interfaces and super_interfaces with type_list
- Create unresolved references for Java imports for cross-file resolution
- Extract interface inheritance via extractInheritance
MCP tools:
- Aggregate callers/callees/impact across ALL matching symbols (e.g. multiple
overloads or same-named methods in different classes)
- New findAllSymbols() helper for multi-symbol lookup
Graph traversal:
- Impact analysis now traverses into container children (class → methods)
so that callers of methods appear in the impact radius of their class
Other:
- Add deleteSpecificResolvedReferences() for precise cleanup after resolution
- Add 'instance-method' to resolvedBy union type
- Version bump to 0.6.8
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminates anonymous error reporting functionality that was collecting stack traces and error context via Sentry. Removes all telemetry-related code, configuration options, and documentation references.
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.
Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.
Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
- Remove getNodesByIds, getNodesByKinds (zero callers)
- Remove getFileHashMap, getFileSyncMap (zero callers)
- Remove getDynamicStmt cache (only used by removed batch methods)
- Revert getStaleFiles to simple implementation (zero callers, no need to optimize)
- Remove intent field from SearchOptions (never read in search logic)
- Add provenance column on edges for tracking how edges were created
- Add project_metadata table for version/provenance tracking
- Make unresolved_refs file_path/language NOT NULL with defaults
- Add composite indexes for unresolved_refs and edges.provenance
- Update v2 migration to handle all new schema additions
- Record schema version on initialize to prevent re-migration
- Add dynamic prepared statement cache (getDynamicStmt) for varying SQL
- Add batch methods: getNodesByIds, getNodesByKinds, getFileHashMap, getFileSyncMap
- Add project metadata methods: getMetadata, setMetadata, getAllMetadata
- Optimize getStats to use single aggregate query
- Optimize getStaleFiles to use temporary table JOIN
- Add provenance parameter to getOutgoingEdges
- Add intent field to SearchOptions type
Node insertion used plain INSERT which crashes on duplicate IDs.
In large C/C++ projects, tree-sitter can produce duplicate nodes for
the same symbol (e.g. typedef struct where both struct_specifier and
type_definition resolve to the same name/kind/line, or multiple
anonymous constructs on the same line).
- Change nodes INSERT to INSERT OR REPLACE (idempotent, same data)
- Change edges INSERT to INSERT OR IGNORE (skip duplicate edges)
The node ID is sha256(filePath:kind:name:line) which already uses full
relative paths, so cross-directory collisions are not the issue.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SQLite performance pragmas: synchronous=NORMAL, 64MB cache,
memory temp store, 256MB mmap (safe with WAL mode)
- Batch insert for unresolved refs: single transaction instead of
N individual inserts per file
- Symbol caching (warmCaches): pre-load all nodes into memory maps
before resolution, eliminating repeated SQLite queries per ref
- Async file I/O: fs.stat/readFile in indexFile() are now non-blocking
- Denormalize filePath/language onto UnresolvedReference: avoids N
node lookups during resolution, with schema migration v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements security improvements inspired by PR #16 (credit: MO2k4):
- Add validatePathWithinRoot() to prevent path traversal attacks in
extraction and context building
- Clamp MCP tool inputs (limit, depth, maxDepth) to sane ranges
- Use atomic writes (temp file + rename) for config saves
- Add symlink cycle detection in directory scanning to prevent infinite loops
- Replace all JSON.parse calls in db/queries.ts with safeJsonParse fallbacks
to handle corrupted database metadata gracefully
- Add cross-process FileLock for DB write operations (indexAll, indexFiles,
sync) to prevent concurrent writes from CLI, MCP server, and git hooks
- Remove unused path import from context/index.ts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
- Add evaluation test suite with TypeScript and Python fixtures
- Fix MCP server to defer CodeGraph init until rootUri received
- Fix call edge extraction by calling resolveReferences() after indexAll/sync
- Fix glob matching for root-level files (e.g., **/*.py now matches auth.py)
- Fix duplicate node extraction for methods inside classes
- Update context tests to use buildContext for semantic search + graph traversal
- Export unused formatter functions to fix build
Evaluation results:
- TypeScript: 96% precision, 79% recall, 85% F1
- Python: 99% precision, 80% recall, 85% F1
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Martin Oehlert