* feat(search): field-qualified queries (kind:/lang:/path:/name:) + fuzzy typo fallback
Two UX improvements that turn a free-text search into something a
real user can drive precisely.
1) Field-qualified queries.
A new query parser (src/search/query-parser.ts) splits the raw query
into structured filters and a free-text remainder:
kind:function name:auth path:src/api authenticate
becomes
{ kinds: ['function'], nameFilters: ['auth'],
pathFilters: ['src/api'], text: 'authenticate' }
Filters compose with the SearchOptions arg (intersection). Unknown
prefixes pass through as plain text so `query "TODO:"` keeps working.
Quoted values (`path:"my dir"`) handle whitespace. When the user
specifies only filters with no text, the search uses a filter-only
candidate scan instead of bailing out.
Recognised today:
kind: any NodeKind value
lang: any Language value (alias: language:)
path: case-insensitive substring of file_path
name: case-insensitive substring of node.name
2) Fuzzy fallback.
When BOTH FTS and LIKE return nothing AND the text is at least 3
chars, the resolver scans the distinct-name set with a bounded
Damerau-Levenshtein-style edit distance (≤2 for ≥5 chars, ≤1 for
4-char queries, off for shorter). Bounded edit-distance early-exits
once the row min exceeds maxDist, so this stays O(distinct-names *
avg-name-length) with a very low constant.
Verified live against ollama/ollama@v0.22.0:
query "kind:function auth" → only function-kind hits
query "lang:go path:server route" → Go files under server/
query "getUssr" (typo) → finds getUser, SetUser
query "confg" (typo) → finds Config
Full test suite: 380 passed.
* fix(search): address reviewer findings — tokenizer mid-token quotes, fuzzy fan-out cap, larger filter-only over-fetch, unit tests
Five fixes from independent review:
- parseQuery tokenizer: quotes that appear MID-token (path:"my dir/
file") were not being recognised — only quotes at the start of a
token were treated as quoted spans. The fixture path:"my dir"
parsed as ['path:"my', 'dir"'] instead of ['path:"my dir"'].
Tokeniser is now a single state machine that scans into a token
until whitespace OR a quote, and recognises quotes anywhere within
the token (skips to the matching close quote).
- searchNodesFuzzy: cap the per-name follow-up SQL queries at
Math.max(limit*2, 50) AFTER edit-distance filtering. Without
this, a project with many similar names (getUser1, getUser2...)
could fan out far beyond limit queries before the inner-loop
break kicks in.
- searchAllByFilters (filter-only no-text path): bumped over-fetch
multiplier from 2× to 5× so a selective post-filter (e.g.
path:src/very/specific/file.ts) doesn't return fewer than limit
results despite the DB having matches.
- 23 new unit tests in __tests__/search-query-parser.test.ts:
parseQuery covers known-field filter, lang/language alias,
multiple kind: ORs, quoted spans (incl. mid-token), URL
passthrough, empty-value passthrough, unknown prefix passthrough,
unknown value passthrough, all-filters-no-text, empty input,
20k-char input. boundedEditDistance covers identity, single
insertion/deletion/substitution, length-difference shortcut,
empty inputs, case-sensitivity, early-exit correctness.
Full test suite: 853 passed (up from 830).
* refactor(search): derive parser kind/lang sets from types.ts as const
Convert NodeKind and Language to runtime-iterable as const arrays
(NODE_KINDS, LANGUAGES) so the query parser imports the canonical
list instead of duplicating it. Also fix the path: JSDoc to say
substring (matches the .includes() impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(extraction): instantiates + decorates graph edges
Two new structural edges that fill gaps in the call graph for
modern JS/TS / Java / C# / Python / Kotlin codebases.
1) `instantiates` edges from `new Foo(...)`:
The bulk-extraction and visitFunctionBody dispatchers only
recognised `call_expression`; `new_expression` (and the equivalent
`object_creation_expression` / `instance_creation_expression` in
other grammars) was silently ignored. Adds INSTANTIATION_KINDS,
extractInstantiation(), and dispatch from BOTH the top-level
visitNode and the per-function-body walker. Children are still
descended so nested calls inside constructor args (`new Foo(bar())`)
get their own `calls` refs.
Output: a `bootstrap` function that does `new UserService(); new
UserController(svc)` now produces two `instantiates` edges to those
class nodes — previously zero edges.
2) `decorates` edges from `@Decorator` annotations:
Tree-sitter places decorator nodes BEFORE the symbol they apply to
in the AST, so the original walk-time dispatch saw the wrong
nodeStack head (file/class instead of class/method). Replaced with
extractDecoratorsFor(declNode, decoratedId) that runs from inside
extractClass / extractFunction / extractMethod after the symbol's
node id is known.
Looks for decorator nodes in two places:
- Direct named children of the declaration (method/property style)
- Preceding siblings in the parent (TypeScript class style:
@Foo class X {} parses as parent { decorator, class_decl })
Sibling check uses startIndex comparison rather than reference
identity — tree-sitter web bindings return fresh JS wrappers from
parent/namedChild navigation, so `===` is unreliable. Took a debug
session to spot this; flagging in the comment so the next reader
doesn't re-introduce the bug.
Output: a `@Controller` class decorator + `@Get` method decorator
on a NestJS-style controller now produce two `decorates` edges
(class→Controller, method→Get) with the correct source nodes.
Verified live on a synthetic NestJS-shape fixture; all 380
existing tests pass.
* fix(extraction): address reviewer findings — decorator boundary, generic constructors, property/field decorators, marker_annotation, tests
Five fixes from independent semantic review:
- extractDecoratorsFor sibling walk now iterates BACKWARD from the
declaration and stops at the first non-decorator/annotation
separator. Previous version walked forward up to declStart and
consumed every decorator-typed sibling — so two adjacent
decorated classes (`@A class Foo {} @B class Bar {}`) had `@A`
spuriously attributed to `Bar`.
- extractInstantiation strips the type-argument suffix from the
constructor field text. `new Map<K, V>()` was producing
referenceName 'Map<K, V>' (the constructor field is a generic_type
node) and resolution always failed.
- extractProperty and extractField now call extractDecoratorsFor
after their createNode calls. NestJS-style `@Inject() private
svc: Foo` and Java field annotations were being silently dropped.
- consider() in extractDecoratorsFor recognises 'marker_annotation'
in addition to 'decorator'/'annotation'. Java's tree-sitter grammar
emits marker_annotation for arg-less annotations like @Override
and @Deprecated; without this every Java marker annotation was
silently skipped.
- 6 new extraction tests covering: instantiates ref for new Foo(),
generic-type stripping (`new Container<string>()` -> 'Container'),
qualified-new keeps trailing identifier (`new ns.Foo()` -> 'Foo'),
decorates ref for @Foo class X {}, regression for adjacent
decorated classes (each gets its OWN decorator), decorates ref
for @Foo method().
Full test suite: 386 passed (was 380, +6 new extraction tests).
* feat(resolution): kind-aware scoring + Python instantiation promotion
Two follow-ups to the new instantiates/decorates ref kinds, surfaced
during review:
1) name-matcher previously only had a kind bonus for `calls`
(preferring function/method). When a class and a function share a
name across modules, an `instantiates` ref would tie or pick the
wrong candidate. Adds:
- `instantiates` → +25 for class/struct/interface
- `decorates` → +25 for function/method, +15 for class
(Python class decorators, Java annotation interfaces)
2) Python (and Ruby) have no `new` keyword — `Foo()` is the standard
instantiation syntax, indistinguishable from a function call at
extraction time. Resolution can tell the difference once the
target is known: when a `calls` ref resolves to a class/struct,
promote it to `instantiates`. Mirrors the existing extends→
implements promotion in createEdges.
Verified: 386 → 389 passing (+3 tests covering the kind biases and
the Python promotion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses the need for automatic graph synchronization on file changes. Implements FileWatcher using native OS file events (FSEvents/inotify/ReadDirectoryChangesW) with 2-second debouncing to prevent thrashing on rapid saves. Filters changes against include/exclude patterns and ignores .codegraph directory modifications. Integrates with CodeGraph API (watch/unwatch/isWatching methods) and MCP server for automatic activation. Updates documentation to reflect shift from semantic to full-text search and removal of manual hook installation requirements.
Removes @xenova/transformers dependency, vector storage tables, embedding generation, and semantic search APIs. Simplifies context building to use only FTS search. Eliminates visualizer server, postinstall model download, and related CLI commands. Reduces package size and complexity while maintaining core static analysis capabilities.
Addresses two tree-sitter misparse patterns: (1) fun interfaces with @Throws annotations parse as function_declaration > ERROR instead of user_type, (2) parent interface bodies become ERROR nodes when containing nested fun interfaces, causing methods to be skipped. Updates isFunInterfaceNode to check ERROR-nested user_type children and resolveBody to prefer ERROR bodies starting with `{`.
Addresses Kotlin interfaces/enums extracted as classes, zero function calls, and missing `fun interface` declarations. Adds classifyClassNode to distinguish interfaces/enums from classes, resolveBody hook for non-field grammar, navigation_expression call handling, getReceiverType for extension functions, and visitNode hook to detect `fun interface` misparse patterns from tree-sitter-kotlin's lack of Kotlin 1.4+ syntax support. Verified against Koin and LeakCanary codebases.
Addresses Ruby methods inside modules missing owner in qualified_name by adding visitNode hook to extract module AST nodes. Methods inside modules now get Module::method qualified names with proper containment relationships. Includes ExtractorContext wiring with pushScope/popScope for language hooks and updates isInsideClassLikeNode to include module kind for nested method handling.
Addresses PHP's base_clause syntax for class inheritance (extends) and implements clause for interface implementation. Adds trait_declaration support and separates property_declaration into fieldTypes. Improves PHP method call extraction by handling member_call_expression and scoped_call_expression with proper receiver name processing, including $ prefix stripping and self/this/parent/static receiver filtering.
Addresses Rust's impl block syntax where trait implementations (`impl Trait for Type`) and trait supertraits (`trait Sub: Super`) create inheritance relationships. Adds getReceiverType to extract method receiver types from impl blocks, enabling proper method-to-struct relationships and qualified name resolution. Verified against Deno codebase and moved from "Needs Verification" to completed language support.
Addresses Swift's inheritance_specifier syntax where type relationships are specified after colons (e.g. `class UploadRequest: DataRequest, Sendable`). Extracts user_type > type_identifier children from inheritance_specifier nodes as 'extends' references to properly model Swift's inheritance, protocol conformance, and struct conformance patterns in the code graph.
Addresses cases where multi-word queries like "search execution from request to shard" return generic single-term matches instead of highly relevant classes matching multiple terms. Applies co-occurrence boosting before truncation to prioritize nodes matching 2+ query terms, adds compound term matching to catch classes like "SearchShardsRequest" that contain multiple query terms at any position, and widens per-term accumulation pools to prevent relevant multi-term matches from being filtered out early.
Introduces automated testing infrastructure to measure CodeGraph performance across searchNodes and findRelevantContext APIs. Includes recall/MRR scoring metrics, predefined test cases for symbol lookup and context exploration, and JSON report generation. Enhances context building with acronym extraction, definition prefix matching, and improved FTS filtering to exclude imports by default.
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
Adds type annotation parsing to create references edges for parameter types, return types, and variable type annotations in TypeScript and other typed languages. Expands symbol extraction from queries to capture lowercase identifiers and filters out more common English words. Removes obsolete search utility tests.
Testing showed semantic search produces significantly better results for
natural language queries that Claude writes. FTS alone often ranks
properties above their parent classes and misses conceptual matches.
Embeddings are now always on — the vector manager is created eagerly,
with model download and embedding generation still happening lazily.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Name matching was creating false `calls` edges between unrelated modules
in monorepos because `findBestMatch()` had no concept of directory
proximity — functions with common names (e.g. `navigate`) in different
apps scored identically and resolved to whichever came first.
Adds path proximity scoring (shared directory segments) so same-module
candidates strongly win over cross-boundary ones, and lowers confidence
for distant matches so import-based resolution takes precedence.
Closes#67
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes#54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone"
on large codebases because all 16 tree-sitter WASM grammar modules were compiled
upfront by V8, exhausting the WASM Zone allocator.
Changes:
- initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()),
no longer eagerly loads all grammar files
- New loadGrammarsForLanguages() loads only grammars for languages actually present
in the project (e.g. a Dart project loads ~2-3 grammars instead of 16)
- Orchestrator detects needed languages after file scan, before parsing begins
- Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce
WASM memory pressure when embeddings are enabled
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.
Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.
Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
Some .dpr templates use "program;" without a name, which produces an
empty moduleName in the AST. Fall back to the filename (without extension)
to prevent nodes with empty names that cause downstream FK constraint errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- extraction/index.ts: use picomatch with static import (replacing
dynamic require) and keep normalizePath for other call sites
- utils.ts: keep normalizePath from main, take PR's PID-based FileLock
Both functions have zero callers — dead code on arrival. Remove them
and their tests (9 tests) to keep the module focused on what's
actually used: search term extraction, path relevance scoring, and
kind bonuses.
- Remove extractFunctionVariable() and its dispatch (already handled by extractVariable)
- Remove dead getGrammar() export (zero callers)
- Deduplicate indexFile by delegating to indexFileWithContent
- Remove redundant arrow function variable extraction tests (covered by existing suite)
- Create file-kind nodes for each parsed source file
- Add isInsideClassLikeNode() for method vs function detection
- Extract arrow functions and function expressions from variable declarators
- Batch file I/O with FILE_IO_BATCH_SIZE=10 using Promise.all
- Add symlink cycle detection with visitedDirs Set in scanDirectory
- Add lazy grammar loading with exported getGrammar() function
- Add indexFileWithContent() for pre-read content processing
- Add tests for file nodes and arrow function extraction
- Add validateProjectPath() to reject sensitive system directories
- Add isPathWithinRoot/isPathWithinRootReal for symlink-aware path checks
- Replace hand-rolled glob-to-regex with picomatch to prevent ReDoS
- Add isSafeRegex() to reject custom patterns with nested quantifiers
- Replace FileLock with PID-tracking version that detects stale locks
- Add symlink detection in removeDirectory/listDirectoryContents
- Add subdirectory name validation in ensureSubdirectory
- Add atomicWriteFileSync and corrupted file backup in config-writer
- Add MCP input validation (validateString) for all tool handlers
- Fix CLAUDE.md section replacement to handle ### subsections correctly
Normalize paths to forward slashes in matchesGlob() and scanDirectory()
so glob exclude patterns work on Windows. Add getGitIgnoredDirectories()
using git ls-files to skip .gitignore'd directories during indexing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Covers arrow function extraction, best-candidate resolution, graph
traversal direction fix, MCP symbol disambiguation, output truncation,
CLI uninit command, and more. Tests requiring better-sqlite3 native
bindings are conditionally skipped.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
Extend extraction to index two additional categories of symbols
that were previously invisible:
1. Type aliases (e.g. `export type X = ...` in TypeScript,
`type X` in Go, `type X = ...` in Rust, `typealias X` in Swift,
`type_alias` in Kotlin). Adds `typeAliasTypes` to the
LanguageExtractor interface with values for all 13 languages.
2. Exported variable declarations that aren't functions, including:
- Zustand stores: `export const useX = create(...)`
- XState machines: `export const xMachine = createMachine(...)`
- Zod schemas: `export const schema = z.object(...)`
- Config objects: `export const config = { ... }`
- Constants: `export const MAX = 3`
- Arrays: `export const NAMES = [...] as const`
The extractExportedVariables() method is called when visiting
export_statement nodes. It skips variable_declarator values that
are already handled by functionTypes (arrow_function,
function_expression) to avoid duplicate extraction.
Adds 11 new test cases (59 total extraction tests, 215 total).
Tested on production monorepo: nodes increased from 958 to 1,172
(+22%), with 109 new variable nodes and 105 new type_alias nodes.
Only 4 files remain at 0 nodes — all are re-export barrels or
ambient declaration files with no extractable symbols.
Arrow functions and function expressions assigned to variables
(e.g. `export const useAuth = () => { ... }`) were not being indexed
because the arrow_function AST node has no `name` field — the name
lives on the parent variable_declarator node.
Additionally, `isExported()` for TypeScript and JavaScript extractors
only checked 10 characters back from the node's start position, which
missed `export` for deeply nested nodes like arrow functions inside
variable declarations inside export statements.
Changes:
- extractFunction(): When an arrow_function or function_expression
resolves to '<anonymous>', look up the parent variable_declarator
for the name before skipping.
- isExported() (TS + JS): Walk the parent chain to find an
export_statement ancestor instead of substring matching.
- Add 6 test cases covering arrow function exports, function
expression exports, non-exported arrow functions, anonymous
arrow functions, multiple exports, and JavaScript files.
Tested on a real monorepo (238 files): node count increased from
779 to 958 (+23%), with 94 new nodes in packages/ that previously
had 0 coverage.
- Add evaluation test suite with TypeScript and Python fixtures
- Fix MCP server to defer CodeGraph init until rootUri received
- Fix call edge extraction by calling resolveReferences() after indexAll/sync
- Fix glob matching for root-level files (e.g., **/*.py now matches auth.py)
- Fix duplicate node extraction for methods inside classes
- Update context tests to use buildContext for semantic search + graph traversal
- Export unused formatter functions to fix build
Evaluation results:
- TypeScript: 96% precision, 79% recall, 85% F1
- Python: 99% precision, 80% recall, 85% F1
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Olaf MonienandClaude Opus 4.6
Martin Oehlert
Tanner Balluff