`git ls-files -co --exclude-standard` only sees the submodule pointer in
the main repo's index, so projects using submodules indexed 0 files. Now
the tracked list runs with `-c --recurse-submodules` so submodule
contents are included; untracked files are gathered with a separate
`-o --exclude-standard` call (the two flags can't be combined — git only
supports --recurse-submodules with --cached/--stage).
Fixes#147.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two correctness bugs in the core extraction pipeline, surfaced by an
adversarial stress corpus (5k synthetic export-const declarations
plus a deliberate 8MB single-line file):
1) Every `export const X = ...` produced TWO nodes for the same
symbol — one kind:'variable' from extractExportedVariables, plus
one kind:'constant' from extractVariable (called when the walker
descended into the export_statement child). Stress test showed
100% duplication across 5,003 export-const declarations. The
dedicated extractVariable dispatch is the correct one — it picks
kind from isConst, captures the initializer signature, and walks
type annotations; the export-statement helper was redundant
because the language extractors' isExported predicate already
walks parent chains. Remove the export_statement branch from the
dispatch (children are descended into normally) and drop the
private helper.
2) The bulk indexAll path read each file's stats but never compared
stats.size against config.maxFileSize. Vendored generated files
(multi-MB headers, minified bundles, etc.) were indexed regardless
of the user's size cap. The single-file extractFile path enforced
it; only the bulk path was missing the check. Mirror the
single-file behaviour: emit a 'size_exceeded' warning, count the
file as skipped, advance progress, and continue.
On the stress workspace (5,005 synthetic files; 50,000 fns in one
3MB file; 8MB single-line file; 5,000 export-const declarations):
before: 65,014 nodes (100% var/const duplication, every >1MB file
indexed despite maxFileSize=1MB)
after: 10,008 nodes (0 duplicates, large files correctly skipped
with size_exceeded warnings)
Tests calibrated to the duplicate behavior were updated to look for
kind:'constant' on `export const`, which is the correct kind. Full
suite: 380 passed (was 374 passed, 6 failed before this fix).
* feat(extraction): instantiates + decorates graph edges
Two new structural edges that fill gaps in the call graph for
modern JS/TS / Java / C# / Python / Kotlin codebases.
1) `instantiates` edges from `new Foo(...)`:
The bulk-extraction and visitFunctionBody dispatchers only
recognised `call_expression`; `new_expression` (and the equivalent
`object_creation_expression` / `instance_creation_expression` in
other grammars) was silently ignored. Adds INSTANTIATION_KINDS,
extractInstantiation(), and dispatch from BOTH the top-level
visitNode and the per-function-body walker. Children are still
descended so nested calls inside constructor args (`new Foo(bar())`)
get their own `calls` refs.
Output: a `bootstrap` function that does `new UserService(); new
UserController(svc)` now produces two `instantiates` edges to those
class nodes — previously zero edges.
2) `decorates` edges from `@Decorator` annotations:
Tree-sitter places decorator nodes BEFORE the symbol they apply to
in the AST, so the original walk-time dispatch saw the wrong
nodeStack head (file/class instead of class/method). Replaced with
extractDecoratorsFor(declNode, decoratedId) that runs from inside
extractClass / extractFunction / extractMethod after the symbol's
node id is known.
Looks for decorator nodes in two places:
- Direct named children of the declaration (method/property style)
- Preceding siblings in the parent (TypeScript class style:
@Foo class X {} parses as parent { decorator, class_decl })
Sibling check uses startIndex comparison rather than reference
identity — tree-sitter web bindings return fresh JS wrappers from
parent/namedChild navigation, so `===` is unreliable. Took a debug
session to spot this; flagging in the comment so the next reader
doesn't re-introduce the bug.
Output: a `@Controller` class decorator + `@Get` method decorator
on a NestJS-style controller now produce two `decorates` edges
(class→Controller, method→Get) with the correct source nodes.
Verified live on a synthetic NestJS-shape fixture; all 380
existing tests pass.
* fix(extraction): address reviewer findings — decorator boundary, generic constructors, property/field decorators, marker_annotation, tests
Five fixes from independent semantic review:
- extractDecoratorsFor sibling walk now iterates BACKWARD from the
declaration and stops at the first non-decorator/annotation
separator. Previous version walked forward up to declStart and
consumed every decorator-typed sibling — so two adjacent
decorated classes (`@A class Foo {} @B class Bar {}`) had `@A`
spuriously attributed to `Bar`.
- extractInstantiation strips the type-argument suffix from the
constructor field text. `new Map<K, V>()` was producing
referenceName 'Map<K, V>' (the constructor field is a generic_type
node) and resolution always failed.
- extractProperty and extractField now call extractDecoratorsFor
after their createNode calls. NestJS-style `@Inject() private
svc: Foo` and Java field annotations were being silently dropped.
- consider() in extractDecoratorsFor recognises 'marker_annotation'
in addition to 'decorator'/'annotation'. Java's tree-sitter grammar
emits marker_annotation for arg-less annotations like @Override
and @Deprecated; without this every Java marker annotation was
silently skipped.
- 6 new extraction tests covering: instantiates ref for new Foo(),
generic-type stripping (`new Container<string>()` -> 'Container'),
qualified-new keeps trailing identifier (`new ns.Foo()` -> 'Foo'),
decorates ref for @Foo class X {}, regression for adjacent
decorated classes (each gets its OWN decorator), decorates ref
for @Foo method().
Full test suite: 386 passed (was 380, +6 new extraction tests).
* feat(resolution): kind-aware scoring + Python instantiation promotion
Two follow-ups to the new instantiates/decorates ref kinds, surfaced
during review:
1) name-matcher previously only had a kind bonus for `calls`
(preferring function/method). When a class and a function share a
name across modules, an `instantiates` ref would tie or pick the
wrong candidate. Adds:
- `instantiates` → +25 for class/struct/interface
- `decorates` → +25 for function/method, +15 for class
(Python class decorators, Java annotation interfaces)
2) Python (and Ruby) have no `new` keyword — `Foo()` is the standard
instantiation syntax, indistinguishable from a function call at
extraction time. Resolution can tell the difference once the
target is known: when a `calls` ref resolves to a class/struct,
promote it to `instantiates`. Mirrors the existing extends→
implements promotion in createEdges.
Verified: 386 → 389 passing (+3 tests covering the kind biases and
the Python promotion).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses two tree-sitter misparse patterns: (1) fun interfaces with @Throws annotations parse as function_declaration > ERROR instead of user_type, (2) parent interface bodies become ERROR nodes when containing nested fun interfaces, causing methods to be skipped. Updates isFunInterfaceNode to check ERROR-nested user_type children and resolveBody to prefer ERROR bodies starting with `{`.
Addresses Kotlin interfaces/enums extracted as classes, zero function calls, and missing `fun interface` declarations. Adds classifyClassNode to distinguish interfaces/enums from classes, resolveBody hook for non-field grammar, navigation_expression call handling, getReceiverType for extension functions, and visitNode hook to detect `fun interface` misparse patterns from tree-sitter-kotlin's lack of Kotlin 1.4+ syntax support. Verified against Koin and LeakCanary codebases.
Addresses Ruby methods inside modules missing owner in qualified_name by adding visitNode hook to extract module AST nodes. Methods inside modules now get Module::method qualified names with proper containment relationships. Includes ExtractorContext wiring with pushScope/popScope for language hooks and updates isInsideClassLikeNode to include module kind for nested method handling.
Addresses PHP's base_clause syntax for class inheritance (extends) and implements clause for interface implementation. Adds trait_declaration support and separates property_declaration into fieldTypes. Improves PHP method call extraction by handling member_call_expression and scoped_call_expression with proper receiver name processing, including $ prefix stripping and self/this/parent/static receiver filtering.
Addresses Rust's impl block syntax where trait implementations (`impl Trait for Type`) and trait supertraits (`trait Sub: Super`) create inheritance relationships. Adds getReceiverType to extract method receiver types from impl blocks, enabling proper method-to-struct relationships and qualified name resolution. Verified against Deno codebase and moved from "Needs Verification" to completed language support.
Addresses Swift's inheritance_specifier syntax where type relationships are specified after colons (e.g. `class UploadRequest: DataRequest, Sendable`). Extracts user_type > type_identifier children from inheritance_specifier nodes as 'extends' references to properly model Swift's inheritance, protocol conformance, and struct conformance patterns in the code graph.
Fixes#54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone"
on large codebases because all 16 tree-sitter WASM grammar modules were compiled
upfront by V8, exhausting the WASM Zone allocator.
Changes:
- initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()),
no longer eagerly loads all grammar files
- New loadGrammarsForLanguages() loads only grammars for languages actually present
in the project (e.g. a Dart project loads ~2-3 grammars instead of 16)
- Orchestrator detects needed languages after file scan, before parsing begins
- Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce
WASM memory pressure when embeddings are enabled
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.
Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.
Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
Some .dpr templates use "program;" without a name, which produces an
empty moduleName in the AST. Fall back to the filename (without extension)
to prevent nodes with empty names that cause downstream FK constraint errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove extractFunctionVariable() and its dispatch (already handled by extractVariable)
- Remove dead getGrammar() export (zero callers)
- Deduplicate indexFile by delegating to indexFileWithContent
- Remove redundant arrow function variable extraction tests (covered by existing suite)
- Create file-kind nodes for each parsed source file
- Add isInsideClassLikeNode() for method vs function detection
- Extract arrow functions and function expressions from variable declarators
- Batch file I/O with FILE_IO_BATCH_SIZE=10 using Promise.all
- Add symlink cycle detection with visitedDirs Set in scanDirectory
- Add lazy grammar loading with exported getGrammar() function
- Add indexFileWithContent() for pre-read content processing
- Add tests for file nodes and arrow function extraction
Normalize paths to forward slashes in matchesGlob() and scanDirectory()
so glob exclude patterns work on Windows. Add getGitIgnoredDirectories()
using git ls-files to skip .gitignore'd directories during indexing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
Extend extraction to index two additional categories of symbols
that were previously invisible:
1. Type aliases (e.g. `export type X = ...` in TypeScript,
`type X` in Go, `type X = ...` in Rust, `typealias X` in Swift,
`type_alias` in Kotlin). Adds `typeAliasTypes` to the
LanguageExtractor interface with values for all 13 languages.
2. Exported variable declarations that aren't functions, including:
- Zustand stores: `export const useX = create(...)`
- XState machines: `export const xMachine = createMachine(...)`
- Zod schemas: `export const schema = z.object(...)`
- Config objects: `export const config = { ... }`
- Constants: `export const MAX = 3`
- Arrays: `export const NAMES = [...] as const`
The extractExportedVariables() method is called when visiting
export_statement nodes. It skips variable_declarator values that
are already handled by functionTypes (arrow_function,
function_expression) to avoid duplicate extraction.
Adds 11 new test cases (59 total extraction tests, 215 total).
Tested on production monorepo: nodes increased from 958 to 1,172
(+22%), with 109 new variable nodes and 105 new type_alias nodes.
Only 4 files remain at 0 nodes — all are re-export barrels or
ambient declaration files with no extractable symbols.
Arrow functions and function expressions assigned to variables
(e.g. `export const useAuth = () => { ... }`) were not being indexed
because the arrow_function AST node has no `name` field — the name
lives on the parent variable_declarator node.
Additionally, `isExported()` for TypeScript and JavaScript extractors
only checked 10 characters back from the node's start position, which
missed `export` for deeply nested nodes like arrow functions inside
variable declarations inside export statements.
Changes:
- extractFunction(): When an arrow_function or function_expression
resolves to '<anonymous>', look up the parent variable_declarator
for the name before skipping.
- isExported() (TS + JS): Walk the parent chain to find an
export_statement ancestor instead of substring matching.
- Add 6 test cases covering arrow function exports, function
expression exports, non-exported arrow functions, anonymous
arrow functions, multiple exports, and JavaScript files.
Tested on a real monorepo (238 files): node count increased from
779 to 958 (+23%), with 94 new nodes in packages/ that previously
had 0 coverage.
andreinknvandGitHub
Olaf MonienandClaude Opus 4.6
Martin Oehlert
Tanner Balluff