Commit Graph
123 Commits
Author SHA1 Message Date
8eed24327c feat(extraction): instantiates + decorates graph edges (#134)
* feat(extraction): instantiates + decorates graph edges

Two new structural edges that fill gaps in the call graph for
modern JS/TS / Java / C# / Python / Kotlin codebases.

1) `instantiates` edges from `new Foo(...)`:

The bulk-extraction and visitFunctionBody dispatchers only
recognised `call_expression`; `new_expression` (and the equivalent
`object_creation_expression` / `instance_creation_expression` in
other grammars) was silently ignored. Adds INSTANTIATION_KINDS,
extractInstantiation(), and dispatch from BOTH the top-level
visitNode and the per-function-body walker. Children are still
descended so nested calls inside constructor args (`new Foo(bar())`)
get their own `calls` refs.

Output: a `bootstrap` function that does `new UserService(); new
UserController(svc)` now produces two `instantiates` edges to those
class nodes — previously zero edges.

2) `decorates` edges from `@Decorator` annotations:

Tree-sitter places decorator nodes BEFORE the symbol they apply to
in the AST, so the original walk-time dispatch saw the wrong
nodeStack head (file/class instead of class/method). Replaced with
extractDecoratorsFor(declNode, decoratedId) that runs from inside
extractClass / extractFunction / extractMethod after the symbol's
node id is known.

Looks for decorator nodes in two places:
  - Direct named children of the declaration (method/property style)
  - Preceding siblings in the parent (TypeScript class style:
    @Foo class X {} parses as parent { decorator, class_decl })

Sibling check uses startIndex comparison rather than reference
identity — tree-sitter web bindings return fresh JS wrappers from
parent/namedChild navigation, so `===` is unreliable. Took a debug
session to spot this; flagging in the comment so the next reader
doesn't re-introduce the bug.

Output: a `@Controller` class decorator + `@Get` method decorator
on a NestJS-style controller now produce two `decorates` edges
(class→Controller, method→Get) with the correct source nodes.

Verified live on a synthetic NestJS-shape fixture; all 380
existing tests pass.

* fix(extraction): address reviewer findings — decorator boundary, generic constructors, property/field decorators, marker_annotation, tests

Five fixes from independent semantic review:

- extractDecoratorsFor sibling walk now iterates BACKWARD from the
  declaration and stops at the first non-decorator/annotation
  separator. Previous version walked forward up to declStart and
  consumed every decorator-typed sibling — so two adjacent
  decorated classes (`@A class Foo {} @B class Bar {}`) had `@A`
  spuriously attributed to `Bar`.

- extractInstantiation strips the type-argument suffix from the
  constructor field text. `new Map<K, V>()` was producing
  referenceName 'Map<K, V>' (the constructor field is a generic_type
  node) and resolution always failed.

- extractProperty and extractField now call extractDecoratorsFor
  after their createNode calls. NestJS-style `@Inject() private
  svc: Foo` and Java field annotations were being silently dropped.

- consider() in extractDecoratorsFor recognises 'marker_annotation'
  in addition to 'decorator'/'annotation'. Java's tree-sitter grammar
  emits marker_annotation for arg-less annotations like @Override
  and @Deprecated; without this every Java marker annotation was
  silently skipped.

- 6 new extraction tests covering: instantiates ref for new Foo(),
  generic-type stripping (`new Container<string>()` -> 'Container'),
  qualified-new keeps trailing identifier (`new ns.Foo()` -> 'Foo'),
  decorates ref for @Foo class X {}, regression for adjacent
  decorated classes (each gets its OWN decorator), decorates ref
  for @Foo method().

Full test suite: 386 passed (was 380, +6 new extraction tests).

* feat(resolution): kind-aware scoring + Python instantiation promotion

Two follow-ups to the new instantiates/decorates ref kinds, surfaced
during review:

1) name-matcher previously only had a kind bonus for `calls`
   (preferring function/method). When a class and a function share a
   name across modules, an `instantiates` ref would tie or pick the
   wrong candidate. Adds:
     - `instantiates` → +25 for class/struct/interface
     - `decorates`    → +25 for function/method, +15 for class
       (Python class decorators, Java annotation interfaces)

2) Python (and Ruby) have no `new` keyword — `Foo()` is the standard
   instantiation syntax, indistinguishable from a function call at
   extraction time. Resolution can tell the difference once the
   target is known: when a `calls` ref resolves to a class/struct,
   promote it to `instantiates`. Mirrors the existing extends→
   implements promotion in createEdges.

Verified: 386 → 389 passing (+3 tests covering the kind biases and
the Python promotion).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Colby McHenry <me@colbymchenry.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 20:08:55 -05:00
Colby McHenry f402ab8363 feat: Add complete PHP language support with trait handling and property extraction
Addresses PHP traits extracted as classes, missing class properties, skipped constants, and invisible trait usage. Adds classifyClassNode to distinguish traits from classes, fixes property extraction for PHP's property_element AST structure (added 4,366 field nodes), and adds visitNode hook for class constants and trait use declarations (increased trait edges from 636 to 1,514). Also improves Liquid schema name handling and file path reference resolution. Verified against Laravel codebase.
2026-04-07 13:44:14 -05:00
Colby McHenry 07d899b735 feat: Promote "extends" to "implements" for class-to-interface relationships in edge creation
Addresses semantic accuracy in inheritance relationships where classes use "extends" syntax to implement interfaces. Adds target node inspection to detect interface/protocol targets and promotes the edge kind from "extends" to "implements" when the source is a concrete class or struct, ensuring proper representation of implementation vs inheritance relationships in the code graph.
2026-04-07 00:02:47 -05:00
Colby McHenry e12bd7ce91 feat: Include receiver names in method call extraction and improve built-in filtering
Addresses method call resolution ambiguity where bare method names couldn't be distinguished from function calls. Modifies tree-sitter extraction to include receiver names (e.g., "console.log" instead of just "log") while skipping common instance references like self/this. Updates built-in filtering to be language-specific and adds Python built-in method detection based on receiver types and method names.
2026-04-06 20:31:37 -05:00
Colby McHenry 7a3afc9124 feat: Enhance symbol search with co-location boosting and receiver type support
Improves search accuracy by boosting results when multiple query symbols appear in the same file, addressing cases where common names like "run" return too many results. Adds Go method receiver type extraction to qualified names for better searchability (e.g., "scrapeLoop.run"). Optimizes database queries with two-pass approach to handle distinctive vs common symbol names efficiently.
2026-04-06 11:16:56 -05:00
Colby McHenry d256af3a23 feat: Optimize reference resolution with indexed queries and built-in filtering
Replaces O(n) file scanning with O(log n) indexed database lookups by adding getAllNodeNames query and caching node lookups by name/qualified name. Pre-filters references against known symbol names to skip expensive resolution for non-existent symbols. Consolidates Go resolver helper functions into a unified resolveByNameAndKind function and moves built-in symbol sets to module-level constants for better performance.
2026-04-06 10:45:14 -05:00
Colby McHenry 9a2d3d9a13 feat: Fix progress bar hanging and improve UI responsiveness during indexing
Adds strategic yield points and direct stdout writes to prevent progress animation from freezing when the main thread is blocked by synchronous operations. Introduces 'finalizing' phase to smooth transition between parsing and resolving steps, ensuring progress reaches 100% completion.
2026-04-06 09:50:30 -05:00
Colby McHenry e4908e1270 feat: Add database schema v3 with optimized node lookups and improved error handling
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
2026-04-04 10:19:08 -05:00
Colby McHenryandClaude Opus 4.6 d04a911309 feat: Improve Java extraction, multi-symbol aggregation, and impact traversal
Java extraction:
- Handle Java method_invocation AST (receiver.method pattern)
- Support Java extends_interfaces and super_interfaces with type_list
- Create unresolved references for Java imports for cross-file resolution
- Extract interface inheritance via extractInheritance

MCP tools:
- Aggregate callers/callees/impact across ALL matching symbols (e.g. multiple
  overloads or same-named methods in different classes)
- New findAllSymbols() helper for multi-symbol lookup

Graph traversal:
- Impact analysis now traverses into container children (class → methods)
  so that callers of methods appear in the impact radius of their class

Other:
- Add deleteSpecificResolvedReferences() for precise cleanup after resolution
- Add 'instance-method' to resolvedBy union type
- Version bump to 0.6.8

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:30:28 -05:00
Colby McHenryandClaude Opus 4.6 8b541be894 fix: Improve Python resolution accuracy and context relevance
Eliminate cross-language false positives in name resolution and deprioritize
test files in context building. Benchmarked on a Python+Rust codebase where
37% of edges were false positives from Python built-in methods resolving to
Rust functions (e.g., list.extend → Rust extend).

Resolution fixes (index-time):
- Filter Python built-in type method calls (list.extend, dict.update, etc.)
- Filter bare Python built-in method names (append, extend, pop, keys, etc.)
- Add language boundary checks to matchMethodCall strategies 1, 2, and 3
- Penalize cross-language matches: -80 points in findBestMatch (was 0)
- Reduce confidence for single cross-language exact matches (0.5 vs 0.9)
- Prefer same-language candidates in matchFuzzy

Context relevance fixes (query-time):
- Add isTestFile() utility detecting test files across Python/JS/TS/Go/Rust/Java
- Deprioritize test files in scorePathRelevance (-15 penalty)
- Reduce test file scores to 30% in context builder result merging
- Both skip deprioritization when query mentions "test" or "spec"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:29:21 -05:00
Colby McHenry 8b5622d346 Remove Sentry telemetry system from CodeGraph
Eliminates anonymous error reporting functionality that was collecting stack traces and error context via Sentry. Removes all telemetry-related code, configuration options, and documentation references.
2026-04-03 09:54:07 -05:00
Colby McHenryandClaude Opus 4.6 584bd94ecc fix: Prevent false cross-module edges in name-based resolution
Name matching was creating false `calls` edges between unrelated modules
in monorepos because `findBestMatch()` had no concept of directory
proximity — functions with common names (e.g. `navigate`) in different
apps scored identically and resolved to whichever came first.

Adds path proximity scoring (shared directory segments) so same-module
candidates strongly win over cross-boundary ones, and lowers confidence
for distant matches so import-based resolution takes precedence.

Closes #67

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 14:23:13 -05:00
Olaf MonienandClaude Opus 4.6 a86ddf6e6a fix: Resolve Pascal extraction edge cases
- Fix defProc method lookup collisions when multiple classes share
  method names (e.g. Create) by indexing qualified forms
- Remove overly broad 'Id' prefix filter that swallowed user-defined
  symbols like Identifier or IdleTimer
- Use case-insensitive extension matching for DFM/FMX routing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 01:29:49 +01:00
Olaf MonienandClaude Opus 4.6 77135445db Merge origin/main into delphi-support
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 20:23:54 +01:00
Colby McHenry 8346440592 Add WASM fallbacks for tree-sitter and SQLite, fix installer
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.

Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.

Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
2026-02-14 00:56:15 -06:00
Olaf MonienandClaude Opus 4.6 ab9d7188c1 perf: Cache import mappings and index fuzzy matches for resolution
Two major optimizations for the ref resolution phase:

1. Cache extractImportMappings() results per file path — previously
   re-read and re-parsed the source file for every single ref from
   that file (e.g. 100 refs from one file = 100 identical file reads)

2. Replace linear scan in matchFuzzy() with a lazily-built
   case-insensitive Map index — O(1) lookup instead of iterating
   all function/method/class nodes for every unresolved ref.
   Also drop low-value prefix matching (confidence 0.3).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 19:29:18 +01:00
Olaf MonienandClaude Opus 4.6 f5d9a0efe5 perf: Speed up ref resolution for large Pascal codebases
Two optimizations that eliminate the O(n²) bottleneck:

1. Add Pascal built-in filtering to isBuiltInOrExternal() — skips
   resolution attempts for System.*, Vcl.*, Fmx.* and ~60 common
   RTL identifiers (WriteLn, Create, Free, TObject, etc.)

2. Cache getNodesByKind() results during warm cache phase — matchFuzzy()
   was calling the database 3 times per unresolved reference instead
   of using the in-memory cache that warmCaches() already builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:28:37 +01:00
Colby McHenry acd9713632 Optimize reference resolution with in-memory caches
The resolving refs phase stalled on large projects (3400+ files, 38k+ nodes)
because matchFuzzy loaded ALL functions/methods/classes per ref, import
mappings were re-extracted per ref, and fileExists hit disk every call.

Add kindCache, lowerNameCache, importMappingCache, and knownFiles set to
warmCaches(). Rewrite matchFuzzy to use O(1) lowercase index lookup instead
of 3x getNodesByKind scans. Cache import mappings per file. Pre-build file
existence set from the index for O(1) fileExists checks.
2026-02-10 18:14:26 -06:00
Colby McHenryandClaude Opus 4.6 4c983ba082 Add Svelte language support with SvelteKit framework resolver
- Add 'svelte' to Language type, DEFAULT_CONFIG includes, grammars, and config validation
- Add SvelteExtractor that extracts <script> blocks and delegates to TS/JS TreeSitterExtractor
- Add Svelte framework resolver for runes ($state, $derived, $effect, etc.), store auto-subscriptions, SvelteKit module aliases ($app/*, $env/*, $lib/*), and SvelteKit route detection
- Update README to list Svelte and Dart in supported languages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 01:32:52 -06:00
Colby McHenryandClaude Opus 4.6 94c8d5ccaa Port extraction quality improvements from PR #15
- Fix arrow function extraction: explicitly call extractFunction() for
  arrow functions/function expressions in variable declarations instead
  of silently skipping them (all 6 arrow function tests now pass)
- Best-candidate resolution: collect candidates from all strategies and
  return highest confidence match instead of first match
- Fix graph traversal 'both' direction: correctly determine next node
  for mixed incoming/outgoing edges in BFS and DFS

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:55:00 -06:00
Colby McHenryandClaude Opus 4.6 d80900f653 Port performance improvements from PR #15
- SQLite performance pragmas: synchronous=NORMAL, 64MB cache,
  memory temp store, 256MB mmap (safe with WAL mode)
- Batch insert for unresolved refs: single transaction instead of
  N individual inserts per file
- Symbol caching (warmCaches): pre-load all nodes into memory maps
  before resolution, eliminating repeated SQLite queries per ref
- Async file I/O: fs.stat/readFile in indexFile() are now non-blocking
- Denormalize filePath/language onto UnresolvedReference: avoids N
  node lookups during resolution, with schema migration v2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:45:34 -06:00
Colby McHenry d0ee6f7fc4 Enhances code extraction and project indexing
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
2026-02-09 22:18:59 -06:00
Colby McHenry cc6e7a5c89 Init 2026-01-18 16:25:00 -06:00