Commit Graph
33 Commits
Author SHA1 Message Date
Colby McHenry e848e6f22f feat: Add PHP inheritance extraction and improve method call handling
Addresses PHP's base_clause syntax for class inheritance (extends) and implements clause for interface implementation. Adds trait_declaration support and separates property_declaration into fieldTypes. Improves PHP method call extraction by handling member_call_expression and scoped_call_expression with proper receiver name processing, including $ prefix stripping and self/this/parent/static receiver filtering.
2026-04-06 22:10:23 -05:00
Colby McHenry 2d14503258 feat: Add Rust trait inheritance and impl block extraction with method receiver type support
Addresses Rust's impl block syntax where trait implementations (`impl Trait for Type`) and trait supertraits (`trait Sub: Super`) create inheritance relationships. Adds getReceiverType to extract method receiver types from impl blocks, enabling proper method-to-struct relationships and qualified name resolution. Verified against Deno codebase and moved from "Needs Verification" to completed language support.
2026-04-06 21:50:03 -05:00
Colby McHenry 5046c760cb feat: Add Swift inheritance extraction for class, struct, enum, and protocol relationships
Addresses Swift's inheritance_specifier syntax where type relationships are specified after colons (e.g. `class UploadRequest: DataRequest, Sendable`). Extracts user_type > type_identifier children from inheritance_specifier nodes as 'extends' references to properly model Swift's inheritance, protocol conformance, and struct conformance patterns in the code graph.
2026-04-06 20:52:42 -05:00
Colby McHenry c626dfa989 feat: Improve multi-term search ranking with co-occurrence boosting and compound matching
Addresses cases where multi-word queries like "search execution from request to shard" return generic single-term matches instead of highly relevant classes matching multiple terms. Applies co-occurrence boosting before truncation to prioritize nodes matching 2+ query terms, adds compound term matching to catch classes like "SearchShardsRequest" that contain multiple query terms at any position, and widens per-term accumulation pools to prevent relevant multi-term matches from being filtered out early.
2026-04-06 14:27:13 -05:00
Colby McHenry 13d3ff3613 feat: Add comprehensive evaluation framework for CodeGraph API testing
Introduces automated testing infrastructure to measure CodeGraph performance across searchNodes and findRelevantContext APIs. Includes recall/MRR scoring metrics, predefined test cases for symbol lookup and context exploration, and JSON report generation. Enhances context building with acronym extraction, definition prefix matching, and improved FTS filtering to exclude imports by default.
2026-04-06 13:24:27 -05:00
Colby McHenry e4908e1270 feat: Add database schema v3 with optimized node lookups and improved error handling
Adds expression index on lower(name) for memory-efficient case-insensitive searches, replacing in-memory caches that caused OOM on large codebases. Includes batched reference resolution, enhanced error reporting with detailed breakdown by error type, and improved CLI progress display for scanning phases.
2026-04-04 10:19:08 -05:00
Colby McHenry 4af51f565b feat: Extract type references from annotations and improve symbol query matching
Adds type annotation parsing to create references edges for parameter types, return types, and variable type annotations in TypeScript and other typed languages. Expands symbol extraction from queries to capture lowercase identifiers and filters out more common English words. Removes obsolete search utility tests.
2026-04-03 16:15:44 -05:00
Colby McHenryandClaude Opus 4.6 7e6914ddff feat: Always enable embeddings, remove enableEmbeddings config option
Testing showed semantic search produces significantly better results for
natural language queries that Claude writes. FTS alone often ranks
properties above their parent classes and misses conceptual matches.
Embeddings are now always on — the vector manager is created eagerly,
with model download and embedding generation still happening lazily.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:03:41 -05:00
Colby McHenryandClaude Opus 4.6 584bd94ecc fix: Prevent false cross-module edges in name-based resolution
Name matching was creating false `calls` edges between unrelated modules
in monorepos because `findBestMatch()` had no concept of directory
proximity — functions with common names (e.g. `navigate`) in different
apps scored identically and resolved to whichever came first.

Adds path proximity scoring (shared directory segments) so same-module
candidates strongly win over cross-boundary ones, and lowers confidence
for distant matches so import-based resolution takes precedence.

Closes #67

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-01 14:23:13 -05:00
Colby McHenry 15b5e56322 fix: Lazy grammar loading and quantized embeddings to prevent V8 WASM OOM
Fixes #54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone"
on large codebases because all 16 tree-sitter WASM grammar modules were compiled
upfront by V8, exhausting the WASM Zone allocator.

Changes:
- initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()),
  no longer eagerly loads all grammar files
- New loadGrammarsForLanguages() loads only grammars for languages actually present
  in the project (e.g. a Dart project loads ~2-3 grammars instead of 16)
- Orchestrator detects needed languages after file scan, before parsing begins
- Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce
  WASM memory pressure when embeddings are enabled
2026-03-18 14:21:54 -05:00
Olaf MonienandClaude Opus 4.6 77135445db Merge origin/main into delphi-support
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 20:23:54 +01:00
Colby McHenry 8346440592 Add WASM fallbacks for tree-sitter and SQLite, fix installer
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.

Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.

Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
2026-02-14 00:56:15 -06:00
Colby McHenry ce504c642a Fix performance issues. 2026-02-11 16:11:50 -06:00
Olaf MonienandClaude Opus 4.6 763e280829 fix: Fallback to filename when Pascal module name is empty
Some .dpr templates use "program;" without a name, which produces an
empty moduleName in the AST. Fall back to the filename (without extension)
to prevent nodes with empty names that cause downstream FK constraint errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:50:58 +01:00
Olaf MonienandClaude Opus 4.6 425bfceea5 feat: Add Pascal/Delphi language support with DFM/FMX extraction
Implements tree-sitter-based extraction for Pascal/Delphi source files
(.pas, .dpr, .dpk, .lpr) and a custom regex-based DfmExtractor for
form files (.dfm, .fmx). Covers classes, records, interfaces, methods,
properties, fields, enums, constants, type aliases, uses-imports,
visibility sections, inheritance, call extraction, and DFM component
hierarchies with event handler references.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:58:13 +01:00
Colby McHenry 98034cc76e Resolve merge conflicts with main
- extraction/index.ts: use picomatch with static import (replacing
  dynamic require) and keep normalizePath for other call sites
- utils.ts: keep normalizePath from main, take PR's PID-based FileLock
2026-02-10 16:21:23 -06:00
Colby MchenryandGitHub e6531c50c0 Merge pull request #26 from MO2k4/feat/search-query-utils
feat: Search query utilities + multi-signal scoring
2026-02-10 16:14:12 -06:00
Colby McHenry a15ad69288 Remove unused detectApiIntent and inferRouteDirectories
Both functions have zero callers — dead code on arrival. Remove them
and their tests (9 tests) to keep the module focused on what's
actually used: search term extraction, path relevance scoring, and
kind bonuses.
2026-02-10 16:05:46 -06:00
Colby McHenry 36af284e10 Remove redundant code and deduplicate indexFile
- Remove extractFunctionVariable() and its dispatch (already handled by extractVariable)
- Remove dead getGrammar() export (zero callers)
- Deduplicate indexFile by delegating to indexFileWithContent
- Remove redundant arrow function variable extraction tests (covered by existing suite)
2026-02-10 15:55:43 -06:00
Colby McHenry ce220a573c Resolve merge conflict with main (gitignore + symlink rename)
Keeps the PR's visitedDirs rename and main's gitIgnoredDirs addition.
2026-02-10 15:44:56 -06:00
Martin Oehlert 0f2eda8da3 feat: file nodes, arrow function extraction, parallel I/O
- Create file-kind nodes for each parsed source file
- Add isInsideClassLikeNode() for method vs function detection
- Extract arrow functions and function expressions from variable declarators
- Batch file I/O with FILE_IO_BATCH_SIZE=10 using Promise.all
- Add symlink cycle detection with visitedDirs Set in scanDirectory
- Add lazy grammar loading with exported getGrammar() function
- Add indexFileWithContent() for pre-read content processing
- Add tests for file nodes and arrow function extraction
2026-02-10 11:47:07 +01:00
Martin Oehlert c80378d73c feat: search query utilities and multi-signal scoring
- Add src/search/query-utils.ts with extractSearchTerms, scorePathRelevance,
  kindBonus, detectApiIntent, inferRouteDirectories
- Add multi-signal scoring to searchNodes (kind bonus + path relevance)
- Improve FTS query sanitization (strip :^ chars, filter boolean operators)
- Add comprehensive search tests
2026-02-10 11:40:41 +01:00
Martin Oehlert 399d78b938 security: path validation, ReDoS prevention, picomatch, PID-based file lock
- Add validateProjectPath() to reject sensitive system directories
- Add isPathWithinRoot/isPathWithinRootReal for symlink-aware path checks
- Replace hand-rolled glob-to-regex with picomatch to prevent ReDoS
- Add isSafeRegex() to reject custom patterns with nested quantifiers
- Replace FileLock with PID-tracking version that detects stale locks
- Add symlink detection in removeDirectory/listDirectoryContents
- Add subdirectory name validation in ensureSubdirectory
- Add atomicWriteFileSync and corrupted file backup in config-writer
- Add MCP input validation (validateString) for all tool handlers
- Fix CLAUDE.md section replacement to handle ### subsections correctly
2026-02-10 11:22:04 +01:00
Colby McHenryandClaude Opus 4.6 62a6cf38fe Fix .gitignore support and Windows path separator bug in scanner
Normalize paths to forward slashes in matchesGlob() and scanDirectory()
so glob exclude patterns work on Windows. Add getGitIgnoredDirectories()
using git ls-files to skip .gitignore'd directories during indexing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 04:08:10 -06:00
Colby McHenryandClaude Opus 4.6 c8c7785626 Add tests for PR #19 improvements (Phases 3-5)
Covers arrow function extraction, best-candidate resolution, graph
traversal direction fix, MCP symbol disambiguation, output truncation,
CLI uninit command, and more. Tests requiring better-sqlite3 native
bindings are conditionally skipped.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 00:19:48 -06:00
Colby McHenryandClaude Opus 4.6 41c49ce83d Add security hardening test suite (28 tests)
Tests cover all security utilities introduced in the previous commit:
- validatePathWithinRoot: path traversal prevention (7 tests)
- safeJsonParse: corrupted JSON fallback handling (6 tests)
- clamp: input range clamping (5 tests)
- FileLock: cross-process file locking (7 tests)
- Atomic config writes: temp file + rename pattern (3 tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:21:36 -06:00
Colby McHenryandClaude Opus 4.6 38fac1ff28 Merge PR #18: Fix arrow function/export indexing + type alias extraction
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 22:36:26 -06:00
Colby McHenry d0ee6f7fc4 Enhances code extraction and project indexing
Adds support for Dart and Liquid languages with tree-sitter parsing.
Improves accuracy of code symbol extraction for existing languages.
Indexes project files to enhance code navigation features.
Migrates build system to facilitate code contributions.
Removes git hook functionality.
Integrates Sentry for error tracking and reporting.
Enhances project initialization and configuration loading.
2026-02-09 22:18:59 -06:00
Tanner Balluff ea9d0bf549 Add type alias and exported variable extraction
Extend extraction to index two additional categories of symbols
that were previously invisible:

1. Type aliases (e.g. `export type X = ...` in TypeScript,
   `type X` in Go, `type X = ...` in Rust, `typealias X` in Swift,
   `type_alias` in Kotlin). Adds `typeAliasTypes` to the
   LanguageExtractor interface with values for all 13 languages.

2. Exported variable declarations that aren't functions, including:
   - Zustand stores: `export const useX = create(...)`
   - XState machines: `export const xMachine = createMachine(...)`
   - Zod schemas: `export const schema = z.object(...)`
   - Config objects: `export const config = { ... }`
   - Constants: `export const MAX = 3`
   - Arrays: `export const NAMES = [...] as const`

   The extractExportedVariables() method is called when visiting
   export_statement nodes. It skips variable_declarator values that
   are already handled by functionTypes (arrow_function,
   function_expression) to avoid duplicate extraction.

Adds 11 new test cases (59 total extraction tests, 215 total).

Tested on production monorepo: nodes increased from 958 to 1,172
(+22%), with 109 new variable nodes and 105 new type_alias nodes.
Only 4 files remain at 0 nodes — all are re-export barrels or
ambient declaration files with no extractable symbols.
2026-02-09 14:48:09 -05:00
Tanner Balluff b6d5346c3c Fix arrow function and function expression export indexing
Arrow functions and function expressions assigned to variables
(e.g. `export const useAuth = () => { ... }`) were not being indexed
because the arrow_function AST node has no `name` field — the name
lives on the parent variable_declarator node.

Additionally, `isExported()` for TypeScript and JavaScript extractors
only checked 10 characters back from the node's start position, which
missed `export` for deeply nested nodes like arrow functions inside
variable declarations inside export statements.

Changes:
- extractFunction(): When an arrow_function or function_expression
  resolves to '<anonymous>', look up the parent variable_declarator
  for the name before skipping.
- isExported() (TS + JS): Walk the parent chain to find an
  export_statement ancestor instead of substring matching.
- Add 6 test cases covering arrow function exports, function
  expression exports, non-exported arrow functions, anonymous
  arrow functions, multiple exports, and JavaScript files.

Tested on a real monorepo (238 files): node count increased from
779 to 958 (+23%), with 94 new nodes in packages/ that previously
had 0 coverage.
2026-02-09 14:35:15 -05:00
Colby McHenryandClaude Opus 4.5 2fffcf4736 Add embeddings initialization to evaluation tests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 18:56:10 -06:00
Colby McHenryandClaude Opus 4.5 6b672f9152 Add evaluation framework and fix call graph extraction
- Add evaluation test suite with TypeScript and Python fixtures
- Fix MCP server to defer CodeGraph init until rootUri received
- Fix call edge extraction by calling resolveReferences() after indexAll/sync
- Fix glob matching for root-level files (e.g., **/*.py now matches auth.py)
- Fix duplicate node extraction for methods inside classes
- Update context tests to use buildContext for semantic search + graph traversal
- Export unused formatter functions to fix build

Evaluation results:
- TypeScript: 96% precision, 79% recall, 85% F1
- Python: 99% precision, 80% recall, 85% F1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 18:48:22 -06:00
Colby McHenry cc6e7a5c89 Init 2026-01-18 16:25:00 -06:00