Commit Graph
208 Commits
Author SHA1 Message Date
Olaf MonienandClaude Opus 4.6 a86ddf6e6a fix: Resolve Pascal extraction edge cases
- Fix defProc method lookup collisions when multiple classes share
  method names (e.g. Create) by indexing qualified forms
- Remove overly broad 'Id' prefix filter that swallowed user-defined
  symbols like Identifier or IdleTimer
- Use case-insensitive extension matching for DFM/FMX routing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 01:29:49 +01:00
Olaf MonienandClaude Opus 4.6 77135445db Merge origin/main into delphi-support
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 20:23:54 +01:00
Rafael Vescovi 267c9e1379 fix: load WASM grammars sequentially to avoid Node 20+ race condition
web-tree-sitter has a known race condition when loading multiple WASM
grammars concurrently on Node.js 19+ (V8 10.8+). External scanner
symbols from one grammar can overwrite another's GOT entries, causing
"bad export type" errors for TypeScript, TSX, and other languages.

Replace Promise.allSettled(entries.map(...)) with a sequential for...of
loop so each grammar fully initializes before the next one starts.

Ref: https://github.com/tree-sitter/tree-sitter/issues/2338
2026-02-17 10:33:39 -06:00
Colby McHenry a9148e674e Fix git issue 2026-02-14 02:14:48 -06:00
Colby McHenry 8346440592 Add WASM fallbacks for tree-sitter and SQLite, fix installer
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.

Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.

Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
2026-02-14 00:56:15 -06:00
Olaf MonienandClaude Opus 4.6 e377a957b9 docs: Add performance improvements and testing instructions to DELPHI-SUPPORT.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 00:39:06 +01:00
Olaf MonienandClaude Opus 4.6 ab9d7188c1 perf: Cache import mappings and index fuzzy matches for resolution
Two major optimizations for the ref resolution phase:

1. Cache extractImportMappings() results per file path — previously
   re-read and re-parsed the source file for every single ref from
   that file (e.g. 100 refs from one file = 100 identical file reads)

2. Replace linear scan in matchFuzzy() with a lazily-built
   case-insensitive Map index — O(1) lookup instead of iterating
   all function/method/class nodes for every unresolved ref.
   Also drop low-value prefix matching (confidence 0.3).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 19:29:18 +01:00
Olaf MonienandClaude Opus 4.6 44ea6af043 fix: Remove _libs/** and libs/** from global exclude defaults
These are project-specific directories and should be configured per
project in .codegraph/config.json, not hardcoded globally. Delphi-
specific excludes (__history, __recovery, *.dcu) remain as global
defaults since they are analogous to __pycache__ for Python.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 11:16:41 +01:00
Olaf MonienandClaude Opus 4.6 5a279d3132 perf: Use indexed lookup in extractPascalDefProc instead of linear scan
Replace this.nodes.find() with a lazily-built Map<name, nodeId> for
matching implementation bodies to their declarations. Reduces per-file
complexity from O(methods × nodes) to O(nodes) for index build + O(1)
per lookup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 10:09:51 +01:00
Olaf MonienandClaude Opus 4.6 f414ed7e3e perf: Add Delphi-specific exclude patterns to defaults
Add _libs/**, libs/**, __history/**, __recovery/**, and *.dcu to
default exclude list. Delphi projects store external dependencies
in _libs/ or libs/ (often as Git submodules) and these should not
be indexed — same as node_modules for JS projects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:40:56 +01:00
Olaf MonienandClaude Opus 4.6 f5d9a0efe5 perf: Speed up ref resolution for large Pascal codebases
Two optimizations that eliminate the O(n²) bottleneck:

1. Add Pascal built-in filtering to isBuiltInOrExternal() — skips
   resolution attempts for System.*, Vcl.*, Fmx.* and ~60 common
   RTL identifiers (WriteLn, Create, Free, TObject, etc.)

2. Cache getNodesByKind() results during warm cache phase — matchFuzzy()
   was calling the database 3 times per unresolved reference instead
   of using the in-memory cache that warmCaches() already builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:28:37 +01:00
Colby McHenry 429359c25f version bump 2026-02-11 16:12:45 -06:00
Colby McHenry ce504c642a Fix performance issues. 2026-02-11 16:11:50 -06:00
Colby McHenry a7fc5853a2 Exit child processes on windows 2026-02-11 15:15:42 -06:00
Olaf MonienandClaude Opus 4.6 763e280829 fix: Fallback to filename when Pascal module name is empty
Some .dpr templates use "program;" without a name, which produces an
empty moduleName in the AST. Fall back to the filename (without extension)
to prevent nodes with empty names that cause downstream FK constraint errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:50:58 +01:00
Olaf MonienandClaude Opus 4.6 482cbee0d0 docs: Add Pascal/Delphi to supported languages in README and CLAUDE.md
Update language count (17+ → 18+), add Pascal/Delphi row to the
supported languages table, and include Pascal in CLAUDE.md language list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:06:28 +01:00
Olaf MonienandClaude Opus 4.6 9f415002e4 docs: Replace planning docs with DELPHI-SUPPORT.md for PR
Remove Delphi/ planning folder (concept docs, fixtures, resolution notes)
that served as implementation guidance. Add DELPHI-SUPPORT.md describing
the integration for stakeholder review: motivation, feature matrix,
architecture decisions, and test coverage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:04:55 +01:00
Olaf MonienandClaude Opus 4.6 2442c58932 fix: Move tree-sitter-pascal to optionalDependencies with pinned commit
Align with project convention: all tree-sitter grammars are optional.
Pin to commit 042119e for reproducible builds (npm v0.0.1 is outdated
and incompatible with Node.js v24 due to NAN bindings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:00:14 +01:00
Colby McHenry eee081e2a7 detached process 2026-02-11 02:58:59 -06:00
Olaf MonienandClaude Opus 4.6 425bfceea5 feat: Add Pascal/Delphi language support with DFM/FMX extraction
Implements tree-sitter-based extraction for Pascal/Delphi source files
(.pas, .dpr, .dpk, .lpr) and a custom regex-based DfmExtractor for
form files (.dfm, .fmx). Covers classes, records, interfaces, methods,
properties, fields, enums, constants, type aliases, uses-imports,
visibility sections, inheritance, call extraction, and DFM component
hierarchies with event handler references.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:58:13 +01:00
Colby McHenry 09a8d24bd8 version bump 2026-02-10 18:18:51 -06:00
Colby McHenry 4c7827675a Auto stash before merge of "main" and "origin/main" 2026-02-10 18:16:06 -06:00
Colby MchenryandGitHub 7239a6f7d3 Merge pull request #29 from colbymchenry/optimize-reference-resolution
Optimize reference resolution with in-memory caches
2026-02-10 18:15:36 -06:00
Colby McHenry acd9713632 Optimize reference resolution with in-memory caches
The resolving refs phase stalled on large projects (3400+ files, 38k+ nodes)
because matchFuzzy loaded ALL functions/methods/classes per ref, import
mappings were re-extracted per ref, and fileExists hit disk every call.

Add kindCache, lowerNameCache, importMappingCache, and knownFiles set to
warmCaches(). Rewrite matchFuzzy to use O(1) lowercase index lookup instead
of 3x getNodesByKind scans. Cache import mappings per file. Pre-build file
existence set from the index for O(1) fileExists checks.
2026-02-10 18:14:26 -06:00
Olaf Monien 8dfc99572f Delphi-Konzept: DFM/FMX Support hinzugefügt
- Neues Dokument 07-DFM-FMX-Support.md: Format-Referenz, Extraktions-Strategie, DfmExtractor-Konzept
- Neues Fixture MainForm.dfm: Verschachtelte Komponenten, Event-Handler (FormCreate, btnLoginClick, etc.)
- Integration Guide erweitert: DfmExtractor-Implementierung (analog LiquidExtractor), Coding-Style-Hinweise
- NodeKind-Mapping erweitert: DFM-Komponenten als component, Event-Handler als references, Verschachtelung als contains
- Checklist: DFM/FMX von 'Zukunft' auf konkreten Plan (Phase 1b) hochgestuft
- README aktualisiert: DFM/FMX-Support in Ziel, Inhalt und Grammar-Abschnitt aufgenommen
2026-02-11 01:06:16 +01:00
Olaf Monien dcca22c78f Delphi-Konzept: Reale tree-sitter-pascal AST-Typen, Integration Guide, erweiterte Fixtures
- Alle Docs auf verifizierte AST-Knotentypen aktualisiert (declClass, declIntf, declProc, etc.)
- Scaffold/ entfernt und durch konkreten Integration Guide ersetzt (06-Integration-Guide.md)
- Neue Docs: AST-Referenz (03), NodeKind-Mapping (05), Integration Guide (06)
- Checklist um bekannte Einschränkungen erweitert (with-Statements, .dfm/.fmx, Generics)
- UAuth.pas erweitert: Sichtbarkeitsbereiche, Felder, Properties, Constructor/Destructor
- UTypes.pas neu: Enums, Records, Type-Aliase, Konstanten, Class Methods, verschachtelte Typen
- UnitResolution.md mit konkreten AST-Strukturen aktualisiert
- README.md an neue Architektur-Beschreibung angepasst
2026-02-11 00:17:29 +01:00
Olaf Monien 645fd71e45 Delphi concept 2026-02-10 23:55:42 +01:00
Colby McHenry df937ceca1 Add site-packages and dist-packages to default exclude patterns
Fixes #28 - Python site-packages directories (e.g.
audio_tools/python/Lib/site-packages/) were not excluded by default,
causing massive index bloat and FOREIGN KEY failures when indexing
large libraries like tensorflow. The FK crash itself was already fixed
via INSERT OR IGNORE, but excluding these directories prevents the
bloat in the first place.
2026-02-10 16:47:13 -06:00
Colby MchenryandGitHub adaff67219 Merge pull request #25 from MO2k4/feat/db-performance-schema-v2
feat: Schema v2 migration + database performance
2026-02-10 16:36:21 -06:00
Colby McHenry 1c023c41cc Remove unused batch methods, dynamic stmt cache, and intent field
- Remove getNodesByIds, getNodesByKinds (zero callers)
- Remove getFileHashMap, getFileSyncMap (zero callers)
- Remove getDynamicStmt cache (only used by removed batch methods)
- Revert getStaleFiles to simple implementation (zero callers, no need to optimize)
- Remove intent field from SearchOptions (never read in search logic)
2026-02-10 16:34:13 -06:00
Colby MchenryandGitHub 04e1f3e122 Merge pull request #24 from MO2k4/security/path-validation-redos
security: Path validation, ReDoS prevention, picomatch
2026-02-10 16:22:39 -06:00
Colby McHenry 98034cc76e Resolve merge conflicts with main
- extraction/index.ts: use picomatch with static import (replacing
  dynamic require) and keep normalizePath for other call sites
- utils.ts: keep normalizePath from main, take PR's PID-based FileLock
2026-02-10 16:21:23 -06:00
Colby MchenryandGitHub e6531c50c0 Merge pull request #26 from MO2k4/feat/search-query-utils
feat: Search query utilities + multi-signal scoring
2026-02-10 16:14:12 -06:00
Colby McHenry a15ad69288 Remove unused detectApiIntent and inferRouteDirectories
Both functions have zero callers — dead code on arrival. Remove them
and their tests (9 tests) to keep the module focused on what's
actually used: search term extraction, path relevance scoring, and
kind bonuses.
2026-02-10 16:05:46 -06:00
Colby MchenryandGitHub e2a95ee9ab Merge pull request #27 from MO2k4/feat/extraction-quality
feat: File nodes, arrow functions, parallel I/O
2026-02-10 16:00:23 -06:00
Colby McHenry 36af284e10 Remove redundant code and deduplicate indexFile
- Remove extractFunctionVariable() and its dispatch (already handled by extractVariable)
- Remove dead getGrammar() export (zero callers)
- Deduplicate indexFile by delegating to indexFileWithContent
- Remove redundant arrow function variable extraction tests (covered by existing suite)
2026-02-10 15:55:43 -06:00
Colby McHenry ce220a573c Resolve merge conflict with main (gitignore + symlink rename)
Keeps the PR's visitedDirs rename and main's gitIgnoredDirs addition.
2026-02-10 15:44:56 -06:00
Martin Oehlert 0f2eda8da3 feat: file nodes, arrow function extraction, parallel I/O
- Create file-kind nodes for each parsed source file
- Add isInsideClassLikeNode() for method vs function detection
- Extract arrow functions and function expressions from variable declarators
- Batch file I/O with FILE_IO_BATCH_SIZE=10 using Promise.all
- Add symlink cycle detection with visitedDirs Set in scanDirectory
- Add lazy grammar loading with exported getGrammar() function
- Add indexFileWithContent() for pre-read content processing
- Add tests for file nodes and arrow function extraction
2026-02-10 11:47:07 +01:00
Martin Oehlert c80378d73c feat: search query utilities and multi-signal scoring
- Add src/search/query-utils.ts with extractSearchTerms, scorePathRelevance,
  kindBonus, detectApiIntent, inferRouteDirectories
- Add multi-signal scoring to searchNodes (kind bonus + path relevance)
- Improve FTS query sanitization (strip :^ chars, filter boolean operators)
- Add comprehensive search tests
2026-02-10 11:40:41 +01:00
Martin Oehlert 53beb1bd2f feat: schema v2 migration, database performance, batch query methods
- Add provenance column on edges for tracking how edges were created
- Add project_metadata table for version/provenance tracking
- Make unresolved_refs file_path/language NOT NULL with defaults
- Add composite indexes for unresolved_refs and edges.provenance
- Update v2 migration to handle all new schema additions
- Record schema version on initialize to prevent re-migration
- Add dynamic prepared statement cache (getDynamicStmt) for varying SQL
- Add batch methods: getNodesByIds, getNodesByKinds, getFileHashMap, getFileSyncMap
- Add project metadata methods: getMetadata, setMetadata, getAllMetadata
- Optimize getStats to use single aggregate query
- Optimize getStaleFiles to use temporary table JOIN
- Add provenance parameter to getOutgoingEdges
- Add intent field to SearchOptions type
2026-02-10 11:31:05 +01:00
Martin Oehlert 399d78b938 security: path validation, ReDoS prevention, picomatch, PID-based file lock
- Add validateProjectPath() to reject sensitive system directories
- Add isPathWithinRoot/isPathWithinRootReal for symlink-aware path checks
- Replace hand-rolled glob-to-regex with picomatch to prevent ReDoS
- Add isSafeRegex() to reject custom patterns with nested quantifiers
- Replace FileLock with PID-tracking version that detects stale locks
- Add symlink detection in removeDirectory/listDirectoryContents
- Add subdirectory name validation in ensureSubdirectory
- Add atomicWriteFileSync and corrupted file backup in config-writer
- Add MCP input validation (validateString) for all tool handlers
- Fix CLAUDE.md section replacement to handle ### subsections correctly
2026-02-10 11:22:04 +01:00
Colby McHenryandClaude Opus 4.6 62a6cf38fe Fix .gitignore support and Windows path separator bug in scanner
Normalize paths to forward slashes in matchesGlob() and scanDirectory()
so glob exclude patterns work on Windows. Add getGitIgnoredDirectories()
using git ls-files to skip .gitignore'd directories during indexing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 04:08:10 -06:00
Colby McHenry 4825661e02 upgrade 2026-02-10 03:49:06 -06:00
Colby McHenryandClaude Opus 4.6 75e9bfca04 Downgrade tree-sitter to 0.21.1 to eliminate all peer dependency warnings
All grammar packages now have compatible peer deps with tree-sitter 0.21.x,
resulting in zero warnings during npm install. Downgraded grammars:
c 0.24.1→0.23.2, php 0.24.2→0.23.11, python 0.23.6→0.23.4,
rust 0.24.0→0.23.1, swift 0.7.1→0.6.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 03:07:39 -06:00
Colby McHenryandClaude Opus 4.6 a4c1d32160 Upgrade tree-sitter-c, php, and rust to versions compatible with tree-sitter 0.22.4
Reduces peer dependency warnings during install by upgrading grammars that
have newer versions accepting ^0.22.x: c 0.23.4→0.24.1, php 0.23.11→0.24.2,
rust 0.23.2→0.24.0. Python 0.23.6 and swift 0.7.1 already compatible.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 03:01:59 -06:00
Colby McHenryandClaude Opus 4.6 9a122b041b Move tree-sitter grammars to optionalDependencies for Windows compatibility
tree-sitter-kotlin doesn't ship prebuilt binaries for win32-x64, causing
npm install to fail on Windows without Visual Studio. All grammars are now
optional since the runtime already handles missing parsers gracefully.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 02:51:32 -06:00
Colby McHenryandClaude Opus 4.6 7fe2973e73 Move platform badges to separate row in README
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 02:29:41 -06:00
Colby McHenry 834ae663e9 readme update 2026-02-10 02:28:26 -06:00
Colby McHenry f5746981bd Merge branch 'svelte-support' 2026-02-10 02:26:54 -06:00
Colby McHenry 2ad03d2f7d updates 2026-02-10 02:26:47 -06:00