Commit Graph
521 Commits
Author SHA1 Message Date
Colby McHenry 15b5e56322 fix: Lazy grammar loading and quantized embeddings to prevent V8 WASM OOM
Fixes #54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone"
on large codebases because all 16 tree-sitter WASM grammar modules were compiled
upfront by V8, exhausting the WASM Zone allocator.

Changes:
- initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()),
  no longer eagerly loads all grammar files
- New loadGrammarsForLanguages() loads only grammars for languages actually present
  in the project (e.g. a Dart project loads ~2-3 grammars instead of 16)
- Orchestrator detects needed languages after file scan, before parsing begins
- Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce
  WASM memory pressure when embeddings are enabled
2026-03-18 14:21:54 -05:00
Mark HandGitHub eb25a166d1 exclude .pio/ folder for Platform.io IoT libs 2026-02-24 19:11:10 -08:00
Colby MchenryandGitHub 16db37566d Merge pull request #46 from colbymchenry/fix/serve-stdout-mcp-protocol
fix: Write serve banner to stderr, not stdout
2026-02-20 11:27:31 -06:00
Colby McHenry 7442b27002 fix: Write serve command banner to stderr instead of stdout
The serve command without --mcp prints a help banner. Writing this to
stdout breaks MCP stdio clients (like Cursor) that expect only JSON-RPC
on stdout. Move all banner output to stderr so stdout stays clean.

Fixes #43
2026-02-19 16:02:52 -06:00
Colby MchenryandGitHub 7f516b9308 Merge pull request #45 from colbymchenry/fix/silent-failed-install
fix: Stop silent install failures — always run npm install -g, add preuninstall cleanup
2026-02-19 15:58:51 -06:00
Colby McHenry dff98a638c Bumps version to 0.6.2
Bumps version to 0.6.2 in both package.json and lockfile to align release metadata and prevent silent install failures on fresh installs.
Relates to silent install fix
2026-02-19 15:57:01 -06:00
Colby McHenry a46d6b7487 fix: Always run npm install -g, skip command -v check
command -v codegraph is unreliable inside npx because npx puts a
temporary binary in PATH. The check always passes, so the global
install is always skipped — which is the root cause of #37 and #38.

Fix: remove the check entirely, always run npm install -g.
2026-02-19 15:51:03 -06:00
Colby McHenry 67f60b9b2f fix: Use bare codegraph everywhere, add preuninstall cleanup
- Revert configs (MCP, hooks) to use bare `codegraph` command
- Remove all npx references from configs and messaging
- Add preuninstall script that runs on `npm uninstall -g` to clean up
  MCP server, permissions, hooks, and CLAUDE.md section
- Show uninstall instructions in post-install next steps
2026-02-19 15:46:34 -06:00
Colby McHenry 88e1f2df7f fix: Bring back global install attempt with loud failure messaging
Global install is still attempted for bare `codegraph` convenience,
but now verifies the command is actually in PATH after install. If it
fails, users get clear actionable messages instead of silent swallowing.

Configs (MCP server, hooks) always use npx regardless — those never
break even if global install fails.
2026-02-19 15:38:59 -06:00
Colby McHenry 675aab386a fix: Always use npx — stop silent global install failures
Remove the npm install -g attempt from the installer that silently
fails on many systems (permissions, PATH, node version managers).
All configs (MCP server, hooks, next-steps) now always use
npx @colbymchenry/codegraph. Global install offered as an optional tip.

Fixes #37, #38
2026-02-19 15:28:56 -06:00
Colby MchenryandGitHub 3be779f6f6 Merge pull request #41 from omonien/delphi-support
feat: Add Pascal/Delphi support (Tree-sitter & DFM extraction)
2026-02-19 15:00:05 -06:00
Colby MchenryandGitHub 544d193086 Merge branch 'main' into delphi-support 2026-02-19 14:59:53 -06:00
Colby MchenryandGitHub 5d699ab2c5 Merge pull request #40 from ravescovi/fix/sequential-grammar-loading
fix: load WASM grammars sequentially to avoid Node 20+ race condition
2026-02-19 14:52:05 -06:00
Olaf MonienandClaude Opus 4.6 a86ddf6e6a fix: Resolve Pascal extraction edge cases
- Fix defProc method lookup collisions when multiple classes share
  method names (e.g. Create) by indexing qualified forms
- Remove overly broad 'Id' prefix filter that swallowed user-defined
  symbols like Identifier or IdleTimer
- Use case-insensitive extension matching for DFM/FMX routing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 01:29:49 +01:00
Olaf MonienandClaude Opus 4.6 77135445db Merge origin/main into delphi-support
Integrate main branch changes (WASM grammar architecture, centralized
resolution caches, SQLite adapter) with delphi-support branch. Pascal
grammar is now built as WASM and shipped in src/extraction/wasm/ for
consistency with the WASM-based grammar loading approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-17 20:23:54 +01:00
Rafael Vescovi 267c9e1379 fix: load WASM grammars sequentially to avoid Node 20+ race condition
web-tree-sitter has a known race condition when loading multiple WASM
grammars concurrently on Node.js 19+ (V8 10.8+). External scanner
symbols from one grammar can overwrite another's GOT entries, causing
"bad export type" errors for TypeScript, TSX, and other languages.

Replace Promise.allSettled(entries.map(...)) with a sequential for...of
loop so each grammar fully initializes before the next one starts.

Ref: https://github.com/tree-sitter/tree-sitter/issues/2338
2026-02-17 10:33:39 -06:00
Colby McHenry a9148e674e Fix git issue 2026-02-14 02:14:48 -06:00
Colby McHenry 8346440592 Add WASM fallbacks for tree-sitter and SQLite, fix installer
Replace native tree-sitter with web-tree-sitter + tree-sitter-wasms for
universal cross-platform support. Add node-sqlite3-wasm as a fallback
when better-sqlite3 native bindings aren't available. Move better-sqlite3
and sqlite-vss to optionalDependencies so installs never fail.

Fix installer to use npx fallback when global npm install fails, so MCP
config, hooks, and quick-start instructions all work without the bare
codegraph command in PATH.

Fix tests: update schema version expectation, fix db test paths and
method names, extract MAX_OUTPUT_LENGTH as module constant, normalize
Windows path separators in import resolver.
2026-02-14 00:56:15 -06:00
Olaf MonienandClaude Opus 4.6 e377a957b9 docs: Add performance improvements and testing instructions to DELPHI-SUPPORT.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 00:39:06 +01:00
Olaf MonienandClaude Opus 4.6 ab9d7188c1 perf: Cache import mappings and index fuzzy matches for resolution
Two major optimizations for the ref resolution phase:

1. Cache extractImportMappings() results per file path — previously
   re-read and re-parsed the source file for every single ref from
   that file (e.g. 100 refs from one file = 100 identical file reads)

2. Replace linear scan in matchFuzzy() with a lazily-built
   case-insensitive Map index — O(1) lookup instead of iterating
   all function/method/class nodes for every unresolved ref.
   Also drop low-value prefix matching (confidence 0.3).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 19:29:18 +01:00
Olaf MonienandClaude Opus 4.6 44ea6af043 fix: Remove _libs/** and libs/** from global exclude defaults
These are project-specific directories and should be configured per
project in .codegraph/config.json, not hardcoded globally. Delphi-
specific excludes (__history, __recovery, *.dcu) remain as global
defaults since they are analogous to __pycache__ for Python.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 11:16:41 +01:00
Olaf MonienandClaude Opus 4.6 5a279d3132 perf: Use indexed lookup in extractPascalDefProc instead of linear scan
Replace this.nodes.find() with a lazily-built Map<name, nodeId> for
matching implementation bodies to their declarations. Reduces per-file
complexity from O(methods × nodes) to O(nodes) for index build + O(1)
per lookup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 10:09:51 +01:00
Olaf MonienandClaude Opus 4.6 f414ed7e3e perf: Add Delphi-specific exclude patterns to defaults
Add _libs/**, libs/**, __history/**, __recovery/**, and *.dcu to
default exclude list. Delphi projects store external dependencies
in _libs/ or libs/ (often as Git submodules) and these should not
be indexed — same as node_modules for JS projects.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:40:56 +01:00
Olaf MonienandClaude Opus 4.6 f5d9a0efe5 perf: Speed up ref resolution for large Pascal codebases
Two optimizations that eliminate the O(n²) bottleneck:

1. Add Pascal built-in filtering to isBuiltInOrExternal() — skips
   resolution attempts for System.*, Vcl.*, Fmx.* and ~60 common
   RTL identifiers (WriteLn, Create, Free, TObject, etc.)

2. Cache getNodesByKind() results during warm cache phase — matchFuzzy()
   was calling the database 3 times per unresolved reference instead
   of using the in-memory cache that warmCaches() already builds.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 00:28:37 +01:00
Colby McHenry 429359c25f version bump 2026-02-11 16:12:45 -06:00
Colby McHenry ce504c642a Fix performance issues. 2026-02-11 16:11:50 -06:00
Colby McHenry a7fc5853a2 Exit child processes on windows 2026-02-11 15:15:42 -06:00
Olaf MonienandClaude Opus 4.6 763e280829 fix: Fallback to filename when Pascal module name is empty
Some .dpr templates use "program;" without a name, which produces an
empty moduleName in the AST. Fall back to the filename (without extension)
to prevent nodes with empty names that cause downstream FK constraint errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:50:58 +01:00
Olaf MonienandClaude Opus 4.6 482cbee0d0 docs: Add Pascal/Delphi to supported languages in README and CLAUDE.md
Update language count (17+ → 18+), add Pascal/Delphi row to the
supported languages table, and include Pascal in CLAUDE.md language list.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:06:28 +01:00
Olaf MonienandClaude Opus 4.6 9f415002e4 docs: Replace planning docs with DELPHI-SUPPORT.md for PR
Remove Delphi/ planning folder (concept docs, fixtures, resolution notes)
that served as implementation guidance. Add DELPHI-SUPPORT.md describing
the integration for stakeholder review: motivation, feature matrix,
architecture decisions, and test coverage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:04:55 +01:00
Olaf MonienandClaude Opus 4.6 2442c58932 fix: Move tree-sitter-pascal to optionalDependencies with pinned commit
Align with project convention: all tree-sitter grammars are optional.
Pin to commit 042119e for reproducible builds (npm v0.0.1 is outdated
and incompatible with Node.js v24 due to NAN bindings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:00:14 +01:00
Colby McHenry eee081e2a7 detached process 2026-02-11 02:58:59 -06:00
Olaf MonienandClaude Opus 4.6 425bfceea5 feat: Add Pascal/Delphi language support with DFM/FMX extraction
Implements tree-sitter-based extraction for Pascal/Delphi source files
(.pas, .dpr, .dpk, .lpr) and a custom regex-based DfmExtractor for
form files (.dfm, .fmx). Covers classes, records, interfaces, methods,
properties, fields, enums, constants, type aliases, uses-imports,
visibility sections, inheritance, call extraction, and DFM component
hierarchies with event handler references.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:58:13 +01:00
Colby McHenry 09a8d24bd8 version bump 2026-02-10 18:18:51 -06:00
Colby McHenry 4c7827675a Auto stash before merge of "main" and "origin/main" 2026-02-10 18:16:06 -06:00
Colby MchenryandGitHub 7239a6f7d3 Merge pull request #29 from colbymchenry/optimize-reference-resolution
Optimize reference resolution with in-memory caches
2026-02-10 18:15:36 -06:00
Colby McHenry acd9713632 Optimize reference resolution with in-memory caches
The resolving refs phase stalled on large projects (3400+ files, 38k+ nodes)
because matchFuzzy loaded ALL functions/methods/classes per ref, import
mappings were re-extracted per ref, and fileExists hit disk every call.

Add kindCache, lowerNameCache, importMappingCache, and knownFiles set to
warmCaches(). Rewrite matchFuzzy to use O(1) lowercase index lookup instead
of 3x getNodesByKind scans. Cache import mappings per file. Pre-build file
existence set from the index for O(1) fileExists checks.
2026-02-10 18:14:26 -06:00
Olaf Monien 8dfc99572f Delphi-Konzept: DFM/FMX Support hinzugefügt
- Neues Dokument 07-DFM-FMX-Support.md: Format-Referenz, Extraktions-Strategie, DfmExtractor-Konzept
- Neues Fixture MainForm.dfm: Verschachtelte Komponenten, Event-Handler (FormCreate, btnLoginClick, etc.)
- Integration Guide erweitert: DfmExtractor-Implementierung (analog LiquidExtractor), Coding-Style-Hinweise
- NodeKind-Mapping erweitert: DFM-Komponenten als component, Event-Handler als references, Verschachtelung als contains
- Checklist: DFM/FMX von 'Zukunft' auf konkreten Plan (Phase 1b) hochgestuft
- README aktualisiert: DFM/FMX-Support in Ziel, Inhalt und Grammar-Abschnitt aufgenommen
2026-02-11 01:06:16 +01:00
Olaf Monien dcca22c78f Delphi-Konzept: Reale tree-sitter-pascal AST-Typen, Integration Guide, erweiterte Fixtures
- Alle Docs auf verifizierte AST-Knotentypen aktualisiert (declClass, declIntf, declProc, etc.)
- Scaffold/ entfernt und durch konkreten Integration Guide ersetzt (06-Integration-Guide.md)
- Neue Docs: AST-Referenz (03), NodeKind-Mapping (05), Integration Guide (06)
- Checklist um bekannte Einschränkungen erweitert (with-Statements, .dfm/.fmx, Generics)
- UAuth.pas erweitert: Sichtbarkeitsbereiche, Felder, Properties, Constructor/Destructor
- UTypes.pas neu: Enums, Records, Type-Aliase, Konstanten, Class Methods, verschachtelte Typen
- UnitResolution.md mit konkreten AST-Strukturen aktualisiert
- README.md an neue Architektur-Beschreibung angepasst
2026-02-11 00:17:29 +01:00
Olaf Monien 645fd71e45 Delphi concept 2026-02-10 23:55:42 +01:00
Colby McHenry df937ceca1 Add site-packages and dist-packages to default exclude patterns
Fixes #28 - Python site-packages directories (e.g.
audio_tools/python/Lib/site-packages/) were not excluded by default,
causing massive index bloat and FOREIGN KEY failures when indexing
large libraries like tensorflow. The FK crash itself was already fixed
via INSERT OR IGNORE, but excluding these directories prevents the
bloat in the first place.
2026-02-10 16:47:13 -06:00
Colby MchenryandGitHub adaff67219 Merge pull request #25 from MO2k4/feat/db-performance-schema-v2
feat: Schema v2 migration + database performance
2026-02-10 16:36:21 -06:00
Colby McHenry 1c023c41cc Remove unused batch methods, dynamic stmt cache, and intent field
- Remove getNodesByIds, getNodesByKinds (zero callers)
- Remove getFileHashMap, getFileSyncMap (zero callers)
- Remove getDynamicStmt cache (only used by removed batch methods)
- Revert getStaleFiles to simple implementation (zero callers, no need to optimize)
- Remove intent field from SearchOptions (never read in search logic)
2026-02-10 16:34:13 -06:00
Colby MchenryandGitHub 04e1f3e122 Merge pull request #24 from MO2k4/security/path-validation-redos
security: Path validation, ReDoS prevention, picomatch
2026-02-10 16:22:39 -06:00
Colby McHenry 98034cc76e Resolve merge conflicts with main
- extraction/index.ts: use picomatch with static import (replacing
  dynamic require) and keep normalizePath for other call sites
- utils.ts: keep normalizePath from main, take PR's PID-based FileLock
2026-02-10 16:21:23 -06:00
Colby MchenryandGitHub e6531c50c0 Merge pull request #26 from MO2k4/feat/search-query-utils
feat: Search query utilities + multi-signal scoring
2026-02-10 16:14:12 -06:00
Colby McHenry a15ad69288 Remove unused detectApiIntent and inferRouteDirectories
Both functions have zero callers — dead code on arrival. Remove them
and their tests (9 tests) to keep the module focused on what's
actually used: search term extraction, path relevance scoring, and
kind bonuses.
2026-02-10 16:05:46 -06:00
Colby MchenryandGitHub e2a95ee9ab Merge pull request #27 from MO2k4/feat/extraction-quality
feat: File nodes, arrow functions, parallel I/O
2026-02-10 16:00:23 -06:00
Colby McHenry 36af284e10 Remove redundant code and deduplicate indexFile
- Remove extractFunctionVariable() and its dispatch (already handled by extractVariable)
- Remove dead getGrammar() export (zero callers)
- Deduplicate indexFile by delegating to indexFileWithContent
- Remove redundant arrow function variable extraction tests (covered by existing suite)
2026-02-10 15:55:43 -06:00
Colby McHenry ce220a573c Resolve merge conflict with main (gitignore + symlink rename)
Keeps the PR's visitedDirs rename and main's gitIgnoredDirs addition.
2026-02-10 15:44:56 -06:00