fix: Lazy grammar loading and quantized embeddings to prevent V8 WASM OOM

Fixes #54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone"
on large codebases because all 16 tree-sitter WASM grammar modules were compiled
upfront by V8, exhausting the WASM Zone allocator.

Changes:
- initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()),
  no longer eagerly loads all grammar files
- New loadGrammarsForLanguages() loads only grammars for languages actually present
  in the project (e.g. a Dart project loads ~2-3 grammars instead of 16)
- Orchestrator detects needed languages after file scan, before parsing begins
- Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce
  WASM memory pressure when embeddings are enabled
This commit is contained in:
Colby McHenry
2026-03-18 14:21:54 -05:00
parent 16db37566d
commit 15b5e56322
6 changed files with 81 additions and 28 deletions
+1 -1
View File
@@ -61,7 +61,7 @@ export {
CODEGRAPH_DIR,
} from './directory';
export { IndexProgress, IndexResult, SyncResult } from './extraction';
export { detectLanguage, isLanguageSupported, getSupportedLanguages, initGrammars } from './extraction';
export { detectLanguage, isLanguageSupported, isGrammarLoaded, getSupportedLanguages, initGrammars, loadGrammarsForLanguages, loadAllGrammars } from './extraction';
export { ResolutionResult } from './resolution';
export { EmbeddingProgress } from './vectors';
export {