fix: Lazy grammar loading and quantized embeddings to prevent V8 WASM OOM
Fixes #54 — `codegraph init -i` crashes with "Fatal process out of memory: Zone" on large codebases because all 16 tree-sitter WASM grammar modules were compiled upfront by V8, exhausting the WASM Zone allocator. Changes: - initGrammars() now only initializes the tree-sitter WASM runtime (Parser.init()), no longer eagerly loads all grammar files - New loadGrammarsForLanguages() loads only grammars for languages actually present in the project (e.g. a Dart project loads ~2-3 grammars instead of 16) - Orchestrator detects needed languages after file scan, before parsing begins - Embedding pipeline now uses quantized model (~67MB vs ~270MB) to further reduce WASM memory pressure when embeddings are enabled
This commit is contained in:
@@ -25,10 +25,12 @@ import {
|
||||
clearParserCache,
|
||||
getUnavailableGrammarErrors,
|
||||
initGrammars,
|
||||
loadAllGrammars,
|
||||
} from '../src/extraction/grammars';
|
||||
|
||||
beforeAll(async () => {
|
||||
await initGrammars();
|
||||
await loadAllGrammars();
|
||||
});
|
||||
|
||||
// Create a temporary directory for each test
|
||||
|
||||
Reference in New Issue
Block a user