feat: Add adaptive timeouts and WASM memory error recovery for robust parsing

Implements file-size-based timeouts (base 10s + 10s per 100KB), more frequent worker recycling (250 files), and automatic retry logic for WASM memory corruption failures. Workers now crash immediately on memory errors to prevent cascading failures, with failed files automatically retried on fresh workers with clean heaps.
This commit is contained in:
Colby McHenry
2026-04-04 22:49:36 -05:00
parent c19c0ca8fd
commit 1271ad9161
3 changed files with 83 additions and 7 deletions
+10 -1
View File
@@ -172,8 +172,17 @@ export class TreeSitterExtractor {
this.visitNode(this.tree.rootNode);
this.nodeStack.pop();
} catch (error) {
const msg = error instanceof Error ? error.message : String(error);
// WASM memory errors leave the module in a corrupted state — all subsequent
// parses would also fail. Re-throw so the worker can detect and crash,
// forcing a clean restart with a fresh heap.
if (msg.includes('memory access out of bounds') || msg.includes('out of memory')) {
throw error;
}
this.errors.push({
message: `Parse error: ${error instanceof Error ? error.message : String(error)}`,
message: `Parse error: ${msg}`,
filePath: this.filePath,
severity: 'error',
code: 'parse_error',