d8f2eeaddf
Real-world validation of #1575 on indexes damaged by the released v1.5.0 binary surfaced both of these. getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter limit but appended each chunk's RESULT rows with a spread — every row becomes a call argument, so a dense recovery sync (the #1541 self-heal re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit and killed resolution mid-sync with "Maximum call stack size exceeded", leaving the graph 226k edges short until another sync resumed the orphans (and that sweep resolves measurably worse than the batched path — see the follow-up issue). The failed-ref retry loader had the identical pattern on unbounded result rows. Both append with a loop now (#1558). The #1575 stripped-salvage warning also never rendered: init's summary prints only index_partial warnings and counts only hard errors, so a run with salvaged files still read as fully clean — and with no hard errors the detail wasn't written to errors.log either. Salvage entries now carry code 'salvaged_stripped', the summary prints a visible warning naming the files, and errors.log is written for salvage-only runs. Validated on real corpora with full-graph dumps: healthy-path inits stay byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a determinism control); a realistically-damaged index (41 wiped + 5 missing files, damage generated by the released binary) heals in one plain sync to identical per-file counts and an edge set within the normal incremental residual; pathological mass damage (52% of the repo) completes without crashing. New regression test reproduces the RangeError on the old code with 200k pending refs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
75 lines
2.8 KiB
TypeScript
75 lines
2.8 KiB
TypeScript
/**
|
|
* getUnresolvedReferencesByFiles must survive dense result sets (#1558).
|
|
*
|
|
* The input file-path list is chunked under SQLite's parameter limit, but the
|
|
* ROWS a chunk returns are unbounded — and appending them with
|
|
* `rows.push(...chunkRows)` passes every row as a call argument, so a dense
|
|
* chunk (a recovery sync re-indexing many files at once, e.g. the #1541
|
|
* self-heal) exceeded V8's argument limit and killed the whole sync with
|
|
* "Maximum call stack size exceeded" after the store phase, leaving every
|
|
* re-indexed file's references unresolved. Reproduced for real on a
|
|
* cpython-stdlib-sized heal (919 files, 234k refs). The append is now a loop;
|
|
* this pins it with a result set well past V8's argument ceiling (~124k).
|
|
*/
|
|
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
|
import * as fs from 'node:fs';
|
|
import * as path from 'node:path';
|
|
import * as os from 'node:os';
|
|
import { CodeGraph } from '../src';
|
|
import type { UnresolvedReference } from '../src/types';
|
|
|
|
describe('unresolved-ref loads with dense result sets (#1558)', () => {
|
|
let dir: string;
|
|
let cg: CodeGraph;
|
|
|
|
beforeEach(async () => {
|
|
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'refs-spread-'));
|
|
fs.writeFileSync(path.join(dir, 'anchor.py'), 'def anchor():\n return 1\n');
|
|
cg = await CodeGraph.init(dir);
|
|
await cg.indexAll();
|
|
});
|
|
|
|
afterEach(() => {
|
|
cg.destroy();
|
|
fs.rmSync(dir, { recursive: true, force: true });
|
|
});
|
|
|
|
it('returns 200k pending refs from few files without exhausting the call stack', () => {
|
|
const queries = (cg as unknown as {
|
|
queries: {
|
|
insertUnresolvedRefsBatch(refs: UnresolvedReference[]): void;
|
|
getUnresolvedReferencesByFiles(paths: string[]): UnresolvedReference[];
|
|
};
|
|
}).queries;
|
|
|
|
const FILES = 200;
|
|
const TOTAL = 200_000;
|
|
const paths: string[] = Array.from({ length: FILES }, (_, i) => `src/f${i}.py`);
|
|
// unresolved_refs.from_node_id is FK-constrained — anchor on a real node.
|
|
const anchorId = cg.getNodesInFile('anchor.py')[0]!.id;
|
|
|
|
const batch: UnresolvedReference[] = [];
|
|
for (let i = 0; i < TOTAL; i++) {
|
|
batch.push({
|
|
fromNodeId: anchorId,
|
|
referenceName: `ref_${i}`,
|
|
referenceKind: 'call',
|
|
line: (i % 1000) + 1,
|
|
column: 0,
|
|
filePath: paths[i % FILES]!,
|
|
language: 'python',
|
|
});
|
|
if (batch.length === 20_000) {
|
|
queries.insertUnresolvedRefsBatch(batch);
|
|
batch.length = 0;
|
|
}
|
|
}
|
|
if (batch.length > 0) queries.insertUnresolvedRefsBatch(batch);
|
|
|
|
// All 200 paths fit in ONE SQLite parameter chunk, so a single query
|
|
// returns all 200k rows — the exact shape that blew the argument limit.
|
|
const rows = queries.getUnresolvedReferencesByFiles(paths);
|
|
expect(rows.length).toBe(TOTAL);
|
|
});
|
|
});
|