fix(db): loop-append dense unresolved-ref result rows; make stripped-salvage visible (#1558) (#1576)

Real-world validation of #1575 on indexes damaged by the released v1.5.0
binary surfaced both of these.

getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter
limit but appended each chunk's RESULT rows with a spread — every row
becomes a call argument, so a dense recovery sync (the #1541 self-heal
re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit
and killed resolution mid-sync with "Maximum call stack size exceeded",
leaving the graph 226k edges short until another sync resumed the orphans
(and that sweep resolves measurably worse than the batched path — see the
follow-up issue). The failed-ref retry loader had the identical pattern on
unbounded result rows. Both append with a loop now (#1558).

The #1575 stripped-salvage warning also never rendered: init's summary
prints only index_partial warnings and counts only hard errors, so a run
with salvaged files still read as fully clean — and with no hard errors the
detail wasn't written to errors.log either. Salvage entries now carry code
'salvaged_stripped', the summary prints a visible warning naming the files,
and errors.log is written for salvage-only runs.

Validated on real corpora with full-graph dumps: healthy-path inits stay
byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a
determinism control); a realistically-damaged index (41 wiped + 5 missing
files, damage generated by the released binary) heals in one plain sync to
identical per-file counts and an edge set within the normal incremental
residual; pathological mass damage (52% of the repo) completes without
crashing. New regression test reproduces the RangeError on the old code
with 200k pending refs.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-08-20 12:36:22 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 26045b3159
commit d8f2eeaddf
5 changed files with 106 additions and 5 deletions
+20 -3
View File
@@ -405,6 +405,16 @@ function printIndexResult(clack: typeof import('@clack/prompts'), result: IndexR
for (const w of result.errors.filter((e) => e.code === 'index_partial')) {
clack.log.warn(w.message);
}
// Files salvaged from comment-stripped source after repeated parser
// failures are indexed but possibly incomplete — say so here, or the run
// reads as fully clean and the index quietly disagrees with a later
// re-parse of the same bytes (#1565).
const salvaged = result.errors.filter((e) => e.code === 'salvaged_stripped');
if (salvaged.length > 0) {
const sample = salvaged.slice(0, 3).map((e) => e.filePath).filter(Boolean).join(', ');
const more = salvaged.length > 3 ? ', ...' : '';
clack.log.warn(`${formatNumber(salvaged.length)} file(s) indexed from comment-stripped source after repeated parse failures ${getGlyphs().dash} symbols may be incomplete (${sample}${more})`);
}
} else if (hasErrors) {
clack.log.error(`Indexing failed ${getGlyphs().dash} all ${formatNumber(result.filesErrored)} files had errors`);
} else {
@@ -443,9 +453,16 @@ function printIndexResult(clack: typeof import('@clack/prompts'), result: IndexR
clack.log.info(`The index is fully usable ${getGlyphs().dash} only the failed files are missing.`);
}
} else if (projectPath) {
const logPath = path.join(getCodeGraphDir(projectPath), 'errors.log');
if (fs.existsSync(logPath)) {
fs.unlinkSync(logPath);
// No hard errors. Salvaged-file warnings still belong in the log — it
// carries the per-file detail behind the one-line summary above.
if (result.errors.some((e) => e.code === 'salvaged_stripped')) {
writeErrorLog(projectPath, result.errors);
clack.log.info('See .codegraph/errors.log for details');
} else {
const logPath = path.join(getCodeGraphDir(projectPath), 'errors.log');
if (fs.existsSync(logPath)) {
fs.unlinkSync(logPath);
}
}
}
}
+10 -2
View File
@@ -2359,7 +2359,12 @@ export class QueryBuilder {
const chunkRows = this.db
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'pending' AND file_path IN (${placeholders})`)
.all(...chunk) as UnresolvedRefRow[];
rows.push(...chunkRows);
// Append with a loop, never a spread: the INPUT chunk is bounded, but
// the RESULT rows per chunk are not — a dense recovery sync (e.g. the
// #1541 self-heal re-indexing hundreds of files) returns more rows than
// V8 allows as arguments, and `push(...chunkRows)` dies with "Maximum
// call stack size exceeded", aborting resolution mid-sync (#1558).
for (const row of chunkRows) rows.push(row);
}
return rows.map((row) => ({
@@ -2541,7 +2546,10 @@ export class QueryBuilder {
const chunkRows = this.db
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'failed' AND name_tail IN (${placeholders})`)
.all(...chunk) as UnresolvedRefRow[];
rows.push(...chunkRows);
// Loop, not spread — same V8 argument-limit hazard as
// getUnresolvedReferencesByFiles (#1558): a large definition delta can
// select an unbounded number of failed rows per chunk.
for (const row of chunkRows) rows.push(row);
}
return rows.map((row) => ({
+1
View File
@@ -2102,6 +2102,7 @@ export class ExtractionOrchestrator {
// on, and a silently "clean" file here is how an index quietly
// disagrees with a later per-file sync of the same bytes (#1565).
errEntry.severity = 'warning';
errEntry.code = 'salvaged_stripped';
errEntry.message = `Indexed from comment-stripped source after repeated parse failures (symbols may be incomplete until the file is re-indexed): ${errEntry.message}`;
filesErrored--;
filesIndexed++;