fix(db): loop-append dense unresolved-ref result rows; make stripped-salvage visible (#1558) (#1576)
Real-world validation of #1575 on indexes damaged by the released v1.5.0 binary surfaced both of these. getUnresolvedReferencesByFiles chunked its INPUT under SQLite's parameter limit but appended each chunk's RESULT rows with a spread — every row becomes a call argument, so a dense recovery sync (the #1541 self-heal re-indexing 919 files produced 234,440 rows) exceeded V8's argument limit and killed resolution mid-sync with "Maximum call stack size exceeded", leaving the graph 226k edges short until another sync resumed the orphans (and that sweep resolves measurably worse than the batched path — see the follow-up issue). The failed-ref retry loader had the identical pattern on unbounded result rows. Both append with a loop now (#1558). The #1575 stripped-salvage warning also never rendered: init's summary prints only index_partial warnings and counts only hard errors, so a run with salvaged files still read as fully clean — and with no hard errors the detail wasn't written to errors.log either. Salvage entries now carry code 'salvaged_stripped', the summary prints a visible warning naming the files, and errors.log is written for salvage-only runs. Validated on real corpora with full-graph dumps: healthy-path inits stay byte-identical to the pre-#1575 baseline (cpython Lib, Alamofire, with a determinism control); a realistically-damaged index (41 wiped + 5 missing files, damage generated by the released binary) heals in one plain sync to identical per-file counts and an edge set within the normal incremental residual; pathological mass damage (52% of the repo) completes without crashing. New regression test reproduces the RangeError on the old code with 200k pending refs. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
26045b3159
commit
d8f2eeaddf
+20
-3
@@ -405,6 +405,16 @@ function printIndexResult(clack: typeof import('@clack/prompts'), result: IndexR
|
||||
for (const w of result.errors.filter((e) => e.code === 'index_partial')) {
|
||||
clack.log.warn(w.message);
|
||||
}
|
||||
// Files salvaged from comment-stripped source after repeated parser
|
||||
// failures are indexed but possibly incomplete — say so here, or the run
|
||||
// reads as fully clean and the index quietly disagrees with a later
|
||||
// re-parse of the same bytes (#1565).
|
||||
const salvaged = result.errors.filter((e) => e.code === 'salvaged_stripped');
|
||||
if (salvaged.length > 0) {
|
||||
const sample = salvaged.slice(0, 3).map((e) => e.filePath).filter(Boolean).join(', ');
|
||||
const more = salvaged.length > 3 ? ', ...' : '';
|
||||
clack.log.warn(`${formatNumber(salvaged.length)} file(s) indexed from comment-stripped source after repeated parse failures ${getGlyphs().dash} symbols may be incomplete (${sample}${more})`);
|
||||
}
|
||||
} else if (hasErrors) {
|
||||
clack.log.error(`Indexing failed ${getGlyphs().dash} all ${formatNumber(result.filesErrored)} files had errors`);
|
||||
} else {
|
||||
@@ -443,9 +453,16 @@ function printIndexResult(clack: typeof import('@clack/prompts'), result: IndexR
|
||||
clack.log.info(`The index is fully usable ${getGlyphs().dash} only the failed files are missing.`);
|
||||
}
|
||||
} else if (projectPath) {
|
||||
const logPath = path.join(getCodeGraphDir(projectPath), 'errors.log');
|
||||
if (fs.existsSync(logPath)) {
|
||||
fs.unlinkSync(logPath);
|
||||
// No hard errors. Salvaged-file warnings still belong in the log — it
|
||||
// carries the per-file detail behind the one-line summary above.
|
||||
if (result.errors.some((e) => e.code === 'salvaged_stripped')) {
|
||||
writeErrorLog(projectPath, result.errors);
|
||||
clack.log.info('See .codegraph/errors.log for details');
|
||||
} else {
|
||||
const logPath = path.join(getCodeGraphDir(projectPath), 'errors.log');
|
||||
if (fs.existsSync(logPath)) {
|
||||
fs.unlinkSync(logPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+10
-2
@@ -2359,7 +2359,12 @@ export class QueryBuilder {
|
||||
const chunkRows = this.db
|
||||
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'pending' AND file_path IN (${placeholders})`)
|
||||
.all(...chunk) as UnresolvedRefRow[];
|
||||
rows.push(...chunkRows);
|
||||
// Append with a loop, never a spread: the INPUT chunk is bounded, but
|
||||
// the RESULT rows per chunk are not — a dense recovery sync (e.g. the
|
||||
// #1541 self-heal re-indexing hundreds of files) returns more rows than
|
||||
// V8 allows as arguments, and `push(...chunkRows)` dies with "Maximum
|
||||
// call stack size exceeded", aborting resolution mid-sync (#1558).
|
||||
for (const row of chunkRows) rows.push(row);
|
||||
}
|
||||
|
||||
return rows.map((row) => ({
|
||||
@@ -2541,7 +2546,10 @@ export class QueryBuilder {
|
||||
const chunkRows = this.db
|
||||
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'failed' AND name_tail IN (${placeholders})`)
|
||||
.all(...chunk) as UnresolvedRefRow[];
|
||||
rows.push(...chunkRows);
|
||||
// Loop, not spread — same V8 argument-limit hazard as
|
||||
// getUnresolvedReferencesByFiles (#1558): a large definition delta can
|
||||
// select an unbounded number of failed rows per chunk.
|
||||
for (const row of chunkRows) rows.push(row);
|
||||
}
|
||||
|
||||
return rows.map((row) => ({
|
||||
|
||||
@@ -2102,6 +2102,7 @@ export class ExtractionOrchestrator {
|
||||
// on, and a silently "clean" file here is how an index quietly
|
||||
// disagrees with a later per-file sync of the same bytes (#1565).
|
||||
errEntry.severity = 'warning';
|
||||
errEntry.code = 'salvaged_stripped';
|
||||
errEntry.message = `Indexed from comment-stripped source after repeated parse failures (symbols may be incomplete until the file is re-indexed): ${errEntry.message}`;
|
||||
filesErrored--;
|
||||
filesIndexed++;
|
||||
|
||||
Reference in New Issue
Block a user