fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) (#1249)

* chore: ignore .kommandr/ directory

* fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240)

Incremental sync scoped reference resolution to the changed files' own
refs, and a completed pass deleted every ref it failed to resolve — so
a symbol change in one file could never repair references in UNCHANGED
files, in either direction, until a full re-index:

- New-export case: a.ts imports/calls `greet` before b.ts defines it.
  The failed refs were deleted at index time; when b.ts later gained
  `greet`, nothing revisited a.ts — the calls/imports edges stayed
  missing while status reported a clean index.
- Removal case: when a re-index (or file deletion) dropped a symbol,
  the incoming edges cascade-deleted and the callers — whose resolved
  refs had been consumed — never got a chance to rebind to an
  alternative definition or reconnect when the symbol returned.

Fix, sharing one lifecycle:

- Schema v8: unresolved_refs gains status ('pending'/'failed') and
  name_tail (last dotted segment, so `h.greet` is findable by `greet`).
  Both resolver persist paths now park unresolvable refs as failed
  instead of deleting them. All pending-work readers (batched drain,
  non-progress guard, #1187 orphan sweep, status pendingRefs) filter to
  pending, preserving their invariants and keeping status honest.
- Sync retry: after scoped resolution, failed refs whose name tail
  matches a symbol name now present in the changed files are re-resolved
  through a per-ref-yielding path (watchdog-safe, #1091 class). Names
  matching >500 failed refs are skipped as external/builtin noise (#999
  rationale).
- Removal side: createEdges stamps each resolution edge with its
  originating reference (metadata.refName, + refKind when kind promotion
  rewrote it). When the #899 restore misses a target or sync deletes a
  file, the dropped edge is resurrected as exactly that ref — re-resolved
  in the same sync (rebinding to an alternative definition) or parked
  failed until the symbol reappears. Edges without the stamp (pre-upgrade,
  synthesized) still drop silently: reconstructing from the target's plain
  name would strip receiver context and risk a rebind a full re-index
  would never make.
- Pure-removal syncs clear resolver caches so a long-lived daemon can't
  resolve resurrected refs against the pre-removal graph.

Validated: issue repro now yields a graph byte-identical to a full
re-index; move/remove-readd/file-deletion scenarios all rebind or heal;
baseline-vs-new A/B on express and gin shows identical node/edge counts
and no timing regression (DB grows ~25% from the parked ref rows — pure
cache, reset by any full re-index). 8 regression tests added.

Fixes #1240

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-10 12:19:08 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 386bff0f84
commit 9d0cd3a7d1
12 changed files with 570 additions and 43 deletions
+32 -1
View File
@@ -9,7 +9,7 @@ import { SqliteDatabase } from './sqlite-adapter';
/**
* Current schema version
*/
export const CURRENT_SCHEMA_VERSION = 7;
export const CURRENT_SCHEMA_VERSION = 8;
/**
* Migration definition
@@ -119,6 +119,37 @@ const migrations: Migration[] = [
`);
},
},
{
version: 8,
description:
'Track attempted-but-unresolvable refs as status=failed so sync can retry them when a changed file adds a matching symbol (#1240)',
up: (db) => {
// DDL only — instant on any size database. No backfill needed: rows are
// only ever queried by name_tail once they carry status='failed', and
// both fields are written together by markReferencesFailed. Legacy rows
// (all 'pending' after this migration) are orphans from interrupted runs
// that the #1187 sweep grinds down on the next sync, marking survivors
// failed with their tails as it goes. The tail index is partial: on a
// healthy index the pending set is empty and the failed set is the only
// population worth indexing. Keep the definitions in lockstep with
// schema.sql. ALTER TABLE has no IF NOT EXISTS, so guard each column for
// idempotency — a database created from current schema.sql already has
// both (matters when migrations are re-run from an older recorded
// version, as the v6 regression test does).
const cols = db.prepare('PRAGMA table_info(unresolved_refs)').all() as Array<{ name: string }>;
const hasColumn = (name: string) => cols.some((c) => c.name === name);
if (!hasColumn('status')) {
db.exec("ALTER TABLE unresolved_refs ADD COLUMN status TEXT NOT NULL DEFAULT 'pending'");
}
if (!hasColumn('name_tail')) {
db.exec("ALTER TABLE unresolved_refs ADD COLUMN name_tail TEXT NOT NULL DEFAULT ''");
}
db.exec(`
CREATE INDEX IF NOT EXISTS idx_unresolved_status ON unresolved_refs(status);
CREATE INDEX IF NOT EXISTS idx_unresolved_failed_tail ON unresolved_refs(name_tail) WHERE status = 'failed';
`);
},
},
];
/**
+127 -9
View File
@@ -109,6 +109,20 @@ interface UnresolvedRefRow {
candidates: string | null;
file_path: string;
language: string;
status: string;
name_tail: string;
}
/**
* Last segment of a (possibly dotted/qualified) reference name — the part a
* new symbol's plain node name could match: 'util.greet' → 'greet',
* 'mod::fn' → 'fn', 'greet' → 'greet'. Written to unresolved_refs.name_tail
* when a ref is marked failed, so the #1240 retry lookup can match dotted
* refs against newly-added node names.
*/
function referenceNameTail(referenceName: string): string {
const idx = Math.max(referenceName.lastIndexOf('.'), referenceName.lastIndexOf(':'));
return idx >= 0 ? referenceName.slice(idx + 1) : referenceName;
}
/**
@@ -1627,19 +1641,26 @@ export class QueryBuilder {
* re-index (issue #899). Same edge-kind rules as
* {@link getDependentFilePaths}: all kinds except `contains`.
*/
getCrossFileIncomingEdgesWithTarget(filePath: string): Array<Edge & { targetName: string; targetKind: NodeKind }> {
const sql = `SELECT e.*, tgt.name AS target_name, tgt.kind AS target_kind
getCrossFileIncomingEdgesWithTarget(
filePath: string
): Array<Edge & { targetName: string; targetKind: NodeKind; sourceFilePath: string; sourceLanguage: Language }> {
const sql = `SELECT e.*, tgt.name AS target_name, tgt.kind AS target_kind,
src.file_path AS source_file_path, src.language AS source_language
FROM edges e
JOIN nodes tgt ON tgt.id = e.target
JOIN nodes src ON src.id = e.source
WHERE tgt.file_path = ?
AND e.kind != 'contains'
AND src.file_path != ?`;
const rows = this.db.prepare(sql).all(filePath, filePath) as Array<EdgeRow & { target_name: string; target_kind: NodeKind }>;
const rows = this.db.prepare(sql).all(filePath, filePath) as Array<
EdgeRow & { target_name: string; target_kind: NodeKind; source_file_path: string; source_language: Language }
>;
return rows.map(row => ({
...rowToEdge(row),
targetName: row.target_name,
targetKind: row.target_kind,
sourceFilePath: row.source_file_path,
sourceLanguage: row.source_language,
}));
}
@@ -1827,12 +1848,16 @@ export class QueryBuilder {
}
/**
* Get the count of unresolved references without loading them into memory
* Get the count of PENDING (never-attempted) references without loading
* them into memory. Rows marked status='failed' — attempted by a completed
* pass, no match — are excluded: they are not outstanding work, only retry
* candidates for the #1240 sweep, so they must not trip the #1187 orphan
* sweep or the `status` pending-refs warning.
*/
getUnresolvedReferencesCount(): number {
if (!this.stmts.getUnresolvedCount) {
this.stmts.getUnresolvedCount = this.db.prepare(
'SELECT COUNT(*) as count FROM unresolved_refs'
"SELECT COUNT(*) as count FROM unresolved_refs WHERE status = 'pending'"
);
}
const row = this.stmts.getUnresolvedCount.get() as { count: number };
@@ -1840,13 +1865,15 @@ export class QueryBuilder {
}
/**
* Get a batch of unresolved references using LIMIT/OFFSET pagination.
* Used to process references in bounded memory chunks.
* Get a batch of PENDING unresolved references using LIMIT/OFFSET
* pagination. Used to process references in bounded memory chunks; failed
* rows are excluded so the batched drain loop terminates once every row
* has been attempted.
*/
getUnresolvedReferencesBatch(offset: number, limit: number): UnresolvedReference[] {
if (!this.stmts.getUnresolvedBatch) {
this.stmts.getUnresolvedBatch = this.db.prepare(
'SELECT * FROM unresolved_refs LIMIT ? OFFSET ?'
"SELECT * FROM unresolved_refs WHERE status = 'pending' LIMIT ? OFFSET ?"
);
}
const rows = this.stmts.getUnresolvedBatch.all(limit, offset) as UnresolvedRefRow[];
@@ -1913,7 +1940,7 @@ export class QueryBuilder {
const chunk = filePaths.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const chunkRows = this.db
.prepare(`SELECT * FROM unresolved_refs WHERE file_path IN (${placeholders})`)
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'pending' AND file_path IN (${placeholders})`)
.all(...chunk) as UnresolvedRefRow[];
rows.push(...chunkRows);
}
@@ -1971,6 +1998,97 @@ export class QueryBuilder {
deleteMany(refs);
}
/**
* Mark refs a completed resolution pass could not resolve as status='failed'
* instead of deleting them (#1240). Failed rows are invisible to the pending
* count/batch readers (so drain loops and the #1187 orphan sweep still
* terminate) but stay queryable by name_tail so a later sync can retry them
* when a changed file introduces a symbol that could satisfy them. name_tail
* is (re)written here so rows inserted before the v8 migration get their
* tail the first time they're attempted.
*/
markReferencesFailed(refs: Array<{ fromNodeId: string; referenceName: string; referenceKind: string }>): void {
if (refs.length === 0) return;
const stmt = this.db.prepare(
"UPDATE unresolved_refs SET status = 'failed', name_tail = ? WHERE from_node_id = ? AND reference_name = ? AND reference_kind = ?"
);
const markMany = this.db.transaction((items: typeof refs) => {
for (const ref of items) {
stmt.run(referenceNameTail(ref.referenceName), ref.fromNodeId, ref.referenceName, ref.referenceKind);
}
});
markMany(refs);
}
/**
* Failed refs whose name tail matches one of the given symbol names — the
* candidates a sync should retry after files carrying those names changed
* (#1240). Names matching more than `perNameCeiling` failed refs are
* skipped entirely: at that population a name is external/builtin noise
* (`get`, `map`, …) that one new definition won't resolve — the same
* rationale as resolution's AMBIGUOUS_NAME_CEILING (#999) — and retrying an
* arbitrary subset would be both wasted work and incoherent coverage.
*/
getRetryableFailedReferences(names: string[], perNameCeiling: number = 500): UnresolvedReference[] {
if (names.length === 0) return [];
// Pass 1: per-tail counts, chunked under the SQLite parameter limit.
const retryNames: string[] = [];
for (let i = 0; i < names.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = names.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const counts = this.db
.prepare(
`SELECT name_tail, COUNT(*) as count FROM unresolved_refs WHERE status = 'failed' AND name_tail IN (${placeholders}) GROUP BY name_tail`
)
.all(...chunk) as Array<{ name_tail: string; count: number }>;
for (const row of counts) {
if (row.count <= perNameCeiling) retryNames.push(row.name_tail);
}
}
if (retryNames.length === 0) return [];
// Pass 2: load the surviving rows.
const rows: UnresolvedRefRow[] = [];
for (let i = 0; i < retryNames.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = retryNames.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const chunkRows = this.db
.prepare(`SELECT * FROM unresolved_refs WHERE status = 'failed' AND name_tail IN (${placeholders})`)
.all(...chunk) as UnresolvedRefRow[];
rows.push(...chunkRows);
}
return rows.map((row) => ({
fromNodeId: row.from_node_id,
referenceName: row.reference_name,
referenceKind: row.reference_kind as EdgeKind,
line: row.line,
column: row.col,
candidates: row.candidates ? safeJsonParse(row.candidates, undefined) : undefined,
filePath: row.file_path,
language: row.language as Language,
}));
}
/**
* Distinct node names present in the given files — the symbol names a sync
* pass uses to look up retryable failed refs after those files changed.
*/
getNodeNamesByFiles(filePaths: string[]): string[] {
if (filePaths.length === 0) return [];
const names = new Set<string>();
for (let i = 0; i < filePaths.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = filePaths.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const rows = this.db
.prepare(`SELECT DISTINCT name FROM nodes WHERE file_path IN (${placeholders})`)
.all(...chunk) as Array<{ name: string }>;
for (const row of rows) names.add(row.name);
}
return [...names];
}
// ===========================================================================
// Statistics
// ===========================================================================
+13 -1
View File
@@ -67,7 +67,15 @@ CREATE TABLE IF NOT EXISTS files (
errors TEXT -- JSON array
);
-- Unresolved References: References that need resolution after full indexing
-- Unresolved References: References that need resolution after full indexing.
-- status lifecycle: rows are inserted 'pending' by extraction; a completed
-- resolution pass either deletes a row (resolved) or marks it 'failed'
-- (attempted, no match — kept so a later sync can retry it when a changed
-- file introduces a symbol that could satisfy it, #1240). name_tail is the
-- last segment of reference_name ('util.greet' → 'greet'), written when a
-- row is marked failed, so the retry lookup matches new node names against
-- dotted refs too. Rows follow their from_node via ON DELETE CASCADE, so
-- re-extracting or deleting a file clears its stale rows in any status.
CREATE TABLE IF NOT EXISTS unresolved_refs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
from_node_id TEXT NOT NULL,
@@ -78,6 +86,8 @@ CREATE TABLE IF NOT EXISTS unresolved_refs (
candidates TEXT, -- JSON array
file_path TEXT NOT NULL DEFAULT '',
language TEXT NOT NULL DEFAULT 'unknown',
status TEXT NOT NULL DEFAULT 'pending',
name_tail TEXT NOT NULL DEFAULT '',
FOREIGN KEY (from_node_id) REFERENCES nodes(id) ON DELETE CASCADE
);
@@ -172,6 +182,8 @@ CREATE INDEX IF NOT EXISTS idx_unresolved_from_node ON unresolved_refs(from_node
CREATE INDEX IF NOT EXISTS idx_unresolved_name ON unresolved_refs(reference_name);
CREATE INDEX IF NOT EXISTS idx_unresolved_file_path ON unresolved_refs(file_path);
CREATE INDEX IF NOT EXISTS idx_unresolved_from_name ON unresolved_refs(from_node_id, reference_name);
CREATE INDEX IF NOT EXISTS idx_unresolved_status ON unresolved_refs(status);
CREATE INDEX IF NOT EXISTS idx_unresolved_failed_tail ON unresolved_refs(name_tail) WHERE status = 'failed';
CREATE INDEX IF NOT EXISTS idx_edges_provenance ON edges(provenance);
-- Project metadata for version/provenance tracking
+70 -4
View File
@@ -16,6 +16,8 @@ import {
ExtractionResult,
ExtractionError,
Edge,
UnresolvedReference,
ReferenceKind,
} from '../types';
import { QueryBuilder } from '../db/queries';
import { extractFromSource } from './tree-sitter';
@@ -1360,6 +1362,38 @@ function scanDirectoryWalk(
return files;
}
/**
* Resurrect a resolution edge that is about to be dropped (its target symbol
* was removed, renamed, or its whole file deleted) as the ORIGINAL unresolved
* reference that created it, read from the refName/refKind stamp
* `createEdges` writes into edge metadata. Inserted as status='pending', the
* ref is consumed by the same sync's resolution sweep: it rebinds to an
* alternative definition if one exists, or parks as status='failed' where the
* #1240 retry finds it if the symbol later reappears.
*
* Returns null — drop silently, the pre-#1240 behavior — for edges without a
* refName stamp (created before the stamp existed, or synthesized): rebuilding
* a ref from the target's plain node name would strip the receiver/qualifier
* context the original text carried (`h.greet` → `greet`) and could rebind
* somewhere a full re-index never would. Silent beats wrong.
*/
function resurrectRefFromDroppedEdge(
e: Edge & { sourceFilePath: string; sourceLanguage: Language }
): UnresolvedReference | null {
const refName = e.metadata?.refName;
if (typeof refName !== 'string' || refName.length === 0) return null;
const refKind = typeof e.metadata?.refKind === 'string' ? (e.metadata.refKind as ReferenceKind) : e.kind;
return {
fromNodeId: e.source,
referenceName: refName,
referenceKind: refKind,
line: e.line ?? 0,
column: e.column ?? 0,
filePath: e.sourceFilePath,
language: e.sourceLanguage,
};
}
/**
* Extraction orchestrator
*/
@@ -2193,25 +2227,41 @@ export class ExtractionOrchestrator {
// (filePath, kind, name). Node ids include the source line, so any line
// shift in the callee file (e.g. a docstring-only edit above the symbol)
// changes every target id and a naive re-insert by old id would drop them
// all. `insertEdges` still filters to endpoints that exist, so edges whose
// caller (source) was deleted, or whose callee (target) was renamed/removed
// during the re-index (no match in `newTargetIds`), are dropped. This
// closes the #899 edge-drop on `sync`.
// all. `insertEdges` still filters to endpoints that exist. This closes
// the #899 edge-drop on `sync`.
//
// Edges whose callee (target) was renamed/removed during the re-index (no
// match in `newNodesByKindName`) are not silently dropped anymore: each is
// resurrected as its ORIGINAL unresolved ref (stamped on the edge as
// metadata.refName/refKind at creation) so the same sync's resolution
// sweep can rebind it to an alternative definition elsewhere, or park it
// as status='failed' to be retried when the symbol reappears — the
// removal-side counterpart of #1240. Edges without refName (built before
// the stamp existed, or synthesized) still drop silently: reconstructing
// a ref from the target's plain name would strip receiver/qualifier
// context and risk a rebind a full re-index would never make.
if (crossFileIncomingEdges.length > 0) {
const newNodesByKindName = new Map<string, string>();
for (const n of validNodes) {
newNodesByKindName.set(`${n.kind}\0${n.name}`, n.id);
}
const reinserted: Edge[] = [];
const resurrected: UnresolvedReference[] = [];
for (const e of crossFileIncomingEdges) {
const newTargetId = newNodesByKindName.get(`${e.targetKind}\0${e.targetName}`);
if (newTargetId) {
reinserted.push({ source: e.source, target: newTargetId, kind: e.kind, metadata: e.metadata, line: e.line, column: e.column, provenance: e.provenance });
} else {
const ref = resurrectRefFromDroppedEdge(e);
if (ref) resurrected.push(ref);
}
}
if (reinserted.length > 0) {
this.queries.insertEdges(reinserted);
}
if (resurrected.length > 0) {
this.queries.insertUnresolvedRefsBatch(resurrected);
}
}
// Insert unresolved references in batch with denormalized filePath/language
@@ -2300,6 +2350,22 @@ export class ExtractionOrchestrator {
let reconcileChecks = 0;
for (const tracked of trackedFiles) {
if (!currentSet.has(tracked.path) || !fs.existsSync(path.join(this.rootDir, tracked.path))) {
// Before the cascade deletes them, resurrect incoming cross-file
// resolution edges as their original refs (#1240 removal case): the
// callers live in files this sync will NOT revisit, so this is their
// only chance to rebind to an alternative definition — or to park as
// failed until the symbol reappears somewhere. (A deleted file whose
// CALLERS are also being deleted is fine: their nodes cascade later
// in this loop and take the resurrected rows with them.)
const incoming = this.queries.getCrossFileIncomingEdgesWithTarget(tracked.path);
if (incoming.length > 0) {
const resurrected = incoming
.map((e) => resurrectRefFromDroppedEdge(e))
.filter((r): r is UnresolvedReference => r !== null);
if (resurrected.length > 0) {
this.queries.insertUnresolvedRefsBatch(resurrected);
}
}
this.queries.deleteFile(tracked.path);
filesRemoved++;
}
+46 -7
View File
@@ -664,6 +664,15 @@ export class CodeGraph {
// (regex over *.module.ts only).
if (result.filesAdded > 0 || result.filesModified > 0) {
this.resolver.runPostExtract();
} else if (result.filesRemoved > 0) {
// A pure-removal sync still resolves refs below — the deletion path
// resurrects the removed file's incoming edges as pending refs
// (#1240 removal case) and the orphan sweep consumes them. In a
// long-lived process (daemon) the resolver's name caches were
// warmed against the pre-removal graph; drop them so resolution
// sees the post-removal state. (runPostExtract above clears caches
// itself, so the changed-files branch is already covered.)
this.resolver.clearCaches();
}
// Resolve references if files were updated
@@ -688,6 +697,34 @@ export class CodeGraph {
total,
});
});
// Retry previously-failed refs the changed files may now satisfy
// (#1240). Scoped resolution above only re-resolves refs FROM the
// changed files — but when a changed file gains an export/symbol,
// refs in UNCHANGED files that failed against the old graph can
// now resolve, and nothing else ever revisits them (their rows
// were parked as status='failed' by an earlier completed pass).
// Look them up by the symbol names the changed files now carry
// and re-resolve just that set. On a sync where no failed ref
// matches, this is one indexed lookup.
const tRetry = Date.now();
const retryable = this.queries.getRetryableFailedReferences(
this.queries.getNodeNamesByFiles(result.changedFilePaths)
);
if (retryable.length > 0) {
options.onProgress?.({
phase: 'resolving',
current: 0,
total: retryable.length,
});
await this.resolver.resolveAndPersistListYielding(retryable);
options.onProgress?.({
phase: 'resolving',
current: retryable.length,
total: retryable.length,
});
}
if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] sync-failed-ref-retry: ${Date.now() - tRetry}ms (${retryable.length} refs)`);
} else {
// No git info — use batched resolution to avoid OOM
const unresolvedCount = this.queries.getUnresolvedReferencesCount();
@@ -714,9 +751,10 @@ export class CodeGraph {
// path above never revisits them (it reads only the changed files'
// rows). Those files' call edges were then missing PERMANENTLY, with
// nothing to see except a too-small blast radius, until a full
// re-index. A completed pass deletes every row it processed (resolved
// or not), so any row still present now is such an orphan — or a row
// parked by an older engine whose scoped pass kept unresolvable refs.
// re-index. A completed pass takes every row it processed out of the
// PENDING set (resolved rows are deleted, unresolvable ones parked as
// status='failed' for the #1240 retry above), so any pending row now
// is such an orphan — or a row from an older engine's scoped pass.
// Grind them down with the batched resolver; this also makes a bare
// `codegraph sync` the recovery command for a wedged index. On a
// healthy index this is one COUNT query.
@@ -968,10 +1006,11 @@ export class CodeGraph {
}
/**
* References extracted but not yet resolved into edges. Zero on a healthy
* index — a completed resolution pass consumes every row. Non-zero at rest
* means a pass was interrupted mid-run (killed indexer, crash — #1187), so
* some files' call edges are missing; the next `sync` sweeps them.
* References extracted but never attempted by a resolution pass. Zero on a
* healthy index — a completed pass consumes every pending row (resolving it
* or parking it as failed, #1240). Non-zero at rest means a pass was
* interrupted mid-run (killed indexer, crash — #1187), so some files' call
* edges are missing; the next `sync` sweeps them.
*/
getPendingReferenceCount(): number {
return this.queries.getUnresolvedReferencesCount();
+81 -18
View File
@@ -974,6 +974,18 @@ export class ReferenceResolver {
metadata: {
confidence: ref.confidence,
resolvedBy: ref.resolvedBy,
// The ORIGINAL reference text (and kind, when edge-kind promotion
// rewrote it — calls→instantiates, extends→implements,
// function_ref→references). If this edge's target is later removed
// by a re-index, the edge is resurrected as exactly this ref and
// re-resolved (#1240 removal case) — a faithful resurrection, so
// re-resolution can never bind anywhere a full re-index wouldn't.
// Reconstruction from the target node's name instead would strip
// receiver/qualifier context (`h.greet` → `greet`) and risk a
// wrong rebind; edges without refName (pre-#1240, synthesized) are
// deliberately NOT resurrected for the same reason.
refName: ref.original.referenceName,
...(ref.original.referenceKind !== kind ? { refKind: ref.original.referenceKind } : {}),
// Uniform marker for function-as-value edges (#756), regardless of
// which strategy resolved them (import vs matchFunctionRef) — lets
// tooling label "callback registration" and lets validation diff
@@ -1012,15 +1024,17 @@ export class ReferenceResolver {
);
}
// Delete unresolvable refs too — parity with resolveAndPersistBatched.
// Keeping them bought nothing: a ref is only ever retried when its file
// is re-extracted, which cascade-deletes and re-inserts its rows anyway.
// And it broke the #1187 orphan sweep's invariant — after a COMPLETED
// pass the table must hold nothing that pass processed, so that any row
// still present belongs to an interrupted run and the sweep can key off
// a bare row count.
// Park unresolvable refs as status='failed' — parity with
// resolveAndPersistBatched. Deleting them was wrong (#1240): a ref whose
// own file never changes is otherwise gone forever, so when a DIFFERENT
// file later gains the export/symbol that would satisfy it, no sync can
// recreate the edge — only a full re-index. Failed rows are excluded from
// the pending readers, which preserves the #1187 orphan sweep's
// invariant in status form: after a COMPLETED pass nothing it processed
// is still 'pending', so any pending row at rest belongs to an
// interrupted run and the sweep can key off the pending count.
if (result.unresolved.length > 0) {
this.queries.deleteSpecificResolvedReferences(
this.queries.markReferencesFailed(
result.unresolved.map((r) => ({
fromNodeId: r.fromNodeId,
referenceName: r.referenceName,
@@ -1032,6 +1046,49 @@ export class ReferenceResolver {
return result;
}
/**
* Yielding counterpart of {@link resolveAndPersist} for a caller-supplied
* ref list — used by sync's failed-ref retry pass (#1240). Same persistence
* semantics: resolved refs become edges and their rows are deleted;
* still-unresolvable refs are (re-)marked failed (a no-op for rows already
* in that status). Yields per-ref because sync can run on the daemon's
* liveness-watchdog thread (#850/#1091) and a retry set is unbounded when
* a large edit lands many popular symbol names at once.
*/
async resolveAndPersistListYielding(refs: UnresolvedReference[]): Promise<ResolutionResult> {
const maybeYield = createYielder();
const result = await this.resolveBatchYielding(refs, maybeYield);
const PERSIST_CHUNK = 1000;
const edges = this.createEdges(result.resolved);
for (let i = 0; i < edges.length; i += PERSIST_CHUNK) {
this.queries.insertEdges(edges.slice(i, i + PERSIST_CHUNK));
await maybeYield();
}
const resolvedKeys = result.resolved.map((r) => ({
fromNodeId: r.original.fromNodeId,
referenceName: r.original.referenceName,
referenceKind: r.original.referenceKind,
}));
for (let i = 0; i < resolvedKeys.length; i += PERSIST_CHUNK) {
this.queries.deleteSpecificResolvedReferences(resolvedKeys.slice(i, i + PERSIST_CHUNK));
await maybeYield();
}
const unresolvedKeys = result.unresolved.map((r) => ({
fromNodeId: r.fromNodeId,
referenceName: r.referenceName,
referenceKind: r.referenceKind,
}));
for (let i = 0; i < unresolvedKeys.length; i += PERSIST_CHUNK) {
this.queries.markReferencesFailed(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
await maybeYield();
}
return result;
}
/**
* Second resolution pass for chained static-factory / fluent calls whose
* chained method is defined on a SUPERTYPE the receiver's type conforms to —
@@ -1162,8 +1219,10 @@ export class ReferenceResolver {
byMethod: {} as Record<string, number>,
};
// Process in batches. We always read from offset 0 because resolved refs
// are deleted after each batch, shifting the remaining rows forward.
// Process in batches. We always read from offset 0 because every ref the
// batch processed leaves the pending set (resolved rows are deleted,
// unresolvable ones flip to status='failed'), shifting the remaining
// pending rows forward.
let prevRemaining = Number.POSITIVE_INFINITY;
while (true) {
const batch = this.queries.getUnresolvedReferencesBatch(0, batchSize);
@@ -1198,14 +1257,17 @@ export class ReferenceResolver {
await maybeYield();
}
// Delete unresolvable refs from this batch to avoid re-processing them
// Park unresolvable refs from this batch as status='failed' so they
// leave the pending set (the batch reader and non-progress guard below
// only see pending rows) but stay retryable when a later sync adds a
// symbol that could satisfy them (#1240).
const unresolvedKeys = result.unresolved.map((r) => ({
fromNodeId: r.fromNodeId,
referenceName: r.referenceName,
referenceKind: r.referenceKind,
}));
for (let i = 0; i < unresolvedKeys.length; i += PERSIST_CHUNK) {
this.queries.deleteSpecificResolvedReferences(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
this.queries.markReferencesFailed(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
await maybeYield();
}
@@ -1232,13 +1294,14 @@ export class ReferenceResolver {
// The count-based guard below catches the true no-progress case.
// Non-progress guard (defense-in-depth). Because we re-read from offset 0
// each pass, the unresolved_refs table MUST shrink every iteration — both
// resolved and unresolved refs are deleted above. If it didn't shrink, a
// each pass, the PENDING population MUST shrink every iteration — resolved
// refs are deleted and unresolvable ones are marked failed above, and both
// leave the pending set the batch reader sees. If it didn't shrink, a
// resolver returned a match whose `original.referenceName` differs from the
// stored row, so the keyed delete no-ops, and we'd re-read + re-resolve +
// re-insert the same rows forever (the runaway that grew a 99-file repo to
// 5M edges / 1.4 GB before the Go-fallback fix). Stop rather than grow the
// graph without bound.
// stored row, so the keyed delete/update no-ops, and we'd re-read +
// re-resolve + re-insert the same rows forever (the runaway that grew a
// 99-file repo to 5M edges / 1.4 GB before the Go-fallback fix). Stop
// rather than grow the graph without bound.
const remaining = this.queries.getUnresolvedReferencesCount();
if (remaining >= prevRemaining) break;
prevRemaining = remaining;