fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) (#1249)
* chore: ignore .kommandr/ directory * fix(sync): resolve cross-file refs when an edit adds or removes the satisfying symbol (#1240) Incremental sync scoped reference resolution to the changed files' own refs, and a completed pass deleted every ref it failed to resolve — so a symbol change in one file could never repair references in UNCHANGED files, in either direction, until a full re-index: - New-export case: a.ts imports/calls `greet` before b.ts defines it. The failed refs were deleted at index time; when b.ts later gained `greet`, nothing revisited a.ts — the calls/imports edges stayed missing while status reported a clean index. - Removal case: when a re-index (or file deletion) dropped a symbol, the incoming edges cascade-deleted and the callers — whose resolved refs had been consumed — never got a chance to rebind to an alternative definition or reconnect when the symbol returned. Fix, sharing one lifecycle: - Schema v8: unresolved_refs gains status ('pending'/'failed') and name_tail (last dotted segment, so `h.greet` is findable by `greet`). Both resolver persist paths now park unresolvable refs as failed instead of deleting them. All pending-work readers (batched drain, non-progress guard, #1187 orphan sweep, status pendingRefs) filter to pending, preserving their invariants and keeping status honest. - Sync retry: after scoped resolution, failed refs whose name tail matches a symbol name now present in the changed files are re-resolved through a per-ref-yielding path (watchdog-safe, #1091 class). Names matching >500 failed refs are skipped as external/builtin noise (#999 rationale). - Removal side: createEdges stamps each resolution edge with its originating reference (metadata.refName, + refKind when kind promotion rewrote it). When the #899 restore misses a target or sync deletes a file, the dropped edge is resurrected as exactly that ref — re-resolved in the same sync (rebinding to an alternative definition) or parked failed until the symbol reappears. Edges without the stamp (pre-upgrade, synthesized) still drop silently: reconstructing from the target's plain name would strip receiver context and risk a rebind a full re-index would never make. - Pure-removal syncs clear resolver caches so a long-lived daemon can't resolve resurrected refs against the pre-removal graph. Validated: issue repro now yields a graph byte-identical to a full re-index; move/remove-readd/file-deletion scenarios all rebind or heal; baseline-vs-new A/B on express and gin shows identical node/edge counts and no timing regression (DB grows ~25% from the parked ref rows — pure cache, reset by any full re-index). 8 regression tests added. Fixes #1240 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Fable 5
Claude Fable 5
parent
386bff0f84
commit
9d0cd3a7d1
+81
-18
@@ -974,6 +974,18 @@ export class ReferenceResolver {
|
||||
metadata: {
|
||||
confidence: ref.confidence,
|
||||
resolvedBy: ref.resolvedBy,
|
||||
// The ORIGINAL reference text (and kind, when edge-kind promotion
|
||||
// rewrote it — calls→instantiates, extends→implements,
|
||||
// function_ref→references). If this edge's target is later removed
|
||||
// by a re-index, the edge is resurrected as exactly this ref and
|
||||
// re-resolved (#1240 removal case) — a faithful resurrection, so
|
||||
// re-resolution can never bind anywhere a full re-index wouldn't.
|
||||
// Reconstruction from the target node's name instead would strip
|
||||
// receiver/qualifier context (`h.greet` → `greet`) and risk a
|
||||
// wrong rebind; edges without refName (pre-#1240, synthesized) are
|
||||
// deliberately NOT resurrected for the same reason.
|
||||
refName: ref.original.referenceName,
|
||||
...(ref.original.referenceKind !== kind ? { refKind: ref.original.referenceKind } : {}),
|
||||
// Uniform marker for function-as-value edges (#756), regardless of
|
||||
// which strategy resolved them (import vs matchFunctionRef) — lets
|
||||
// tooling label "callback registration" and lets validation diff
|
||||
@@ -1012,15 +1024,17 @@ export class ReferenceResolver {
|
||||
);
|
||||
}
|
||||
|
||||
// Delete unresolvable refs too — parity with resolveAndPersistBatched.
|
||||
// Keeping them bought nothing: a ref is only ever retried when its file
|
||||
// is re-extracted, which cascade-deletes and re-inserts its rows anyway.
|
||||
// And it broke the #1187 orphan sweep's invariant — after a COMPLETED
|
||||
// pass the table must hold nothing that pass processed, so that any row
|
||||
// still present belongs to an interrupted run and the sweep can key off
|
||||
// a bare row count.
|
||||
// Park unresolvable refs as status='failed' — parity with
|
||||
// resolveAndPersistBatched. Deleting them was wrong (#1240): a ref whose
|
||||
// own file never changes is otherwise gone forever, so when a DIFFERENT
|
||||
// file later gains the export/symbol that would satisfy it, no sync can
|
||||
// recreate the edge — only a full re-index. Failed rows are excluded from
|
||||
// the pending readers, which preserves the #1187 orphan sweep's
|
||||
// invariant in status form: after a COMPLETED pass nothing it processed
|
||||
// is still 'pending', so any pending row at rest belongs to an
|
||||
// interrupted run and the sweep can key off the pending count.
|
||||
if (result.unresolved.length > 0) {
|
||||
this.queries.deleteSpecificResolvedReferences(
|
||||
this.queries.markReferencesFailed(
|
||||
result.unresolved.map((r) => ({
|
||||
fromNodeId: r.fromNodeId,
|
||||
referenceName: r.referenceName,
|
||||
@@ -1032,6 +1046,49 @@ export class ReferenceResolver {
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Yielding counterpart of {@link resolveAndPersist} for a caller-supplied
|
||||
* ref list — used by sync's failed-ref retry pass (#1240). Same persistence
|
||||
* semantics: resolved refs become edges and their rows are deleted;
|
||||
* still-unresolvable refs are (re-)marked failed (a no-op for rows already
|
||||
* in that status). Yields per-ref because sync can run on the daemon's
|
||||
* liveness-watchdog thread (#850/#1091) and a retry set is unbounded when
|
||||
* a large edit lands many popular symbol names at once.
|
||||
*/
|
||||
async resolveAndPersistListYielding(refs: UnresolvedReference[]): Promise<ResolutionResult> {
|
||||
const maybeYield = createYielder();
|
||||
const result = await this.resolveBatchYielding(refs, maybeYield);
|
||||
|
||||
const PERSIST_CHUNK = 1000;
|
||||
const edges = this.createEdges(result.resolved);
|
||||
for (let i = 0; i < edges.length; i += PERSIST_CHUNK) {
|
||||
this.queries.insertEdges(edges.slice(i, i + PERSIST_CHUNK));
|
||||
await maybeYield();
|
||||
}
|
||||
|
||||
const resolvedKeys = result.resolved.map((r) => ({
|
||||
fromNodeId: r.original.fromNodeId,
|
||||
referenceName: r.original.referenceName,
|
||||
referenceKind: r.original.referenceKind,
|
||||
}));
|
||||
for (let i = 0; i < resolvedKeys.length; i += PERSIST_CHUNK) {
|
||||
this.queries.deleteSpecificResolvedReferences(resolvedKeys.slice(i, i + PERSIST_CHUNK));
|
||||
await maybeYield();
|
||||
}
|
||||
|
||||
const unresolvedKeys = result.unresolved.map((r) => ({
|
||||
fromNodeId: r.fromNodeId,
|
||||
referenceName: r.referenceName,
|
||||
referenceKind: r.referenceKind,
|
||||
}));
|
||||
for (let i = 0; i < unresolvedKeys.length; i += PERSIST_CHUNK) {
|
||||
this.queries.markReferencesFailed(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
|
||||
await maybeYield();
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Second resolution pass for chained static-factory / fluent calls whose
|
||||
* chained method is defined on a SUPERTYPE the receiver's type conforms to —
|
||||
@@ -1162,8 +1219,10 @@ export class ReferenceResolver {
|
||||
byMethod: {} as Record<string, number>,
|
||||
};
|
||||
|
||||
// Process in batches. We always read from offset 0 because resolved refs
|
||||
// are deleted after each batch, shifting the remaining rows forward.
|
||||
// Process in batches. We always read from offset 0 because every ref the
|
||||
// batch processed leaves the pending set (resolved rows are deleted,
|
||||
// unresolvable ones flip to status='failed'), shifting the remaining
|
||||
// pending rows forward.
|
||||
let prevRemaining = Number.POSITIVE_INFINITY;
|
||||
while (true) {
|
||||
const batch = this.queries.getUnresolvedReferencesBatch(0, batchSize);
|
||||
@@ -1198,14 +1257,17 @@ export class ReferenceResolver {
|
||||
await maybeYield();
|
||||
}
|
||||
|
||||
// Delete unresolvable refs from this batch to avoid re-processing them
|
||||
// Park unresolvable refs from this batch as status='failed' so they
|
||||
// leave the pending set (the batch reader and non-progress guard below
|
||||
// only see pending rows) but stay retryable when a later sync adds a
|
||||
// symbol that could satisfy them (#1240).
|
||||
const unresolvedKeys = result.unresolved.map((r) => ({
|
||||
fromNodeId: r.fromNodeId,
|
||||
referenceName: r.referenceName,
|
||||
referenceKind: r.referenceKind,
|
||||
}));
|
||||
for (let i = 0; i < unresolvedKeys.length; i += PERSIST_CHUNK) {
|
||||
this.queries.deleteSpecificResolvedReferences(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
|
||||
this.queries.markReferencesFailed(unresolvedKeys.slice(i, i + PERSIST_CHUNK));
|
||||
await maybeYield();
|
||||
}
|
||||
|
||||
@@ -1232,13 +1294,14 @@ export class ReferenceResolver {
|
||||
// The count-based guard below catches the true no-progress case.
|
||||
|
||||
// Non-progress guard (defense-in-depth). Because we re-read from offset 0
|
||||
// each pass, the unresolved_refs table MUST shrink every iteration — both
|
||||
// resolved and unresolved refs are deleted above. If it didn't shrink, a
|
||||
// each pass, the PENDING population MUST shrink every iteration — resolved
|
||||
// refs are deleted and unresolvable ones are marked failed above, and both
|
||||
// leave the pending set the batch reader sees. If it didn't shrink, a
|
||||
// resolver returned a match whose `original.referenceName` differs from the
|
||||
// stored row, so the keyed delete no-ops, and we'd re-read + re-resolve +
|
||||
// re-insert the same rows forever (the runaway that grew a 99-file repo to
|
||||
// 5M edges / 1.4 GB before the Go-fallback fix). Stop rather than grow the
|
||||
// graph without bound.
|
||||
// stored row, so the keyed delete/update no-ops, and we'd re-read +
|
||||
// re-resolve + re-insert the same rows forever (the runaway that grew a
|
||||
// 99-file repo to 5M edges / 1.4 GB before the Go-fallback fix). Stop
|
||||
// rather than grow the graph without bound.
|
||||
const remaining = this.queries.getUnresolvedReferencesCount();
|
||||
if (remaining >= prevRemaining) break;
|
||||
prevRemaining = remaining;
|
||||
|
||||
Reference in New Issue
Block a user