fix(resolution): sweep orphaned unresolved refs so an interrupted index heals on sync (#1187) (#1191)
An indexing run killed mid-"Resolving refs" (crash, Ctrl-C, the #1122 watchdog kill) left the refs it never reached parked in unresolved_refs. The git-scoped sync fast path only re-resolves changed files' refs, so those files' call edges were missing permanently — a too-small blast radius clustering by package/module (the #1187 field report: 3 of 10 caller files for a Spring @Resource-injected method) — until a full re-index. - sync() now sweeps leftover unresolved refs with the batched resolver after its scoped pass, including on no-change syncs, so a bare `codegraph sync` recovers a wedged index (and heals pre-fix indexes on the first post-upgrade sync) - the scoped pass deletes unresolvable rows too (parity with the batched path), making "rows at rest" a sound orphan signal - drop the batched loop's early break that abandoned all later batches when one batch was all-unresolvable (its rows WERE consumed — that early stop could orphan the rest of the table at init) - surface the state: `codegraph status` warns, `status --json` gains index.pendingRefs, and MCP codegraph_status tells agents the blast radius is incomplete until the next sync Verified end-to-end on a 2,414-file synthetic Spring repo: SIGKILL mid-resolution reproduces the reporter's exact 3-of-10-callers state; a bare sync now heals it to 10/10 with the edge count converging to the clean-init total; a healthy-index sync stays a no-op. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
7f325134e0
commit
4c15f84aa4
@@ -35,6 +35,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
### Fixes
|
||||
|
||||
- `codegraph init` and `codegraph index` no longer get killed by the safety watchdog at the "Resolving refs" step on large method-name-heavy codebases (big Java/enterprise monorepos were the main victims, especially on slower machines). Resolution used to come up for air only every 500 references, so a dense stretch of expensive ones could starve the watchdog long enough for it to assume the process was stuck and kill a perfectly healthy index. Resolution now checkpoints after every reference, and two of the expensive steps got much cheaper: repeated method lookups on the same type are now cached, and source files are no longer re-split line-by-line for every call being resolved — indexing such repos is several times faster as a result. Generated or minified single-line files are also skipped during receiver-type inference instead of being scanned per call. Thanks @UchihaYong and @wangmeng-95 for the reports. (#1122)
|
||||
- An index left incomplete by an interrupted run now heals itself on the next sync instead of silently staying wrong forever. If indexing died partway through resolving references (a crash, Ctrl-C, or the watchdog kill fixed above), the affected files still looked indexed but their caller/impact edges were missing — a too-small blast radius clustering by package or module, e.g. a Spring `@Resource`-injected method reporting 3 of its 10 real caller files — and because incremental syncs only re-resolve files that changed, the damage was permanent until a full re-index. Any sync (a watched file change, or a bare `codegraph sync`) now detects the leftover references and finishes resolving them, `codegraph status` warns when an index is in that state instead of passing it off as healthy, and a rare early-stop that could abandon resolution on repos whose first files reference only external libraries is fixed too. Thanks @KnifeOfLife for the report and the package-correlation observation that pinned it down. (#1187)
|
||||
- The automatic context hook for Claude Code now fires for structural questions asked in nearly thirty languages — French, Spanish, Portuguese, German, Italian, Dutch, Polish, Czech, Romanian, Hungarian, Greek, Swedish, Danish, Norwegian, Finnish, Russian, Ukrainian, Turkish, Indonesian, Vietnamese, Thai, Hindi, Arabic, Farsi, Hebrew, Japanese, Korean, and both simplified and traditional Chinese — instead of just English and simplified Chinese. Previously a natural question like "comment marche la state machine des commandes ?" injected nothing unless it happened to contain a code-shaped symbol name, making the hook look broken for non-English teams. English questions phrased with derived word forms ("explain the architecture…", "what are the dependencies…") now fire too, and prompts in any other language still fire when they name a symbol from the index. Thanks @anthonyle-roy-lgtm for the report. (#1126)
|
||||
- Lua and Luau method calls with capitalized names (`obj:Method()` — the standard Roblox convention) now link to the right method. Because Lua's method-call syntax looks identical to a Luau type annotation, a capitalized call like `lg:Log()` was misread as declaring the variable's type, so whenever two or more classes shared a method name (`Init`, `Update`, `Destroy`, …) the call was silently dropped from callers, impact/blast-radius, and flow traces. Lowercase method names were unaffected. Thanks @inth3shadows for the precise root-cause analysis and repro. (#1124)
|
||||
- Removed dead code left behind by the discontinued managed-reasoning feature. Its `codegraph login` flow was unplugged before ever shipping in a release, but the unused module still shipped inside the platform bundles, and a security review flagged its Windows browser-open step (it routed the login URL through `cmd`, which would have been unsafe had the flow ever been wired back up). The leftover module and its tests are now fully deleted. Thanks @inth3shadows for the report. (#1114)
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
/**
|
||||
* Orphaned unresolved-refs sweep (#1187)
|
||||
*
|
||||
* A resolution pass that dies mid-run (watchdog SIGKILL, Ctrl-C, crash)
|
||||
* leaves the refs it never reached in unresolved_refs. The git-scoped sync
|
||||
* fast path only ever reads the changed files' rows, so those orphans — and
|
||||
* the call edges they represent — used to be missing permanently until a
|
||||
* full re-index. Field report: a Spring monorepo where blast radius showed
|
||||
* 3 of 10 caller files for a method behind @Resource field injection.
|
||||
*
|
||||
* These tests pin the healing behavior: a completed pass consumes every row
|
||||
* it processes (resolved or not), and sync sweeps any leftovers even when
|
||||
* no files changed.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
|
||||
describe('Orphaned refs sweep (#1187)', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
|
||||
beforeEach(() => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-orphan-sweep-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (cg) {
|
||||
cg.destroy();
|
||||
}
|
||||
if (fs.existsSync(testDir)) {
|
||||
fs.rmSync(testDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
/** Distinct files with a `calls` edge into the node. */
|
||||
function callerFiles(target: { id: string }): string[] {
|
||||
return [...new Set(cg.getCallers(target.id).map((c) => c.node.filePath))].sort();
|
||||
}
|
||||
|
||||
/**
|
||||
* Leave `relPath` in the exact on-disk state a resolution pass killed
|
||||
* mid-run leaves behind: content re-extracted (nodes + refs re-inserted,
|
||||
* old edges cascade-deleted, content hash stamped current) but resolution
|
||||
* never run. The content tweak is needed because re-extraction of
|
||||
* byte-identical content is a no-op; the hash stamp means a later sync
|
||||
* sees NO changed files.
|
||||
*/
|
||||
async function interruptAfterExtraction(relPath: string): Promise<void> {
|
||||
fs.appendFileSync(path.join(testDir, relPath), '\n// interrupted-run edit\n');
|
||||
await cg.indexFiles([relPath]);
|
||||
}
|
||||
|
||||
function findMethod(name: string) {
|
||||
const hit = cg
|
||||
.searchNodes(name)
|
||||
.find((r) => (r.node.kind === 'method' || r.node.kind === 'function') && r.node.name === name);
|
||||
expect(hit, `expected an indexed definition of ${name}`).toBeDefined();
|
||||
return hit!.node;
|
||||
}
|
||||
|
||||
describe('sync() heals an interrupted resolution run', () => {
|
||||
beforeEach(async () => {
|
||||
// The #1187 shape: a concrete @Component class called through Spring
|
||||
// @Resource field injection from another package.
|
||||
const supportDir = path.join(testDir, 'src', 'support');
|
||||
const notifyDir = path.join(testDir, 'src', 'notify');
|
||||
fs.mkdirSync(supportDir, { recursive: true });
|
||||
fs.mkdirSync(notifyDir, { recursive: true });
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(supportDir, 'MemberDescriptionSupport.java'),
|
||||
[
|
||||
'package com.demo.support;',
|
||||
'',
|
||||
'public class MemberDescriptionSupport {',
|
||||
' public String getSuperVipName() {',
|
||||
' return "SVIP";',
|
||||
' }',
|
||||
'}',
|
||||
'',
|
||||
].join('\n')
|
||||
);
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(notifyDir, 'NotifyBuilder.java'),
|
||||
[
|
||||
'package com.demo.notify;',
|
||||
'',
|
||||
'import com.demo.support.MemberDescriptionSupport;',
|
||||
'',
|
||||
'public class NotifyBuilder {',
|
||||
' private MemberDescriptionSupport memberDescriptionSupport;',
|
||||
'',
|
||||
' public String buildParams() {',
|
||||
' return memberDescriptionSupport.getSuperVipName();',
|
||||
' }',
|
||||
'}',
|
||||
'',
|
||||
].join('\n')
|
||||
);
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
});
|
||||
|
||||
it('resolves leftover refs on a sync with NO file changes', async () => {
|
||||
const target = findMethod('getSuperVipName');
|
||||
|
||||
// Healthy baseline: the caller edge exists, no refs pending.
|
||||
expect(callerFiles(target)).toContain('src/notify/NotifyBuilder.java');
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
|
||||
// Simulate the interrupted run: re-extract the caller (cascade-deleting
|
||||
// its old nodes and edges, re-inserting its refs) and stop before
|
||||
// resolution — exactly the state a killed "Resolving refs" phase
|
||||
// leaves behind.
|
||||
await interruptAfterExtraction('src/notify/NotifyBuilder.java');
|
||||
expect(cg.getPendingReferenceCount()).toBeGreaterThan(0);
|
||||
expect(callerFiles(target)).not.toContain('src/notify/NotifyBuilder.java');
|
||||
|
||||
// The file on disk is unchanged, so this sync re-extracts nothing —
|
||||
// pre-fix it returned without touching resolution and the edge stayed
|
||||
// missing forever.
|
||||
const result = await cg.sync();
|
||||
expect(result.filesAdded).toBe(0);
|
||||
expect(result.filesModified).toBe(0);
|
||||
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
expect(callerFiles(target)).toContain('src/notify/NotifyBuilder.java');
|
||||
});
|
||||
|
||||
it('is idempotent: a second no-change sync stays clean', async () => {
|
||||
await interruptAfterExtraction('src/notify/NotifyBuilder.java');
|
||||
await cg.sync();
|
||||
const target = findMethod('getSuperVipName');
|
||||
const healed = callerFiles(target);
|
||||
|
||||
const again = await cg.sync();
|
||||
expect(again.filesAdded + again.filesModified + again.filesRemoved).toBe(0);
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
expect(callerFiles(target)).toEqual(healed);
|
||||
});
|
||||
});
|
||||
|
||||
describe('completed passes consume every processed row', () => {
|
||||
it('resolveReferences() deletes unresolvable rows (parity with the batched path)', async () => {
|
||||
const srcDir = path.join(testDir, 'src');
|
||||
fs.mkdirSync(srcDir, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(srcDir, 'app.ts'),
|
||||
[
|
||||
'export function helper() { return 1; }',
|
||||
'export function main() {',
|
||||
' helper();',
|
||||
' totallyUndefinedCall();', // resolves to nothing anywhere
|
||||
'}',
|
||||
'',
|
||||
].join('\n')
|
||||
);
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
|
||||
// Re-extract without resolving: both the resolvable helper() ref and
|
||||
// the unresolvable one are back in the table.
|
||||
await interruptAfterExtraction('src/app.ts');
|
||||
expect(cg.getPendingReferenceCount()).toBeGreaterThan(0);
|
||||
|
||||
// The non-batched full pass (which also backs the git-scoped sync
|
||||
// path) must consume BOTH: pre-fix it deleted only resolved rows, so
|
||||
// unresolvable ones parked forever and defeated the orphan sweep's
|
||||
// "non-empty table means interrupted run" invariant.
|
||||
cg.resolveReferences();
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
});
|
||||
|
||||
it('batched resolution does not stop at an all-unresolvable batch', async () => {
|
||||
const srcDir = path.join(testDir, 'src');
|
||||
fs.mkdirSync(srcDir, { recursive: true });
|
||||
// File A: only unresolvable refs. Extracted first, so its rows sort
|
||||
// first and fill the whole first batch.
|
||||
fs.writeFileSync(
|
||||
path.join(srcDir, 'a.ts'),
|
||||
[
|
||||
'export function a() {',
|
||||
' ghostOne();',
|
||||
' ghostTwo();',
|
||||
' ghostThree();',
|
||||
'}',
|
||||
'',
|
||||
].join('\n')
|
||||
);
|
||||
// File B: a resolvable ref whose rows sort after A's.
|
||||
fs.writeFileSync(
|
||||
path.join(srcDir, 'b.ts'),
|
||||
[
|
||||
"import { target } from './c';",
|
||||
'export function b() { target(); }',
|
||||
'',
|
||||
].join('\n')
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(srcDir, 'c.ts'),
|
||||
'export function target() { return 2; }\n'
|
||||
);
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
|
||||
// Re-queue A's refs then B's, in that order.
|
||||
await interruptAfterExtraction('src/a.ts');
|
||||
await interruptAfterExtraction('src/b.ts');
|
||||
expect(cg.getPendingReferenceCount()).toBeGreaterThan(0);
|
||||
|
||||
// Batch size 2 puts only A's unresolvable refs in the first batch.
|
||||
// The old early break ended the whole run there, leaving B's ref an
|
||||
// orphan even though the batch's rows WERE consumed (progress).
|
||||
const resolver = (cg as unknown as { resolver: { resolveAndPersistBatched(p?: unknown, b?: number): Promise<unknown> } }).resolver;
|
||||
await resolver.resolveAndPersistBatched(undefined, 2);
|
||||
|
||||
expect(cg.getPendingReferenceCount()).toBe(0);
|
||||
const target = findMethod('target');
|
||||
expect(callerFiles(target)).toContain('src/b.ts');
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -807,6 +807,9 @@ program
|
||||
const buildInfo = cg.getIndexBuildInfo();
|
||||
const reindexRecommended = cg.isIndexStale();
|
||||
const indexState = cg.getIndexState();
|
||||
// Zero on a healthy index; non-zero at rest means a resolution pass was
|
||||
// interrupted, so some files' call edges are missing (#1187).
|
||||
const pendingRefs = cg.getPendingReferenceCount();
|
||||
|
||||
// JSON output mode
|
||||
if (options.json) {
|
||||
@@ -842,6 +845,10 @@ program
|
||||
// (a run was killed mid-index — the index is truncated) |
|
||||
// 'failed' | null (predates the marker).
|
||||
state: indexState,
|
||||
// References awaiting resolution. Non-zero at rest means an
|
||||
// interrupted resolution pass left edges missing; the next
|
||||
// sync sweeps them (#1187).
|
||||
pendingRefs,
|
||||
},
|
||||
}));
|
||||
cg.destroy();
|
||||
@@ -862,6 +869,9 @@ program
|
||||
} else if (indexState === 'failed') {
|
||||
warn('The last index run failed — results may be incomplete. Re-run "codegraph index".');
|
||||
}
|
||||
if (pendingRefs > 0) {
|
||||
warn(`${formatNumber(pendingRefs)} references from an interrupted run are awaiting resolution — some callers/impact edges are missing. Run "codegraph sync" to resolve them.`);
|
||||
}
|
||||
console.log();
|
||||
|
||||
// Index stats
|
||||
|
||||
+44
-2
@@ -608,7 +608,8 @@ export class CodeGraph {
|
||||
}
|
||||
|
||||
// Resolve references if files were updated
|
||||
if (result.filesAdded > 0 || result.filesModified > 0) {
|
||||
const filesChanged = result.filesAdded > 0 || result.filesModified > 0;
|
||||
if (filesChanged) {
|
||||
if (result.changedFilePaths) {
|
||||
// Scope resolution to changed files (git fast path — bounded set)
|
||||
const unresolvedRefs = this.queries.getUnresolvedReferencesByFiles(result.changedFilePaths);
|
||||
@@ -644,7 +645,38 @@ export class CodeGraph {
|
||||
});
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Orphan sweep (#1187). A resolution pass that dies mid-run — the #850
|
||||
// daemon liveness watchdog's SIGKILL (#1122), Ctrl-C, a crash — leaves
|
||||
// the refs it never reached in unresolved_refs, and the git-scoped fast
|
||||
// path above never revisits them (it reads only the changed files'
|
||||
// rows). Those files' call edges were then missing PERMANENTLY, with
|
||||
// nothing to see except a too-small blast radius, until a full
|
||||
// re-index. A completed pass deletes every row it processed (resolved
|
||||
// or not), so any row still present now is such an orphan — or a row
|
||||
// parked by an older engine whose scoped pass kept unresolvable refs.
|
||||
// Grind them down with the batched resolver; this also makes a bare
|
||||
// `codegraph sync` the recovery command for a wedged index. On a
|
||||
// healthy index this is one COUNT query.
|
||||
const orphanCount = this.queries.getUnresolvedReferencesCount();
|
||||
if (orphanCount > 0) {
|
||||
options.onProgress?.({
|
||||
phase: 'resolving',
|
||||
current: 0,
|
||||
total: orphanCount,
|
||||
});
|
||||
|
||||
await this.resolveReferencesBatched((current, total) => {
|
||||
options.onProgress?.({
|
||||
phase: 'resolving',
|
||||
current,
|
||||
total,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (filesChanged || orphanCount > 0) {
|
||||
// Second pass: chained calls whose method lives on a supertype the
|
||||
// receiver conforms to (protocol-extension / inherited). Needs the
|
||||
// implements/extends edges built above (#750).
|
||||
@@ -655,7 +687,7 @@ export class CodeGraph {
|
||||
}
|
||||
|
||||
// Refresh planner stats + checkpoint the WAL after bulk writes.
|
||||
if (result.filesAdded > 0 || result.filesModified > 0 || result.filesRemoved > 0) {
|
||||
if (filesChanged || result.filesRemoved > 0 || orphanCount > 0) {
|
||||
this.db.runMaintenance();
|
||||
}
|
||||
|
||||
@@ -873,6 +905,16 @@ export class CodeGraph {
|
||||
return this.resolver.resolveAndPersistBatched(onProgress);
|
||||
}
|
||||
|
||||
/**
|
||||
* References extracted but not yet resolved into edges. Zero on a healthy
|
||||
* index — a completed resolution pass consumes every row. Non-zero at rest
|
||||
* means a pass was interrupted mid-run (killed indexer, crash — #1187), so
|
||||
* some files' call edges are missing; the next `sync` sweeps them.
|
||||
*/
|
||||
getPendingReferenceCount(): number {
|
||||
return this.queries.getUnresolvedReferencesCount();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get detected frameworks in the project
|
||||
*/
|
||||
|
||||
@@ -4038,6 +4038,19 @@ export class ToolHandler {
|
||||
);
|
||||
}
|
||||
|
||||
// Non-zero at rest means a resolution pass was interrupted mid-run, so
|
||||
// some files' call/impact edges are missing until the next sync sweeps
|
||||
// the leftovers (#1187). Surface it — an agent trusting an incomplete
|
||||
// blast radius is worse than one that knows to re-sync.
|
||||
const pendingRefs = cg.getPendingReferenceCount();
|
||||
if (pendingRefs > 0) {
|
||||
lines.push(
|
||||
`**Pending resolution:** ⚠ ${pendingRefs} references from an interrupted ` +
|
||||
`index run — some caller/impact edges are missing until the next sync ` +
|
||||
`(any file change triggers it, or run \`codegraph sync\`)`
|
||||
);
|
||||
}
|
||||
|
||||
lines.push('', '**Nodes by Kind:**');
|
||||
|
||||
for (const [kind, count] of Object.entries(stats.nodesByKind)) {
|
||||
|
||||
+24
-5
@@ -968,6 +968,23 @@ export class ReferenceResolver {
|
||||
);
|
||||
}
|
||||
|
||||
// Delete unresolvable refs too — parity with resolveAndPersistBatched.
|
||||
// Keeping them bought nothing: a ref is only ever retried when its file
|
||||
// is re-extracted, which cascade-deletes and re-inserts its rows anyway.
|
||||
// And it broke the #1187 orphan sweep's invariant — after a COMPLETED
|
||||
// pass the table must hold nothing that pass processed, so that any row
|
||||
// still present belongs to an interrupted run and the sweep can key off
|
||||
// a bare row count.
|
||||
if (result.unresolved.length > 0) {
|
||||
this.queries.deleteSpecificResolvedReferences(
|
||||
result.unresolved.map((r) => ({
|
||||
fromNodeId: r.fromNodeId,
|
||||
referenceName: r.referenceName,
|
||||
referenceKind: r.referenceKind,
|
||||
}))
|
||||
);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -1152,11 +1169,13 @@ export class ReferenceResolver {
|
||||
// Yield so progress UI can render between batches
|
||||
await new Promise(resolve => setImmediate(resolve));
|
||||
|
||||
// If nothing was resolved or removed in this batch, we'd loop forever
|
||||
// on the same rows. Break to avoid infinite loop.
|
||||
if (result.resolved.length === 0 && result.unresolved.length === batch.length) {
|
||||
break;
|
||||
}
|
||||
// NOTE: there used to be an extra early break here when a batch resolved
|
||||
// nothing (`result.unresolved.length === batch.length`). That was wrong:
|
||||
// an all-unresolvable batch still DELETES its rows (progress), yet the
|
||||
// break abandoned every batch after it in the same run — on a repo whose
|
||||
// first 5000 refs are all external/stdlib calls, resolution stopped at
|
||||
// batch one and left the rest of the table as permanent orphans (#1187).
|
||||
// The count-based guard below catches the true no-progress case.
|
||||
|
||||
// Non-progress guard (defense-in-depth). Because we re-read from offset 0
|
||||
// each pass, the unresolved_refs table MUST shrink every iteration — both
|
||||
|
||||
Reference in New Issue
Block a user