fix(sync): refresh the watcher's scope when codegraph.json or a .gitignore changes (#1590) (#1594)

Fixes #1590.

## What was wrong

The live file watcher built its scope matcher — built-in defaults + `.gitignore` + the `codegraph.json` `exclude`/`include` rules — once in `start()` and kept it for the watcher's lifetime. The MCP server is long-lived, so a `codegraph.json` created or edited after it started was invisible to the watcher, while `codegraph sync` (a fresh process with a fresh matcher) honoured it immediately. From the user's side: the CLI removed a newly excluded file, and the daemon re-indexed it a few seconds later, which reads as "`exclude` doesn't work". As the report points out, `extensions` on the very same config file *was* read live (its loader is mtime-cached), so two fields of one file behaved differently.

There was a second half to it. The watcher's scoped fast path hands the exact edited paths to sync, and that path stat'ed and re-parsed them without consulting the scope matcher at all — so the stale view of scope leaked straight into the index.

## What this does

**Watcher — rebuild on a scope change, then reconcile in full.** An event for the root `codegraph.json` or `.gitignore` rebuilds the matcher, marks the next sync as a full reconcile, and schedules it. A scope change has no per-file events: newly excluded files must be *removed* from the index and newly included ones *added*, and only the scan-diff (which builds its own fresh matcher) knows which those are. Two ordering details are deliberate:

- the two root files are checked *before* the matcher is consulted, so a user pattern that happens to cover them (`*.json`, `.*`) can't hide their own edits;
- a nested `.gitignore` (an embedded child repo's own rules, or a subdirectory rule the git-backed scan honours) is checked *after* the matcher, so the thousands of package-local `.gitignore`s an `npm install` writes under an ignored `node_modules/` can never trigger a rebuild storm.

Rebuilding runs embedded-repo discovery (one `git ls-files`), which is fine per config edit and never happens per event. Replacing the field serves both watch strategies: the recursive handler and the per-directory `shouldIgnoreDir` walk read it on every call.

**Scoped sync — re-check the paths it was handed.** The orchestrator now runs scoped paths through the same scope matcher and source-extension gate the full walk applies. An out-of-scope path is treated as absent: removed if tracked, never parsed on trust. The matcher is memoized on the mtimes of the two root files it derives from (two `stat`s per sync while nothing changed), so the scoped path keeps skipping O(repo) work — paying embedded-repo discovery per sync would defeat its whole point.

## Tests

- `watcher.test.ts` — a `codegraph.json` edit schedules a full sync, after which an edit inside the newly excluded tree is dropped by the live matcher (not pending, no sync) while an in-scope edit still syncs scoped; a root `.gitignore` edit behaves the same; a nested `.gitignore` forces a full sync; a `.gitignore` under `node_modules/` schedules nothing; dropping the exclude again readmits the tree.
- `sync.test.ts` — end-to-end through `CodeGraph`: a scoped sync of a path that `codegraph.json` now excludes removes it (`filesRemoved: 1`, nothing parsed — the symbol added to the file never appears), stays out on a repeat, and is re-added through the same scoped path once the exclude is dropped.
- All five new tests fail on `main`; the `node_modules` guard passes both ways as expected.
- Full suite: 189 files, 3184 passed / 9 skipped.
- CLI half of the issue's repro (init with `exclude`, edit the config + the file, `codegraph sync`): the newly excluded file is removed and its new symbol never enters the index.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
This commit is contained in:
Colby Mchenry
2026-08-26 10:38:38 -05:00
committed by GitHub
parent 838006c947
commit cf1b0e341a
5 changed files with 265 additions and 8 deletions
+54 -2
View File
@@ -27,7 +27,7 @@ import { StoreWriter, StoreBundle, finalizeStoreBundle } from './store-writer';
import { materializeKernelResult } from './kernel';
import { detectGeneratedFile } from './generated-detection';
import { detectLanguage, isSourceFile, isLanguageSupported, isFileLevelOnlyLanguage, initGrammars, loadGrammarsForLanguages, readGrammarWasmBytes } from './grammars';
import { loadExtensionOverrides, loadIncludeIgnoredPatterns, loadExcludePatterns, loadIncludePatterns } from '../project-config';
import { loadExtensionOverrides, loadIncludeIgnoredPatterns, loadExcludePatterns, loadIncludePatterns, PROJECT_CONFIG_FILENAME } from '../project-config';
import { isCodeGraphDataDir } from '../directory';
import { logDebug, logWarn } from '../errors';
import { validatePathWithinRoot, normalizePath } from '../utils';
@@ -1448,12 +1448,48 @@ export class ExtractionOrchestrator {
* hasn't run yet so single-file re-index paths can detect on the spot.
*/
private detectedFrameworkNames: string[] | null = null;
/**
* Scope matcher for SCOPED syncs, memoized on the mtimes of the two root
* files it is derived from (`codegraph.json`, `.gitignore`). See
* {@link scopedSyncMatcher}.
*/
private scopedMatcher: { key: string; matcher: ScopeIgnore } | null = null;
constructor(rootDir: string, queries: QueryBuilder) {
this.rootDir = rootDir;
this.queries = queries;
}
/**
* The scope matcher a scoped sync applies to the paths it was handed — the
* same `buildScopeIgnore` the full scan uses, so an explicitly-passed path
* that is OUT of scope (a user `exclude` in `codegraph.json`, a `.gitignore`
* rule, a built-in default) is treated exactly as the full walk would treat
* it: absent, hence removed if tracked, never parsed (#1590).
*
* Memoized on the root config + root `.gitignore` mtimes: building the
* matcher runs embedded-repo discovery (`git ls-files`), which would defeat
* the scoped path's whole point (skipping O(repo) work) if paid per sync.
* Two `stat`s per sync while nothing changed. An embedded repo created
* between config edits joins the scoped matcher on the next full sync, the
* same lifecycle the watcher's own matcher already has.
*/
private scopedSyncMatcher(): ScopeIgnore {
const key = [PROJECT_CONFIG_FILENAME, '.gitignore']
.map((name) => {
try {
return String(fs.statSync(path.join(this.rootDir, name)).mtimeMs);
} catch {
return '-';
}
})
.join('|');
if (this.scopedMatcher && this.scopedMatcher.key === key) return this.scopedMatcher.matcher;
const matcher = buildScopeIgnore(this.rootDir);
this.scopedMatcher = { key, matcher };
return matcher;
}
/**
* Build a filesystem-backed ResolutionContext sufficient for framework
* detection. Graph-query methods (getNodesByName etc.) return empty because
@@ -2700,7 +2736,23 @@ export class ExtractionOrchestrator {
// reads `filesChecked === 0 && durationMs === 0` as the
// lock-unavailable signature (#449).
const unique = [...new Set(scopedPaths)];
currentFiles = unique.filter((p) => fs.existsSync(path.join(this.rootDir, p)));
// A scoped path is "present" only if it exists AND is in scope — the
// same two gates the full walk applies (source extension, scope
// matcher). Without the scope gate a caller's stale view of scope
// leaked straight into the index: the watcher re-parsed a file the
// user had just excluded in `codegraph.json` while `codegraph sync`
// removed it (#1590). Out-of-scope paths fall out of `currentFiles`,
// so a tracked one takes the removal branch below, exactly as a full
// sync would treat it. (`include`-forced paths pass: ScopeIgnore
// applies the include precedence itself.)
const scope = this.scopedSyncMatcher();
const overrides = loadExtensionOverrides(this.rootDir);
currentFiles = unique.filter(
(p) =>
isSourceFile(p, overrides) &&
!scope.ignores(p) &&
fs.existsSync(path.join(this.rootDir, p))
);
trackedFiles = [];
for (const p of unique) {
const rec = this.queries.getFileByPath(p);