Fixes #1590. ## What was wrong The live file watcher built its scope matcher — built-in defaults + `.gitignore` + the `codegraph.json` `exclude`/`include` rules — once in `start()` and kept it for the watcher's lifetime. The MCP server is long-lived, so a `codegraph.json` created or edited after it started was invisible to the watcher, while `codegraph sync` (a fresh process with a fresh matcher) honoured it immediately. From the user's side: the CLI removed a newly excluded file, and the daemon re-indexed it a few seconds later, which reads as "`exclude` doesn't work". As the report points out, `extensions` on the very same config file *was* read live (its loader is mtime-cached), so two fields of one file behaved differently. There was a second half to it. The watcher's scoped fast path hands the exact edited paths to sync, and that path stat'ed and re-parsed them without consulting the scope matcher at all — so the stale view of scope leaked straight into the index. ## What this does **Watcher — rebuild on a scope change, then reconcile in full.** An event for the root `codegraph.json` or `.gitignore` rebuilds the matcher, marks the next sync as a full reconcile, and schedules it. A scope change has no per-file events: newly excluded files must be *removed* from the index and newly included ones *added*, and only the scan-diff (which builds its own fresh matcher) knows which those are. Two ordering details are deliberate: - the two root files are checked *before* the matcher is consulted, so a user pattern that happens to cover them (`*.json`, `.*`) can't hide their own edits; - a nested `.gitignore` (an embedded child repo's own rules, or a subdirectory rule the git-backed scan honours) is checked *after* the matcher, so the thousands of package-local `.gitignore`s an `npm install` writes under an ignored `node_modules/` can never trigger a rebuild storm. Rebuilding runs embedded-repo discovery (one `git ls-files`), which is fine per config edit and never happens per event. Replacing the field serves both watch strategies: the recursive handler and the per-directory `shouldIgnoreDir` walk read it on every call. **Scoped sync — re-check the paths it was handed.** The orchestrator now runs scoped paths through the same scope matcher and source-extension gate the full walk applies. An out-of-scope path is treated as absent: removed if tracked, never parsed on trust. The matcher is memoized on the mtimes of the two root files it derives from (two `stat`s per sync while nothing changed), so the scoped path keeps skipping O(repo) work — paying embedded-repo discovery per sync would defeat its whole point. ## Tests - `watcher.test.ts` — a `codegraph.json` edit schedules a full sync, after which an edit inside the newly excluded tree is dropped by the live matcher (not pending, no sync) while an in-scope edit still syncs scoped; a root `.gitignore` edit behaves the same; a nested `.gitignore` forces a full sync; a `.gitignore` under `node_modules/` schedules nothing; dropping the exclude again readmits the tree. - `sync.test.ts` — end-to-end through `CodeGraph`: a scoped sync of a path that `codegraph.json` now excludes removes it (`filesRemoved: 1`, nothing parsed — the symbol added to the file never appears), stays out on a repeat, and is re-added through the same scoped path once the exclude is dropped. - All five new tests fail on `main`; the `node_modules` guard passes both ways as expected. - Full suite: 189 files, 3184 passed / 9 skipped. - CLI half of the issue's repro (init with `exclude`, edit the config + the file, `codegraph sync`): the newly excluded file is removed and its new symbol never enters the index. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01LxZj6W6Y1SHXwvpT3uwJpK
This commit is contained in:
+53
-6
@@ -34,7 +34,7 @@
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { isSourceFile, buildScopeIgnore, type ScopeIgnore } from '../extraction';
|
||||
import { loadExtensionOverrides } from '../project-config';
|
||||
import { loadExtensionOverrides, PROJECT_CONFIG_FILENAME } from '../project-config';
|
||||
import { logDebug, logWarn } from '../errors';
|
||||
import { normalizePath } from '../utils';
|
||||
import { isCodeGraphDataDir } from '../directory';
|
||||
@@ -328,11 +328,13 @@ export class FileWatcher {
|
||||
* deterministically gate on watcher readiness.
|
||||
*/
|
||||
private readyWaiters: Array<() => void> = [];
|
||||
// The shared scope matcher (built-in defaults + project .gitignore, with
|
||||
// embedded child repos matched by their OWN rules — #514), built once at
|
||||
// start(). Same source of truth the indexer uses, so watcher scope can
|
||||
// never diverge from index scope. An embedded repo created after start()
|
||||
// joins the scope on the next watcher restart / re-index.
|
||||
// The shared scope matcher (built-in defaults + project .gitignore + the
|
||||
// `codegraph.json` exclude/include rules, with embedded child repos matched
|
||||
// by their OWN rules — #514), built at start() and REBUILT whenever one of
|
||||
// the files it is derived from changes (see `refreshScope`, #1590). Same
|
||||
// source of truth the indexer uses, so watcher scope can never diverge from
|
||||
// index scope. An embedded repo created after start() joins the scope on
|
||||
// the next scope refresh / watcher restart / re-index.
|
||||
private ignoreMatcher: ScopeIgnore | null = null;
|
||||
|
||||
private readonly projectRoot: string;
|
||||
@@ -573,7 +575,24 @@ export class FileWatcher {
|
||||
private handleChange(rel: string): void {
|
||||
if (!rel || rel === '.' || rel.startsWith('..')) return;
|
||||
if (this.isAlwaysIgnored(rel)) return;
|
||||
// The two root files the scope matcher is derived from are handled BEFORE
|
||||
// the matcher is consulted: a user `exclude` pattern that happens to cover
|
||||
// them (`*.json`, `.*`) must not be able to hide their own edits (#1590).
|
||||
if (rel === PROJECT_CONFIG_FILENAME || rel === '.gitignore') {
|
||||
this.refreshScope(rel);
|
||||
return;
|
||||
}
|
||||
if (this.ignoreMatcher && this.ignoreMatcher.ignores(rel)) return;
|
||||
// A nested `.gitignore` (an embedded child repo's own rules, #514, or a
|
||||
// subdirectory rule the git-backed full scan honors) is only a scope
|
||||
// change when it sits INSIDE the current scope — checked after the matcher
|
||||
// on purpose, so the thousands of package-local `.gitignore`s an
|
||||
// `npm install` writes under an ignored `node_modules/` never trigger a
|
||||
// rebuild storm.
|
||||
if (rel.endsWith('/.gitignore')) {
|
||||
this.refreshScope(rel);
|
||||
return;
|
||||
}
|
||||
if (!isSourceFile(rel, loadExtensionOverrides(this.projectRoot))) {
|
||||
this.maybeScheduleForRemovedDir(rel);
|
||||
return;
|
||||
@@ -591,6 +610,34 @@ export class FileWatcher {
|
||||
this.scheduleSync();
|
||||
}
|
||||
|
||||
/**
|
||||
* A scope-defining file changed (`codegraph.json`, a `.gitignore`): rebuild
|
||||
* the ignore matcher and make the next sync a FULL reconcile (#1590).
|
||||
*
|
||||
* The matcher used to be built once in `start()` and kept for the watcher's
|
||||
* lifetime — in a long-lived MCP daemon that meant a `codegraph.json`
|
||||
* created or edited after startup was invisible to the live watcher, while
|
||||
* `codegraph sync` (a fresh process) honoured it immediately: the CLI
|
||||
* removed a newly excluded file and the watcher re-added it seconds later.
|
||||
* `loadExtensionOverrides()` on the same filter line was already read live
|
||||
* (mtime-cached), so two fields of the same config file disagreed.
|
||||
*
|
||||
* Rebuilding costs one `git ls-files` pass (embedded-repo discovery), which
|
||||
* is fine per config edit — never per event. Replacing the field is enough
|
||||
* for both strategies: the recursive handler and the per-directory
|
||||
* `shouldIgnoreDir` walk read `this.ignoreMatcher` on every call. The full
|
||||
* scan is required because a scope change has no per-file events: newly
|
||||
* excluded files must be REMOVED from the index and newly included ones
|
||||
* added, and only the scan-diff (which builds its own fresh matcher) knows
|
||||
* which those are.
|
||||
*/
|
||||
private refreshScope(rel: string): void {
|
||||
logDebug('Scope config changed; rebuilding watcher scope', { file: rel });
|
||||
this.ignoreMatcher = buildScopeIgnore(this.projectRoot);
|
||||
this.needsFullScan = true;
|
||||
this.scheduleSync();
|
||||
}
|
||||
|
||||
/**
|
||||
* A deleted DIRECTORY arrives as one event on the directory's own path —
|
||||
* no source extension, so the source-file filter drops it, and the files
|
||||
|
||||
Reference in New Issue
Block a user