fix(init): surface and offer to opt in gitignored child repos on an empty index (#1156) (#1208)

A Git super-repo whose `.gitignore` excludes its child repositories indexed
~nothing at the parent: CodeGraph respects `.gitignore` by default (#970,
#1065), so the excluded children were skipped and `codegraph init` printed
"Done" with 0 nodes — even though `init` inside each child worked fine. The
empty index was silent and unexplained.

`init`/`index` now detect the gitignored child repos they skipped when an
index comes up empty of symbols, name them, and — in an interactive terminal
— offer to index them (writing an `includeIgnored` entry to codegraph.json and
re-indexing on the spot); non-interactive runs print the exact codegraph.json
snippet to add. Gated on nodesCreated === 0, so a project that deliberately
keeps gitignored reference clones out of a working index is never nagged.

- extraction: findUnindexedIgnoredRepos — the inverse of discoverEmbeddedRepoRoots
  (bounded, skips default-ignored dirs, respects existing includeIgnored)
- project-config: addIncludeIgnoredPatterns — create/merge codegraph.json,
  idempotent, refuses to clobber malformed JSON
- cli: wire the detect-name-offer flow into both `init` and `index`
- tests: +13 covering detection, config writing, and the no-nag gate

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-07 13:01:38 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent a9e8fa48a1
commit e65a39746c
6 changed files with 345 additions and 30 deletions
+1
View File
@@ -36,6 +36,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Fixes
- `codegraph init` at a parent repository whose `.gitignore` excludes its child repositories no longer silently indexes nothing and reports success. The "super-repo of gitignored child repos" layout — a top-level Git repo that `.gitignore`s each `service-*/` or `packages/*` child so `git status` stays quiet — used to index only the parent's few top-level files and print "Done" with 0 nodes, even though running `codegraph init` inside any child worked fine (CodeGraph respects `.gitignore` by default, so the excluded children were skipped). Now, when an index comes up empty, CodeGraph detects the gitignored child repositories that were skipped, names them, and — in an interactive terminal — offers to index them (writing an `includeIgnored` entry to `codegraph.json` and re-indexing on the spot); non-interactive runs print the exact `codegraph.json` snippet to add. Projects that legitimately keep gitignored reference clones out of a working index are never nagged: the offer only appears when the index would otherwise be empty. Thanks @small-thanks for the report. (#1156)
- The MyBatis mapper reader is sturdier on real-world XML. Single-quoted attribute values (`id='getById'`, legal XML and common in older mappers) are no longer skipped, so those statements make it into the graph. Statements and `<include>`s that were commented out with `<!-- ... -->` no longer produce phantom symbols. And two vendor-split statements — the same `id` with `databaseId="oracle"` / `databaseId="mysql"` — written on a single line no longer silently drop one of the pair. Thanks @ESPINS for the report, the reproductions, and the fixes. (#1182)
- `codegraph init` and `codegraph index` no longer get killed by the safety watchdog at the "Resolving refs" step on large method-name-heavy codebases (big Java/enterprise monorepos were the main victims, especially on slower machines). Resolution used to come up for air only every 500 references, so a dense stretch of expensive ones could starve the watchdog long enough for it to assume the process was stuck and kill a perfectly healthy index. Resolution now checkpoints after every reference, and two of the expensive steps got much cheaper: repeated method lookups on the same type are now cached, and source files are no longer re-split line-by-line for every call being resolved — indexing such repos is several times faster as a result. Generated or minified single-line files are also skipped during receiver-type inference instead of being scanned per call. Thanks @UchihaYong and @wangmeng-95 for the reports. (#1122)
- An index left incomplete by an interrupted run now heals itself on the next sync instead of silently staying wrong forever. If indexing died partway through resolving references (a crash, Ctrl-C, or the watchdog kill fixed above), the affected files still looked indexed but their caller/impact edges were missing — a too-small blast radius clustering by package or module, e.g. a Spring `@Resource`-injected method reporting 3 of its 10 real caller files — and because incremental syncs only re-resolve files that changed, the damage was permanent until a full re-index. Any sync (a watched file change, or a bare `codegraph sync`) now detects the leftover references and finishes resolving them, `codegraph status` warns when an index is in that state instead of passing it off as healthy, and a rare early-stop that could abandon resolution on repos whose first files reference only external libraries is fixed too. Thanks @KnifeOfLife for the report and the package-correlation observation that pinned it down. (#1187)
+56 -1
View File
@@ -14,7 +14,7 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import * as fs from 'node:fs';
import * as path from 'node:path';
import * as os from 'node:os';
import { loadIncludeIgnoredPatterns, loadExtensionOverrides, clearProjectConfigCache } from '../src/project-config';
import { loadIncludeIgnoredPatterns, loadExtensionOverrides, clearProjectConfigCache, addIncludeIgnoredPatterns } from '../src/project-config';
describe('includeIgnored loader (codegraph.json)', () => {
let dir: string;
@@ -87,3 +87,58 @@ describe('includeIgnored loader (codegraph.json)', () => {
expect(loadIncludeIgnoredPatterns(dir)).toEqual([]);
});
});
describe('addIncludeIgnoredPatterns (codegraph.json writer, #1156)', () => {
let dir: string;
beforeEach(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-addincludeignored-'));
clearProjectConfigCache();
});
afterEach(() => {
clearProjectConfigCache();
fs.rmSync(dir, { recursive: true, force: true });
});
const readConfig = () => JSON.parse(fs.readFileSync(path.join(dir, 'codegraph.json'), 'utf-8'));
it('creates codegraph.json when none exists', () => {
expect(addIncludeIgnoredPatterns(dir, ['mtc-a/', 'mtc-b/'])).toBe(2);
expect(loadIncludeIgnoredPatterns(dir)).toEqual(['mtc-a/', 'mtc-b/']);
});
it('merges into an existing list, preserving other keys and de-duping', () => {
fs.writeFileSync(
path.join(dir, 'codegraph.json'),
JSON.stringify({ extensions: { '.foo': 'typescript' }, includeIgnored: ['mtc-a/'] }),
);
expect(addIncludeIgnoredPatterns(dir, ['mtc-a/', 'mtc-b/'])).toBe(1); // only mtc-b/ is new
const parsed = readConfig();
expect(parsed.includeIgnored).toEqual(['mtc-a/', 'mtc-b/']);
expect(parsed.extensions).toEqual({ '.foo': 'typescript' }); // untouched
});
it('is idempotent — re-adding the same patterns adds nothing', () => {
addIncludeIgnoredPatterns(dir, ['mtc-a/']);
expect(addIncludeIgnoredPatterns(dir, ['mtc-a/'])).toBe(0);
expect(loadIncludeIgnoredPatterns(dir)).toEqual(['mtc-a/']);
});
it('replaces a non-array includeIgnored value rather than crashing', () => {
fs.writeFileSync(path.join(dir, 'codegraph.json'), JSON.stringify({ includeIgnored: 'oops' }));
expect(addIncludeIgnoredPatterns(dir, ['mtc-a/'])).toBe(1);
expect(loadIncludeIgnoredPatterns(dir)).toEqual(['mtc-a/']);
});
it('refuses to clobber a malformed existing codegraph.json (throws, leaves file intact)', () => {
const bad = '{ not: valid json ';
fs.writeFileSync(path.join(dir, 'codegraph.json'), bad);
expect(() => addIncludeIgnoredPatterns(dir, ['mtc-a/'])).toThrow();
expect(fs.readFileSync(path.join(dir, 'codegraph.json'), 'utf-8')).toBe(bad);
});
it('writes pretty-printed, newline-terminated JSON', () => {
addIncludeIgnoredPatterns(dir, ['mtc-a/']);
const raw = fs.readFileSync(path.join(dir, 'codegraph.json'), 'utf-8');
expect(raw.endsWith('\n')).toBe(true);
expect(raw).toContain('\n "includeIgnored"'); // 2-space indent
});
});
+78 -1
View File
@@ -26,7 +26,7 @@ import * as path from 'path';
import * as os from 'os';
import { execFileSync } from 'child_process';
import CodeGraph from '../src/index';
import { scanDirectory, buildScopeIgnore, discoverEmbeddedRepoRoots } from '../src/extraction';
import { scanDirectory, buildScopeIgnore, discoverEmbeddedRepoRoots, findUnindexedIgnoredRepos } from '../src/extraction';
import { clearProjectConfigCache } from '../src/project-config';
function git(cwd: string, ...args: string[]): void {
@@ -361,4 +361,81 @@ describe('multi-repo workspaces (#514) + .gitignore-respect default (#970, #976)
expect(discoverEmbeddedRepoRoots(child)).toEqual([]);
});
});
describe('findUnindexedIgnoredRepos: the skipped-child-repos hint (#1156)', () => {
// The reported layout: a super-repo whose `.gitignore` excludes its child
// repos, so `init` at the parent correctly indexes ~nothing. This detector
// is the inverse of `discoverEmbeddedRepoRoots` — it names exactly the repos
// the default scan skipped so the CLI can offer to opt them in.
it('names the gitignored child repos a default index skipped', () => {
write(path.join(ws, 'mtc-activity/src/a.ts'), 'export const a = 1;\n');
write(path.join(ws, 'mtc-admin/src/b.ts'), 'export const b = 2;\n');
makeRepo(path.join(ws, 'mtc-activity'));
makeRepo(path.join(ws, 'mtc-admin'));
write(path.join(ws, '.gitignore'), 'mtc-*/\n');
write(path.join(ws, 'AGENTS.md'), '# docs\n');
makeRepo(ws);
// Nothing of the child repos indexes by default — the symptom being fixed.
expect(scanDirectory(ws).some((f) => f.startsWith('mtc-'))).toBe(false);
// ...but the detector names them (trailing-slashed, valid includeIgnored patterns).
expect(findUnindexedIgnoredRepos(ws).sort()).toEqual(['mtc-activity/', 'mtc-admin/']);
});
it('excludes repos already opted in via includeIgnored (only the rest remain)', () => {
write(path.join(ws, 'mtc-activity/src/a.ts'), 'export const a = 1;\n');
write(path.join(ws, 'mtc-admin/src/b.ts'), 'export const b = 2;\n');
makeRepo(path.join(ws, 'mtc-activity'));
makeRepo(path.join(ws, 'mtc-admin'));
write(path.join(ws, '.gitignore'), 'mtc-*/\n');
writeConfig({ includeIgnored: ['mtc-activity/'] });
makeRepo(ws);
expect(findUnindexedIgnoredRepos(ws)).toEqual(['mtc-admin/']);
});
it('returns [] when every gitignored repo is already opted in (nothing to nag)', () => {
write(path.join(ws, 'pkgs/a/src/a.ts'), 'export const a = 1;\n');
makeRepo(path.join(ws, 'pkgs/a'));
write(path.join(ws, '.gitignore'), '/pkgs/\n');
writeConfig({ includeIgnored: ['pkgs/'] });
makeRepo(ws);
expect(findUnindexedIgnoredRepos(ws)).toEqual([]);
});
it('does NOT report nested repos that are NOT gitignored (they already index)', () => {
// Scenario A: an untracked, non-ignored nested repo is indexed via the
// untracked-embedded path, so there is nothing to hint about.
write(path.join(ws, 'sub/src/a.ts'), 'export const a = 1;\n');
makeRepo(path.join(ws, 'sub'));
write(path.join(ws, 'app.ts'), 'export const app = 0;\n');
makeRepo(ws); // sub/ stays untracked, not ignored
expect(findUnindexedIgnoredRepos(ws)).toEqual([]);
});
it('skips a gitignored node_modules even when it holds a git repo', () => {
write(path.join(ws, 'node_modules/dep/index.js'), 'module.exports = 1;\n');
makeRepo(path.join(ws, 'node_modules/dep'));
write(path.join(ws, '.gitignore'), 'node_modules/\n');
makeRepo(ws);
expect(findUnindexedIgnoredRepos(ws)).toEqual([]);
});
it('finds repos nested inside a gitignored data dir, not just top-level ones', () => {
write(path.join(ws, 'refs/lib-a/x.ts'), 'export const x = 1;\n');
makeRepo(path.join(ws, 'refs/lib-a'));
write(path.join(ws, '.gitignore'), '/refs/\n');
makeRepo(ws);
expect(findUnindexedIgnoredRepos(ws)).toEqual(['refs/lib-a/']);
});
it('returns [] for a non-git directory', () => {
write(path.join(ws, 'a.ts'), 'export const a = 1;\n'); // no git init at all
expect(findUnindexedIgnoredRepos(ws)).toEqual([]);
});
});
});
+116 -28
View File
@@ -412,6 +412,82 @@ function printIndexResult(clack: typeof import('@clack/prompts'), result: IndexR
}
}
/**
* When an `init`/`index` produced an EMPTY graph and the reason is that the
* project's own `.gitignore` excludes nested git repositories the "super-repo
* gitignores its child repos" layout (#1156), where `init` at the parent
* correctly indexes ~nothing while `init` inside each child works name those
* repos and offer to index them. An interactive terminal gets a yes/no prompt
* that writes `includeIgnored` to codegraph.json and re-indexes; a
* non-interactive run just prints the one-line opt-in snippet. The caller gates
* this on `nodesCreated === 0`, so a project that DID index real content is
* never nagged about the gitignored reference clones it deliberately keeps out
* (#970, #1065). Best-effort throughout: detection never breaks the command.
*/
async function offerIndexIgnoredRepos(
clack: typeof import('@clack/prompts'),
projectPath: string,
reindex: () => Promise<IndexResult>,
opts: { interactive: boolean },
): Promise<IndexResult | undefined> {
let repos: string[];
try {
const { findUnindexedIgnoredRepos } = await import('../extraction');
repos = findUnindexedIgnoredRepos(projectPath);
} catch {
return; // detection is advisory — never let it break the command
}
if (repos.length === 0) return;
const { PROJECT_CONFIG_FILENAME } = await import('../project-config');
const isOne = repos.length === 1;
const SHOWN = 6;
const names = repos.slice(0, SHOWN).map((r) => r.replace(/\/$/, ''));
const extra = repos.length > SHOWN ? ` (+${formatNumber(repos.length - SHOWN)} more)` : '';
const snippet = `{ "includeIgnored": [${repos.map((p) => JSON.stringify(p)).join(', ')}] }`;
clack.log.warn(
`Your .gitignore excludes ${isOne ? 'a nested git repository' : `${formatNumber(repos.length)} nested git repositories`} here, ` +
`so ${isOne ? 'it was' : 'they were'} not indexed: ${names.join(', ')}${extra}.`,
);
const manualHint = () => {
clack.log.info(
`If ${isOne ? "it's" : "they're"} your code, add ${isOne ? 'it' : 'them'} to ${PROJECT_CONFIG_FILENAME} and re-index:`,
);
clack.log.info(` ${snippet}`);
};
if (!opts.interactive || !process.stdin.isTTY) {
manualHint();
return;
}
const yes = await clack.confirm({
message: `Index ${isOne ? 'it' : `these ${formatNumber(repos.length)}`} now? Adds ${isOne ? 'it' : 'them'} to ${PROJECT_CONFIG_FILENAME}.`,
initialValue: true,
});
if (clack.isCancel(yes) || !yes) {
manualHint();
return;
}
let added: number;
try {
const { addIncludeIgnoredPatterns } = await import('../project-config');
added = addIncludeIgnoredPatterns(projectPath, repos);
} catch (err) {
clack.log.error(`Could not update ${PROJECT_CONFIG_FILENAME}: ${err instanceof Error ? err.message : String(err)}`);
manualHint();
return;
}
clack.log.success(`Added ${formatNumber(added)} ${added === 1 ? 'entry' : 'entries'} to ${PROJECT_CONFIG_FILENAME} ${getGlyphs().dash} re-indexing…`);
const result = await reindex();
printIndexResult(clack, result, projectPath);
return result;
}
/**
* Write detailed error log to .codegraph/errors.log
*/
@@ -522,28 +598,34 @@ program
// accepted (so existing muscle memory and scripts don't break) but is a
// no-op — initializing always builds the initial index.
// Supervise the index: self-terminate if orphaned or wedged (#999).
const supervision = installCommandSupervision('init');
let result: IndexResult;
try {
if (options.verbose) {
result = await cg.indexAll({
onProgress: createVerboseProgress(),
verbose: true,
});
} else {
// A closure so we can re-run the exact same supervised, progress-rendered
// index if the user opts gitignored child repos in below (#1156).
const runIndex = async (): Promise<IndexResult> => {
const supervision = installCommandSupervision('init');
try {
if (options.verbose) {
return await cg.indexAll({ onProgress: createVerboseProgress(), verbose: true });
}
process.stdout.write(`${colors.dim}${getGlyphs().rail}${colors.reset}\n`);
const progress = createShimmerProgress();
result = await cg.indexAll({
onProgress: progress.onProgress,
});
const r = await cg.indexAll({ onProgress: progress.onProgress });
await progress.stop();
return r;
} finally {
supervision.stop();
}
} finally {
supervision.stop();
}
};
const result = await runIndex();
printIndexResult(clack, result, projectPath);
await recordIndexTelemetry(cg, result);
// An empty graph at a git super-repo usually means `.gitignore` excludes
// the child repos that hold the code — surface them and offer to opt in
// rather than leaving the user with a silent 0-node "Done". (#1156)
if (result.nodesCreated === 0) {
await offerIndexIgnoredRepos(clack, projectPath, runIndex, { interactive: true });
}
try {
const { offerWatchFallback } = await import('../installer');
await offerWatchFallback(clack, projectPath);
@@ -672,26 +754,32 @@ program
const clack = await importESM('@clack/prompts');
clack.intro('Indexing project');
let result: IndexResult;
if (options.verbose) {
result = await cg.indexAll({
onProgress: createVerboseProgress(),
verbose: true,
});
} else {
// A closure so a re-index (after opting gitignored child repos in, #1156)
// renders identically. Supervision already wraps the whole command.
const renderIndex = async (): Promise<IndexResult> => {
if (options.verbose) {
return await cg.indexAll({ onProgress: createVerboseProgress(), verbose: true });
}
process.stdout.write(`${colors.dim}${getGlyphs().rail}${colors.reset}\n`);
const progress = createShimmerProgress();
result = await cg.indexAll({
onProgress: progress.onProgress,
});
const r = await cg.indexAll({ onProgress: progress.onProgress });
await progress.stop();
}
return r;
};
const result = await renderIndex();
printIndexResult(clack, result, projectPath);
await recordIndexTelemetry(cg, result);
if (!result.success) {
// Empty graph at a git super-repo → likely `.gitignore`d child repos;
// name them and offer to opt in instead of a silent 0-node result (#1156).
let finalResult = result;
if (result.nodesCreated === 0) {
finalResult = (await offerIndexIgnoredRepos(clack, projectPath, renderIndex, { interactive: true })) ?? result;
}
if (!finalResult.success) {
process.exit(1);
}
+42
View File
@@ -798,6 +798,48 @@ export function discoverEmbeddedRepoRoots(rootDir: string): string[] {
return out;
}
/**
* Cap on how many skipped gitignored repos the CLI hint enumerates a huge
* gitignored data dir full of clones must never turn the hint scan into a long
* walk. Enough to make the point; the caller says "+N more" past this.
*/
const UNINDEXED_IGNORED_REPO_HINT_CAP = 100;
/**
* The INVERSE of the gitignored side of {@link discoverEmbeddedRepoRoots}:
* nested git repositories under a gitignored directory that the project has NOT
* opted into via `codegraph.json` `includeIgnored`. These are real repos the
* default `init`/`index` deliberately skips because `.gitignore` excludes them
* (#970, #976) most visibly the "super-repo `.gitignore`s its child repos"
* layout (#1156), where `init` at the parent correctly indexes ~nothing while
* `init` inside each child works. The CLI uses this to turn that silent empty
* index into an actionable hint: it names the skipped repos and offers to opt
* them in. Paths are `rootDir`-relative and trailing-slashed (valid
* `includeIgnored` patterns as-is). Returns `[]` for a non-git root (a
* filesystem walk already descends into nested repos there), skips built-in
* default-ignored dirs (`node_modules`, ), and is bounded so it never stalls
* on a giant ignored tree.
*/
export function findUnindexedIgnoredRepos(rootDir: string): string[] {
try {
execFileSync('git', ['rev-parse', '--git-dir'], { cwd: rootDir, encoding: 'utf-8', timeout: 5000, stdio: ['pipe', 'pipe', 'pipe'], windowsHide: true });
} catch {
return [];
}
const defaults = defaultsOnlyIgnore();
const includeIgnored = loadIncludeIgnoredMatcher(rootDir);
const repos: string[] = [];
for (const dir of listIgnoredDirs(rootDir)) {
if (defaults.ignores(dir)) continue; // node_modules etc. — never project code
if (includeIgnored?.ignores(normalizePath(dir))) continue; // already opted in — nothing to nag about
for (const repo of findNestedGitRepos(path.join(rootDir, dir), dir)) {
repos.push(repo);
if (repos.length >= UNINDEXED_IGNORED_REPO_HINT_CAP) return repos;
}
}
return repos;
}
/**
* Discover embedded repos hidden by `repoDir`'s OWN gitignore rules: for each
* gitignored directory, search for nested `.git` roots. Returns repo paths
+52
View File
@@ -342,3 +342,55 @@ export function loadIncludePatterns(rootDir: string): string[] {
export function clearProjectConfigCache(): void {
cache.clear();
}
/**
* Add gitignore-style patterns to a project's `codegraph.json` `includeIgnored`
* list, creating the file if absent and preserving every other key. Used by the
* CLI to opt a "super-repo of gitignored child repos" (#1156) into the index on
* the user's say-so. Returns the count of patterns actually ADDED (ones already
* present are skipped, so a re-run is idempotent).
*
* A plain-JSON round-trip: a `codegraph.json` carrying comments (not valid JSON)
* already fails to load with a warning, so rather than silently clobber such a
* file this throws when an existing config won't parse the caller falls back
* to printing the manual snippet. Invalidates the config cache so a subsequent
* index in the same process sees the new patterns.
*/
export function addIncludeIgnoredPatterns(rootDir: string, patterns: string[]): number {
const file = path.join(rootDir, PROJECT_CONFIG_FILENAME);
let config: Record<string, unknown> = {};
let raw: string | null = null;
try {
raw = fs.readFileSync(file, 'utf-8');
} catch {
raw = null; // missing file — create a fresh one below
}
if (raw !== null) {
let parsed: unknown;
try {
parsed = JSON.parse(raw);
} catch {
throw new Error(`${PROJECT_CONFIG_FILENAME} is not valid JSON — fix it by hand, then re-run.`);
}
if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) {
config = parsed as Record<string, unknown>;
}
}
const existing = Array.isArray(config.includeIgnored)
? (config.includeIgnored as unknown[]).filter((p): p is string => typeof p === 'string')
: [];
const merged = [...existing];
const seen = new Set(existing);
let added = 0;
for (const p of patterns) {
if (seen.has(p)) continue;
seen.add(p);
merged.push(p);
added++;
}
config.includeIgnored = merged;
fs.writeFileSync(file, JSON.stringify(config, null, 2) + '\n');
clearProjectConfigCache();
return added;
}