feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
b37f191f5a
commit
16e17495f4
@@ -298,8 +298,21 @@ describe('Best-Candidate Resolution', () => {
|
||||
|
||||
describe('Schema v2 Migration', () => {
|
||||
it.skipIf(!HAS_SQLITE)('should have correct current schema version', async () => {
|
||||
const { CURRENT_SCHEMA_VERSION } = await import('../src/db/migrations');
|
||||
expect(CURRENT_SCHEMA_VERSION).toBe(8);
|
||||
const { CURRENT_SCHEMA_VERSION, getPendingMigrations } = await import('../src/db/migrations');
|
||||
const { DatabaseConnection } = await import('../src/db');
|
||||
|
||||
// The constant must track the migration table, not a literal — a literal
|
||||
// just makes every schema change edit this test (v9/#1500 was the latest).
|
||||
// A fresh database records the current version, so nothing is pending;
|
||||
// ask a version-0 database instead to see the full migration list.
|
||||
const dbPath = path.join(createTempDir(), 'schema-version.db');
|
||||
const conn = DatabaseConnection.initialize(dbPath);
|
||||
const raw = conn.getDb();
|
||||
raw.prepare('DELETE FROM schema_versions').run();
|
||||
const highest = Math.max(...getPendingMigrations(raw).map((m) => m.version));
|
||||
conn.close();
|
||||
|
||||
expect(CURRENT_SCHEMA_VERSION).toBe(highest);
|
||||
});
|
||||
|
||||
it.skipIf(!HAS_SQLITE)('should have migration for version 2', async () => {
|
||||
|
||||
Reference in New Issue
Block a user