feat(extraction): content-based generated-file detection (CG-5, #1500)
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
co-authored by
Claude Opus 5
Claude Opus 5
parent
b37f191f5a
commit
16e17495f4
@@ -16,7 +16,7 @@ import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import { DatabaseConnection } from '../src/db';
|
||||
import { QueryBuilder } from '../src/db/queries';
|
||||
import { runMigrations, getCurrentVersion } from '../src/db/migrations';
|
||||
import { runMigrations, getCurrentVersion, CURRENT_SCHEMA_VERSION } from '../src/db/migrations';
|
||||
import { Node, Edge } from '../src/types';
|
||||
|
||||
function makeNode(id: string, name = id): Node {
|
||||
@@ -344,7 +344,11 @@ describe('migration v6: dedup edges + add identity index on upgrade (#1034)', ()
|
||||
runMigrations(raw, 5);
|
||||
|
||||
expect(count()).toBe(2); // duplicate collapsed, the distinct `calls` edge kept
|
||||
expect(getCurrentVersion(raw)).toBe(8);
|
||||
// Migrations ran to completion. Tracked against the constant, not a
|
||||
// literal, so adding a migration doesn't require editing this assertion —
|
||||
// and so replaying every migration over a current-schema database (which
|
||||
// is what this test does) stays covered as new ones land.
|
||||
expect(getCurrentVersion(raw)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
const idx = raw
|
||||
.prepare("SELECT name FROM sqlite_master WHERE type='index' AND name='idx_edges_identity'")
|
||||
.get();
|
||||
|
||||
@@ -12,6 +12,7 @@ import { CodeGraph } from '../src';
|
||||
import { Node, Edge } from '../src/types';
|
||||
import { isInitialized, getCodeGraphDir, validateDirectory, codeGraphDirName, isCodeGraphDataDir } from '../src/directory';
|
||||
import { DatabaseConnection, getDatabasePath, removeDatabaseFiles } from '../src/db';
|
||||
import { CURRENT_SCHEMA_VERSION } from '../src/db/migrations';
|
||||
|
||||
// Create a temporary directory for each test
|
||||
function createTempDir(): string {
|
||||
@@ -370,7 +371,9 @@ describe('Database Connection', () => {
|
||||
|
||||
const version = db.getSchemaVersion();
|
||||
expect(version).not.toBeNull();
|
||||
expect(version?.version).toBe(8);
|
||||
// A freshly initialized database records the current version outright
|
||||
// (schema.sql already contains every migration's end state).
|
||||
expect(version?.version).toBe(CURRENT_SCHEMA_VERSION);
|
||||
|
||||
db.close();
|
||||
});
|
||||
|
||||
@@ -4,10 +4,23 @@
|
||||
* list is a contract: if a future edit drops `.pb.go`, the cosmos-sdk
|
||||
* trace endpoint regresses to the gRPC stub (see
|
||||
* `project_go_multi_module_audit` memory + the audit in #N/A).
|
||||
*
|
||||
* The content-header half (#1500) is a second contract: the marker table is
|
||||
* precision-first, because a false positive silently demotes hand-written code
|
||||
* in EVERY ranking path. Measured on a shallow clone of kubernetes/client-go
|
||||
* (2,453 Go files): the path check flags 0, the content check flags 2,001 —
|
||||
* exactly the set that greps to the canonical banner, no false positives and
|
||||
* no misses. Every one of those files has an ordinary name.
|
||||
*/
|
||||
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { isGeneratedFile } from '../src/extraction/generated-detection';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import {
|
||||
isGeneratedFile,
|
||||
hasGeneratedHeader,
|
||||
detectGeneratedFile,
|
||||
} from '../src/extraction/generated-detection';
|
||||
|
||||
describe('isGeneratedFile', () => {
|
||||
it('classifies Go protobuf / gRPC / pulsar / mock outputs as generated', () => {
|
||||
@@ -45,3 +58,149 @@ describe('isGeneratedFile', () => {
|
||||
expect(isGeneratedFile('app/db.py')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('hasGeneratedHeader — per-marker coverage (#1500)', () => {
|
||||
// One case per banner the marker table claims to recognize. Each string is
|
||||
// the real thing a generator emits, not a paraphrase — if a regex is
|
||||
// narrowed, the case that motivated it fails by name.
|
||||
const GENERATED: ReadonlyArray<[string, string]> = [
|
||||
[
|
||||
'Go — the #1500 case: ordinary filename, banner below the package clause',
|
||||
'package payroll\n\n// Code generated by fkit. DO NOT EDIT.\n\nimport "context"\n\nfunc CreatePayroll(ctx context.Context) error { return nil }\n',
|
||||
],
|
||||
[
|
||||
'Go — protoc-gen-go',
|
||||
'// Code generated by protoc-gen-go. DO NOT EDIT.\n// versions:\n// protoc-gen-go v1.28.0\n\npackage pb\n',
|
||||
],
|
||||
[
|
||||
'Go — banner under build tags',
|
||||
'//go:build !windows\n// +build !windows\n\n// Code generated by MockGen. DO NOT EDIT.\npackage mocks\n',
|
||||
],
|
||||
[
|
||||
'Go — banner under an Apache-2.0 license preamble',
|
||||
'// Copyright 2021 The Foo Authors.\n// Licensed under the Apache License, Version 2.0 (the "License");\n// you may not use this file except in compliance with the License.\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an "AS IS" BASIS.\n\n// Code generated by sqlc. DO NOT EDIT.\n// source: query.sql\n\npackage db\n',
|
||||
],
|
||||
[
|
||||
'protoc — Java banner ("DO NOT EDIT!")',
|
||||
'// Generated by the protocol buffer compiler. DO NOT EDIT!\n// source: foo.proto\n\npackage com.example;\n',
|
||||
],
|
||||
[
|
||||
'protoc — Python banner behind a coding cookie',
|
||||
'# -*- coding: utf-8 -*-\n# Generated by the protocol buffer compiler. DO NOT EDIT!\n# source: foo.proto\n',
|
||||
],
|
||||
[
|
||||
'C# — Roslyn / designer <auto-generated> block',
|
||||
'//------------------------------------------------------------------------------\n// <auto-generated>\n// This code was generated by a tool.\n// </auto-generated>\n//------------------------------------------------------------------------------\n',
|
||||
],
|
||||
['C# — EF self-closing <auto-generated />', '// <auto-generated />\nusing System;\n'],
|
||||
[
|
||||
'JS — Meta/Relay @generated with a SignedSource',
|
||||
'/**\n * @generated SignedSource<<0123456789abcdef0123456789abcdef>>\n * @flow\n */\n',
|
||||
],
|
||||
[
|
||||
'TS — protobuf-es / Buf @generated',
|
||||
'// @generated by protoc-gen-es v1.2.0 with parameter "target=ts"\n// @generated from file foo.proto (package example, syntax proto3)\n',
|
||||
],
|
||||
[
|
||||
'Thrift — "Autogenerated by Thrift Compiler"',
|
||||
'/**\n * Autogenerated by Thrift Compiler (0.14.1)\n *\n * DO NOT EDIT UNLESS YOU ARE SURE THAT YOU KNOW WHAT YOU ARE DOING\n */\n',
|
||||
],
|
||||
[
|
||||
'OpenAPI Generator — "This class is auto generated by"',
|
||||
'/*\n * Pet Store API\n *\n * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).\n * Do not edit the class manually.\n */\n',
|
||||
],
|
||||
[
|
||||
'FlatBuffers — "automatically generated by … do not modify"',
|
||||
'// automatically generated by the FlatBuffers compiler, do not modify\n\npackage MyGame;\n',
|
||||
],
|
||||
[
|
||||
'Rust — bindgen block comment',
|
||||
'/* automatically generated by rust-bindgen 0.59.2 */\n\npub const FOO: u32 = 1;\n',
|
||||
],
|
||||
['ANTLR — "Generated from … -- DO NOT EDIT"', '// Generated from Expr.g4 by ANTLR 4.9.2 -- DO NOT EDIT\npackage parser;\n'],
|
||||
[
|
||||
'banner on an unprefixed line INSIDE a block comment',
|
||||
'/*\n Code generated by ent. DO NOT EDIT.\n*/\npackage ent\n',
|
||||
],
|
||||
[
|
||||
'Python — banner inside a module docstring',
|
||||
'"""Generated by the protocol buffer compiler. DO NOT EDIT!"""\nimport sys\n',
|
||||
],
|
||||
['YAML/shell — "#" comment leader', '# This file is generated by kustomize. Do not edit.\napiVersion: v1\n'],
|
||||
['SQL — "--" comment leader', '-- Code generated by sqlc. DO NOT EDIT.\nCREATE TABLE foo (id INT);\n'],
|
||||
['HTML/XML — "<!--" comment leader', '<!-- Autogenerated by docgen. Do not edit. -->\n<html></html>\n'],
|
||||
];
|
||||
|
||||
it.each(GENERATED)('flags: %s', (_label, source) => {
|
||||
expect(hasGeneratedHeader(source)).toBe(true);
|
||||
});
|
||||
|
||||
// Precision cases. Each is a shape that a looser marker table WOULD flag.
|
||||
const HAND_WRITTEN: ReadonlyArray<[string, string]> = [
|
||||
[
|
||||
'ordinary Go source',
|
||||
'package keeper\n\nimport "context"\n\n// SendCoins moves coins between accounts.\nfunc (k Keeper) SendCoins(ctx context.Context) error { return nil }\n',
|
||||
],
|
||||
[
|
||||
'a generator\'s own source, which merely talks about generating',
|
||||
'// This package generates SQL migrations from the schema.\n// The generated output lives under db/migrations.\npackage gen\n',
|
||||
],
|
||||
[
|
||||
'prose using "automatically generated" without naming a tool',
|
||||
'"""Report builder.\n\nThe summary table is automatically generated at runtime from the\nrows below; callers should not edit it in place.\n"""\n',
|
||||
],
|
||||
[
|
||||
'a generator holding the banner as a string constant in its BODY',
|
||||
'package main\n\n// Package main implements the fkit CRUD generator.\n\nimport "fmt"\n\nfunc header() string {\n\treturn "// Code generated by fkit. DO NOT EDIT."\n}\n',
|
||||
],
|
||||
['an email address that happens to contain "@generated"', '// Contact: build@generated.example.com for issues.\npackage main\n'],
|
||||
['"DO NOT EDIT" with no generation claim', '// DO NOT EDIT THIS FILE BY HAND — run `make fmt` instead.\npackage main\n'],
|
||||
['empty file', ''],
|
||||
];
|
||||
|
||||
it.each(HAND_WRITTEN)('does not flag: %s', (_label, source) => {
|
||||
expect(hasGeneratedHeader(source)).toBe(false);
|
||||
});
|
||||
|
||||
it('only looks at the header — a banner buried 80 lines down is not a banner', () => {
|
||||
const filler = Array.from({ length: 80 }, (_, i) => `// filler line ${i}`).join('\n');
|
||||
expect(hasGeneratedHeader(`${filler}\n// Code generated by foo. DO NOT EDIT.\npackage main\n`)).toBe(false);
|
||||
// …but the same banner within the window is caught.
|
||||
const shortFiller = Array.from({ length: 20 }, (_, i) => `// filler line ${i}`).join('\n');
|
||||
expect(hasGeneratedHeader(`${shortFiller}\n// Code generated by foo. DO NOT EDIT.\npackage main\n`)).toBe(true);
|
||||
});
|
||||
|
||||
it('requires a comment line — the same words in executable code are not a banner', () => {
|
||||
// No comment leader, no open block: this is a bare statement.
|
||||
expect(hasGeneratedHeader('const banner = "Code generated by tool. DO NOT EDIT.";\n')).toBe(false);
|
||||
});
|
||||
|
||||
it('does not classify the detector module itself (the pattern table must stay below the header window)', () => {
|
||||
const self = fs.readFileSync(
|
||||
path.join(__dirname, '..', 'src', 'extraction', 'generated-detection.ts'),
|
||||
'utf-8'
|
||||
);
|
||||
expect(hasGeneratedHeader(self)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('detectGeneratedFile — the union the indexer persists', () => {
|
||||
it('is true when only the PATH says so', () => {
|
||||
expect(detectGeneratedFile('x/bank/types/tx.pb.go', 'package types\n')).toBe(true);
|
||||
});
|
||||
|
||||
it('is true when only the CONTENT says so — the #1500 acceptance case', () => {
|
||||
// A Go file named `payroll.go` sitting beside hand-written workflow
|
||||
// use-cases. Nothing in the path gives it away.
|
||||
expect(
|
||||
detectGeneratedFile('internal/payroll/payroll.go', 'package payroll\n\n// Code generated by fkit. DO NOT EDIT.\n\nfunc Create() {}\n')
|
||||
).toBe(true);
|
||||
expect(isGeneratedFile('internal/payroll/payroll.go')).toBe(false);
|
||||
});
|
||||
|
||||
it('is false for a hand-written file with an ordinary name', () => {
|
||||
expect(
|
||||
detectGeneratedFile('internal/payroll/workflow.go', 'package payroll\n\n// RunPayrollWorkflow drives the monthly run.\nfunc RunPayrollWorkflow() {}\n')
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
/**
|
||||
* Index-time persistence of the generated-file flag (#1500).
|
||||
*
|
||||
* `isGeneratedFile` is path-only, so a Go monorepo's generated CRUD — ordinary
|
||||
* filenames, a `// Code generated by … DO NOT EDIT.` banner in the header — is
|
||||
* invisible to it and outranks the hand-written use-case beside it. The fix
|
||||
* decides the verdict ONCE during extraction (content is already in memory for
|
||||
* parsing) and persists it on `files.generated`, so ranking reads a column
|
||||
* instead of re-reading file headers per request.
|
||||
*
|
||||
* This suite pins the whole path: extraction writes it, `sync` re-decides it,
|
||||
* the migration adds the column to an old database, and the bounded lookup
|
||||
* that ranking uses unions it with the filename convention.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, beforeAll, afterAll, afterEach } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src';
|
||||
import { QueryBuilder } from '../src/db/queries';
|
||||
import { createDatabase, type SqliteDatabase } from '../src/db/sqlite-adapter';
|
||||
import { runMigrations, getCurrentVersion, CURRENT_SCHEMA_VERSION } from '../src/db/migrations';
|
||||
|
||||
/** The FKIT-style generated CRUD from the issue: ordinary name, banner inside. */
|
||||
const GENERATED_PAYROLL = `package payroll
|
||||
|
||||
// Code generated by fkit. DO NOT EDIT.
|
||||
|
||||
type PayrollRecord struct {
|
||||
ID string
|
||||
Amount int
|
||||
}
|
||||
|
||||
func CreatePayrollRecord(r PayrollRecord) error { return nil }
|
||||
func UpdatePayrollRecord(r PayrollRecord) error { return nil }
|
||||
func DeletePayrollRecord(id string) error { return nil }
|
||||
`;
|
||||
|
||||
/** The hand-written use-case that must NOT be demoted. */
|
||||
const HANDWRITTEN_WORKFLOW = `package payroll
|
||||
|
||||
// RunPayrollWorkflow computes the monthly run and persists each record.
|
||||
func RunPayrollWorkflow(records []PayrollRecord) error {
|
||||
for _, r := range records {
|
||||
if err := CreatePayrollRecord(r); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
`;
|
||||
|
||||
describe('generated flag — written at index time', () => {
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
|
||||
beforeAll(async () => {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-genflag-'));
|
||||
fs.writeFileSync(path.join(dir, 'payroll.go'), GENERATED_PAYROLL);
|
||||
fs.writeFileSync(path.join(dir, 'workflow.go'), HANDWRITTEN_WORKFLOW);
|
||||
// A path-convention generated file, so both signals are exercised together.
|
||||
fs.writeFileSync(path.join(dir, 'payroll.pb.go'), 'package payroll\n\ntype PayrollProto struct{}\n');
|
||||
cg = await CodeGraph.init(dir, { index: true });
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
cg?.close();
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('flags an ORDINARY-named Go file carrying the DO-NOT-EDIT banner (the acceptance case)', () => {
|
||||
expect(cg.getFile('payroll.go')?.generated).toBe(true);
|
||||
});
|
||||
|
||||
it('leaves the hand-written use-case beside it unflagged', () => {
|
||||
expect(cg.getFile('workflow.go')?.generated).toBe(false);
|
||||
});
|
||||
|
||||
it('still flags the filename convention', () => {
|
||||
expect(cg.getFile('payroll.pb.go')?.generated).toBe(true);
|
||||
});
|
||||
|
||||
it('counts the flagged files', () => {
|
||||
expect(cg.getGeneratedFileCount()).toBe(2);
|
||||
});
|
||||
|
||||
it('exposes a bounded predicate that unions both signals', () => {
|
||||
const isGen = cg.generatedFilePredicate(['payroll.go', 'workflow.go', 'payroll.pb.go']);
|
||||
expect(isGen('payroll.go')).toBe(true); // content only
|
||||
expect(isGen('payroll.pb.go')).toBe(true); // path (and content)
|
||||
expect(isGen('workflow.go')).toBe(false);
|
||||
});
|
||||
|
||||
it('falls back to the filename check for a path outside the queried set', () => {
|
||||
const isGen = cg.generatedFilePredicate([]);
|
||||
// Not in the bounded set, but the path convention still decides.
|
||||
expect(isGen('some/other/tx.pb.go')).toBe(true);
|
||||
expect(isGen('some/other/keeper.go')).toBe(false);
|
||||
});
|
||||
|
||||
it('re-decides on sync: removing the banner clears the flag', async () => {
|
||||
fs.writeFileSync(
|
||||
path.join(dir, 'payroll.go'),
|
||||
GENERATED_PAYROLL.replace('// Code generated by fkit. DO NOT EDIT.\n\n', '')
|
||||
);
|
||||
await cg.sync();
|
||||
expect(cg.getFile('payroll.go')?.generated).toBe(false);
|
||||
|
||||
// …and adding it back re-flags it, so a stale 1 can never linger.
|
||||
fs.writeFileSync(path.join(dir, 'payroll.go'), GENERATED_PAYROLL);
|
||||
await cg.sync();
|
||||
expect(cg.getFile('payroll.go')?.generated).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('generated flag — schema migration to v9', () => {
|
||||
let dir: string;
|
||||
let db: SqliteDatabase | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
db?.close();
|
||||
db = null;
|
||||
if (dir) fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
/** A pre-v9 `files` table: no `generated` column, no partial index. */
|
||||
function makeLegacyDb(): SqliteDatabase {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-genmigrate-'));
|
||||
const conn = createDatabase(path.join(dir, 'legacy.db')).db;
|
||||
conn.exec(`
|
||||
CREATE TABLE schema_versions (version INTEGER PRIMARY KEY, applied_at INTEGER NOT NULL, description TEXT);
|
||||
INSERT INTO schema_versions VALUES (8, 0, 'legacy');
|
||||
CREATE TABLE files (
|
||||
path TEXT PRIMARY KEY,
|
||||
content_hash TEXT NOT NULL,
|
||||
language TEXT NOT NULL,
|
||||
size INTEGER NOT NULL,
|
||||
modified_at INTEGER NOT NULL,
|
||||
indexed_at INTEGER NOT NULL,
|
||||
node_count INTEGER DEFAULT 0,
|
||||
errors TEXT
|
||||
);
|
||||
INSERT INTO files VALUES ('x/bank/types/tx.pb.go', 'h1', 'go', 10, 0, 0, 1, NULL);
|
||||
INSERT INTO files VALUES ('internal/payroll/payroll.go', 'h2', 'go', 10, 0, 0, 1, NULL);
|
||||
`);
|
||||
db = conn;
|
||||
return conn;
|
||||
}
|
||||
|
||||
const columnNames = (conn: SqliteDatabase): string[] =>
|
||||
(conn.prepare('PRAGMA table_info(files)').all() as Array<{ name: string }>).map((c) => c.name);
|
||||
|
||||
it('adds the column and the partial index without touching existing rows', () => {
|
||||
const conn = makeLegacyDb();
|
||||
|
||||
expect(getCurrentVersion(conn)).toBe(8);
|
||||
runMigrations(conn, 8);
|
||||
expect(getCurrentVersion(conn)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
|
||||
expect(columnNames(conn)).toContain('generated');
|
||||
|
||||
const indexes = (conn.prepare('PRAGMA index_list(files)').all() as Array<{ name: string }>).map((i) => i.name);
|
||||
expect(indexes).toContain('idx_files_generated');
|
||||
|
||||
// NO backfill: the flag is derived from file CONTENT, which the migration
|
||||
// cannot see (files stores a hash, not bytes). Rows stay 0 until a
|
||||
// re-index, and readers union with the path check so behavior is unchanged
|
||||
// rather than regressed. This is why the CHANGELOG says "requires a
|
||||
// re-index".
|
||||
expect((conn.prepare('SELECT COUNT(*) AS n FROM files WHERE generated = 1').get() as { n: number }).n).toBe(0);
|
||||
expect((conn.prepare('SELECT COUNT(*) AS n FROM files').get() as { n: number }).n).toBe(2);
|
||||
});
|
||||
|
||||
it('is idempotent — replaying v9 over a database that already has the column does not throw', () => {
|
||||
const conn = makeLegacyDb();
|
||||
runMigrations(conn, 8);
|
||||
|
||||
// ALTER TABLE has no IF NOT EXISTS, so v9 guards on PRAGMA table_info.
|
||||
// Replay happens for real whenever the recorded version trails the on-disk
|
||||
// shape — a database created straight from current schema.sql already HAS
|
||||
// the column, and the v6 regression test rewinds `schema_versions` and
|
||||
// re-runs. Rewind the same way here; without the guard this is
|
||||
// "duplicate column name: generated".
|
||||
conn.prepare('DELETE FROM schema_versions WHERE version >= 9').run();
|
||||
expect(() => runMigrations(conn, 8)).not.toThrow();
|
||||
expect(columnNames(conn).filter((c) => c === 'generated')).toHaveLength(1);
|
||||
expect(getCurrentVersion(conn)).toBe(CURRENT_SCHEMA_VERSION);
|
||||
});
|
||||
|
||||
it('an un-backfilled database still down-ranks by the path convention', () => {
|
||||
const conn = makeLegacyDb();
|
||||
runMigrations(conn, 8);
|
||||
|
||||
const queries = new QueryBuilder(conn);
|
||||
const paths = ['x/bank/types/tx.pb.go', 'internal/payroll/payroll.go'];
|
||||
// Nothing carries the content flag yet…
|
||||
expect(queries.getGeneratedPathsAmong(paths).size).toBe(0);
|
||||
// …but the union predicate still knows `.pb.go`.
|
||||
const isGen = queries.generatedPredicateFor(paths);
|
||||
expect(isGen('x/bank/types/tx.pb.go')).toBe(true);
|
||||
expect(isGen('internal/payroll/payroll.go')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -298,8 +298,21 @@ describe('Best-Candidate Resolution', () => {
|
||||
|
||||
describe('Schema v2 Migration', () => {
|
||||
it.skipIf(!HAS_SQLITE)('should have correct current schema version', async () => {
|
||||
const { CURRENT_SCHEMA_VERSION } = await import('../src/db/migrations');
|
||||
expect(CURRENT_SCHEMA_VERSION).toBe(8);
|
||||
const { CURRENT_SCHEMA_VERSION, getPendingMigrations } = await import('../src/db/migrations');
|
||||
const { DatabaseConnection } = await import('../src/db');
|
||||
|
||||
// The constant must track the migration table, not a literal — a literal
|
||||
// just makes every schema change edit this test (v9/#1500 was the latest).
|
||||
// A fresh database records the current version, so nothing is pending;
|
||||
// ask a version-0 database instead to see the full migration list.
|
||||
const dbPath = path.join(createTempDir(), 'schema-version.db');
|
||||
const conn = DatabaseConnection.initialize(dbPath);
|
||||
const raw = conn.getDb();
|
||||
raw.prepare('DELETE FROM schema_versions').run();
|
||||
const highest = Math.max(...getPendingMigrations(raw).map((m) => m.version));
|
||||
conn.close();
|
||||
|
||||
expect(CURRENT_SCHEMA_VERSION).toBe(highest);
|
||||
});
|
||||
|
||||
it.skipIf(!HAS_SQLITE)('should have migration for version 2', async () => {
|
||||
|
||||
@@ -408,6 +408,10 @@ describe('MCP Input Validation', () => {
|
||||
}));
|
||||
const fakeCg = {
|
||||
searchNodes: () => many,
|
||||
// Search down-ranks generated files, and since #1500 that verdict comes
|
||||
// from the index (path convention ∪ content banner) rather than the
|
||||
// filename alone. No database here — none of these paths is generated.
|
||||
generatedFilePredicate: () => () => false,
|
||||
};
|
||||
const fakeHandler = new ToolHandler(fakeCg as unknown as CodeGraph);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user