Colby McHenryandClaude Opus 5
16e17495f4
`isGeneratedFile` was path-only, but Go's own convention is a CONTENT marker (`// Code generated by <tool>. DO NOT EDIT.`), not a filename one. A Go monorepo with generated CRUD in ordinarily-named files sitting beside hand-written use-cases was therefore invisible to every generated-file down-rank in the codebase — that is #1500. Measured on kubernetes/client-go (2,453 Go files): the canonical banner appears in 2,001 of them, the path check flags 0, the new content check flags exactly those 2,001 — no false positives, no misses. Design: decide at INDEX time (content is already in memory for parsing), persist on `files.generated`, read from the DB. Explore never reads file headers per request. - `hasGeneratedHeader(content)` recognizes the standard banners — Go's, protoc's, `@generated`, `<auto-generated>`, Thrift, OpenAPI Generator, FlatBuffers, bindgen, ANTLR. Precision-first and fenced three ways: an 8KB/60-line header window, a comment-line requirement (leader or open block comment), and markers tight enough that prose can't trip them. A generator's own source, holding the banner as a string constant in its body, is not flagged; neither is this module itself (pinned by test). - `isGeneratedFile(path)` is unchanged — cheap, sync, still the fallback. - Schema v9 adds `files.generated` + a PARTIAL index. DDL only, no backfill: the flag derives from content the migration cannot see, so rows stay 0 until a re-index and every reader unions the flag with the path check — an un-migrated index keeps pre-#1500 behavior rather than regressing. Re-index required; noted in the CHANGELOG. - `generatedPredicateFor(paths)` gives ranking a bounded probe + O(1) lookups. Bounded, not cached: no invalidation, so a ranking call can never serve a verdict the last sync already replaced. Wired into explore ranking, findSymbolMatches, findAllSymbols, search (MCP + CLI), the context formatter, and the dominant-file/route-file hygiene filters. Cost (acceptance bar was no measurable index-time regression): a single unanchored `/generat/i` test over the header rejects ~every hand-written file before any line splitting. 4.6 µs/file on client-go (worst case — 82% generated). End-to-end `codegraph init` on client-go, n=3 alternating arms: 5.73s median with detection vs 5.76s path-only baseline; the arms cross over between runs, so the difference is inside run-to-run noise. Scope note: generated status remains a stable TIEBREAK at equal score, exactly where it was. Making it a strong negative signal is CG-10, which this unblocks by making the signal correct and available. Two pre-existing tests hard-coded schema version 8; both now track CURRENT_SCHEMA_VERSION (or the migration table) so future migrations don't require editing them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
124 lines
6.3 KiB
Markdown
124 lines
6.3 KiB
Markdown
# Generated-file detection — path convention plus content banner
|
|
|
|
CG-5, issue #1500. Companion to `explore-budget-allocation.md` (CG-4's instrument) and a
|
|
prerequisite for the scoring overhaul (CG-10).
|
|
|
|
## The problem
|
|
|
|
`isGeneratedFile` was path-only. It matches the `<basename>.<tool>.<ext>` convention —
|
|
`.pb.go`, `_grpc.pb.go`, `.g.dart`, `_pb2.py` — which is where most codegen output lives,
|
|
and which was enough for the cosmos-sdk audit that motivated it.
|
|
|
|
It is not enough for Go. **Go's convention is a content marker, not a filename one:**
|
|
|
|
```go
|
|
// Code generated by <tool>. DO NOT EDIT.
|
|
```
|
|
|
|
codified by `go generate`, honored by gofmt, golangci-lint and GitHub linguist, and emitted
|
|
by protoc-gen-go, mockgen, sqlc, ent, wire, stringer — and by in-house generators. #1500 is
|
|
exactly this: a Go monorepo with generated FKIT CRUD in ordinarily-named files
|
|
(`payroll.go`) sitting beside hand-written workflow use-cases. Nothing in the path gives it
|
|
away, so every generated-file down-rank in the codebase was a no-op on it.
|
|
|
|
### How large the gap is
|
|
|
|
Measured on a shallow clone of `kubernetes/client-go` (2,453 Go files):
|
|
|
|
| signal | files flagged |
|
|
|---|---|
|
|
| ground truth (grep the canonical banner in the first 60 lines) | 2,001 |
|
|
| path convention (`isGeneratedFile`) | **0** |
|
|
| content banner (`hasGeneratedHeader`) | **2,001** — 0 false positives, 0 misses |
|
|
|
|
82% of that repository is generated code with ordinary filenames, and the path check saw
|
|
none of it. This is not a long-tail case.
|
|
|
|
## Design
|
|
|
|
**Decide at index time, store on the file record, read from the DB.** Explore must never
|
|
read file headers per request.
|
|
|
|
- `isGeneratedFile(path)` — unchanged. Path-only, pure, synchronous, free to call in a sort
|
|
comparator. Kept for callers with no database in hand.
|
|
- `hasGeneratedHeader(content)` — the content signal (below).
|
|
- `detectGeneratedFile(path, content)` — the union, which is what the indexer persists.
|
|
- `files.generated INTEGER NOT NULL DEFAULT 0` (schema v9) + a **partial** index
|
|
`idx_files_generated ON files(path) WHERE generated = 1`, so lookups cost the generated
|
|
minority, not the repo.
|
|
- `QueryBuilder.generatedPredicateFor(paths)` / `CodeGraph.generatedFilePredicate(paths)` —
|
|
one bounded probe up front, `O(1)` per comparison after, unioned with the path check.
|
|
|
|
### Why a bounded lookup and not a cached set
|
|
|
|
Every consumer already holds a short candidate list — a ranked file group, an FTS result
|
|
page, a `LIMIT 20` aggregate. Intersecting that list against the partial index needs no
|
|
whole-repo set to materialize and, more importantly, **no cache to invalidate**: a ranking
|
|
call can never serve a verdict the last sync already replaced. The alternative (a lazily
|
|
materialized `Set` of all generated paths) has to be invalidated on every file write and
|
|
goes stale on the read-only pool workers, in exchange for saving a sub-millisecond query.
|
|
|
|
### Precision over recall
|
|
|
|
A false positive silently demotes hand-written code in every ranking path, so the marker
|
|
table is precision-first and the scan is fenced three ways:
|
|
|
|
1. **Header window only** — first 8,192 chars / 60 lines. Generous enough for build tags
|
|
plus an Apache-2.0 preamble above the banner; tight enough that a generator's own source,
|
|
which holds the banner as a *string constant in its body*, is not flagged.
|
|
2. **Comment lines only** — the marker must sit on a line with a comment leader (`//`, `#`,
|
|
`--`, `<!--`, `%`, `;`, `'`, …) or inside an open block comment (`/* */`, `<!-- -->`,
|
|
`"""`, `'''`, `=begin`, `<# #>`), tracked with a small state machine over the window.
|
|
Generators always emit banners as comments; requiring it rules out identifiers and
|
|
string literals that merely contain the words.
|
|
3. **Tight markers** — `automatically generated` alone is prose ("the table is automatically
|
|
generated at runtime"); `automatically generated **by**` is a banner. `DO NOT EDIT` alone
|
|
is a style directive; paired with a generation claim it is a banner.
|
|
|
|
The module deliberately keeps its own quoted banner literals **below** the header window so
|
|
it does not classify itself; `generated-detection.test.ts` pins that, so moving the pattern
|
|
table upward fails a test rather than silently demoting this file.
|
|
|
|
### Migration: no backfill, by necessity
|
|
|
|
v9 is DDL only. The flag derives from file **content**, which the migration cannot see —
|
|
`files` stores a hash, not bytes. Migrated rows stay 0 until a re-index, and because every
|
|
reader unions the flag with the path check, an un-backfilled database keeps exactly the
|
|
pre-#1500 behavior instead of regressing. `sync` heals it file-by-file as files change.
|
|
This is why the CHANGELOG entry says a re-index is required.
|
|
|
|
## Cost
|
|
|
|
The acceptance bar was "no measurable index-time cost regression."
|
|
|
|
A single unanchored `/generat/i` test over the header rejects ~every hand-written file
|
|
before any line splitting happens. `String.prototype.slice` on a long string yields a V8
|
|
sliced view, not a copy, so the fast path allocates nothing.
|
|
|
|
- **Microbenchmark** (`detectGeneratedFile` over a whole corpus, 5 passes):
|
|
4.6 µs/file on client-go (2,453 files, 14.2 MB, 82% generated — the worst case, where the
|
|
gate passes and the full line scan runs), 7.3 µs/file on this repo's `src`.
|
|
- **End-to-end** `codegraph init` on client-go, n=3 alternating arms
|
|
(current build vs. the same build with the content scan stubbed out):
|
|
|
|
| arm | runs (s) | median |
|
|
|---|---|---|
|
|
| with content detection | 5.66, 5.73, 5.89 | **5.73** |
|
|
| path-only baseline | 5.52, 5.76, 5.88 | **5.76** |
|
|
|
|
The arms cross over between runs — the difference is inside run-to-run noise.
|
|
|
|
## What this task does NOT change
|
|
|
|
Generated status remains a **stable tiebreak at equal score**, exactly where it was
|
|
(`src/mcp/tools.ts` file sort, `findSymbolMatches`, `findAllSymbols`, search formatting,
|
|
`getDominantFile`/`getTopRouteFile`/`getRoutingManifest`, the context formatter). A
|
|
generated file with a higher raw score still outranks a hand-written one. Turning generated
|
|
status into a **strong negative signal** is CG-10, which this task unblocks by making the
|
|
signal correct and available.
|
|
|
|
Verified end-to-end on a two-file Go package where a generated `payroll.go` and a
|
|
hand-written `workflow.go` both define `ProcessPayroll`: with the flag set the hand-written
|
|
file ranks first; clearing the flag in the same index (i.e. pre-#1500 behavior) puts the
|
|
generated file first.
|