fix(explore): damp ambient declaration files on flow queries (CG-28)

A file that declares nothing but types and that nothing in the index depends
on — a hand-written ambient `.d.ts` of global shims, vendored typings, module
augmentation — cannot answer a flow question: no bodies, no call edges, no
behaviour, nothing typed by it. But the identifiers it declares are exactly the
generic ones a prose question uses (`Body`, `Message`, `ImageMetadata`,
`ReadableStream`), so on term overlap it out-scored the implementation. Measured
on the new fixture: rank #1 and 51% of delivered source, with the flow's own
entry file pushed out of the response entirely.

Measured first, per the issue: the Wrangler `worker-configuration.d.ts` that
opened this is already handled by CG-25's banner detection, worth 15-46 points
of envelope share across four flow queries. CG-25 credited; only the un-bannered
case needed anything.

`rankPenalty` now multiplies score and graph mass by 0.5 for such files, taken
as the STRONGER of it and the generated penalty rather than multiplied — one
property two signals see must not be charged twice. Detection is structural, not
by extension, and four conditions deep. Two of them were forced by measurement:
requiring every symbol to be type-level takes the corpus flag rate from 1-18%
(which swept in Kotlin sealed classes, Rust mod.rs re-exports and django's
locale tables) down to 0-4%; requiring that nothing depends on the file
separates an ambient shim from a working types module, and without it the rule
demoted displacement-ts's pipeline `types.ts` and broke the CG-31 gate.

A query that NAMES a declared type is exempt, so a question about a type still
reaches its declaration at full weight. Precise tokens only, so "…the file
body…" cannot exempt a `Body` interface it never meant to name; this needs its
own set because `namedSeedIds` is callable-only and a type never becomes one.

Regression evidence in docs/benchmarks/explore-declaration-only-cg28.md:
6-repo envelope sweep byte-identical against a clean baseline build, zero
ambient files reach the candidate set on VS Code across five queries, corpus
flag rate 0-0.74%, both allocation fixtures PASS, full suite 2,978 green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-08-06 14:35:56 -05:00
co-authored by Claude Opus 5
parent 463f6e7844
commit 9efae0f8f2
18 changed files with 1582 additions and 3 deletions
+95
View File
@@ -1944,6 +1944,101 @@ export class QueryBuilder {
return (filePath: string) => flagged.has(filePath) || isGeneratedFile(filePath);
}
/**
* Which of `filePaths` are AMBIENT DECLARATION files — they declare nothing
* but types, and nothing in the index depends on them (CG-28). A hand-written
* ambient `.d.ts` of global shims, a vendored typings file, module
* augmentation: reachable only by name, structurally attached to nothing.
*
* Structural, not extension-based, so a hand-written `types.ts` and a `.d.ts`
* are judged by the same rule and a `.d.ts` that does declare a class or a
* const is (correctly) not caught. Four conditions, all required:
*
* 1. it declares at least one symbol — an empty or unparsed file is not a
* declaration file, it is a file we know nothing about;
* 2. EVERY declared symbol is a type-level kind (interface / type alias /
* enum / namespace). The narrowness is deliberate and measured: a rule
* of "no callables" alone flags 118% of a repo, including Kotlin sealed
* classes, Rust `mod.rs` re-exports and django's locale constant tables —
* real source that must not be demoted. This rule flags 04%;
* 3. no symbol in it originates a `calls`/`instantiates` edge — the direct
* evidence that nothing here has a body;
* 4. NOTHING ELSE IN THE INDEX points at it. This is the condition that
* separates an ambient shim from a working type module, and it is why
* the flag is narrow enough to be safe: `displacement-ts`'s pipeline
* `types.ts` passes 13 identically but carries 13 inbound imports and
* 21 references, so the files that answer a query about the pipeline are
* typed BY it — it is part of that answer's structure. An ambient
* `declare global` shim has zero. Deliberately index-wide rather than
* restricted to the candidate list: the file that imports it is usually
* not itself a candidate.
*
* Bounded-lookup like {@link getGeneratedPathsAmong}: callers hold a ranked
* candidate list, so this is a partial-index probe over a handful of paths.
*/
getAmbientDeclarationPathsAmong(filePaths: Iterable<string>): Set<string> {
const unique = [...new Set(filePaths)];
const found = new Set<string>();
if (unique.length === 0) return found;
for (let i = 0; i < unique.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = unique.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
// `file`/`import`/`export`/`parameter` are structural bookkeeping, not
// things the file declares, so they neither qualify nor disqualify.
const rows = this.db
.prepare(`
SELECT file_path,
SUM(CASE WHEN kind NOT IN ('file','import','export','parameter')
THEN 1 ELSE 0 END) AS declared,
SUM(CASE WHEN kind IN ('interface','type_alias','enum','enum_member','namespace')
THEN 1 ELSE 0 END) AS typeDeclared
FROM nodes
WHERE file_path IN (${placeholders})
GROUP BY file_path
`)
.all(...chunk) as Array<{ file_path: string; declared: number; typeDeclared: number }>;
let candidates = rows
.filter((r) => r.declared > 0 && r.declared === r.typeDeclared)
.map((r) => r.file_path);
if (candidates.length === 0) continue;
const disqualify = (sql: string): void => {
if (candidates.length === 0) return;
const hit = new Set(
(this.db
.prepare(sql.replace('$IN$', candidates.map(() => '?').join(',')))
.all(...candidates) as Array<{ file_path: string }>).map((r) => r.file_path),
);
candidates = candidates.filter((p) => !hit.has(p));
};
// (3) originates behaviour
disqualify(`
SELECT DISTINCT n.file_path AS file_path
FROM edges e JOIN nodes n ON n.id = e.source
WHERE e.kind IN ('calls','instantiates') AND n.file_path IN ($IN$)
`);
// (4) something outside the file depends on it
disqualify(`
SELECT DISTINCT t.file_path AS file_path
FROM edges e JOIN nodes t ON t.id = e.target JOIN nodes s ON s.id = e.source
WHERE t.file_path IN ($IN$) AND s.file_path <> t.file_path
`);
for (const path of candidates) found.add(path);
}
return found;
}
/**
* A reusable `(path) => boolean` ambient-declaration test over a bounded
* candidate list — the shape a ranking comparator wants: one query up front,
* O(1) per comparison.
*/
ambientDeclarationPredicateFor(filePaths: Iterable<string>): (filePath: string) => boolean {
const flagged = this.getAmbientDeclarationPathsAmong(filePaths);
return (filePath: string) => flagged.has(filePath);
}
/** How many indexed files carry the generated flag. Surfaced by `status`. */
countGeneratedFiles(): number {
const row = this.db
+13
View File
@@ -1550,6 +1550,19 @@ export class CodeGraph {
return this.queries.generatedPredicateFor(filePaths);
}
/**
* A `(path) => boolean` ambient-declaration test over a BOUNDED candidate
* list: true for a file that declares nothing but types, originates no call
* edge, and that nothing in the index depends on — an ambient `.d.ts` of
* global shims, vendored typings, module augmentation (CG-28). Structural
* rather than extension-based, and deliberately narrow: see
* `QueryBuilder.getAmbientDeclarationPathsAmong` for why each condition is
* there, in particular why a `types.ts` the codebase imports is NOT flagged.
*/
ambientDeclarationFilePredicate(filePaths: Iterable<string>): (filePath: string) => boolean {
return this.queries.ambientDeclarationPredicateFor(filePaths);
}
/** How many indexed files are flagged tool-generated. Reported by `status`. */
getGeneratedFileCount(): number {
return this.queries.countGeneratedFiles();
+8
View File
@@ -66,6 +66,12 @@ export interface ExploreCandidateMeta {
spine: boolean;
lowValue: boolean;
generated: boolean;
/**
* Nothing but type declarations in this file, and nothing in the index
* depends on it (CG-28) — it cannot answer a flow question, so it ranks on
* discounted signals unless the query named one of the types it declares.
*/
ambientDeclaration: boolean;
/**
* Multiplier `rankPenalty` applied to BOTH `score` and `graphScore` (1 = no
* penalty). Generated and test/i18n files rank on discounted signals, so the
@@ -579,6 +585,7 @@ export class ExploreDiagnostics {
spine: r.spine,
lowValue: r.lowValue,
generated: r.generated,
ambientDeclaration: r.ambientDeclaration,
penalty: round6(r.penalty),
kinds: r.kinds,
allowance: r.allowance,
@@ -796,5 +803,6 @@ function flagString(f: ExploreDiagnosticFile): string {
if (f.spine) flags.push('spine');
if (f.lowValue) flags.push('low-value');
if (f.generated) flags.push('generated');
if (f.ambientDeclaration) flags.push('ambient-decl');
return flags.join(' ') || '-';
}
+70 -3
View File
@@ -409,6 +409,36 @@ const GENERATED_RANK_PENALTY = 0.3;
* that case: down-weighted rather than removed.
*/
const LOW_VALUE_RANK_PENALTY = 0.5;
/**
* Ambient declaration files a hand-written `.d.ts` of global shims, vendored
* typings, module augmentation (CG-28). Declares nothing but types, and nothing
* in the index depends on it.
*
* Such a file cannot answer a FLOW question no matter how much its identifiers
* overlap the query: no bodies, no call edges, no behaviour, and nothing typed
* by it. Its ceiling of usefulness is a type signature, and one follow-up
* explore fetches that. But the identifiers it declares are exactly the generic
* ones a prose question uses (`Body`, `Message`, `ImageMetadata`,
* `ReadableStream`), so on term overlap it out-scores the implementation and
* takes the envelope measured at rank #1 and 51% of delivered source, with
* the flow's own entry file getting none.
*
* Softer than {@link GENERATED_RANK_PENALTY} on purpose: "generated" is a claim
* about provenance the file itself makes, while this is an inference about what
* a file can be USEFUL for. A demoted declaration file that is still the best
* candidate should keep its place; the penalty only has to stop it beating real
* implementation. It does NOT stack with the generated penalty (see rankPenalty)
* penalising twice for the same property is how a file gets cliffed out of
* answers where it is genuinely relevant.
*/
const AMBIENT_DECLARATION_RANK_PENALTY = 0.5;
/**
* The type-level NodeKinds. Must stay in step with the kind list in
* `QueryBuilder.getAmbientDeclarationPathsAmong` that query decides which
* files are ambient declarations, this set decides which symbols in them the
* agent can name to lift the penalty back off.
*/
const DECLARATION_KINDS = new Set(['interface', 'type_alias', 'enum', 'enum_member', 'namespace']);
/**
* Score floor: `clamp(topScore * FRACTION, ABSOLUTE, MAX)`.
@@ -3275,6 +3305,9 @@ export class ToolHandler {
// and crowd out the real answer file (grpc's `dialoptions.go`). Corroborated
// overloads (the query also named the type) all earn it. (#1064)
const tierSeedIds = new Set<string>();
// Files declaring a TYPE the query named by name — the counter-case guard
// for the declaration-only penalty (CG-28). Populated in the token loop.
const namedTypeFiles = new Set<string>();
{
const FILE_EXT = /\.(?:java|kt|kts|ts|tsx|js|jsx|mjs|cjs|cs|py|go|rb|php|swift|rs|cpp|cc|cxx|c|h|hpp|scala|lua|dart|vue|svelte|astro|erl|hrl)$/i;
const CALLABLE = new Set(['method', 'function', 'component', 'constructor']);
@@ -3341,6 +3374,21 @@ export class ToolHandler {
// codegraph_node's findSymbolMatches.) Qualified tokens keep findAllSymbols.
const isQual = /[.\/]|::/.test(t);
const raw = isQual ? this.findAllSymbols(cg, t).nodes : cg.getNodesByName(t);
// A query that NAMES a declared type is a question ABOUT that type, and
// must still reach its declaration file at full weight — so record the
// files those declarations live in and exempt them from the
// declaration-only penalty below (CG-28). Only PRECISE tokens count, by
// the same NL-stopword reasoning as the seeding above: "…the file body…"
// must not exempt a `Body` interface it never meant to name. Kept
// separate from `namedSeedIds`, which is callable-only by construction —
// a type never becomes a named seed, so it cannot be the guard here.
if (isPreciseToken(t)) {
for (const n of raw) {
if (DECLARATION_KINDS.has(n.kind) && n.name.toLowerCase() === t.toLowerCase()) {
namedTypeFiles.add(n.filePath);
}
}
}
let cands = raw
.filter((n) => CALLABLE.has(n.kind) && !isTestPath(n.filePath))
.sort((a, b) => (bodyLines(b) > 1 ? 1 : 0) - (bodyLines(a) > 1 ? 1 : 0) || bodyLines(b) - bodyLines(a));
@@ -3572,10 +3620,18 @@ export class ToolHandler {
// DO-NOT-EDIT banner and nothing in its name) down-ranks the same way
// `.pb.go` always has (#1500). Covers the whole subgraph, not just the
// grouped files, because the graph-mass penalty below is keyed on it too.
const isGeneratedCandidate = cg.generatedFilePredicate(new Set([
const penaltyCandidates = new Set([
...fileGroups.keys(),
...[...subgraph.nodes.values()].map((n) => n.filePath),
]));
]);
const isGeneratedCandidate = cg.generatedFilePredicate(penaltyCandidates);
// Second bounded probe over the same set: files declaring nothing but types
// that nothing in the index depends on (CG-28). A query that NAMED one of
// those types is asking about the declaration, so its file is exempt and
// ranks at full weight.
const isAmbientDeclaration = cg.ambientDeclarationFilePredicate(penaltyCandidates);
const isDampedDeclaration = (filePath: string): boolean =>
isAmbientDeclaration(filePath) && !namedTypeFiles.has(filePath);
/**
* Rank penalty for a file, applied to its relevance score AND (below) to its
@@ -3583,9 +3639,19 @@ export class ToolHandler {
* score alone would leave the #1500 case unfixed: the generated CRUD carries
* MORE graph mass than the hand-written use-case, and graph mass outranks
* score in the comparator.
*
* Generated and ambient-declaration are taken as the STRONGER of the two,
* never multiplied: a generated `.d.ts` has one property "not the
* implementation" that both signals happen to see, and charging it twice is
* how a file gets cliffed out of answers where it is genuinely relevant
* (CG-28). The low-value multiplier is orthogonal (a test file that is also
* generated is two independent reasons) and still compounds.
*/
const rankPenalty = (filePath: string): number =>
(isGeneratedCandidate(filePath) ? GENERATED_RANK_PENALTY : 1)
Math.min(
isGeneratedCandidate(filePath) ? GENERATED_RANK_PENALTY : 1,
isDampedDeclaration(filePath) ? AMBIENT_DECLARATION_RANK_PENALTY : 1,
)
* (isLowValue(filePath) ? LOW_VALUE_RANK_PENALTY : 1);
for (const [filePath, group] of fileGroups) {
@@ -3931,6 +3997,7 @@ export class ToolHandler {
spine: group.nodes.some((n) => flow.pathNodeIds.has(n.id)),
lowValue: isLowValue(fp),
generated: isGeneratedCandidate(fp),
ambientDeclaration: isAmbientDeclaration(fp),
penalty: rankPenalty(fp),
kinds: kindMix(group.nodes),
});