* feat(config): add codegraph.json "deprioritize" for ranking-only path down-weighting matchesNonProductionDir hardcodes example/sample/fixture/benchmark/demo, so a peripheral tree only the project knows about — optional-skills/, scripts/ — gets no de-prioritization. When helpers there carry generic symbol names, an exact name match hands them a large bonus and they crowd out the product code that answers the query (#982). deprioritize is the RANKING counterpart to exclude: those paths stay indexed and findable, they just stop outranking first-party code. It is deliberately distinct from the corpus-frequency discount, which keys on a name being common and is near-inert on #982's own repro where only two symbols are named usage. The -15 path penalty alone is not enough, and measuring showed why: on that repro a usage() helper sits at 74.8 against 51.2 for the top product symbol, so -15 lands at 59.8 and still leads. The path penalty is additive and the name bonus it must counter is additive and larger. A de-prioritized path is saying its symbol NAMES are not the answer, so the exact-name bonus is damped to 0.25x there as well — damped, not zeroed, so the tree still ranks when it genuinely is what you asked for. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(config): read deprioritize lazily and apply it in explore too Review of the first cut found two real defects. The matcher was built once in wireLayers(), which runs only from the constructor and from reopenIfReplaced(). The MCP server keeps one CodeGraph per project root alive for its whole lifetime, so editing codegraph.json appeared to do nothing until the process restarted -- exclude and include do not behave that way. The predicate now reads loadDeprioritizePatterns() per call (mtime-cached, one stat) and memoizes the compiled matcher on the pattern array's identity. A regression test writes the config after opening the project and fails on the old code. Explore passed no matcher to scorePathRelevance at either of its two call sites, so the setting only half-applied -- and #982's reproduction rows B, C and D are all codegraph explore, which made this the surface the issue actually reports on. Both sites now pass it. Explore's hard early-continue filters and its non-production budget cap are deliberately NOT joined: those REMOVE content, and deprioritize is a ranking lever by definition. README narrowed accordingly -- it previously claimed this extends the built-in list, which overstated it. Also from review: scorePathRelevance takes a boolean rather than a predicate (the caller already evaluated it, and it was being invoked twice per result), the predicate body is exception-guarded so a bad path can never take a search down, the misplaced const moved out from between imports, two vacuous test assertions tightened, and tests added for the single-penalty invariant, the deliberate isTestQuery asymmetry, and a query that genuinely targets the de-prioritized tree. Refs #982 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SKXAJMrVrdHS5Uco6ABtky * fix(search): derive the deprioritize name-bonus damping instead of picking it (#982) The 0.25 scale was a guess. On a 62k-node django index it measurably breaks the "discount, don't erase" rule the lever is built on: exact-name queries for symbols that live only in the de-prioritized tree (child, parent, method) fall behind mere prefix matches (children, all_parents, method_decorator). The prefix arm of nameMatchBonus tops out below 40, and a de-prioritized node also takes the -15 path penalty, so 80 * SCALE - 15 > 40 is the bound that keeps a damped exact match ahead of a prefix match at any corpus shape. 0.75 clears it; crowd-out removal is nearly identical to 0.5 (39 vs 40 of 88 peripheral top-10 slots cleared on django), so the deeper discount bought almost nothing and cost the invariant. Two tests pin the bound, including one that fails at the old 0.25. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Max Hsu
Claude Opus 5
+51
-7
@@ -55,6 +55,23 @@ function isLowValueFile(filePath: string, generated?: ReadonlySet<string>): bool
|
||||
|
||||
const SQLITE_PARAM_CHUNK_SIZE = 500;
|
||||
|
||||
/**
|
||||
* How much of the exact-name bonus a `deprioritize`d path keeps (#982). Damped
|
||||
* rather than zeroed: a query that genuinely targets that tree must still rank
|
||||
* it, the same "discount, don't erase" rule the path penalty follows.
|
||||
*
|
||||
* Derived rather than picked. `nameMatchBonus`'s prefix arm tops out below
|
||||
* `10 + 30 = 40`, and a de-prioritized node also takes the -15 path penalty, so
|
||||
* `80 * SCALE - 15 > 40` is what stops a damped WHOLE-QUERY exact match from
|
||||
* losing to a mere prefix match. 0.75 clears it (45). Measured on a 62k-node
|
||||
* django index: at 0.25 that invariant breaks in practice — `child`, `parent`
|
||||
* and `method` lose rank 1 to `children`, `all_parents` and `method_decorator`
|
||||
* — while crowd-out removal is almost flat between 0.75 and 0.5 (39 vs 40 of 88
|
||||
* peripheral top-10 slots cleared), so a deeper discount buys little and costs
|
||||
* the invariant. Pinned by a test.
|
||||
*/
|
||||
export const DEPRIORITIZED_NAME_BONUS_SCALE = 0.75;
|
||||
|
||||
/**
|
||||
* Database row types (snake_case from SQLite)
|
||||
*/
|
||||
@@ -204,6 +221,7 @@ export class QueryBuilder {
|
||||
// whole project, not a symbol, so it carries no discriminative signal (#720).
|
||||
// Set once by the CodeGraph instance; empty by default (no down-weighting).
|
||||
private projectNameTokens: Set<string> = new Set();
|
||||
private isDeprioritizedPath: ((filePath: string) => boolean) | undefined;
|
||||
|
||||
// Node cache for frequently accessed nodes (LRU-style, max 1000 entries)
|
||||
private nodeCache: Map<string, Node> = new Map();
|
||||
@@ -328,6 +346,21 @@ export class QueryBuilder {
|
||||
return this.projectNameTokens;
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the predicate that marks a path as de-prioritized by the project's
|
||||
* `codegraph.json` `deprioritize` patterns (#982). Ranking-only: those paths
|
||||
* stay indexed and findable, they just stop outranking first-party code.
|
||||
* Called once when the project opens; undefined disables the lever.
|
||||
*/
|
||||
setDeprioritizedPathMatcher(matcher: ((filePath: string) => boolean) | undefined): void {
|
||||
this.isDeprioritizedPath = matcher;
|
||||
}
|
||||
|
||||
/** The `deprioritize` predicate (#982), so other rankers apply the same lever. */
|
||||
getDeprioritizedPathMatcher(): ((filePath: string) => boolean) | undefined {
|
||||
return this.isDeprioritizedPath;
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Node Operations
|
||||
// ===========================================================================
|
||||
@@ -1295,13 +1328,24 @@ export class QueryBuilder {
|
||||
// Apply multi-signal scoring
|
||||
if (results.length > 0 && (text || query)) {
|
||||
const scoringQuery = text || query;
|
||||
results = results.map(r => ({
|
||||
...r,
|
||||
score: r.score
|
||||
+ kindBonus(r.node.kind)
|
||||
+ scorePathRelevance(r.node.filePath, scoringQuery, this.projectNameTokens)
|
||||
+ nameMatchBonus(r.node.name, scoringQuery),
|
||||
}));
|
||||
results = results.map(r => {
|
||||
// A path the project de-prioritized is saying its symbol NAMES are not
|
||||
// the answer, so the exact-name bonus has to be damped too. The -15 path
|
||||
// penalty alone cannot do it: the bonus is additive and larger (measured
|
||||
// on #982's repro, a `usage()` helper sat at 74.8 vs 51.2 for the top
|
||||
// product symbol — -15 lands at 59.8, still ahead). Damped, not zeroed,
|
||||
// so the tree stays findable when it genuinely is what you asked for.
|
||||
// Evaluated once and reused: the predicate stats the config file.
|
||||
const deprioritized = this.isDeprioritizedPath?.(r.node.filePath) ?? false;
|
||||
const nameBonus = nameMatchBonus(r.node.name, scoringQuery);
|
||||
return {
|
||||
...r,
|
||||
score: r.score
|
||||
+ kindBonus(r.node.kind)
|
||||
+ scorePathRelevance(r.node.filePath, scoringQuery, this.projectNameTokens, deprioritized)
|
||||
+ (deprioritized ? Math.round(nameBonus * DEPRIORITIZED_NAME_BONUS_SCALE) : nameBonus),
|
||||
};
|
||||
});
|
||||
results.sort((a, b) => b.score - a.score);
|
||||
// Trim to requested limit after rescoring
|
||||
if (results.length > limit) {
|
||||
|
||||
Reference in New Issue
Block a user