fix(retrieval): multi-hump field-name queries reach their definers (#1319)
Three compounding defects (#1196) made a query bag of object-literal keys (`profileInfo isTrialEligible quotaInfo billingMethod`) return unrelated results while the defining files never surfaced: 1. Step 5b title-cased interior humps (profileInfo -> Profileinfo) and then compared case-SENSITIVELY, dropping every row SQLite's case-insensitive LIKE had just recovered. The hump lookup is now case-insensitive with an explicit uppercase-at-match requirement. 2. Step 5b/5c's kind whitelist held only type-like kinds — dead code on method-centric codebases. Callable kinds (function/method/component) are fetched as a SEPARATE LIKE batch so hot single-word terms can't crowd classes out of the length-ordered 200-row batch. 3. explore's named-symbol seeding was exact-name only; a field token seeded nothing. A camelCase token with ZERO exact defs now seeds its camel-infix definers (callables, hump-boundary or prefix, shortest first, capped at 3) — bare lowercase words keep the #1252 stopword guard untouched. The reporter's acceptance query is a pinned e2e test (definer files present, exact-name seeding unaffected). excalidraw probe: the canonical flow query (mutateElement renderStaticScene) is byte- identical; NL queries shift toward more-central callables (useUIAppState/getDefaultAppState over observer periphery). Fixes #1196 Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
a5a8942d1c
commit
1de7e8f8b5
@@ -18,6 +18,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
### Fixes
|
||||
|
||||
- Searching or exploring by field names now finds the code that defines them. A query made of object keys or API field names (`profileInfo isTrialEligible quotaInfo billingMethod`) used to return unrelated results while the defining files never appeared, because three retrieval steps each dropped multi-word camelCase terms: an internal case-comparison bug, a match step that only considered classes (never functions or methods), and exploration seeding that required exact symbol-name matches. All three are fixed — `codegraph_explore` with a bag of field names now surfaces the controllers and services that assemble those fields. (#1196)
|
||||
- `codegraph.json`'s `includeIgnored` works again for the "folder of repos" layout: when one `.gitignore` rule covers a parent directory (`/repos/`) holding several embedded git repositories, opting in the individual repos (`"includeIgnored": ["repos/a/"]` — the exact spelling `codegraph init`'s own hint suggests) previously matched nothing and indexed zero files, looping the same suggestion back at you. Both spellings now work — name the parent directory to opt in everything under it, or name individual repos to opt in just those — and the hint no longer re-suggests repos that are already configured. (#1295)
|
||||
- Method calls on literals (`", ".join(...)` in Python, `"x".split(...)` in JavaScript, and the like) no longer produce call edges to unrelated project functions that happen to share the builtin's name — a codebase with a function called `join`, `get`, or `update` could show phantom callers from every string-builtin use. Additionally, a function nested inside another function is now only matched as a call target from inside its container, since it isn't reachable from anywhere else. Blast-radius and affected-test results get cleaner on Python and JavaScript codebases especially. (#1230)
|
||||
- Go method calls through a struct field (`target.conn.Exec(...)`) no longer bind to unrelated same-named local methods when the field's type is external — `conn *sql.DB` calls were being attributed to a local interface that happened to declare `Exec`, fabricating internal dependencies. Chained field calls now resolve by inferring the field's declared type from the struct definition: in-project types (including unexported ones like chi's `tree *node`) gain correct, validated call edges that never existed before, and external types (standard library, third-party modules) are left unlinked instead of guessed. (#1276)
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
/**
|
||||
* Multi-word FIELD-NAME query retrieval (#1196).
|
||||
*
|
||||
* A query bag of object-literal keys / API field names (`profileInfo
|
||||
* isTrialEligible quotaInfo billingMethod`) has no nodes of its own — the
|
||||
* definers are methods whose names contain each token at a camel-hump
|
||||
* boundary (`profileInfo` → `getProfileInfoV2`). Three compounding defects
|
||||
* made those definers unreachable:
|
||||
* 1. the CamelCase-boundary LIKE step title-cased interior humps
|
||||
* (`profileInfo` → `Profileinfo`) and then compared case-SENSITIVELY,
|
||||
* dropping every row SQLite's case-insensitive LIKE had just found;
|
||||
* 2. that step's kind whitelist held only type-like kinds, so on
|
||||
* method-centric codebases it contributed nothing at all;
|
||||
* 3. explore's named-symbol seeding was exact-name only, so a field token
|
||||
* seeded no files and the output budget went to unrelated neighbors.
|
||||
*/
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
|
||||
describe('field-name query retrieval (#1196)', () => {
|
||||
let testDir: string;
|
||||
let cg: CodeGraph;
|
||||
let handler: ToolHandler;
|
||||
|
||||
beforeEach(async () => {
|
||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-1196-'));
|
||||
fs.mkdirSync(path.join(testDir, 'controller'), { recursive: true });
|
||||
fs.mkdirSync(path.join(testDir, 'service'), { recursive: true });
|
||||
|
||||
fs.writeFileSync(
|
||||
path.join(testDir, 'controller', 'profileController.js'),
|
||||
`const billing = require('../service/billing');
|
||||
|
||||
class ProfileController {
|
||||
getProfileInfo(userId) {
|
||||
return { profileInfo: { id: userId }, isTrialEligible: this.checkTrialEligibility(userId) };
|
||||
}
|
||||
getProfileInfoV2(userId) {
|
||||
const quotaInfo = this.loadQuotaInfo(userId);
|
||||
return { profileInfo: { id: userId }, quotaInfo, billingMethod: billing.getBillingMethod(userId) };
|
||||
}
|
||||
checkTrialEligibility(userId) { return userId > 100; }
|
||||
loadQuotaInfo(userId) { return { used: 1, max: 10, userId }; }
|
||||
}
|
||||
module.exports = new ProfileController();
|
||||
`
|
||||
);
|
||||
fs.writeFileSync(
|
||||
path.join(testDir, 'service', 'billing.js'),
|
||||
`function _getCustomerBillingMethods(userId) {
|
||||
return [{ type: 'card', userId }];
|
||||
}
|
||||
function getBillingMethod(userId) {
|
||||
return _getCustomerBillingMethods(userId)[0];
|
||||
}
|
||||
module.exports = { getBillingMethod };
|
||||
`
|
||||
);
|
||||
// Noise files so the definers aren't the only content.
|
||||
for (let i = 1; i <= 5; i++) {
|
||||
fs.writeFileSync(
|
||||
path.join(testDir, 'service', `noise${i}.js`),
|
||||
`function unrelatedHelper${i}() { return ${i}; }\nmodule.exports = { unrelatedHelper${i} };\n`
|
||||
);
|
||||
}
|
||||
|
||||
cg = CodeGraph.initSync(testDir);
|
||||
await cg.indexAll();
|
||||
handler = new ToolHandler(cg);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (cg) cg.destroy();
|
||||
if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('a bag of field-name tokens surfaces the files that DEFINE those fields', async () => {
|
||||
const res = await handler.execute('codegraph_explore', {
|
||||
query: 'profileInfo isTrialEligible quotaInfo billingMethod',
|
||||
});
|
||||
const text = res.content[0]!.text as string;
|
||||
|
||||
// The two definer files the reporter saw entirely absent.
|
||||
expect(text).toContain('profileController.js');
|
||||
expect(text).toContain('billing.js');
|
||||
// The camel-infix definers themselves are shown.
|
||||
expect(text).toMatch(/getProfileInfo(V2)?/);
|
||||
expect(text).toContain('BillingMethod');
|
||||
});
|
||||
|
||||
it('exact-name seeding still wins when the token IS a real symbol', async () => {
|
||||
// `getBillingMethod` names a real function — the fallback must not
|
||||
// dilute or replace exact seeding.
|
||||
const res = await handler.execute('codegraph_explore', { query: 'getBillingMethod' });
|
||||
const text = res.content[0]!.text as string;
|
||||
expect(text).toContain('billing.js');
|
||||
expect(text).toContain('getBillingMethod');
|
||||
});
|
||||
});
|
||||
+40
-11
@@ -749,6 +749,13 @@ export class ContextBuilder {
|
||||
if (symbolsFromQuery.length > 0) {
|
||||
const camelDefinitionKinds: NodeKind[] = ['class', 'interface', 'struct', 'trait',
|
||||
'protocol', 'enum', 'type_alias'];
|
||||
// Callable kinds participate too: in service-layer codebases the
|
||||
// camel-infix definers of a queried FIELD are methods/functions
|
||||
// (`profileInfo` → `getProfileInfoV2`), not classes — the type-only
|
||||
// whitelist made this whole step dead code there (#1196). Fetched as a
|
||||
// SEPARATE LIKE batch so one hot single-word term can't crowd classes
|
||||
// out of the length-ordered 200-row batch.
|
||||
const camelCallableKinds: NodeKind[] = ['function', 'method', 'component'];
|
||||
const camelSearchedTerms = new Set<string>();
|
||||
const searchIdSet = new Set(searchResults.map(r => r.node.id));
|
||||
// Track per-node term hits for multi-term boosting
|
||||
@@ -766,18 +773,32 @@ export class ContextBuilder {
|
||||
// have hundreds of substring matches. The LIKE scan cost is the same
|
||||
// regardless of LIMIT (SQLite scans all matches to sort), so we fetch
|
||||
// generously and let path-relevance scoring pick the best ones.
|
||||
const likeResults = this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelDefinitionKinds,
|
||||
excludePrefix: true,
|
||||
});
|
||||
const likeResults = [
|
||||
...this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelDefinitionKinds,
|
||||
excludePrefix: true,
|
||||
}),
|
||||
...this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelCallableKinds,
|
||||
excludePrefix: true,
|
||||
}),
|
||||
];
|
||||
|
||||
// Filter to CamelCase boundaries, score by path relevance, and take top N
|
||||
const termCandidates: SearchResult[] = [];
|
||||
for (const r of likeResults) {
|
||||
const name = r.node.name;
|
||||
const idx = name.indexOf(titleCased);
|
||||
// Case-INSENSITIVE hump lookup: title-casing lowercases interior
|
||||
// humps (`profileInfo` → `Profileinfo`), which SQLite's LIKE still
|
||||
// matched but a case-sensitive indexOf here silently dropped —
|
||||
// making every multi-hump query term unfindable by this step
|
||||
// (#1196). The match must still LAND on an uppercase char, so a
|
||||
// plain lowercase infix can't slip through.
|
||||
const idx = name.toLowerCase().indexOf(termKey);
|
||||
if (idx <= 0) continue;
|
||||
if (!/[A-Z]/.test(name.charAt(idx))) continue;
|
||||
// Accept CamelCase boundary (lowercase before match) OR
|
||||
// acronym boundary (uppercase before match, e.g., RPCProtocol)
|
||||
if (!/[a-zA-Z]/.test(name.charAt(idx - 1))) continue;
|
||||
@@ -841,11 +862,19 @@ export class ContextBuilder {
|
||||
const titleCased = sym.charAt(0).toUpperCase() + sym.slice(1).toLowerCase();
|
||||
if (titleCased.length < 3) continue;
|
||||
|
||||
const likeResults = this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelDefinitionKinds,
|
||||
excludePrefix: false,
|
||||
});
|
||||
const likeResults = [
|
||||
...this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelDefinitionKinds,
|
||||
excludePrefix: false,
|
||||
}),
|
||||
// Same separate callable batch as Step 5b (#1196).
|
||||
...this.queries.findNodesByNameSubstring(titleCased, {
|
||||
limit: 200,
|
||||
kinds: camelCallableKinds,
|
||||
excludePrefix: false,
|
||||
}),
|
||||
];
|
||||
|
||||
for (const r of likeResults) {
|
||||
if (searchIdSet.has(r.node.id)) continue;
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
import * as path from 'path';
|
||||
import {
|
||||
Node,
|
||||
NodeKind,
|
||||
Edge,
|
||||
FileRecord,
|
||||
ExtractionResult,
|
||||
@@ -1220,6 +1221,20 @@ export class CodeGraph {
|
||||
return this.queries.getNodesByNamePrefix(prefix, limit);
|
||||
}
|
||||
|
||||
/**
|
||||
* Nodes whose name CONTAINS `substring` (LIKE scan, ASCII-case-insensitive,
|
||||
* shortest-first). The camel-infix lookup FTS can't do — `profileInfo`
|
||||
* inside `getProfileInfoV2` is one FTS token (#1196).
|
||||
*/
|
||||
getNodesByNameSubstring(
|
||||
substring: string,
|
||||
options: { kinds?: NodeKind[]; limit?: number; excludePrefix?: boolean } = {}
|
||||
): Node[] {
|
||||
return this.queries
|
||||
.findNodesByNameSubstring(substring, options)
|
||||
.map((r) => r.node);
|
||||
}
|
||||
|
||||
/**
|
||||
* Search nodes by text
|
||||
*/
|
||||
|
||||
@@ -2653,6 +2653,31 @@ export class ToolHandler {
|
||||
let cands = raw
|
||||
.filter((n) => CALLABLE.has(n.kind) && !isTestPath(n.filePath))
|
||||
.sort((a, b) => (bodyLines(b) > 1 ? 1 : 0) - (bodyLines(a) > 1 ? 1 : 0) || bodyLines(b) - bodyLines(a));
|
||||
// Field-name seeding fallback (#1196): a camelCase token that names NO
|
||||
// definition of its own is usually an object-literal key / API field
|
||||
// (`profileInfo`) — no node exists, so it contributed zero seeds and
|
||||
// the files that DEFINE it (`getProfileInfoV2` in profileController)
|
||||
// never surfaced. Seed its camel-infix definers instead: callables
|
||||
// whose name contains the token at a hump boundary or as a prefix.
|
||||
// Exact-empty + camel-shaped only (bare words keep the NL-stopword
|
||||
// guard below), shortest-first, capped so a hot infix can't flood.
|
||||
if (cands.length === 0 && !isQual && /[a-z][A-Z]/.test(t)) {
|
||||
const lcToken = t.toLowerCase();
|
||||
cands = cg
|
||||
.getNodesByNameSubstring(t, {
|
||||
kinds: ['function', 'method', 'component'],
|
||||
limit: 60,
|
||||
})
|
||||
.filter((n) => CALLABLE.has(n.kind) && !isTestPath(n.filePath))
|
||||
.filter((n) => {
|
||||
const idx = n.name.toLowerCase().indexOf(lcToken);
|
||||
if (idx < 0) return false;
|
||||
if (idx === 0) return n.name.length > t.length; // prefix definer
|
||||
return /[A-Z]/.test(n.name.charAt(idx)); // camel-hump boundary
|
||||
})
|
||||
.sort((a, b) => a.name.length - b.name.length)
|
||||
.slice(0, 3);
|
||||
}
|
||||
// Bare lowercase words only seed defs their query-siblings corroborate
|
||||
// (see the NL-stopword guard above). Filtering CANDS (not picks) applies
|
||||
// the guard uniformly to both branches below, including the >3-def
|
||||
|
||||
Reference in New Issue
Block a user