fix(explore): reliably pin extension-less kebab-case file basenames in queries
Previously, naming a kebab-case file without its extension (e.g., `background-image-table` vs. `background-image-table.tsx`) in a `codegraph_explore` query would shred the name into fragments (`background`, `image`, `table`), admitting irrelevant sibling files and crowding out the intended target. This change introduces a new resolution pass in `extractQueryPaths` specifically for extension-less kebab basenames. Queries now accurately identify and pin these files. Unresolved hyphenated prose (e.g., `cross-call`) is left in the query for FTS without being flagged as an unknown path. Resolution prioritizes explicit slashed/dotted paths and respects an ambiguity budget for common stems to prevent over-pinning.
This commit is contained in:
@@ -22,6 +22,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
### Fixes
|
||||
|
||||
- Naming a file by its path in a `codegraph_explore` query now works reliably: the path is resolved against the index and that file is guaranteed a place at the top of the answer. Previously the path was broken into fragments — bracketed route segments like SvelteKit's `[id]` made this worst — and pieces like `page` or `runs` matched every sibling file, so the file you actually named could be crowded out of the answer entirely. A path that doesn't match any indexed file is now called out instead of silently ignored.
|
||||
- Naming a kebab-case file **without its extension** in a `codegraph_explore` query — `background-image-table` rather than `background-image-table.tsx`, the way import paths and prose spell it — now returns that exact file too. Previously the name was split at the hyphens, and in a kebab-cased frontend those pieces (`background`, `image`, `table`) are among the most common words in the codebase, so look-alike sibling files filled the answer while the named file never appeared. Hyphenated words that don't name an indexed file, like "cross-call" or "non-blocking", are left alone.
|
||||
- Plainly-worded `codegraph_explore` questions now find camelCase code: a query like "auto-scroll to bottom" can reach a function named `scrollFeedToBottom`, because query words are matched against the words inside identifiers, not just whole names.
|
||||
- Variables and constants now count when `codegraph_explore` picks its starting symbols, so state held in plain variables — `$state`-style variables in Svelte, for example — no longer gets overlooked.
|
||||
- C, C++, Objective-C and Rust unions are now indexed as first-class `union` nodes. A `union` declaration previously produced no symbol at all, so it never appeared in search or `codegraph_explore`, and anything attached to it disappeared with it — in Rust, every `impl SomeTrait for MyUnion` lost its edge, the methods from that impl were left pointing at a type the graph did not contain, and asking which types implement a trait quietly skipped the union ones. A union-shaped dispatch table in C now resolves its function pointers like a struct-shaped one. A `typedef union { … } Name;` in C keeps the typedef's name and is no longer mistaken for a plain type alias. Thanks @ctype-lab. Re-index after upgrading to pick up unions in existing projects. (#1515)
|
||||
|
||||
@@ -92,6 +92,23 @@ describe('path pinning (fix 1)', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('extension-less kebab basenames (the amnisphere gap)', () => {
|
||||
const KEBAB_TARGET = 'src/lib/background-image-table.ts';
|
||||
|
||||
it('a bare kebab basename — no slash, no extension — pins and renders its file', async () => {
|
||||
// Pre-fix this query never opened the path gate; FTS shredded the token
|
||||
// into `background`/`image`/`table` and served the fragment decoy instead.
|
||||
const out = await explore('background-image-table Source column');
|
||||
expect(hasSection(out, KEBAB_TARGET)).toBe(true);
|
||||
expect(out).toContain('pinned from the query');
|
||||
});
|
||||
|
||||
it('kebab prose that names no file is not reported as an unresolved path', async () => {
|
||||
const out = await explore('how does cross-call dedup interact with feed scroll pinning');
|
||||
expect(out).not.toContain('No indexed file uniquely matches');
|
||||
});
|
||||
});
|
||||
|
||||
describe('segment supplement + variable seeding (fixes 2–3)', () => {
|
||||
it('word-level scroll terms reach the camelCase scroll code without a path', async () => {
|
||||
const out = await explore('feed auto-scroll to bottom pinning behavior');
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
/** Table of background images for a training set — source-column rendering. */
|
||||
|
||||
export interface BackgroundImageRow {
|
||||
id: string;
|
||||
sourceUrl: string;
|
||||
label: string;
|
||||
}
|
||||
|
||||
let tableRows: BackgroundImageRow[] = [];
|
||||
|
||||
export function loadTableRows(rows: BackgroundImageRow[]): void {
|
||||
tableRows = rows;
|
||||
}
|
||||
|
||||
export function renderSourceColumn(row: BackgroundImageRow): string {
|
||||
return `${row.label}: ${row.sourceUrl}`;
|
||||
}
|
||||
|
||||
export function sortRowsBySource(): BackgroundImageRow[] {
|
||||
return [...tableRows].sort((a, b) => a.sourceUrl.localeCompare(b.sourceUrl));
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
/** Uploaded-background registry — shares the `background` fragment with the table file. */
|
||||
|
||||
let backgrounds: string[] = [];
|
||||
|
||||
export function addBackground(url: string): void {
|
||||
backgrounds.push(url);
|
||||
}
|
||||
|
||||
export function listBackgrounds(): string[] {
|
||||
return [...backgrounds];
|
||||
}
|
||||
@@ -22,6 +22,21 @@ const INDEX = [
|
||||
'src/lib/task-runner-manager.ts',
|
||||
'src/lib/stores/sqlite-store.ts',
|
||||
'src/lib/stores/postgresql-store.ts',
|
||||
// Kebab-case frontend shapes (the amnisphere extension-less-basename bug):
|
||||
'src/components/training-set-page/training-set-page.tsx',
|
||||
'src/components/training-set-page/training-set-page-background-images.tsx',
|
||||
'src/components/training-set-page/training-set-page.module.scss',
|
||||
'src/components/training-set-page/background-image-table.tsx',
|
||||
'src/components/modal/add-to-training-set/add-to-training-set.tsx',
|
||||
'src/pages/library-page-layout.tsx',
|
||||
'src/api/job-manager/backgrounds.ts',
|
||||
'src/x/generic-modal.tsx',
|
||||
'src/y/generic-modal.tsx',
|
||||
'scripts/pre-commit',
|
||||
'src/a/user-profile.tsx',
|
||||
'src/b/user-profile.tsx',
|
||||
'src/c/user-profile.tsx',
|
||||
'src/d/user-profile.tsx',
|
||||
];
|
||||
|
||||
describe('queryMightContainPaths — the cheap pre-gate', () => {
|
||||
@@ -35,6 +50,18 @@ describe('queryMightContainPaths — the cheap pre-gate', () => {
|
||||
// `.isPackaged` is 10 chars — past the 8-char extension cap.
|
||||
expect(queryMightContainPaths('what reads app.isPackaged here')).toBe(false);
|
||||
});
|
||||
|
||||
it('fires on extension-less kebab basenames — with or without wrapping', () => {
|
||||
expect(queryMightContainPaths('background-image-table Source column')).toBe(true);
|
||||
expect(queryMightContainPaths('the `library-page-layout` wrapper')).toBe(true);
|
||||
expect(queryMightContainPaths('usage, add-to-training-set.')).toBe(true);
|
||||
});
|
||||
|
||||
it('stays quiet on flags, snake_case, and snake-with-a-dash hybrids', () => {
|
||||
expect(queryMightContainPaths('run it with --no-cache maybe')).toBe(false);
|
||||
expect(queryMightContainPaths('where is background_image_table used')).toBe(false);
|
||||
expect(queryMightContainPaths('the foo_bar-baz helper')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractQueryPaths — resolution and stripping', () => {
|
||||
@@ -127,3 +154,85 @@ describe('extractQueryPaths — resolution and stripping', () => {
|
||||
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [] });
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractQueryPaths — extension-less kebab basenames', () => {
|
||||
it('pins the file a bare kebab basename names and consumes the token', () => {
|
||||
const out = extractQueryPaths('background-image-table Source column', INDEX);
|
||||
expect(out.pinnedFiles)
|
||||
.toEqual(['src/components/training-set-page/background-image-table.tsx']);
|
||||
expect(out.strippedQuery).toBe('Source column');
|
||||
expect(out.unresolvedPathSpans).toEqual([]);
|
||||
});
|
||||
|
||||
it('resolves with no slash or extension anywhere in the query (session-4 shape)', () => {
|
||||
const out = extractQueryPaths(
|
||||
'TrainingSetPage train modal library-page-layout AddToTrainingSetModal usage', INDEX,
|
||||
);
|
||||
expect(out.pinnedFiles).toEqual(['src/pages/library-page-layout.tsx']);
|
||||
// Identifier-shaped tokens stay for the named-symbol seeder.
|
||||
expect(out.strippedQuery).toBe('TrainingSetPage train modal AddToTrainingSetModal usage');
|
||||
});
|
||||
|
||||
it('pins every named file in a mixed dotted + kebab query (session-1 shape)', () => {
|
||||
const out = extractQueryPaths(
|
||||
'add-to-training-set training-set-page-background-images backgrounds.ts background-image-table Source column',
|
||||
INDEX,
|
||||
);
|
||||
expect(out.pinnedFiles).toEqual([
|
||||
// The dotted pass runs first, so the explicit basename pins ahead of the kebabs.
|
||||
'src/api/job-manager/backgrounds.ts',
|
||||
'src/components/modal/add-to-training-set/add-to-training-set.tsx',
|
||||
'src/components/training-set-page/training-set-page-background-images.tsx',
|
||||
'src/components/training-set-page/background-image-table.tsx',
|
||||
]);
|
||||
expect(out.strippedQuery).toBe('Source column');
|
||||
});
|
||||
|
||||
it('leaves kebab prose that names no indexed file untouched — and unreported', () => {
|
||||
const q = 'how does cross-call dedup make explore non-blocking';
|
||||
const out = extractQueryPaths(q, INDEX);
|
||||
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [] });
|
||||
});
|
||||
|
||||
it('leaves a stem shared by too many files alone — one hot name must not pin half the repo', () => {
|
||||
const q = 'refactor the user-profile rendering';
|
||||
const out = extractQueryPaths(q, INDEX);
|
||||
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [] });
|
||||
});
|
||||
|
||||
it('pins all files sharing a stem when within the ambiguity budget', () => {
|
||||
const out = extractQueryPaths('generic-modal close behavior', INDEX);
|
||||
expect(out.pinnedFiles).toEqual(['src/x/generic-modal.tsx', 'src/y/generic-modal.tsx']);
|
||||
});
|
||||
|
||||
it('matches case-insensitively and through wrapping punctuation', () => {
|
||||
expect(extractQueryPaths('see `Background-Image-Table`.', INDEX).pinnedFiles)
|
||||
.toEqual(['src/components/training-set-page/background-image-table.tsx']);
|
||||
});
|
||||
|
||||
it('stems drop only the last extension — a kebab token cannot pin a .module.scss sibling', () => {
|
||||
const out = extractQueryPaths('training-set-page props flow', INDEX);
|
||||
expect(out.pinnedFiles)
|
||||
.toEqual(['src/components/training-set-page/training-set-page.tsx']);
|
||||
});
|
||||
|
||||
it('pins an extension-less indexed file by its exact name', () => {
|
||||
expect(extractQueryPaths('what does the pre-commit hook run', INDEX).pinnedFiles)
|
||||
.toEqual(['scripts/pre-commit']);
|
||||
});
|
||||
|
||||
it('skips tokens the dotted pass consumed and dedupes a file named both ways', () => {
|
||||
const out = extractQueryPaths('src/lib/chat-manager.ts vs chat-manager internals', INDEX);
|
||||
expect(out.pinnedFiles).toEqual(['src/lib/chat-manager.ts']);
|
||||
expect(out.strippedQuery).toBe('vs internals');
|
||||
});
|
||||
|
||||
it('explicit paths win the shared maxPins budget over kebab tokens', () => {
|
||||
const out = extractQueryPaths(
|
||||
'background-image-table then src/lib/chat-manager.ts', INDEX, { maxPins: 1 },
|
||||
);
|
||||
expect(out.pinnedFiles).toEqual(['src/lib/chat-manager.ts']);
|
||||
// The kebab token was not consumed once the budget was spent — it stays for FTS.
|
||||
expect(out.strippedQuery).toBe('background-image-table then');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -15,10 +15,13 @@
|
||||
* sibling `+page.svelte` in the repo, which ate the output envelope and
|
||||
* truncated the files the agent actually asked for.
|
||||
*
|
||||
* `extractQueryPaths` finds path-like spans, resolves them against the
|
||||
* INDEXED file list (resolution IS the detector — `and/or`, `gen_server:call/2`
|
||||
* and other slash-bearing non-paths match nothing and are left alone), and
|
||||
* returns the matches as pinned files plus the query with those spans removed.
|
||||
* `extractQueryPaths` finds path-like spans — slashed paths, dotted basenames,
|
||||
* and extension-less kebab basenames (`background-image-table`, the spelling
|
||||
* import paths and prose actually use) — resolves them against the INDEXED
|
||||
* file list (resolution IS the detector — `and/or`, `gen_server:call/2`,
|
||||
* `non-blocking` and other path-shaped non-paths match nothing and are left
|
||||
* alone), and returns the matches as pinned files plus the query with those
|
||||
* spans removed.
|
||||
* Callers treat pinned files as first-class: guaranteed admission, top rank,
|
||||
* funded first. Pure string work — no DB, no fs — so it is trivially testable
|
||||
* and safe inside the query-pool workers.
|
||||
@@ -40,12 +43,18 @@ export interface QueryPathExtraction {
|
||||
|
||||
/**
|
||||
* Cheap pre-gate so callers only fetch the indexed file list when the query
|
||||
* could possibly contain a path: a slash, or a dot-extension-shaped tail
|
||||
* (`chat-manager.ts`). Extensions cap at 8 chars, which keeps `Class.method`
|
||||
* spans (`app.isPackaged`) from qualifying.
|
||||
* could possibly contain a path: a slash, a dot-extension-shaped tail
|
||||
* (`chat-manager.ts`), or a hyphen-joined word (`background-image-table` —
|
||||
* kebab files are named WITHOUT their extension more often than with, so the
|
||||
* shape must open the gate on its own). Extensions cap at 8 chars, which
|
||||
* keeps `Class.method` spans (`app.isPackaged`) from qualifying; the kebab
|
||||
* alternative requires clean non-word boundaries, which keeps `--flags` and
|
||||
* snake_case-with-a-dash hybrids from firing it.
|
||||
*/
|
||||
export function queryMightContainPaths(query: string): boolean {
|
||||
return /[/\\]/.test(query) || /\.[A-Za-z][A-Za-z0-9]{0,7}(?=[\s,;:)\]'"`]|$)/.test(query);
|
||||
return /[/\\]/.test(query)
|
||||
|| /\.[A-Za-z][A-Za-z0-9]{0,7}(?=[\s,;:)\]'"`]|$)/.test(query)
|
||||
|| /(?:^|[^-\w])[A-Za-z0-9]+(?:-[A-Za-z0-9]+)+(?=[^-\w]|$)/.test(query);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -60,6 +69,39 @@ const MAX_CANDIDATE_SPANS = 8;
|
||||
/** `name.ext` shape with a plausible source extension (no slash required). */
|
||||
const DOTTED_BASENAME = /^[^\s/\\]+\.[A-Za-z][A-Za-z0-9]{0,7}$/;
|
||||
|
||||
/**
|
||||
* Extension-less kebab basename (`background-image-table`). Hyphens are
|
||||
* illegal in identifiers, so consuming these tokens can never steal one from
|
||||
* the named-symbol seeder; ≥2 segments keeps single words out.
|
||||
*/
|
||||
const KEBAB_BASENAME = /^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)+$/;
|
||||
|
||||
/** A basename's last dot-extension, same shape DOTTED_BASENAME accepts. */
|
||||
const LAST_EXTENSION = /\.[A-Za-z][A-Za-z0-9]{0,7}$/;
|
||||
|
||||
/**
|
||||
* Lowercased basename stems of the hyphen-named indexed files, stem → paths.
|
||||
* A stem drops only the LAST extension (`a-b.module.scss` → `a-b.module`), so
|
||||
* a bare kebab token can't accidentally pin a same-named stylesheet or
|
||||
* `.d.ts` sibling of the source file it names; an extension-less basename
|
||||
* (`pre-commit`) is its own stem. Hyphen-free basenames are skipped — a
|
||||
* KEBAB_BASENAME token can never equal one, and the filter keeps the map
|
||||
* near-empty in repos that don't name files this way.
|
||||
*/
|
||||
function buildBasenameStems(indexedPaths: readonly string[]): Map<string, string[]> {
|
||||
const stems = new Map<string, string[]>();
|
||||
for (const p of indexedPaths) {
|
||||
const basename = p.slice(Math.max(p.lastIndexOf('/'), p.lastIndexOf('\\')) + 1);
|
||||
if (!basename.includes('-')) continue;
|
||||
const stem = basename.replace(LAST_EXTENSION, '').toLowerCase();
|
||||
if (!stem) continue;
|
||||
const existing = stems.get(stem);
|
||||
if (existing) existing.push(p);
|
||||
else stems.set(stem, [p]);
|
||||
}
|
||||
return stems;
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip prose punctuation wrapped around a token without eating punctuation
|
||||
* that is PART of the path: quotes/backticks always strip; a trailing `)`/`]`
|
||||
@@ -204,6 +246,38 @@ export function extractQueryPaths(
|
||||
// leave the token for the normal matching pipeline.
|
||||
}
|
||||
|
||||
// Second pass — extension-less kebab basenames. `background-image-table`
|
||||
// opens no door above (no slash, no dotted tail), the hyphens disqualify it
|
||||
// from the named-symbol seeder downstream, and FTS shreds it into the most
|
||||
// common words in a kebab-cased repo (`background`, `image`, `table`) —
|
||||
// which admit look-alike SIBLINGS that crowd out the named file. Resolution
|
||||
// stays the detector: a token pins only when its whole lowercased form is
|
||||
// the stem of an indexed basename. Two deliberate asymmetries vs the first
|
||||
// pass: prose that resolves to nothing (`non-blocking`, `cross-call`) is
|
||||
// LEFT IN the query — unlike a slashed span it may be legitimate wording,
|
||||
// so it keeps feeding FTS and is not reported as an unresolved path — and a
|
||||
// stem hotter than maxMatchesPerSpan is likewise left alone (pinning half a
|
||||
// monorepo off one hot name trades precision the wrong way; a directory
|
||||
// segment, which the first pass handles, disambiguates). Runs after the
|
||||
// slashed/dotted pass so explicit paths win the shared maxPins budget, and
|
||||
// examines every remaining token: lookups are O(1) map hits, so the
|
||||
// scan-cost rationale behind MAX_CANDIDATE_SPANS doesn't apply.
|
||||
let basenameStems: Map<string, string[]> | null = null;
|
||||
for (let i = 0; i < tokens.length && pinned.length < maxPins; i++) {
|
||||
if (consumed.has(i)) continue;
|
||||
const stripped = stripWrapping(tokens[i]!);
|
||||
if (stripped.length < 4 || !KEBAB_BASENAME.test(stripped)) continue;
|
||||
basenameStems ??= buildBasenameStems(indexedPaths);
|
||||
const matches = basenameStems.get(stripped.toLowerCase());
|
||||
if (!matches || matches.length > maxMatchesPerSpan) continue;
|
||||
consumed.add(i);
|
||||
for (const m of matches) {
|
||||
if (pinnedSeen.has(m) || pinned.length >= maxPins) continue;
|
||||
pinnedSeen.add(m);
|
||||
pinned.push(m);
|
||||
}
|
||||
}
|
||||
|
||||
if (consumed.size === 0) return passthrough;
|
||||
return {
|
||||
strippedQuery: tokens.filter((_, i) => !consumed.has(i)).join(' '),
|
||||
|
||||
Reference in New Issue
Block a user