`insertEdge` has always used `INSERT OR IGNORE`, but the edges table carried no UNIQUE constraint — only an autoincrement PK and non-unique indexes — so `OR IGNORE` had nothing to conflict on and behaved like a plain INSERT. Whenever two extraction/resolution passes emitted the same edge (e.g. a return type captured by both a type-reference and a value-reference pass), the graph stored byte-identical duplicate rows: ~527 on this repo, inflating edge counts and letting callers/impact list the same relationship twice. Add a UNIQUE identity index on (source, target, kind, IFNULL(line,-1), IFNULL(col,-1)) — in schema.sql for fresh databases and migration v6 (dedup existing rows, then create the index) for existing ones. IFNULL folds the nullable line/col so coordinate-less edges (synthesized / file-level) dedup too; SQLite otherwise treats each NULL as distinct. Distinct call sites (same source/target/kind, different line/col) are preserved — only byte-identical structural duplicates collapse. This is the storage-layer invariant the reporter identified: it makes OR IGNORE keep its promise and catches every double-emit, present and future, rather than chasing each emitting pass. Migration v6 is deterministic (keeps the lowest id per identity group) and idempotent (IF NOT EXISTS index; no-op DELETE once unique). The DELETE's GROUP BY matches the index expression exactly so creation can't fail on a leftover pair. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
2176a7a439
commit
0da2dcec8e
+26
-1
@@ -9,7 +9,7 @@ import { SqliteDatabase } from './sqlite-adapter';
|
||||
/**
|
||||
* Current schema version
|
||||
*/
|
||||
export const CURRENT_SCHEMA_VERSION = 5;
|
||||
export const CURRENT_SCHEMA_VERSION = 6;
|
||||
|
||||
/**
|
||||
* Migration definition
|
||||
@@ -75,6 +75,31 @@ const migrations: Migration[] = [
|
||||
`);
|
||||
},
|
||||
},
|
||||
{
|
||||
version: 6,
|
||||
description:
|
||||
'Dedup duplicate edge rows and add a UNIQUE identity index so INSERT OR IGNORE actually dedups (#1034)',
|
||||
up: (db) => {
|
||||
// `insertEdge` has always used `INSERT OR IGNORE`, but the edges table had
|
||||
// no UNIQUE constraint, so nothing conflicted and byte-identical rows
|
||||
// accumulated whenever two passes emitted the same edge. Collapse each
|
||||
// identity group to its lowest id, then add the constraint that makes
|
||||
// `OR IGNORE` keep its promise. IFNULL folds nullable line/col so
|
||||
// coordinate-less edges dedup too (SQLite treats each NULL as distinct) —
|
||||
// and it MUST match the GROUP BY exactly, or the index creation would
|
||||
// fail on a pair the DELETE left behind. Idempotent: the index is
|
||||
// `IF NOT EXISTS` and the DELETE is a no-op once the table is unique.
|
||||
db.exec(`
|
||||
DELETE FROM edges
|
||||
WHERE id NOT IN (
|
||||
SELECT MIN(id) FROM edges
|
||||
GROUP BY source, target, kind, IFNULL(line, -1), IFNULL(col, -1)
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_edges_identity
|
||||
ON edges(source, target, kind, IFNULL(line, -1), IFNULL(col, -1));
|
||||
`);
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
/**
|
||||
|
||||
@@ -133,6 +133,17 @@ CREATE INDEX IF NOT EXISTS idx_edges_kind ON edges(kind);
|
||||
CREATE INDEX IF NOT EXISTS idx_edges_source_kind ON edges(source, kind);
|
||||
CREATE INDEX IF NOT EXISTS idx_edges_target_kind ON edges(target, kind);
|
||||
|
||||
-- Edge identity uniqueness. An edge IS uniquely (source, target, kind, line,
|
||||
-- col); insertEdge uses `INSERT OR IGNORE`, but without something UNIQUE to
|
||||
-- conflict on it behaved like a plain INSERT, so two passes emitting the same
|
||||
-- edge produced byte-identical duplicate rows that inflated counts and flowed
|
||||
-- into callers/impact (#1034). IFNULL folds the nullable line/col so
|
||||
-- coordinate-less edges (synthesized / file-level) dedup too — SQLite treats
|
||||
-- each NULL as distinct otherwise. Migration v6 dedups existing rows + adds
|
||||
-- this on older databases.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_edges_identity
|
||||
ON edges(source, target, kind, IFNULL(line, -1), IFNULL(col, -1));
|
||||
|
||||
-- File indexes
|
||||
CREATE INDEX IF NOT EXISTS idx_files_language ON files(language);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_modified_at ON files(modified_at);
|
||||
|
||||
Reference in New Issue
Block a user