fix(db): dedup edges with a UNIQUE identity index so INSERT OR IGNORE works (#1034) (#1050)

`insertEdge` has always used `INSERT OR IGNORE`, but the edges table carried
no UNIQUE constraint — only an autoincrement PK and non-unique indexes — so
`OR IGNORE` had nothing to conflict on and behaved like a plain INSERT.
Whenever two extraction/resolution passes emitted the same edge (e.g. a
return type captured by both a type-reference and a value-reference pass),
the graph stored byte-identical duplicate rows: ~527 on this repo, inflating
edge counts and letting callers/impact list the same relationship twice.

Add a UNIQUE identity index on (source, target, kind, IFNULL(line,-1),
IFNULL(col,-1)) — in schema.sql for fresh databases and migration v6 (dedup
existing rows, then create the index) for existing ones. IFNULL folds the
nullable line/col so coordinate-less edges (synthesized / file-level) dedup
too; SQLite otherwise treats each NULL as distinct. Distinct call sites
(same source/target/kind, different line/col) are preserved — only
byte-identical structural duplicates collapse. This is the storage-layer
invariant the reporter identified: it makes OR IGNORE keep its promise and
catches every double-emit, present and future, rather than chasing each
emitting pass.

Migration v6 is deterministic (keeps the lowest id per identity group) and
idempotent (IF NOT EXISTS index; no-op DELETE once unique). The DELETE's
GROUP BY matches the index expression exactly so creation can't fail on a
leftover pair.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-06-28 21:34:33 -05:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 2176a7a439
commit 0da2dcec8e
6 changed files with 150 additions and 4 deletions
+26 -1
View File
@@ -9,7 +9,7 @@ import { SqliteDatabase } from './sqlite-adapter';
/**
* Current schema version
*/
export const CURRENT_SCHEMA_VERSION = 5;
export const CURRENT_SCHEMA_VERSION = 6;
/**
* Migration definition
@@ -75,6 +75,31 @@ const migrations: Migration[] = [
`);
},
},
{
version: 6,
description:
'Dedup duplicate edge rows and add a UNIQUE identity index so INSERT OR IGNORE actually dedups (#1034)',
up: (db) => {
// `insertEdge` has always used `INSERT OR IGNORE`, but the edges table had
// no UNIQUE constraint, so nothing conflicted and byte-identical rows
// accumulated whenever two passes emitted the same edge. Collapse each
// identity group to its lowest id, then add the constraint that makes
// `OR IGNORE` keep its promise. IFNULL folds nullable line/col so
// coordinate-less edges dedup too (SQLite treats each NULL as distinct) —
// and it MUST match the GROUP BY exactly, or the index creation would
// fail on a pair the DELETE left behind. Idempotent: the index is
// `IF NOT EXISTS` and the DELETE is a no-op once the table is unique.
db.exec(`
DELETE FROM edges
WHERE id NOT IN (
SELECT MIN(id) FROM edges
GROUP BY source, target, kind, IFNULL(line, -1), IFNULL(col, -1)
);
CREATE UNIQUE INDEX IF NOT EXISTS idx_edges_identity
ON edges(source, target, kind, IFNULL(line, -1), IFNULL(col, -1));
`);
},
},
];
/**
+11
View File
@@ -133,6 +133,17 @@ CREATE INDEX IF NOT EXISTS idx_edges_kind ON edges(kind);
CREATE INDEX IF NOT EXISTS idx_edges_source_kind ON edges(source, kind);
CREATE INDEX IF NOT EXISTS idx_edges_target_kind ON edges(target, kind);
-- Edge identity uniqueness. An edge IS uniquely (source, target, kind, line,
-- col); insertEdge uses `INSERT OR IGNORE`, but without something UNIQUE to
-- conflict on it behaved like a plain INSERT, so two passes emitting the same
-- edge produced byte-identical duplicate rows that inflated counts and flowed
-- into callers/impact (#1034). IFNULL folds the nullable line/col so
-- coordinate-less edges (synthesized / file-level) dedup too — SQLite treats
-- each NULL as distinct otherwise. Migration v6 dedups existing rows + adds
-- this on older databases.
CREATE UNIQUE INDEX IF NOT EXISTS idx_edges_identity
ON edges(source, target, kind, IFNULL(line, -1), IFNULL(col, -1));
-- File indexes
CREATE INDEX IF NOT EXISTS idx_files_language ON files(language);
CREATE INDEX IF NOT EXISTS idx_files_modified_at ON files(modified_at);