codegraph/docs/design/callback-edge-synthesis.md

# Design + status: general callback / observer edge synthesis

**Status:** SHIPPED (the synthesizer in `callback-synthesizer.ts` is merged and on
`main`). This doc records the original design.
**Motivation:** close the dynamic-dispatch hole that static extraction leaves for
observer / event-emitter / signal patterns, where a *dispatcher* invokes callbacks
registered elsewhere through a shared store — so flows like "how does an update
reach the screen" actually exist in the graph.

> **Update (2026-06-01):** the `codegraph_trace` and `codegraph_context` MCP tools
> were since **removed** — `codegraph_explore` is the single surfacing tool now. Its
> "Flow" section (`buildFlowFromNamedSymbols`) and the `codegraph_node` trail surface
> these synthesized edges; the `trace(a, b)` notation below means "the a→b flow,"
> which you now verify with `codegraph_explore` / `probe-explore.mjs` (the
> `probe-trace.mjs` / `probe-context.mjs` dev probes went away with the tools).

---

## TL;DR for a new session

We synthesize `dispatcher → callback` edges that static parsing misses. It works:

- **Field observer** (excalidraw `Scene.onUpdate`/`triggerUpdate`): synthesizes
  `triggerUpdate → triggerRender`. `trace(mutateElement, triggerRender)` now = 3 hops.
- **EventEmitter** (express `on('mount', …)`/`emit('mount')`): synthesizes `use → onmount`.
- Precision is high: excalidraw got **1** synthesized edge out of 27k (the correct one);
  node count moved +3 after Phase 3 (no explosion).

**Files touched (all uncommitted on `main`):**
- `src/resolution/callback-synthesizer.ts` — the whole-graph synthesis pass (Phase 1 + 2).
- `src/resolution/index.ts` — calls `synthesizeCallbackEdges()` at the end of
  `resolveAndPersistBatched()` (after base edges are persisted) + the import.
- `src/extraction/tree-sitter.ts` — `visitFunctionBody` now extracts **named** nested
  functions (Phase 3), so inline named handlers become linkable nodes.

**How to reproduce / test:**
```bash
npm run build
rm -rf /tmp/codegraph-corpus/excalidraw/.codegraph
( cd /tmp/codegraph-corpus/excalidraw && codegraph init -i )
# synthesized edges (provenance='heuristic', metadata.synthesizedBy in {callback,event-emitter,…,http-client,queue-job,event-bus}):
sqlite3 /tmp/codegraph-corpus/excalidraw/.codegraph/codegraph.db \
  "select s.name||' → '||t.name||'  '||coalesce(e.metadata,'') from edges e \
   join nodes s on e.source=s.id join nodes t on e.target=t.id where e.provenance='heuristic';"
# end-to-end flow (the synthesized edge shows up in explore's Flow section + node trail):
node scripts/agent-eval/probe-explore.mjs /tmp/codegraph-corpus/excalidraw "triggerUpdate triggerRender"
```
Probe scripts (dev-only, in `scripts/agent-eval/`): `probe-node.mjs` (symbol + trail),
`probe-explore.mjs` (relevant source + the flow among named symbols). EventEmitter
fixture lives at `/tmp/cb-fixture/bus.js` (ephemeral — recreate or move into `__tests__/`).

---

## Cross-tier channels (`src/resolution/tier-synthesizer.ts`, 2026-08-28)

The web's RN bridge: one pass, registered before the in-process emitter pass (the more specific edge wins a duplicate
`source>target` pair in the merge), gated on JS-family files, never sourced from a test suite or a generated file.
Three channels, each keyed on a literal on both sides, each edge `kind:'calls'`, `provenance:'heuristic'`, with
`synthesizedBy`, `channel` (`http` | `queue` | `event` | `socket`), `tier` (`client→server` / `server→client`) when the
direction is known, the `event` / `queue` / `method` / `href` it paired on, `line` + `column` of the call, and
`registeredAt` = the other side (route registration, decorator, `.on`):

- **`http-client`** — `fetch` / `$fetch` / `ofetch` / `axios` / `ky` / `got` / `useFetch` / `useSWR`, `<client>.get|post|…(`
  where the receiver is a known client name or a binding made by `axios.create(…)` / `ky.extend(…)` (same file or the file it
  is imported from — `resolveImportPath`, since import mappings carry no resolved path), with a literal / template first
  argument (`new URL('/x', base)` and `{ url, method }` configs read too) → the ONE route `METHOD path` in the index it
  denotes. A hole fills a `:param` / `{id}` / `[id]` / catch-all segment and never a literal one; a hole in front of the path
  (`${API_URL}/users`) matches by the route's tail; a line a framework resolver made a route node on is a registration, not a
  client call; a tie between routes is nothing. No fan-out cap — the match is exact.
- **`queue-job`** — `<queue>.add('job', …)` where the queue is named (`new Queue('email')`, `@InjectQueue('email') x`, in the
  file or its import) or queue-shaped → the `@Process('job')` method of the `@Processor('email')` class (a WorkerHost's
  `process` when there is none), `new Worker('email', handler)` (an inline handler → the enclosing function, else the
  enclosing constant), Bull's `queue.process('job', handler)`. Most specific pairing wins (queue+job > job > the queue's
  default); an unnamed queue pairs only on a unique job name. Fan-out cap 6.
- **`event-bus`** — `.emit|emitAsync('x')` on a bus-shaped receiver (`eventEmitter`, `bus`, `pubsub`, …) → `@OnEvent`
  handlers, `*` / `**` globs honoured; on a socket-shaped receiver (`socket`, `io`, `server`, `client`, `.to(room)`, …) from a
  file without a socket server → `@SubscribeMessage('x')` and server-side `socket.on('x')` (`client→server`); from a file
  with one (`@WebSocketGateway`, `io.on('connection')`, `new Server`) → client-side `socket.on('x', …)`, named or inline
  (→ the enclosing component) (`server→client`). Plain `.on` ↔ `.emit` stays the emitter pass's. Fan-out cap 6.

The Steps view (`ui-server/api/steps.ts`) reads `tier` / `channel` before the languages in `crossing()`, so a hop between
two TS files draws as a bridge (`⇢ POST /api/users`, a boundary like another screen) or an event (`⇠ welcome`); explore's
Flow section labels them (`context/index.ts`, `mcp/tools.ts`). A Next `'use server'` action needs no edge: `steps.ts` marks
the call at request time from the directive. Validated on `bradtraversy/proshop_mern` (30 routes, 23 client→route edges,
all correct on inspection, after Express mounts + chained `router.route()` landed) and `nestjs/nest` (`sample/26-queues`,
`sample/30-event-emitter`); test `__tests__/ui-steps-cross-tier.test.ts`.

## Next.js links (`src/resolution/next-router-synthesizer.ts`, 2026-08-28)

`<Link href="/x">`, `<Link href={`/users/${id}`}>`, `<Link href={{ pathname }}>` and an internal `<a href>` are JSX
attributes — no reference is ever extracted for them — so this pass reads them from the source, attributes each to the
component it is written in, matches the href against the Next page table (`frameworks/nextjs.ts`) and synthesizes a
`navigates` edge (`synthesizedBy:'next-link'`, `href`, `navMethod: 'link' | 'a'`, `registeredAt` = the JSX site). Only files
under a Next app's root, never test files; ≤ 24 links per component (a navigation menu is not a decision); an external `<a>`
is nothing. The Screens view walks back from these edges exactly as from `router.push`; they draw dashed.

## The hole

```ts
class Scene {
  private callbacks = new Set<Callback>();
  onUpdate(cb: Callback) { this.callbacks.add(cb); }          // REGISTRAR
  triggerUpdate() { for (const cb of this.callbacks) cb(); }  // DISPATCHER
}
this.scene.onUpdate(this.triggerRender);                      // REGISTRATION SITE
```

The runtime edge `triggerUpdate → triggerRender` does not exist statically:
`triggerUpdate`'s only literal call is `cb()` (anonymous). Measured: `triggerUpdate`'s
only callee was `randomInteger`; `trace(triggerUpdate, triggerRender)` returned no path.

## Why it's a whole-graph pass, not a `FrameworkResolver.resolve()`

`resolve(ref)` answers "what does this **named** ref point to," one ref at a time. The
callback edge has **no ref to resolve** (`cb()` is anonymous) and needs **cross-file,
multi-site correlation** (registrar, registration, dispatcher). So it's a whole-graph
pass after base resolution, language-level (any OO observer), living in
`src/resolution/callback-synthesizer.ts` — **not** under `frameworks/`.

> Sibling mechanism for the *other* dynamic-dispatch class — **named** attribute/
> descriptor dispatch (e.g. django `self._iterable_class(...)`) — is the
> `claimsReference` hook (`resolution/types.ts` + `resolution/index.ts` pre-filter)
> + a `FrameworkResolver.resolve()` (django ORM resolver in `frameworks/python.ts`).
> That one *does* fit `resolve()` because the ref is named. Both are part of the same
> coverage effort; see the "Related work" section.

---

## As-built algorithm (and where it diverged from the original design)

### Field-observer channels (`fieldChannelEdges`, Phase 1)
1. **Candidates** by method/function **name** — registrar `^(on[A-Z]\w*|subscribe|
   addListener|addEventListener|register|watch|listen|addCallback)$`; dispatcher
   contains `(emit|trigger|notify|dispatch|fire|publish|flush)`.
2. **Confirm by body** (read via `ctx.readFile` + slice node lines): registrar has
   `this.<F>.add|push|set(`; dispatcher has `for (… of [Array.from(]this.<F>)` + a call,
   or `this.<F>.forEach(`.
3. **Pairing — DIVERGENCE:** the design said pair by *class*; the build pairs by
   **same file + same field `F`** (file as a class proxy — getting the containing class
   reliably was harder). Works for the common 1-class-per-file case; revisit for
   multi-class files.
4. **Registrations:** `queries.getIncomingEdges(registrar.id, ['calls'])` → for each,
   read the caller's source at the edge line and **regex-recover the arg**
   (`<registrarName>\s*\(\s*(?:this\.)?(\w+)`). DIVERGENCE: design preferred tree-sitter
   re-parse; build uses regex (named refs only — arrows/inline args are missed here).
5. **Synthesize** `dispatcher → fn` (`getNodesByName(arg)` → method|function). Capped at
   `MAX_CALLBACKS_PER_CHANNEL = 40`.

### EventEmitter channels (`eventEmitterEdges`, Phase 2)
- **File-oriented scan** (`ctx.getAllFiles()` + `readFile`, substring pre-filter on
  `.emit(`/`.on(`/etc). `ON_RE` = `\.(?:on|once|addListener)\(\s*['"]([^'"]+)['"]\s*,\s*
  (?:function\s+(\w+)|(?:this\.)?(\w+))`; `EMIT_RE` = `\.(?:emit|fire|dispatchEvent)\(\s*['"]([^'"]+)['"]`.
- Dispatcher = **enclosing function** of the `emit('e')` call (`enclosingFn` finds the
  tightest function/method/component node containing the line). Handler = `getNodesByName`
  of the on-handler name.
- Correlate by **event-name literal**; synthesize dispatcher → handler.
- **Precision — DIVERGENCE:** design proposed receiver-type matching; build uses an
  **event fan-out cap** (`EVENT_FANOUT_CAP = 6`) — skip events with >6 handlers or
  dispatchers (generic names like `error`/`change` would over-link without type info).

### Provenance — DIVERGENCE
`Edge.provenance` is a fixed enum (`'tree-sitter'|'scip'|'heuristic'`), so synthesized
edges use **`provenance: 'heuristic'`** + `metadata: { synthesizedBy: 'callback'|
'event-emitter', via/event/field }`. The design's `'callback-synthesis'` provenance and
high/medium/low **confidence tiers were NOT implemented** — the fan-out cap +
registrar-name uniqueness + named-only handlers are the precision guards instead.

### Phase 3 — inline callback extraction (`tree-sitter.ts`)
The real blocker for EventEmitter on real repos: inline handlers
(`on('mount', function onmount(){})`) weren't **nodes**, so nothing could link to them.
Root cause: `visitFunctionBody` walked *through* nested functions without extracting them.
Fix: in `visitForCallsAndStructure`, when a body node is a `functionType` and
`extractName` returns a real name, call `extractFunction` (which extracts it and walks
its own body) and return. **Named only** — anonymous arrows fall through to the existing
recursion (so their inner calls stay attributed to the enclosing fn). This bounded it:
excalidraw +3 nodes, no explosion, no regression.

---

## Validation results (actual)

| Repo | Result |
|---|---|
| excalidraw | 1 synthesized edge `triggerUpdate → triggerRender` (of 27,214); `trace(mutateElement, triggerRender)` = 3 hops; nodes 9,286 → 9,289 |
| express | after Phase 3: `use → onmount` `{event-emitter, event:"mount"}` (`onmount` now extracted at `application.js:109`) |
| `/tmp/cb-fixture/bus.js` | `tick → handleRefresh`, `persist → handleSave` (named-method EventEmitter handlers) |
| excalidraw / express | no Phase-1 regression; node counts stable |

---

## Remaining work (prioritized for the next session)

1. **Anonymous-arrow handlers** — `on('e', () => foo())` still produce no edge (no node,
   intentionally not extracted in Phase 3). The fix is **synthesizer link-through-body**:
   parse the arrow's body and link `dispatcher → (calls inside the arrow)`. Highest
   remaining recall win; handles the most common modern callback shape.
2. **Wire into `resolveAndPersist`** (incremental sync) — synthesis currently runs only
   in `resolveAndPersistBatched` (full index). Incremental re-index won't refresh
   synthesized edges.
3. **Receiver-type matching** for EventEmitter precision (replace/augment the fan-out
   cap) — use `type_of` edges so `x.emit('change')` only links to `y.on('change', fn)`
   when `x`,`y` are the same type. Lets the fan-out cap relax.
4. **Tree-sitter arg recovery** (replace the regex in field-channel Stage 4) — robust for
   arrows, multi-arg, line-wrapped calls.
5. **Single-callback fields** (`this.onChange = cb; … this.onChange()`) — scalar-store
   variant of the field observer; not built.
6. **Broad precision/recall audit** — run across the full corpus; tally synthesized edges
   per repo, spot-check, confirm no explosion on EventEmitter-heavy repos.
7. **Tests + CHANGELOG** — the fixture is a ready vitest case for the synthesizer; add
   extractor tests for Phase 3 (named-nested-fn extraction; confirm other languages
   unaffected — the change is in the shared walker), resolver tests for the django side.

## Edge cases / model
- **Over-approximation across instances** is accepted (reachability, not instance
  precision). `unregister`/`off` ignored.
- Synthesized edges are **additive** — never replace static edges; tooling can filter on
  `provenance='heuristic'` + `metadata.synthesizedBy`.

## Related work (same coverage effort)
This is one half of closing dynamic-dispatch coverage. The other artifacts on `main`:
- **Named attribute/descriptor resolver**: `claimsReference` (`resolution/types.ts`,
  pre-filter in `resolution/index.ts`) + django ORM resolver (`frameworks/python.ts`,
  `_iterable_class` → `ModelIterable.__iter__`).
- **Retrieval/UX changes** (separate from coverage): `explore` whole-small-file + glue
  fixes, the `explore` Flow section (`buildFlowFromNamedSymbols`), and `node`-with-trail
  — all in `src/mcp/tools.ts`. (`codegraph_trace` / `codegraph_context` were later
  removed; explore is the one surfacing tool.)
- **Full investigation context + findings:** auto-memory
  `project_codegraph_read_displacement` (why coverage — not prompting/hooks/new-tools —
  is the lever for getting agents to use codegraph over Read).