diff --git a/CHANGELOG.md b/CHANGELOG.md index 8069c2e..1f1a405 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,7 +12,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### New Features - The Claude Code context hook now recognizes prompts that describe code in plain words — in any language — by checking the prompt's words against the symbol names actually in your project's index. Asking about "the state machine des commandes" finds `OrderStateMachine` with no keyword involved. Confidence decides how much gets injected: structural questions and prompts naming a real symbol still get full context up front; a plain-words match gets a short pointer to the matching symbols so the agent queries them itself; everything else stays silent, exactly as before. -- Anonymous usage telemetry now counts how often the context hook injected context, offered a hint, or stayed silent — fixed counter names only; the prompt's content is never stored or sent. This makes the hook's accuracy measurable instead of guessed. +- Anonymous usage telemetry now counts how often the context hook injected context, offered a hint, or stayed silent — fixed counter names only; the prompt's content is never stored or sent. This makes the hook's accuracy measurable instead of guessed. The counters record what actually happened, not what was attempted: a lookup that errors or comes back empty counts as a distinct silent outcome, never as delivered context (#1143, thanks @inth3shadows). ### Fixes diff --git a/docs/design/telemetry.md b/docs/design/telemetry.md index d1e115c..c0263e7 100644 --- a/docs/design/telemetry.md +++ b/docs/design/telemetry.md @@ -75,10 +75,16 @@ Event types: The prompt hook additionally rolls up its gate DECISION as `cli_command` counters named `prompt-hook-gate-`, outcome ∈ `high-keyword` / `high-token` / `medium-segment` / `nudge-projects` / `noop-shape` / - `noop-no-index` / `noop-unverified` — decision names only, never prompt - content. This is the gate's measured recall/precision funnel: a rising - `noop-*` share against the `high`/`medium` tiers is the signal that the - gate (keyword table or segment matching) is missing real questions. + `noop-no-index` / `noop-unverified` / `noop-explore-keyword` / + `noop-explore-token` / `noop-vocab-empty` — decision names only, never + prompt content. This is the gate's measured recall/precision funnel: a + rising `noop-*` share against the `high`/`medium` tiers is the signal that + the gate (keyword table or segment matching) is missing real questions. + A `high-*` outcome means context was actually injected — a gate decision + whose `codegraph_explore` errored or returned nothing records + `noop-explore-` instead (#1143), and a MEDIUM-eligible prompt + hitting a not-yet-backfilled segment vocabulary records `noop-vocab-empty` + rather than polluting `noop-unverified` (#1142). - **`uninstall`** — one per `uninstall`/`uninit` run (churn signal). Props: `targets`. Volume math: rollups mean monthly events ≈ active machines × active days × distinct diff --git a/src/bin/codegraph.ts b/src/bin/codegraph.ts index 30c1475..35e9b72 100644 --- a/src/bin/codegraph.ts +++ b/src/bin/codegraph.ts @@ -1141,8 +1141,14 @@ program process.stdout.write( `\n${body}${others}\n\n`, ); + gate(keyworded ? 'high-keyword' : 'high-token'); + } else { + // A high-* outcome must mean context was actually delivered — + // the funnel's noop-vs-high split is how gate recall is + // measured (#1143). An explore error or empty result is a + // delivery failure, not a gate success. + gate(keyworded ? 'noop-explore-keyword' : 'noop-explore-token'); } - gate(keyworded ? 'high-keyword' : 'high-token'); return; }