Self-hosted telemetry on Cloudflare D1 + password-gated admin dashboard (CG-7) (#1497)
* feat(telemetry): D1 schema + migrations for raw events and daily rollups First step of replacing PostHog with self-hosted telemetry on Cloudflare D1. Creates the codegraph-telemetry database binding and the initial migration; no worker code paths change yet (the ingest write path and the nightly rollup cron land next). Schema is raw events plus daily rollups: `events` holds one row per sanitized event with the envelope broken out into columns and event-specific props as JSON; `daily_machines`, `daily_event_counts` and `daily_dim_counts` are the nightly rollups the dashboard reads; `machine_first_seen` and `machine_days` carry the retention cohorts and are never purged. One generic dimension table covers every bar and pie, so a new breakdown is a cron change rather than a migration. The migration is commented as an audit surface, like the rest of this worker — every column, and which dashboard chart each rollup table serves. Three judgment calls worth flagging, all documented in the file: - `events` gets `(day, event)` instead of the separate `(day)` and `(event, day)` indexes. D1 bills a row write per index touched, so a third index on the hot table costs ~97k writes/day, and `(day, event)` is a covering index for plain day-range scans anyway (verified with EXPLAIN QUERY PLAN). - `daily_event_counts` and `daily_dim_counts` carry a `machines` column, and `machine_days` a `prod` flag. The "users by ..." panels and the production-user count are distinct-machine numbers, not event counts, and they are unrecoverable once raw events are purged. - No CHECK constraint on `event`: the worker's allowlist is the source of truth and the write path is fail-silent, so a rejected INSERT would lose data quietly instead of erroring loudly. Volume note in the migration footer: ~30M row writes/month against the 50M included on Workers Paid. Storage is the tighter constraint — raw events grow ~74 MB/day, so retention should start at 90 days (~6.7 GB) rather than 180, which would exceed D1's 10 GB per-database cap. * feat(telemetry): admin dashboard worker — scaffold + shared-password auth New Cloudflare Worker at telemetry-dashboard/, sibling of telemetry-worker/ and bound read-only to the same D1 database. Serves a static frontend plus a JSON API behind a shared password, on stats.getcodegraph.com. Auth is the simplest thing that is actually safe for exactly two users: one password in a secret, compared in constant time over SHA-256 digests, and an HMAC-signed cookie (HttpOnly; Secure; SameSite=Lax; Path=/) with a one-year expiry so you sign in once per browser. The cookie is a signed assertion, not a lookup key — no session store. Its payload carries a fingerprint of the password it was minted against, so rotating ADMIN_PASSWORD signs everyone out. Login attempts are capped at 5/min per IP via a ratelimit binding. Everything is deny-by-default: assets.run_worker_first routes every request through the worker before the static-asset server sees it, so the dashboard HTML, its JS, its CSS and the chart library are all behind the session check. The login page is rendered inline by the worker rather than served from public/, which leaves no "is this file public?" judgement calls in the asset directory. Unauthenticated pages 302 to /login, unauthenticated /api/* gets 401. A missing secret fails closed rather than opening the dashboard. scripts/smoke-auth.sh is the regression net — 54 assertions against a throwaway `wrangler dev` covering the gate, cookie flags and persistence, forged/flipped/ truncated cookies, open-redirect refusal, brute-force capping, and password rotation invalidating live sessions. Refs CG-11. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(telemetry-dashboard): simplify the chart-library probe in the shell Refs CG-11. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): nightly rollup cron + raw-event retention purge (CG-10) Adds a scheduled() handler to the ingest worker that recomputes daily_event_counts / daily_dim_counts / daily_machines for the just-completed UTC day plus a 2-day overlap (late-arriving offline buffers), then purges raw events past the retention window. Rollup writes are idempotent upserts, so a re-run never double-counts. Also adds an ADMIN_TOKEN-guarded POST /admin/rollup?day=YYYY-MM-DD for backfill/repair, and drops the PostHog forwarding path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): dashboard charts — SQL API over D1 + the Chart.js views (CG-12, CG-13) Replaces the scaffold page with the dashboard proper: 19 panels covering every view of the PostHog dashboard this retires, driven by one filter row. src/api.ts is the read API CG-12 specified: /api/{meta,summary,timeseries, breakdown,activation,retention}, all range-scoped, all parameterized against a closed set of dims and metrics, all shaped labels[] + datasets[] so the frontend does no arithmetic. Rollups answer everything except the activation funnel, which needs raw events and says where they start. The frontend splits into a DOM-free panel registry (public/panels.js) and the page that mounts it (public/app.js), so the render check can drive the same registry the browser rendered from. Panels fail alone, refetch dims rather than flashing, and every chart carries a table twin. Two numbers are labelled rather than rounded off: range-wide "users" per dimension is machine-days (the rollups cannot give distinct machines, and per-day counts are taken as the largest single-event count so one machine's install + index + usage is not counted three times), and recent activation and retention cohorts are marked as still-converting instead of drawn as a cliff. Both colour scales were run through the data-viz validator against the panel surface, not picked by eye; the results are recorded in public/theme.js. Verification, all against the committed fixture (12 machines over 10 days, every expected number worked out by hand from the events, not recorded from a run): scripts/smoke-api.sh 98 assertions scripts/render-check.mjs 79 assertions — real Chromium over CDP, no new deps scripts/smoke-auth.sh 54 assertions (unchanged, still green) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(telemetry): cutover runbook + the end-to-end gate that de-risks it (CG-14) The account-level steps of the PostHog cutover are the maintainer's to run, so this lands the runbook they follow and the check that has to pass first. The runbook (telemetry-worker/README.md) walks the six steps in the order that keeps them reversible: Workers Paid → migrate → deploy → watch 24h → verify the first rollup and the dashboard → only then delete POSTHOG_KEY and cancel the subscription. Step 3 records the outgoing version id because `wrangler rollback` is the escape hatch for the whole verification window, and that window is precisely why the PostHog key is deleted last rather than first. The new gate (scripts/smoke-cutover.sh, `npm run smoke:cutover`) covers the one seam nothing else did. Both workers declare the same D1 database_id, so pointing them at a single --persist-to directory runs the real chain: a client batch → the ingest worker → D1 → the nightly rollup → the dashboard API reading the numbers back. Every other suite stops at one link — smoke-ingest at the events table, smoke-rollup at hand-checked SQL, smoke-api at a hand-written fixture that the cron never touched. That left the dimension names the rollup WRITES versus the ones the dashboard READS agreeing by convention across two branches, where a mismatch is silent: no error, no failed request, just a panel reading zero forever. 61 assertions, all 13 dimensions, and three deliberate traps — a ci machine that is active but not a production user, usage_rollup counts that must be summed rather than tallied, and an uninstall's `targets` that must not leak into the install-scoped breakdown. Writing it caught that the activation funnel's denominator is first-seen machines, not install events (deliberate — a reinstall must not re-enter the funnel), so the suite now pins that distinction rather than assuming it. Also rewords the last PostHog reference in dashboard code: a comment justifying the 14-day retention curve by pointing at a dashboard step 6 deletes. The reasoning now stands on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(telemetry): tell the truth about where events are stored (CG-15) The telemetry docs are a privacy contract, and they still described a managed analytics store that no longer receives anything. Replace that with what actually happens now — events land in our own D1 database on Cloudflare, the endpoint makes no outbound requests, raw events are purged after 90 days and only anonymous daily rollups outlive them. This strengthens the guarantee rather than restating it: there is no second party to share with. - TELEMETRY.md: new "Where it is stored" section; the never-collected IP bullet no longer leans on a vendor-side setting to hold. - docs/design/telemetry.md: ingest section rewritten around D1 + the nightly rollup/retention cron; volume math redone on Workers Paid and the D1 quota (storage, not writes, is what sets the 90-day window); new section documenting the dashboard worker and cross-linking it. - Fixed three drifts from the worker allowlist the sweep surfaced: schema_version was still 1, client_name/client_version was still marked "plumbing to add" though session.ts passes it today, and the legacy sqlite_backend field the worker still accepts was undocumented. - telemetry-worker/README.md: step 6 claimed a repo-wide grep came back clean, which this runbook itself falsifies. Added step 7 — deleting the runbook is what makes that grep true, and is the completion check. - smoke-cutover.sh: the vendor guarantee is now asserted by class (no analytics-ingest endpoint referenced) rather than by one vendor's name, so it keeps working once the name is gone. Verified it still catches a planted forwarding URL. 61/61 pass. Retention is documented as 90 days, not the 180 in the task notes: 180 days of raw events exceeds D1's 10 GB per-database cap, and the code purges at 90. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: untrack local Kommandr issue DB and ignore its sqlite artifacts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Opus 5
Claude Opus 5
parent
f6ac7b36e6
commit
49c11fc2e0
@@ -0,0 +1,208 @@
|
||||
-- Seed data for the dashboard's local checks: 12 machines over 10 days
|
||||
-- (2026-07-01 … 2026-07-10), small enough that every number on every panel can
|
||||
-- be worked out by hand from the events below and checked against the API.
|
||||
--
|
||||
-- npm run seed (writes the LOCAL .wrangler D1 — never the remote one)
|
||||
--
|
||||
-- Only the raw `events` rows are hand-authored. `machine_days`,
|
||||
-- `machine_first_seen` and the three `daily_*` rollups are DERIVED from them at
|
||||
-- the bottom of this file by the same aggregations the writers use in
|
||||
-- telemetry-worker/ (the ingest path and the nightly cron respectively), so the
|
||||
-- fixture can never drift into a state production could not produce.
|
||||
--
|
||||
-- The machines, and what each one does:
|
||||
--
|
||||
-- id first os arch ver ci installs indexes on uninstalls
|
||||
-- m01 07-01 darwin arm64 1.4.0 0 local 07-01, 07-02, 07-04
|
||||
-- m02 07-01 darwin arm64 1.4.0 0 global 07-01
|
||||
-- m03 07-01 linux x64 1.4.0 0 local 07-03
|
||||
-- m04 07-01 win32 x64 1.4.0 0 local never 07-06
|
||||
-- m05 07-02 darwin arm64 1.4.0 0 local 07-02
|
||||
-- m06 07-02 linux x64 1.4.1 0 local never 07-07
|
||||
-- m07 07-03 darwin x64 1.4.1 0 local 07-03
|
||||
-- m08 07-05 linux arm64 1.5.0 0 global 07-06
|
||||
-- m09 07-05 win32 x64 1.5.0 0 local 07-05, 07-07
|
||||
-- m10 07-08 darwin arm64 1.5.0 0 local 07-08
|
||||
-- m11 07-09 linux x64 1.5.0 0 local 07-10
|
||||
-- m12 07-09 linux x64 1.5.0 1 global 07-09 (CI runner)
|
||||
--
|
||||
-- m04 and m06 never index: they are the two machines the activation funnel is
|
||||
-- supposed to lose (12 installs → 10 activated → 83.3%). m12 is the one CI
|
||||
-- machine, so "production users" is 11 where "active machines" is 12.
|
||||
|
||||
DELETE FROM daily_dim_counts;
|
||||
DELETE FROM daily_event_counts;
|
||||
DELETE FROM daily_machines;
|
||||
DELETE FROM machine_days;
|
||||
DELETE FROM machine_first_seen;
|
||||
DELETE FROM events;
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- install — 12, one per machine on its first day
|
||||
-- ---------------------------------------------------------------------------
|
||||
INSERT INTO events (received_at, ts, day, event, machine_id, codegraph_version, os, arch, node_major, ci, schema_version, props)
|
||||
VALUES
|
||||
('2026-07-01T09:00:00Z','2026-07-01T09:00:00Z','2026-07-01','install','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"scope":"local","kind":"fresh","targets":["claude","cursor"]}'),
|
||||
('2026-07-01T09:05:00Z','2026-07-01T09:05:00Z','2026-07-01','install','00000000-0000-4000-8000-000000000002','1.4.0','darwin','arm64',22,0,2,'{"scope":"global","kind":"fresh","targets":["claude"]}'),
|
||||
('2026-07-01T10:00:00Z','2026-07-01T10:00:00Z','2026-07-01','install','00000000-0000-4000-8000-000000000003','1.4.0','linux','x64',20,0,2,'{"scope":"local","kind":"fresh","targets":["codex"]}'),
|
||||
('2026-07-01T11:00:00Z','2026-07-01T11:00:00Z','2026-07-01','install','00000000-0000-4000-8000-000000000004','1.4.0','win32','x64',22,0,2,'{"scope":"local","kind":"fresh","targets":["claude","opencode"]}'),
|
||||
('2026-07-02T09:00:00Z','2026-07-02T09:00:00Z','2026-07-02','install','00000000-0000-4000-8000-000000000005','1.4.0','darwin','arm64',22,0,2,'{"scope":"local","kind":"fresh","targets":["claude"]}'),
|
||||
('2026-07-02T14:00:00Z','2026-07-02T14:00:00Z','2026-07-02','install','00000000-0000-4000-8000-000000000006','1.4.1','linux','x64',20,0,2,'{"scope":"local","kind":"fresh","targets":["cursor"]}'),
|
||||
('2026-07-03T08:00:00Z','2026-07-03T08:00:00Z','2026-07-03','install','00000000-0000-4000-8000-000000000007','1.4.1','darwin','x64',22,0,2,'{"scope":"local","kind":"upgrade","targets":["claude"]}'),
|
||||
('2026-07-05T08:00:00Z','2026-07-05T08:00:00Z','2026-07-05','install','00000000-0000-4000-8000-000000000008','1.5.0','linux','arm64',22,0,2,'{"scope":"global","kind":"fresh","targets":["claude","codex"]}'),
|
||||
('2026-07-05T09:00:00Z','2026-07-05T09:00:00Z','2026-07-05','install','00000000-0000-4000-8000-000000000009','1.5.0','win32','x64',22,0,2,'{"scope":"local","kind":"fresh","targets":["claude"]}'),
|
||||
('2026-07-08T08:00:00Z','2026-07-08T08:00:00Z','2026-07-08','install','00000000-0000-4000-8000-000000000010','1.5.0','darwin','arm64',22,0,2,'{"scope":"local","kind":"fresh","targets":["cursor"]}'),
|
||||
('2026-07-09T08:00:00Z','2026-07-09T08:00:00Z','2026-07-09','install','00000000-0000-4000-8000-000000000011','1.5.0','linux','x64',22,0,2,'{"scope":"local","kind":"fresh","targets":["claude"]}'),
|
||||
('2026-07-09T08:30:00Z','2026-07-09T08:30:00Z','2026-07-09','install','00000000-0000-4000-8000-000000000012','1.5.0','linux','x64',22,1,2,'{"scope":"global","kind":"fresh","targets":["claude"]}');
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- index — 13 runs
|
||||
-- languages typescript 7 · javascript 2 · python 2 · go 2 · rust 2 · csharp 2 · java 1 (18 rows)
|
||||
-- file_count_bucket <100 2 · 100-1k 5 · 1k-10k 4 · 10k+ 2
|
||||
-- duration_bucket <10s 5 · 10-60s 4 · 1-5m 2 · 5m+ 2
|
||||
-- ---------------------------------------------------------------------------
|
||||
INSERT INTO events (received_at, ts, day, event, machine_id, codegraph_version, os, arch, node_major, ci, schema_version, props)
|
||||
VALUES
|
||||
('2026-07-01T09:10:00Z','2026-07-01T09:10:00Z','2026-07-01','index','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"languages":["typescript","javascript"],"file_count_bucket":"100-1k","duration_bucket":"<10s"}'),
|
||||
('2026-07-01T09:20:00Z','2026-07-01T09:20:00Z','2026-07-01','index','00000000-0000-4000-8000-000000000002','1.4.0','darwin','arm64',22,0,2,'{"languages":["typescript"],"file_count_bucket":"<100","duration_bucket":"<10s"}'),
|
||||
('2026-07-02T10:00:00Z','2026-07-02T10:00:00Z','2026-07-02','index','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"languages":["typescript","javascript"],"file_count_bucket":"100-1k","duration_bucket":"10-60s"}'),
|
||||
('2026-07-02T11:00:00Z','2026-07-02T11:00:00Z','2026-07-02','index','00000000-0000-4000-8000-000000000005','1.4.0','darwin','arm64',22,0,2,'{"languages":["python"],"file_count_bucket":"1k-10k","duration_bucket":"10-60s"}'),
|
||||
('2026-07-03T09:00:00Z','2026-07-03T09:00:00Z','2026-07-03','index','00000000-0000-4000-8000-000000000003','1.4.0','linux','x64',20,0,2,'{"languages":["go"],"file_count_bucket":"100-1k","duration_bucket":"<10s"}'),
|
||||
('2026-07-03T10:00:00Z','2026-07-03T10:00:00Z','2026-07-03','index','00000000-0000-4000-8000-000000000007','1.4.1','darwin','x64',22,0,2,'{"languages":["typescript","rust"],"file_count_bucket":"10k+","duration_bucket":"5m+"}'),
|
||||
('2026-07-04T10:00:00Z','2026-07-04T10:00:00Z','2026-07-04','index','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"languages":["typescript"],"file_count_bucket":"100-1k","duration_bucket":"<10s"}'),
|
||||
('2026-07-05T09:30:00Z','2026-07-05T09:30:00Z','2026-07-05','index','00000000-0000-4000-8000-000000000009','1.5.0','win32','x64',22,0,2,'{"languages":["csharp"],"file_count_bucket":"1k-10k","duration_bucket":"1-5m"}'),
|
||||
('2026-07-06T09:00:00Z','2026-07-06T09:00:00Z','2026-07-06','index','00000000-0000-4000-8000-000000000008','1.5.0','linux','arm64',22,0,2,'{"languages":["rust","go"],"file_count_bucket":"1k-10k","duration_bucket":"1-5m"}'),
|
||||
('2026-07-07T09:00:00Z','2026-07-07T09:00:00Z','2026-07-07','index','00000000-0000-4000-8000-000000000009','1.5.0','win32','x64',22,0,2,'{"languages":["csharp"],"file_count_bucket":"1k-10k","duration_bucket":"10-60s"}'),
|
||||
('2026-07-08T08:10:00Z','2026-07-08T08:10:00Z','2026-07-08','index','00000000-0000-4000-8000-000000000010','1.5.0','darwin','arm64',22,0,2,'{"languages":["typescript"],"file_count_bucket":"<100","duration_bucket":"<10s"}'),
|
||||
('2026-07-09T09:00:00Z','2026-07-09T09:00:00Z','2026-07-09','index','00000000-0000-4000-8000-000000000012','1.5.0','linux','x64',22,1,2,'{"languages":["java"],"file_count_bucket":"10k+","duration_bucket":"5m+"}'),
|
||||
('2026-07-10T09:00:00Z','2026-07-10T09:00:00Z','2026-07-10','index','00000000-0000-4000-8000-000000000011','1.5.0','linux','x64',22,0,2,'{"languages":["python","typescript"],"file_count_bucket":"100-1k","duration_bucket":"10-60s"}');
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- uninstall — 2
|
||||
-- ---------------------------------------------------------------------------
|
||||
INSERT INTO events (received_at, ts, day, event, machine_id, codegraph_version, os, arch, node_major, ci, schema_version, props)
|
||||
VALUES
|
||||
('2026-07-06T12:00:00Z','2026-07-06T12:00:00Z','2026-07-06','uninstall','00000000-0000-4000-8000-000000000004','1.4.0','win32','x64',22,0,2,'{"targets":["claude","opencode"]}'),
|
||||
('2026-07-07T12:00:00Z','2026-07-07T12:00:00Z','2026-07-07','uninstall','00000000-0000-4000-8000-000000000006','1.4.1','linux','x64',20,0,2,'{"targets":["cursor"]}');
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- usage_rollup — 5 rows, 85 calls (the `count` prop is summed, never the rows)
|
||||
-- codegraph_explore 82 · index 3 | Claude Code 70 · Cursor 12
|
||||
-- ---------------------------------------------------------------------------
|
||||
INSERT INTO events (received_at, ts, day, event, machine_id, codegraph_version, os, arch, node_major, ci, schema_version, props)
|
||||
VALUES
|
||||
('2026-07-03T02:00:00Z','2026-07-02T12:00:00Z','2026-07-02','usage_rollup','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"kind":"mcp_tool","name":"codegraph_explore","count":40,"error_count":1,"client_name":"Claude Code"}'),
|
||||
('2026-07-04T02:00:00Z','2026-07-03T12:00:00Z','2026-07-03','usage_rollup','00000000-0000-4000-8000-000000000001','1.4.0','darwin','arm64',22,0,2,'{"kind":"mcp_tool","name":"codegraph_explore","count":25,"client_name":"Claude Code"}'),
|
||||
('2026-07-04T02:00:00Z','2026-07-03T12:00:00Z','2026-07-03','usage_rollup','00000000-0000-4000-8000-000000000005','1.4.0','darwin','arm64',22,0,2,'{"kind":"cli_command","name":"index","count":3}'),
|
||||
('2026-07-07T02:00:00Z','2026-07-06T12:00:00Z','2026-07-06','usage_rollup','00000000-0000-4000-8000-000000000009','1.5.0','win32','x64',22,0,2,'{"kind":"mcp_tool","name":"codegraph_explore","count":12,"client_name":"Cursor"}'),
|
||||
('2026-07-11T02:00:00Z','2026-07-10T12:00:00Z','2026-07-10','usage_rollup','00000000-0000-4000-8000-000000000011','1.5.0','linux','x64',22,0,2,'{"kind":"mcp_tool","name":"codegraph_explore","count":5,"client_name":"Claude Code"}');
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Derived: what the ingest worker writes on every batch
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- prod is 0 only when EVERY event a machine sent that day carried ci = 1, which
|
||||
-- is what makes m12 the only non-production machine-day.
|
||||
INSERT INTO machine_days (machine_id, day, prod)
|
||||
SELECT machine_id, day, max(CASE WHEN ci = 1 THEN 0 ELSE 1 END) FROM events GROUP BY machine_id, day;
|
||||
|
||||
INSERT INTO machine_first_seen (machine_id, first_day)
|
||||
SELECT machine_id, min(day) FROM events GROUP BY machine_id;
|
||||
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- Derived: what the nightly cron writes
|
||||
-- ---------------------------------------------------------------------------
|
||||
-- These mirror ROLLUP_STATEMENTS in telemetry-worker/src/rollup.ts, with the
|
||||
-- single-day filter dropped so one pass seeds the whole fixture range.
|
||||
|
||||
INSERT INTO daily_machines (day, machines, prod_machines)
|
||||
SELECT day, count(*), coalesce(sum(prod), 0) FROM machine_days GROUP BY day;
|
||||
|
||||
INSERT INTO daily_event_counts (day, event, count, machines)
|
||||
SELECT day, event,
|
||||
CASE WHEN event = 'usage_rollup'
|
||||
THEN sum(coalesce(json_extract(props, '$.count'), 0))
|
||||
ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events GROUP BY day, event;
|
||||
|
||||
-- Envelope dimensions — carried by every event.
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'os', CAST(os AS TEXT),
|
||||
CASE WHEN event = 'usage_rollup' THEN sum(coalesce(json_extract(props, '$.count'), 0)) ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events WHERE os IS NOT NULL AND os <> '' GROUP BY day, event, os;
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'arch', CAST(arch AS TEXT),
|
||||
CASE WHEN event = 'usage_rollup' THEN sum(coalesce(json_extract(props, '$.count'), 0)) ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events WHERE arch IS NOT NULL AND arch <> '' GROUP BY day, event, arch;
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'codegraph_version', CAST(codegraph_version AS TEXT),
|
||||
CASE WHEN event = 'usage_rollup' THEN sum(coalesce(json_extract(props, '$.count'), 0)) ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events WHERE codegraph_version IS NOT NULL AND codegraph_version <> '' GROUP BY day, event, codegraph_version;
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'node_major', CAST(node_major AS TEXT),
|
||||
CASE WHEN event = 'usage_rollup' THEN sum(coalesce(json_extract(props, '$.count'), 0)) ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events WHERE node_major IS NOT NULL GROUP BY day, event, node_major;
|
||||
|
||||
-- Event-specific scalar props.
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'file_count_bucket', CAST(json_extract(props, '$.file_count_bucket') AS TEXT), count(*), count(DISTINCT machine_id)
|
||||
FROM events WHERE event = 'index' AND json_extract(props, '$.file_count_bucket') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.file_count_bucket');
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'duration_bucket', CAST(json_extract(props, '$.duration_bucket') AS TEXT), count(*), count(DISTINCT machine_id)
|
||||
FROM events WHERE event = 'index' AND json_extract(props, '$.duration_bucket') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.duration_bucket');
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'scope', CAST(json_extract(props, '$.scope') AS TEXT), count(*), count(DISTINCT machine_id)
|
||||
FROM events WHERE event = 'install' AND json_extract(props, '$.scope') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.scope');
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'kind', CAST(json_extract(props, '$.kind') AS TEXT),
|
||||
CASE WHEN event = 'usage_rollup' THEN sum(coalesce(json_extract(props, '$.count'), 0)) ELSE count(*) END,
|
||||
count(DISTINCT machine_id)
|
||||
FROM events WHERE event IN ('install', 'usage_rollup') AND json_extract(props, '$.kind') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.kind');
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'name', CAST(json_extract(props, '$.name') AS TEXT),
|
||||
sum(coalesce(json_extract(props, '$.count'), 0)), count(DISTINCT machine_id)
|
||||
FROM events WHERE event = 'usage_rollup' AND json_extract(props, '$.name') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.name');
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'client_name', CAST(json_extract(props, '$.client_name') AS TEXT),
|
||||
sum(coalesce(json_extract(props, '$.count'), 0)), count(DISTINCT machine_id)
|
||||
FROM events WHERE event = 'usage_rollup' AND json_extract(props, '$.client_name') IS NOT NULL
|
||||
GROUP BY day, event, json_extract(props, '$.client_name');
|
||||
|
||||
-- Array props — one row per element, so a TypeScript+Go repo counts under both.
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT e.day, e.event, 'language', CAST(j.value AS TEXT), count(*), count(DISTINCT e.machine_id)
|
||||
FROM events e, json_each(e.props, '$.languages') j
|
||||
WHERE e.event = 'index' AND j.value <> ''
|
||||
GROUP BY e.day, e.event, j.value;
|
||||
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT e.day, e.event, 'target', CAST(j.value AS TEXT), count(*), count(DISTINCT e.machine_id)
|
||||
FROM events e, json_each(e.props, '$.targets') j
|
||||
WHERE e.event IN ('install', 'uninstall') AND j.value <> ''
|
||||
GROUP BY e.day, e.event, j.value;
|
||||
|
||||
-- Errors per tool: count is errors, machines is the machines that saw one.
|
||||
INSERT INTO daily_dim_counts (day, event, dim, value, count, machines)
|
||||
SELECT day, event, 'name_error', CAST(json_extract(props, '$.name') AS TEXT),
|
||||
sum(json_extract(props, '$.error_count')), count(DISTINCT machine_id)
|
||||
FROM events
|
||||
WHERE event = 'usage_rollup' AND json_extract(props, '$.name') IS NOT NULL
|
||||
AND coalesce(json_extract(props, '$.error_count'), 0) > 0
|
||||
GROUP BY day, event, json_extract(props, '$.name');
|
||||
@@ -0,0 +1,465 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Renders the dashboard in a real browser against the fixture and checks that
|
||||
* every panel drew, and drew the numbers the API returned.
|
||||
*
|
||||
* smoke-api.sh proves the SQL; this proves the other half — that each panel is
|
||||
* wired to the right endpoint and plots it without mangling it. It reads the
|
||||
* Chart.js instance off each canvas and compares its dataset arrays against the
|
||||
* same endpoint fetched straight from Node, so a panel pointed at the wrong dim
|
||||
* fails here even though both halves are individually fine.
|
||||
*
|
||||
* node scripts/render-check.mjs (or: npm run smoke:render)
|
||||
*
|
||||
* Zero new dependencies: it drives whatever Chromium is already on the machine
|
||||
* over the DevTools protocol (Node 22 has WebSocket built in). With no browser
|
||||
* installed it SKIPS rather than fails — the shell smoke suites stay the
|
||||
* portable floor, and this is the deeper check where a browser exists.
|
||||
*/
|
||||
|
||||
import { spawn } from 'node:child_process';
|
||||
import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
|
||||
const root = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const PORT = Number(process.env.DASH_PORT ?? 8790);
|
||||
const BASE = `http://127.0.0.1:${PORT}`;
|
||||
|
||||
/** The fixture's own window — see scripts/fixture.sql. */
|
||||
const FROM = '2026-07-01';
|
||||
const TO = '2026-07-10';
|
||||
|
||||
let pass = 0;
|
||||
let fail = 0;
|
||||
|
||||
const ok = (what) => {
|
||||
console.log(` ok ${what}`);
|
||||
pass++;
|
||||
};
|
||||
const bad = (what, detail) => {
|
||||
console.log(` FAIL ${what}${detail ? ` (${detail})` : ''}`);
|
||||
fail++;
|
||||
};
|
||||
const check = (what, condition, detail) => (condition ? ok(what) : bad(what, detail));
|
||||
const same = (what, expected, actual) =>
|
||||
check(
|
||||
what,
|
||||
JSON.stringify(expected) === JSON.stringify(actual),
|
||||
`expected ${JSON.stringify(expected)}, got ${JSON.stringify(actual)}`,
|
||||
);
|
||||
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Finding a browser
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Expands one `*` in a path segment, newest match first. */
|
||||
function glob(pattern) {
|
||||
const [head, ...rest] = pattern.split('*');
|
||||
const base = dirname(head);
|
||||
const prefix = head.slice(base.length + 1);
|
||||
if (!existsSync(base)) return [];
|
||||
return readdirSync(base)
|
||||
.filter((name) => name.startsWith(prefix))
|
||||
.sort()
|
||||
.reverse()
|
||||
.map((name) => join(base, name) + rest.join('*'));
|
||||
}
|
||||
|
||||
function findBrowser() {
|
||||
const home = process.env.HOME ?? '';
|
||||
const candidates = [
|
||||
process.env.CHROME_BIN,
|
||||
...glob(`${home}/Library/Caches/ms-playwright/chromium_headless_shell-*/chrome-headless-shell-mac-arm64/chrome-headless-shell`),
|
||||
...glob(`${home}/Library/Caches/ms-playwright/chromium_headless_shell-*/chrome-headless-shell-mac-x64/chrome-headless-shell`),
|
||||
...glob(`${home}/.cache/ms-playwright/chromium_headless_shell-*/chrome-headless-shell-linux/chrome-headless-shell`),
|
||||
...glob(`${home}/.cache/ms-playwright/chromium-*/chrome-linux/chrome`),
|
||||
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
||||
'/Applications/Chromium.app/Contents/MacOS/Chromium',
|
||||
'/usr/bin/chromium',
|
||||
'/usr/bin/chromium-browser',
|
||||
'/usr/bin/google-chrome',
|
||||
];
|
||||
return candidates.find((path) => path && existsSync(path)) ?? null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// A minimal DevTools-protocol client
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
class CDP {
|
||||
constructor(socket) {
|
||||
this.socket = socket;
|
||||
this.nextId = 1;
|
||||
this.pending = new Map();
|
||||
this.events = [];
|
||||
socket.addEventListener('message', (event) => {
|
||||
const message = JSON.parse(event.data);
|
||||
if (message.id !== undefined) {
|
||||
const waiter = this.pending.get(message.id);
|
||||
if (!waiter) return;
|
||||
this.pending.delete(message.id);
|
||||
if (message.error) waiter.reject(new Error(message.error.message));
|
||||
else waiter.resolve(message.result);
|
||||
} else {
|
||||
this.events.push(message);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
static async connect(url) {
|
||||
const socket = new WebSocket(url);
|
||||
await new Promise((resolve, reject) => {
|
||||
socket.addEventListener('open', resolve, { once: true });
|
||||
socket.addEventListener('error', () => reject(new Error(`cannot reach ${url}`)), { once: true });
|
||||
});
|
||||
return new CDP(socket);
|
||||
}
|
||||
|
||||
send(method, params = {}, sessionId) {
|
||||
const id = this.nextId++;
|
||||
return new Promise((resolve, reject) => {
|
||||
this.pending.set(id, { resolve, reject });
|
||||
this.socket.send(JSON.stringify(sessionId ? { id, method, params, sessionId } : { id, method, params }));
|
||||
});
|
||||
}
|
||||
|
||||
/** Runs an expression in the page and returns its value, awaiting promises. */
|
||||
async evaluate(sessionId, expression) {
|
||||
const result = await this.send(
|
||||
'Runtime.evaluate',
|
||||
{ expression, returnByValue: true, awaitPromise: true },
|
||||
sessionId,
|
||||
);
|
||||
if (result.exceptionDetails) {
|
||||
throw new Error(result.exceptionDetails.exception?.description ?? 'page threw');
|
||||
}
|
||||
return result.result.value;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The page probe
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Runs inside the page. Reads what each panel actually rendered — including the
|
||||
* live Chart.js instance behind each canvas — rather than trusting that a
|
||||
* fetch resolved.
|
||||
*/
|
||||
const PROBE = `(() => {
|
||||
const panels = [...document.querySelectorAll('[data-panel]')].map((section) => {
|
||||
const canvas = section.querySelector('canvas');
|
||||
const chart = canvas && window.Chart ? window.Chart.getChart(canvas) : null;
|
||||
return {
|
||||
id: section.dataset.panel,
|
||||
state: section.dataset.state,
|
||||
stale: section.dataset.stale,
|
||||
title: section.querySelector('h2').textContent,
|
||||
figure: section.querySelector('[data-role="figure"]').textContent,
|
||||
note: section.querySelector('.panel-note')?.textContent ?? '',
|
||||
message: section.querySelector('[data-role="state"]').textContent,
|
||||
stat: section.querySelector('.stat-value')?.textContent ?? null,
|
||||
funnelValues: [...section.querySelectorAll('.funnel-value')].map((n) => n.textContent),
|
||||
funnelWidths: [...section.querySelectorAll('.funnel-fill')].map((n) => n.style.width),
|
||||
chart: chart && {
|
||||
type: chart.config.type,
|
||||
labels: chart.data.labels,
|
||||
datasets: chart.data.datasets.map((d) => ({ label: d.label, data: d.data })),
|
||||
legend: chart.options.plugins?.legend?.display !== false,
|
||||
},
|
||||
tableRows: section.querySelectorAll('[data-role="table"] tbody tr').length,
|
||||
tableCols: section.querySelectorAll('[data-role="table"] thead th').length,
|
||||
tableHidden: section.querySelector('[data-role="table"]').hidden,
|
||||
};
|
||||
});
|
||||
return {
|
||||
ready: document.body.dataset.ready === 'true',
|
||||
range: document.getElementById('range-summary').textContent,
|
||||
dataThrough: document.getElementById('data-through').textContent,
|
||||
refreshed: document.getElementById('refreshed-at').textContent,
|
||||
selectedPreset: document.querySelector('button.range.is-selected')?.textContent ?? null,
|
||||
panels,
|
||||
};
|
||||
})()`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Run
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const children = [];
|
||||
let profileDir = null;
|
||||
|
||||
function cleanup() {
|
||||
for (const child of children) {
|
||||
try {
|
||||
child.kill('SIGTERM');
|
||||
} catch {
|
||||
/* already gone */
|
||||
}
|
||||
}
|
||||
if (profileDir) rmSync(profileDir, { recursive: true, force: true });
|
||||
}
|
||||
process.on('exit', cleanup);
|
||||
process.on('SIGINT', () => process.exit(130));
|
||||
|
||||
function run(command, args, options = {}) {
|
||||
const child = spawn(command, args, { cwd: root, stdio: 'ignore', ...options });
|
||||
children.push(child);
|
||||
return child;
|
||||
}
|
||||
|
||||
async function waitFor(what, probe, attempts = 90) {
|
||||
for (let i = 0; i < attempts; i++) {
|
||||
try {
|
||||
if (await probe()) return true;
|
||||
} catch {
|
||||
/* not up yet */
|
||||
}
|
||||
await sleep(1000);
|
||||
}
|
||||
throw new Error(`timed out waiting for ${what}`);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const browserPath = findBrowser();
|
||||
if (!browserPath) {
|
||||
console.log('render-check: no Chromium found — skipping.');
|
||||
console.log(' Set CHROME_BIN, or install Chrome; the shell smoke suites cover the rest.');
|
||||
return 0;
|
||||
}
|
||||
console.log(`Browser: ${browserPath}`);
|
||||
|
||||
console.log('Seeding the local D1 fixture…');
|
||||
const seed = run('./scripts/seed-fixture.sh', [], { stdio: 'inherit' });
|
||||
const seeded = await new Promise((resolve) => seed.on('exit', resolve));
|
||||
if (seeded !== 0) throw new Error('seeding failed');
|
||||
|
||||
console.log(`Starting wrangler dev on :${PORT}…`);
|
||||
run('npx', ['wrangler', 'dev', '--port', String(PORT), '--ip', '127.0.0.1']);
|
||||
await waitFor('wrangler dev', async () => (await fetch(`${BASE}/robots.txt`)).ok);
|
||||
|
||||
const password = readFileSync(join(root, '.dev.vars'), 'utf8').match(/^ADMIN_PASSWORD="(.*)"$/m)?.[1];
|
||||
if (!password) throw new Error('no ADMIN_PASSWORD in .dev.vars');
|
||||
const login = await fetch(`${BASE}/login`, {
|
||||
method: 'POST',
|
||||
body: new URLSearchParams({ password }),
|
||||
redirect: 'manual',
|
||||
});
|
||||
const cookie = login.headers.getSetCookie().find((c) => c.startsWith('cg_admin_session='));
|
||||
if (!cookie) throw new Error('login did not set a session cookie');
|
||||
const [name, value] = cookie.split(';')[0].split('=');
|
||||
|
||||
profileDir = mkdtempSync(join(tmpdir(), 'cg-dash-profile-'));
|
||||
// chrome-headless-shell is headless by construction and rejects the flag;
|
||||
// a full Chrome needs it.
|
||||
const headlessFlag = browserPath.includes('headless') ? [] : ['--headless=new'];
|
||||
run(browserPath, [
|
||||
...headlessFlag,
|
||||
'--disable-gpu',
|
||||
'--no-first-run',
|
||||
'--no-default-browser-check',
|
||||
'--remote-debugging-port=0',
|
||||
`--user-data-dir=${profileDir}`,
|
||||
'about:blank',
|
||||
]);
|
||||
|
||||
let devtoolsPort = null;
|
||||
await waitFor('the browser', () => {
|
||||
const portFile = join(profileDir, 'DevToolsActivePort');
|
||||
if (!existsSync(portFile)) return false;
|
||||
devtoolsPort = Number(readFileSync(portFile, 'utf8').split('\n')[0]);
|
||||
return Number.isFinite(devtoolsPort) && devtoolsPort > 0;
|
||||
}, 30);
|
||||
|
||||
const version = await (await fetch(`http://127.0.0.1:${devtoolsPort}/json/version`)).json();
|
||||
const cdp = await CDP.connect(version.webSocketDebuggerUrl);
|
||||
const { targetId } = await cdp.send('Target.createTarget', { url: 'about:blank' });
|
||||
const { sessionId } = await cdp.send('Target.attachToTarget', { targetId, flatten: true });
|
||||
|
||||
await cdp.send('Page.enable', {}, sessionId);
|
||||
await cdp.send('Runtime.enable', {}, sessionId);
|
||||
await cdp.send('Log.enable', {}, sessionId);
|
||||
await cdp.send('Network.enable', {}, sessionId);
|
||||
await cdp.send('Network.setCookie', { url: BASE, name, value, path: '/', httpOnly: true }, sessionId);
|
||||
|
||||
await cdp.send('Page.navigate', { url: `${BASE}/` }, sessionId);
|
||||
await waitFor('the dashboard to finish rendering', async () => {
|
||||
const view = await cdp.evaluate(sessionId, 'document.body.dataset.ready === "true"');
|
||||
return view === true;
|
||||
}, 60);
|
||||
|
||||
let view = await cdp.evaluate(sessionId, PROBE);
|
||||
|
||||
// -- what loaded ---------------------------------------------------------
|
||||
console.log('\nThe page renders');
|
||||
// The very same registry the page just rendered from, imported here so the
|
||||
// expectations cannot drift from the panels under test.
|
||||
const { PANELS } = await import(pathToFileURL(join(root, 'public', 'panels.js')).href);
|
||||
check(`all ${PANELS.length} panels are on the page`, view.panels.length === PANELS.length, `got ${view.panels.length}`);
|
||||
const broken = view.panels.filter((p) => p.state !== 'ready');
|
||||
check(
|
||||
'every panel reached its ready state',
|
||||
broken.length === 0,
|
||||
broken.map((p) => `${p.id}: ${p.state} ${p.message}`).join(' | '),
|
||||
);
|
||||
check('the default range is the 30-day preset', view.selectedPreset === 'Last 30 days', view.selectedPreset);
|
||||
check('the range is stated in the filter row', /Jun|Jul/.test(view.range), view.range);
|
||||
check('the data horizon is stated', view.dataThrough.includes('Jul 10'), view.dataThrough);
|
||||
check('the refresh time is stated', view.refreshed.startsWith('Last refreshed'), view.refreshed);
|
||||
|
||||
// A CSP violation surfaces here as a `security` log entry, which is the point
|
||||
// of the check: the page must work under `script-src 'self'` with no inline
|
||||
// styles at all. The favicon 404 is expected — there isn't one — and is the
|
||||
// only network noise allowed through.
|
||||
const errors = cdp.events.filter(
|
||||
(e) =>
|
||||
(e.method === 'Log.entryAdded' &&
|
||||
e.params.entry.level === 'error' &&
|
||||
!/favicon/.test(e.params.entry.url ?? '')) ||
|
||||
e.method === 'Runtime.exceptionThrown',
|
||||
);
|
||||
check(
|
||||
'no console errors — the strict CSP allows everything the page needs',
|
||||
errors.length === 0,
|
||||
errors.map((e) => e.params.entry?.text ?? e.params.exceptionDetails?.text).join(' | '),
|
||||
);
|
||||
|
||||
// -- the range picker really re-queries ----------------------------------
|
||||
console.log('\nChanging the range re-queries every panel');
|
||||
await cdp.evaluate(
|
||||
sessionId,
|
||||
`document.body.dataset.ready = "";
|
||||
[...document.querySelectorAll('button.range')].find((b) => b.textContent === 'Last 7 days').click();`,
|
||||
);
|
||||
await waitFor('the 7-day render', async () =>
|
||||
(await cdp.evaluate(sessionId, 'document.body.dataset.ready === "true"')) === true,
|
||||
);
|
||||
view = await cdp.evaluate(sessionId, PROBE);
|
||||
const weekly = view.panels.find((p) => p.id === 'daily-production-users');
|
||||
check('a daily line now holds 7 points', weekly.chart?.labels.length === 7, `${weekly.chart?.labels.length}`);
|
||||
check('the 7-day preset is marked selected', view.selectedPreset === 'Last 7 days', view.selectedPreset);
|
||||
check('every panel re-rendered cleanly', view.panels.every((p) => p.state === 'ready'));
|
||||
|
||||
console.log('\nA custom range works the same way');
|
||||
await cdp.evaluate(
|
||||
sessionId,
|
||||
`document.body.dataset.ready = "";
|
||||
document.querySelector('[data-role="custom-from"]').value = "${FROM}";
|
||||
document.querySelector('[data-role="custom-to"]').value = "${TO}";
|
||||
document.querySelector('[data-role="custom-apply"]').click();`,
|
||||
);
|
||||
await waitFor('the custom-range render', async () =>
|
||||
(await cdp.evaluate(sessionId, 'document.body.dataset.ready === "true"')) === true,
|
||||
);
|
||||
view = await cdp.evaluate(sessionId, PROBE);
|
||||
check('the fixture window is 10 days', view.panels.find((p) => p.id === 'daily-production-users').chart?.labels.length === 10);
|
||||
check('no preset stays highlighted', view.selectedPreset === null, view.selectedPreset);
|
||||
|
||||
// -- every panel plots what the API returned ------------------------------
|
||||
console.log('\nEvery panel plots the API’s own numbers');
|
||||
const query = `from=${FROM}&to=${TO}`;
|
||||
const fetched = new Map();
|
||||
const apiGet = async (path) => {
|
||||
if (!fetched.has(path)) {
|
||||
fetched.set(
|
||||
path,
|
||||
fetch(`${BASE}${path}`, { headers: { cookie: `${name}=${value}` } }).then((r) => r.json()),
|
||||
);
|
||||
}
|
||||
return fetched.get(path);
|
||||
};
|
||||
|
||||
for (const panel of PANELS) {
|
||||
const rendered = view.panels.find((p) => p.id === panel.id);
|
||||
const data = await apiGet(panel.source(query));
|
||||
|
||||
if (panel.kind === 'chart') {
|
||||
const plotted = rendered.chart?.datasets.map((d) => d.data);
|
||||
same(`${panel.id}: plots the endpoint's series`, data.datasets.map((d) => d.data), plotted);
|
||||
// A legend is owed wherever colour carries identity: any multi-series
|
||||
// chart, and every pie (whose slices are identities inside one dataset).
|
||||
// A single line needs none — the panel title already names it.
|
||||
const owed = rendered.chart.type === 'pie' || data.datasets.length > 1;
|
||||
check(
|
||||
`${panel.id}: a legend exactly where colour carries identity`,
|
||||
rendered.chart.legend === owed,
|
||||
`legend ${rendered.chart.legend}, expected ${owed}`,
|
||||
);
|
||||
} else if (panel.kind === 'stat') {
|
||||
same(`${panel.id}: shows the endpoint's number`, panel.stat(data).value, rendered.stat);
|
||||
} else if (panel.kind === 'funnel') {
|
||||
same(
|
||||
`${panel.id}: shows both funnel stages`,
|
||||
panel.funnel(data).stages.map((s) => s.value.toLocaleString('en-US')),
|
||||
rendered.funnelValues,
|
||||
);
|
||||
}
|
||||
|
||||
const table = panel.table(data);
|
||||
check(
|
||||
`${panel.id}: the table twin carries every row`,
|
||||
rendered.tableRows === table.rows.length && rendered.tableCols === table.columns.length,
|
||||
`${rendered.tableRows}×${rendered.tableCols} vs ${table.rows.length}×${table.columns.length}`,
|
||||
);
|
||||
}
|
||||
|
||||
// -- a few numbers checked against the fixture by hand --------------------
|
||||
console.log('\nSpot checks against the fixture, worked out by hand');
|
||||
const byId = Object.fromEntries(view.panels.map((p) => [p.id, p]));
|
||||
same('production users is 11 (m12 is the CI machine)', '11', byId['production-users'].stat);
|
||||
same('installs is 12', '12', byId['installs'].stat);
|
||||
same('uninstalls is 2', '2', byId['uninstalls'].stat);
|
||||
same('indexing runs is 13', '13', byId['indexing-runs'].stat);
|
||||
same('the funnel loses m04 and m06', ['12', '10'], byId['activation-funnel'].funnelValues);
|
||||
const widths = byId['activation-funnel'].funnelWidths;
|
||||
check(
|
||||
'…and draws the drop as a shorter bar',
|
||||
widths[0] === '100%' && widths[1].startsWith('83.3'),
|
||||
widths.join(' / '),
|
||||
);
|
||||
same('the OS pie is machine-days', ['linux', 'darwin', 'win32'], byId.os.chart.labels);
|
||||
same('…and its slices are 9 / 8 / 4', [[9, 8, 4]], byId.os.chart.datasets.map((d) => d.data));
|
||||
check('…with the honest metric named under the title', byId.os.figure === '21 machine-days', byId.os.figure);
|
||||
same('run length keeps its bucket order', ['<10s', '10-60s', '1-5m', '5m+'], byId['run-length'].chart.labels);
|
||||
same('languages lead with typescript', 'typescript', byId.languages.chart.labels[0]);
|
||||
check('retention starts at 100%', byId.retention.chart.datasets[0].data[0] === 100);
|
||||
|
||||
// Colour, spacing and label collisions are not things an assertion catches.
|
||||
// RENDER_SHOT=/tmp/dash.png npm run smoke:render → look at it.
|
||||
if (process.env.RENDER_SHOT) {
|
||||
await cdp.send(
|
||||
'Emulation.setDeviceMetricsOverride',
|
||||
{ width: 1440, height: 900, deviceScaleFactor: 2, mobile: false },
|
||||
sessionId,
|
||||
);
|
||||
await sleep(500);
|
||||
const shot = await cdp.send(
|
||||
'Page.captureScreenshot',
|
||||
{ format: 'png', captureBeyondViewport: true },
|
||||
sessionId,
|
||||
);
|
||||
writeFileSync(process.env.RENDER_SHOT, Buffer.from(shot.data, 'base64'));
|
||||
console.log(`\nScreenshot written to ${process.env.RENDER_SHOT}`);
|
||||
}
|
||||
|
||||
console.log('\nPanel copy follows the house rules');
|
||||
const capsy = view.panels.filter((p) => /^[A-Z0-9 ]{4,}$/.test(p.title));
|
||||
check('no shouty panel titles', capsy.length === 0, capsy.map((p) => p.title).join(', '));
|
||||
check('every panel says what it is counting', view.panels.every((p) => p.note.length > 20));
|
||||
check('tables start closed', view.panels.every((p) => p.tableHidden));
|
||||
|
||||
return fail;
|
||||
}
|
||||
|
||||
try {
|
||||
const failures = await main();
|
||||
console.log(`\n${pass} passed, ${fail} failed`);
|
||||
process.exit(failures === 0 ? 0 : 1);
|
||||
} catch (err) {
|
||||
console.error(`\nrender-check: ${err.message}`);
|
||||
process.exit(1);
|
||||
}
|
||||
Executable
+31
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env bash
|
||||
# Loads scripts/fixture.sql into the LOCAL .wrangler D1 (never the remote one:
|
||||
# --local is on every command here, and nothing in this repo writes production).
|
||||
#
|
||||
# The schema comes from the writer, telemetry-worker/migrations/, because that
|
||||
# is where it belongs — D1 is read-only from this worker.
|
||||
#
|
||||
# ./scripts/seed-fixture.sh (or: npm run seed)
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
DB=codegraph-telemetry
|
||||
MIGRATION=../telemetry-worker/migrations/0001_init.sql
|
||||
|
||||
if [[ ! -f "$MIGRATION" ]]; then
|
||||
echo "seed: cannot find $MIGRATION — run this from a full checkout" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The migration is plain CREATE TABLE, so a second run fails on "table already
|
||||
# exists". That is the expected steady state here, hence the swallowed output —
|
||||
# the fixture load below is the step whose failure actually matters.
|
||||
npx wrangler d1 execute "$DB" --local --file="$MIGRATION" >/dev/null 2>&1
|
||||
|
||||
if ! npx wrangler d1 execute "$DB" --local --file=scripts/fixture.sql >/dev/null; then
|
||||
echo "seed: loading scripts/fixture.sql failed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "seed: fixture loaded into the local $DB (12 machines, 2026-07-01 … 2026-07-10)"
|
||||
Executable
+264
@@ -0,0 +1,264 @@
|
||||
#!/usr/bin/env bash
|
||||
# End-to-end check of the chart API against the committed fixture.
|
||||
#
|
||||
# Every expected number below is worked out by hand from scripts/fixture.sql —
|
||||
# the header comment there lists all twelve machines and what each one does — so
|
||||
# a failure here means the SQL changed its mind, not that a golden file drifted.
|
||||
#
|
||||
# ./scripts/smoke-api.sh (or: npm run smoke:api)
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
# Deliberately NOT $PORT — see smoke-auth.sh.
|
||||
DASH_PORT="${DASH_PORT:-8789}"
|
||||
BASE="http://127.0.0.1:${DASH_PORT}"
|
||||
PASSWORD="$(grep '^ADMIN_PASSWORD=' .dev.vars | cut -d'"' -f2)"
|
||||
JAR="$(mktemp -t cg-api-jar)"
|
||||
LOG="$(mktemp -t cg-api-log)"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
# The fixture's own window. Every assertion is scoped to it, so a later fixture
|
||||
# row outside these days cannot silently change an expected number.
|
||||
FROM=2026-07-01
|
||||
TO=2026-07-10
|
||||
RANGE="from=$FROM&to=$TO"
|
||||
|
||||
cleanup() {
|
||||
[[ -n "${DEV_PID:-}" ]] && kill "$DEV_PID" 2>/dev/null
|
||||
rm -f "$JAR" "$LOG"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
status() { curl -s -o /dev/null -w '%{http_code}' "$@"; }
|
||||
get() { curl -s -b "$JAR" "$BASE$1"; }
|
||||
|
||||
# Resolves a dotted path through the JSON. Numeric segments index arrays, so
|
||||
# `datasets.0.data` works. Node rather than jq: this is a Node project, jq is not.
|
||||
jget() {
|
||||
node -e '
|
||||
let v = JSON.parse(process.argv[1]);
|
||||
for (const key of process.argv[2].split(".")) v = v?.[key];
|
||||
console.log(v === undefined ? "<missing>" : typeof v === "object" && v !== null ? JSON.stringify(v) : String(v));
|
||||
' "$1" "$2"
|
||||
}
|
||||
|
||||
check() { # check <description> <expected> <actual>
|
||||
if [[ "$2" == "$3" ]]; then
|
||||
printf ' ok %s\n' "$1"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL %s (expected %s, got %s)\n' "$1" "$2" "$3"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
field() { # field <description> <path> <expected> <json>
|
||||
check "$1" "$3" "$(jget "$4" "$2")"
|
||||
}
|
||||
|
||||
echo "Seeding the local D1 fixture…"
|
||||
./scripts/seed-fixture.sh || exit 1
|
||||
|
||||
echo "Starting wrangler dev on :${DASH_PORT}…"
|
||||
npx wrangler dev --port "$DASH_PORT" --ip 127.0.0.1 >"$LOG" 2>&1 &
|
||||
DEV_PID=$!
|
||||
READY=""
|
||||
for _ in $(seq 1 90); do
|
||||
if [[ "$(curl -s "$BASE/robots.txt")" == "User-agent: *"* ]]; then READY=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
if [[ -z "$READY" ]]; then
|
||||
echo "wrangler dev never came up on :${DASH_PORT} — log follows"
|
||||
cat "$LOG"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "The gate still holds on every new endpoint"
|
||||
for path in summary meta timeseries breakdown activation retention; do
|
||||
check "GET /api/$path without a cookie → 401" 401 "$(status "$BASE/api/$path")"
|
||||
done
|
||||
|
||||
curl -s -o /dev/null -c "$JAR" -X POST -d "password=$PASSWORD" "$BASE/login"
|
||||
check "signed in" 200 "$(status -b "$JAR" "$BASE/api/session")"
|
||||
|
||||
echo
|
||||
echo "Caching"
|
||||
check "chart data is privately cacheable" "private, max-age=300" \
|
||||
"$(curl -sD - -o /dev/null -b "$JAR" "$BASE/api/summary" | grep -i '^cache-control:' | cut -d' ' -f2- | tr -d '\r')"
|
||||
check "health stays uncached" "no-store" \
|
||||
"$(curl -sD - -o /dev/null -b "$JAR" "$BASE/api/health" | grep -i '^cache-control:' | cut -d' ' -f2- | tr -d '\r')"
|
||||
|
||||
echo
|
||||
echo "/api/meta — what the range picker anchors on"
|
||||
META="$(get "/api/meta")"
|
||||
field "latest day" latest_day 2026-07-10 "$META"
|
||||
field "earliest day" earliest_day 2026-07-01 "$META"
|
||||
field "raw events start" earliest_raw_day 2026-07-01 "$META"
|
||||
field "retention window" retention_days 14 "$META"
|
||||
|
||||
echo
|
||||
echo "/api/summary — the big numbers (12 machines, one of them CI)"
|
||||
SUMMARY="$(get "/api/summary?$RANGE")"
|
||||
field "production users (m12 is CI)" production_users 11 "$SUMMARY"
|
||||
field "active machines" active_machines 12 "$SUMMARY"
|
||||
field "new machines" new_machines 12 "$SUMMARY"
|
||||
field "installs" installs 12 "$SUMMARY"
|
||||
field "uninstalls" uninstalls 2 "$SUMMARY"
|
||||
field "indexing runs" index_runs 13 "$SUMMARY"
|
||||
field "tool calls (SUM of count)" tool_calls 85 "$SUMMARY"
|
||||
field "range echoed back" range.days 10 "$SUMMARY"
|
||||
|
||||
echo
|
||||
echo "/api/timeseries — one dense point per day, zeros where nothing happened"
|
||||
TS="$(get "/api/timeseries?metric=installs_uninstalls&$RANGE")"
|
||||
field "10 labels" labels.0 2026-07-01 "$TS"
|
||||
field "installs" datasets.0.data '[4,2,1,0,2,0,0,1,2,0]' "$TS"
|
||||
field "uninstalls" datasets.1.data '[0,0,0,0,0,1,1,0,0,0]' "$TS"
|
||||
field "legend labels" datasets.1.label Uninstalls "$TS"
|
||||
|
||||
TS="$(get "/api/timeseries?metric=new_installs&$RANGE")"
|
||||
field "new installs by first-seen day" datasets.0.data '[4,2,1,0,2,0,0,1,2,0]' "$TS"
|
||||
|
||||
TS="$(get "/api/timeseries?metric=production_users&$RANGE")"
|
||||
field "daily production users" datasets.0.data '[4,3,4,1,2,3,2,1,1,1]' "$TS"
|
||||
|
||||
TS="$(get "/api/timeseries?metric=indexing_activity&$RANGE")"
|
||||
field "indexing runs" datasets.0.data '[2,2,2,1,1,1,1,1,1,1]' "$TS"
|
||||
field "machines indexing" datasets.1.data '[2,2,2,1,1,1,1,1,1,1]' "$TS"
|
||||
|
||||
TS="$(get "/api/timeseries?metric=tool_calls&$RANGE")"
|
||||
field "calls per day" datasets.0.data '[0,40,28,0,0,12,0,0,0,5]' "$TS"
|
||||
field "machines per day" datasets.1.data '[0,1,2,0,0,1,0,0,0,1]' "$TS"
|
||||
|
||||
TS="$(get "/api/timeseries?metric=duration_buckets&$RANGE")"
|
||||
field "bucket order is the scale" datasets.0.label '<10s' "$TS"
|
||||
field "…and ends at the longest" datasets.3.label '5m+' "$TS"
|
||||
field "<10s over time" datasets.0.data '[2,0,1,1,0,0,0,1,0,0]' "$TS"
|
||||
field "10-60s over time" datasets.1.data '[0,2,0,0,0,0,1,0,0,1]' "$TS"
|
||||
field "1-5m over time" datasets.2.data '[0,0,0,0,1,1,0,0,0,0]' "$TS"
|
||||
field "5m+ over time" datasets.3.data '[0,0,1,0,0,0,0,0,1,0]' "$TS"
|
||||
|
||||
echo
|
||||
echo "/api/breakdown — bars and pies"
|
||||
# machine-days, taking the largest per-event count per day so one machine's
|
||||
# install + index + usage_rollup on one day is not counted three times.
|
||||
BD="$(get "/api/breakdown?dim=os&$RANGE")"
|
||||
field "os labels" labels '["linux","darwin","win32"]' "$BD"
|
||||
field "os machine-days" datasets.0.data '[9,8,4]' "$BD"
|
||||
field "os metric named" datasets.0.label 'Machine-days' "$BD"
|
||||
field "os total" total 21 "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=os&metric=count&$RANGE")"
|
||||
field "os by events sums every event" total 112 "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=language&$RANGE")"
|
||||
field "languages, most-indexed first" labels '["typescript","csharp","go","javascript","python","rust","java"]' "$BD"
|
||||
field "language counts" datasets.0.data '[7,2,2,2,2,2,1]' "$BD"
|
||||
field "language rows total" total 18 "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=file_count_bucket&$RANGE")"
|
||||
field "codebase size keeps bucket order" labels '["<100","100-1k","1k-10k","10k+"]' "$BD"
|
||||
field "codebase size counts" datasets.0.data '[2,5,4,2]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=duration_bucket&$RANGE")"
|
||||
field "run length keeps bucket order" labels '["<10s","10-60s","1-5m","5m+"]' "$BD"
|
||||
field "run length counts" datasets.0.data '[5,4,2,2]' "$BD"
|
||||
field "run length total = index runs" total 13 "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=target&$RANGE")"
|
||||
field "agent targets are the installs" event install "$BD"
|
||||
field "agent target labels" labels '["claude","cursor","codex","opencode"]' "$BD"
|
||||
field "agent target counts" datasets.0.data '[9,3,2,1]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=codegraph_version&$RANGE")"
|
||||
field "versions sort newest first" labels '["1.5.0","1.4.1","1.4.0"]' "$BD"
|
||||
field "version machine-days" datasets.0.data '[8,3,10]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=name&$RANGE")"
|
||||
field "tool names by call volume" labels '["codegraph_explore","index"]' "$BD"
|
||||
field "tool call counts" datasets.0.data '[82,3]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=client_name&$RANGE")"
|
||||
field "agents by call volume" labels '["Claude Code","Cursor"]' "$BD"
|
||||
field "agent call counts" datasets.0.data '[70,12]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=kind&$RANGE")"
|
||||
field "install kinds" labels '["fresh","upgrade"]' "$BD"
|
||||
field "install kind counts" datasets.0.data '[11,1]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=scope&$RANGE")"
|
||||
field "install scopes" datasets.0.data '[9,3]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=name_error&$RANGE")"
|
||||
field "errors by tool" datasets.0.data '[1]' "$BD"
|
||||
|
||||
BD="$(get "/api/breakdown?dim=language&limit=2&$RANGE")"
|
||||
field "the tail folds into Other, never truncates" labels '["typescript","csharp","Other"]' "$BD"
|
||||
field "Other keeps the total honest" total 18 "$BD"
|
||||
field "truncation is declared" truncated true "$BD"
|
||||
|
||||
echo
|
||||
echo "/api/activation — install → first index within 7 days"
|
||||
ACT="$(get "/api/activation?$RANGE")"
|
||||
field "cohort is every machine first seen" installs 12 "$ACT"
|
||||
field "m04 and m06 never indexed" activated 10 "$ACT"
|
||||
field "…so two dropped" dropped 2 "$ACT"
|
||||
field "window" window_days 7 "$ACT"
|
||||
field "daily rate, null where no cohort" datasets.0.data '[75,50,100,null,100,null,null,100,100,null]' "$ACT"
|
||||
field "recent cohorts flagged incomplete" incomplete_from 2026-07-04 "$ACT"
|
||||
field "…and the completed ones are not" rows.2.complete true "$ACT"
|
||||
field "…while the last week is" rows.8.complete false "$ACT"
|
||||
|
||||
# Narrowing the window drops m03 alone: it installed on 07-01 and did not index
|
||||
# until 07-03. Everyone else who ever indexed did it on day 0 or day 1.
|
||||
ACT="$(get "/api/activation?window=1&$RANGE")"
|
||||
field "a 1-day window converts fewer" activated 9 "$ACT"
|
||||
|
||||
echo
|
||||
echo "/api/retention — day 0–14, denominator per day"
|
||||
RET="$(get "/api/retention?$RANGE")"
|
||||
field "cohort size" cohort 12 "$RET"
|
||||
field "15 points" labels.14 'Day 14' "$RET"
|
||||
# Day 2 divides by 10, not 12: m11/m12 arrived on 07-09 and cannot have a day-2
|
||||
# data point yet. Day 10+ is null — nobody in the cohort is old enough at all.
|
||||
field "retention curve" datasets.0.data \
|
||||
'[100,41.7,30,11.1,0,22.2,0,0,0,0,null,null,null,null,null]' "$RET"
|
||||
field "day 2 eligible excludes the newest cohorts" rows.2.eligible 10 "$RET"
|
||||
field "day 10 has nobody old enough" rows.10.eligible 0 "$RET"
|
||||
|
||||
echo
|
||||
echo "Bad input is rejected, never guessed at"
|
||||
check "unknown dim → 400" 400 "$(status -b "$JAR" "$BASE/api/breakdown?dim=machine_id")"
|
||||
check "missing dim → 400" 400 "$(status -b "$JAR" "$BASE/api/breakdown")"
|
||||
check "unknown metric → 400" 400 "$(status -b "$JAR" "$BASE/api/breakdown?dim=os&metric=secrets")"
|
||||
check "unknown series → 400" 400 "$(status -b "$JAR" "$BASE/api/timeseries?metric=everything")"
|
||||
check "limit out of range → 400" 400 "$(status -b "$JAR" "$BASE/api/breakdown?dim=os&limit=0")"
|
||||
check "impossible date → 400" 400 "$(status -b "$JAR" "$BASE/api/summary?from=2026-02-31&to=2026-07-10")"
|
||||
check "malformed date → 400" 400 "$(status -b "$JAR" "$BASE/api/summary?from=yesterday")"
|
||||
check "backwards range → 400" 400 "$(status -b "$JAR" "$BASE/api/summary?from=2026-07-10&to=2026-07-01")"
|
||||
check "window out of range → 400" 400 "$(status -b "$JAR" "$BASE/api/activation?window=99")"
|
||||
check "unknown endpoint → 404" 404 "$(status -b "$JAR" "$BASE/api/everything")"
|
||||
check "event name is a closed shape → 400" 400 \
|
||||
"$(status -b "$JAR" "$BASE/api/breakdown?dim=os&event=install%27%20OR%201=1")"
|
||||
|
||||
CLAMPED="$(get "/api/breakdown?dim=os&from=2019-01-01&to=$TO")"
|
||||
field "a decade-wide range clamps to a year" range.days 366 "$CLAMPED"
|
||||
field "…and says so" range.clamped true "$CLAMPED"
|
||||
field "…kept against the recent end" range.from 2025-07-10 "$CLAMPED"
|
||||
|
||||
echo
|
||||
echo "An empty range renders as empty, not as an error"
|
||||
EMPTY="$(get "/api/summary?from=2025-01-01&to=2025-01-07")"
|
||||
field "no machines" production_users 0 "$EMPTY"
|
||||
field "no installs" installs 0 "$EMPTY"
|
||||
EMPTY="$(get "/api/breakdown?dim=os&from=2025-01-01&to=2025-01-07")"
|
||||
field "no bars" labels '[]' "$EMPTY"
|
||||
EMPTY="$(get "/api/timeseries?metric=production_users&from=2025-01-01&to=2025-01-03")"
|
||||
field "still a dense axis" datasets.0.data '[0,0,0]' "$EMPTY"
|
||||
|
||||
echo
|
||||
printf '%d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
[[ "$FAIL" -eq 0 ]]
|
||||
Executable
+211
@@ -0,0 +1,211 @@
|
||||
#!/usr/bin/env bash
|
||||
# End-to-end check of the auth gate against a local `wrangler dev`.
|
||||
#
|
||||
# Verifies the acceptance criteria for the gate: unauthenticated requests reach
|
||||
# nothing (pages, API, or static assets), a valid cookie reaches everything, and
|
||||
# a tampered cookie is rejected. Run it after touching src/auth.ts or the route
|
||||
# table in src/index.ts.
|
||||
#
|
||||
# ./scripts/smoke-auth.sh
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
# Deliberately NOT $PORT: that is commonly already set to some other local dev
|
||||
# server, and the whole suite would then silently test the wrong app.
|
||||
DASH_PORT="${DASH_PORT:-8788}"
|
||||
BASE="http://127.0.0.1:${DASH_PORT}"
|
||||
PASSWORD="$(grep '^ADMIN_PASSWORD=' .dev.vars | cut -d'"' -f2)"
|
||||
JAR="$(mktemp -t cg-dash-jar)"
|
||||
LOG="$(mktemp -t cg-dash-log)"
|
||||
DEV_VARS_BACKUP="$(mktemp -t cg-dash-vars)"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
cleanup() {
|
||||
[[ -n "${DEV_PID:-}" ]] && kill "$DEV_PID" 2>/dev/null
|
||||
# The rotation phase rewrites .dev.vars; always put the original back.
|
||||
[[ -s "$DEV_VARS_BACKUP" ]] && cp "$DEV_VARS_BACKUP" .dev.vars
|
||||
rm -f "$JAR" "$LOG" "$DEV_VARS_BACKUP"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# `curl -o /dev/null -w '%{http_code}'` plus the headers we care about.
|
||||
status() { curl -s -o /dev/null -w '%{http_code}' "$@"; }
|
||||
body() { curl -s "$@"; }
|
||||
|
||||
check() { # check <description> <expected> <actual>
|
||||
if [[ "$2" == "$3" ]]; then
|
||||
printf ' ok %s\n' "$1"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL %s (expected %s, got %s)\n' "$1" "$2" "$3"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
contains() { # contains <description> <needle> <haystack>
|
||||
if [[ "$3" == *"$2"* ]]; then
|
||||
printf ' ok %s\n' "$1"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL %s (missing %q in %.200q…)\n' "$1" "$2" "$3"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
lacks() { # lacks <description> <needle> <haystack>
|
||||
if [[ "$3" != *"$2"* ]]; then
|
||||
printf ' ok %s\n' "$1"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf ' FAIL %s (found %q)\n' "$1" "$2"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
echo "Seeding local D1 from the ingest worker's migration…"
|
||||
npx wrangler d1 execute codegraph-telemetry --local \
|
||||
--file=../telemetry-worker/migrations/0001_init.sql >/dev/null 2>&1
|
||||
|
||||
echo "Starting wrangler dev on :${DASH_PORT}…"
|
||||
npx wrangler dev --port "$DASH_PORT" --ip 127.0.0.1 >"$LOG" 2>&1 &
|
||||
DEV_PID=$!
|
||||
READY=""
|
||||
for _ in $(seq 1 90); do
|
||||
if [[ "$(body "$BASE/robots.txt")" == "User-agent: *"* ]]; then READY=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
if [[ -z "$READY" ]]; then
|
||||
echo "wrangler dev never came up on :${DASH_PORT} — log follows"
|
||||
cat "$LOG"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Unauthenticated — nothing but the login page and robots.txt"
|
||||
check "GET / → 302 to login" 302 "$(status "$BASE/")"
|
||||
check "GET /index.html → 302 to login" 302 "$(status "$BASE/index.html")"
|
||||
check "GET /styles.css → 302 to login" 302 "$(status "$BASE/styles.css")"
|
||||
check "GET /app.js → 302 to login" 302 "$(status "$BASE/app.js")"
|
||||
check "GET /vendor/chart → 302 to login" 302 "$(status "$BASE/vendor/chart.umd.js")"
|
||||
check "GET /api/health → 401" 401 "$(status "$BASE/api/health")"
|
||||
check "GET /api/session → 401" 401 "$(status "$BASE/api/session")"
|
||||
check "GET /api/anything → 401" 401 "$(status "$BASE/api/whatever")"
|
||||
check "GET /login → 200" 200 "$(status "$BASE/login")"
|
||||
check "GET /robots.txt → 200" 200 "$(status "$BASE/robots.txt")"
|
||||
contains "no data leaks in the 401 body" '"unauthorized"' "$(body "$BASE/api/health")"
|
||||
|
||||
echo
|
||||
echo "Login page"
|
||||
LOGIN_HTML="$(body "$BASE/login")"
|
||||
contains "sentence-case heading" "codegraph telemetry" "$LOGIN_HTML"
|
||||
contains "sentence-case label" ">Password<" "$LOGIN_HTML"
|
||||
contains "sentence-case button" ">Sign in<" "$LOGIN_HTML"
|
||||
lacks "no uppercased labels" "uppercase" "$LOGIN_HTML"
|
||||
lacks "no tracked-out labels" "letter-spacing" "$LOGIN_HTML"
|
||||
contains "label is normal size" "font-size: 16px" "$LOGIN_HTML"
|
||||
check "open redirect refused" "/" \
|
||||
"$(body "$BASE/login?next=%2F%2Fevil.example" | sed -n 's/.*name="next" value="\([^"]*\)".*/\1/p')"
|
||||
check "same-origin next kept" "/api/health" \
|
||||
"$(body "$BASE/login?next=%2Fapi%2Fhealth" | sed -n 's/.*name="next" value="\([^"]*\)".*/\1/p')"
|
||||
|
||||
echo
|
||||
echo "Sign-in"
|
||||
check "wrong password → 401" 401 \
|
||||
"$(status -X POST "$BASE/login" -d "password=definitely-not-it" -d "next=/")"
|
||||
check "wrong password sets no cookie" "" \
|
||||
"$(curl -s -D - -o /dev/null -X POST "$BASE/login" -d "password=nope" | grep -ci 'set-cookie' | sed 's/^0$//')"
|
||||
check "empty password → 400" 400 "$(status -X POST "$BASE/login" -d "password=")"
|
||||
check "cross-origin post → 400" 400 \
|
||||
"$(status -X POST "$BASE/login" -H 'Origin: https://evil.example' -d "password=${PASSWORD}")"
|
||||
# One sign-in, then every cookie assertion reads the captured headers. Doing a
|
||||
# fresh POST per assertion would burn the login rate limit and 429 halfway down.
|
||||
SIGNIN="$(curl -s -D - -o /dev/null -c "$JAR" -X POST "$BASE/login" -d "password=${PASSWORD}" -d "next=/")"
|
||||
check "correct password → 302" "302" "$(printf '%s' "$SIGNIN" | head -1 | awk '{print $2}')"
|
||||
contains "cookie is HttpOnly" "HttpOnly" "$SIGNIN"
|
||||
contains "cookie is Secure" "Secure" "$SIGNIN"
|
||||
contains "cookie is SameSite=Lax" "SameSite=Lax" "$SIGNIN"
|
||||
contains "cookie is ~1 year" "Max-Age=31536000" "$SIGNIN"
|
||||
contains "cookie is site-wide" "Path=/" "$SIGNIN"
|
||||
|
||||
COOKIE="$(grep cg_admin_session "$JAR" | awk '{print $NF}')"
|
||||
PAYLOAD="${COOKIE%%.*}"
|
||||
SIG="${COOKIE#*.}"
|
||||
|
||||
# A persistent cookie carries a real expiry in the jar; a session cookie (gone
|
||||
# on browser restart) carries 0. This is the "survives a restart" criterion.
|
||||
JAR_EXPIRY="$(grep cg_admin_session "$JAR" | awk '{print $5}')"
|
||||
if [[ "$JAR_EXPIRY" -gt "$(( $(date +%s) + 300 * 86400 ))" ]]; then
|
||||
check "cookie persists across browser restarts" "persistent" "persistent"
|
||||
else
|
||||
check "cookie persists across browser restarts" "persistent" "session-only (expiry ${JAR_EXPIRY})"
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "Authenticated — the whole app"
|
||||
check "GET / → 200" 200 "$(status -b "$JAR" "$BASE/")"
|
||||
check "GET /styles.css → 200" 200 "$(status -b "$JAR" "$BASE/styles.css")"
|
||||
check "GET /app.js → 200" 200 "$(status -b "$JAR" "$BASE/app.js")"
|
||||
check "GET /vendor/chart→ 200" 200 "$(status -b "$JAR" "$BASE/vendor/chart.umd.js")"
|
||||
check "GET /api/session → 200" 200 "$(status -b "$JAR" "$BASE/api/session")"
|
||||
check "GET /api/health → 200" 200 "$(status -b "$JAR" "$BASE/api/health")"
|
||||
contains "health reads D1" '"ok":true' "$(body -b "$JAR" "$BASE/api/health")"
|
||||
check "GET /login while signed in → 302" 302 "$(status -b "$JAR" "$BASE/login")"
|
||||
check "unknown API route → 404" 404 "$(status -b "$JAR" "$BASE/api/nope")"
|
||||
check "POST to an API route → 405" 405 "$(status -b "$JAR" -X POST "$BASE/api/health")"
|
||||
|
||||
echo
|
||||
echo "Tampering"
|
||||
# Mutate the FIRST signature character, not the last: base64url's final
|
||||
# character of a 32-byte tag carries only 4 significant bits, so flipping it is
|
||||
# sometimes a no-op on the decoded bytes and the test would pass vacuously.
|
||||
FLIPPED="${PAYLOAD}.$([[ "${SIG:0:1}" == 'A' ]] && echo B || echo A)${SIG:1}"
|
||||
check "flipped signature → 401" 401 "$(status -H "Cookie: cg_admin_session=${FLIPPED}" "$BASE/api/health")"
|
||||
check "truncated signature → 401" 401 "$(status -H "Cookie: cg_admin_session=${PAYLOAD}.${SIG:0:40}" "$BASE/api/health")"
|
||||
check "swapped payload → 401" 401 \
|
||||
"$(status -H "Cookie: cg_admin_session=$(printf '%s' '{"v":1,"iat":0,"exp":9999999999,"pw":"x"}' | base64 | tr -d '=' | tr '+/' '-_').${SIG}" "$BASE/api/health")"
|
||||
check "no signature → 401" 401 "$(status -H "Cookie: cg_admin_session=${PAYLOAD}" "$BASE/api/health")"
|
||||
check "garbage cookie → 401" 401 "$(status -H 'Cookie: cg_admin_session=not-a-token' "$BASE/api/health")"
|
||||
check "empty cookie → 401" 401 "$(status -H 'Cookie: cg_admin_session=' "$BASE/api/health")"
|
||||
check "tampered cookie on a page → 302 to login" 302 \
|
||||
"$(status -H "Cookie: cg_admin_session=${FLIPPED}" "$BASE/")"
|
||||
|
||||
echo
|
||||
echo "Sign-out"
|
||||
check "POST /logout → 302" 302 "$(status -X POST "$BASE/logout")"
|
||||
contains "logout clears the cookie" "Max-Age=0" \
|
||||
"$(curl -s -D - -o /dev/null -X POST "$BASE/logout")"
|
||||
check "GET /logout → 405" 405 "$(status "$BASE/logout")"
|
||||
|
||||
echo
|
||||
echo "Rate limiting (6 attempts in a minute; the 6th should be capped)"
|
||||
LAST=""
|
||||
for _ in 1 2 3 4 5 6 7; do
|
||||
LAST="$(status -X POST "$BASE/login" -d 'password=guess')"
|
||||
done
|
||||
check "brute force capped → 429" 429 "$LAST"
|
||||
|
||||
echo
|
||||
echo "Password rotation (restarting with a different ADMIN_PASSWORD)"
|
||||
cp .dev.vars "$DEV_VARS_BACKUP"
|
||||
sed 's/^ADMIN_PASSWORD=.*/ADMIN_PASSWORD="rotated-password"/' "$DEV_VARS_BACKUP" >.dev.vars
|
||||
kill "$DEV_PID" 2>/dev/null
|
||||
wait "$DEV_PID" 2>/dev/null
|
||||
npx wrangler dev --port "$DASH_PORT" --ip 127.0.0.1 >"$LOG" 2>&1 &
|
||||
DEV_PID=$!
|
||||
for _ in $(seq 1 90); do
|
||||
[[ "$(body "$BASE/robots.txt")" == "User-agent: *"* ]] && break
|
||||
sleep 1
|
||||
done
|
||||
check "cookie from the old password → 401" 401 \
|
||||
"$(status -H "Cookie: cg_admin_session=${COOKIE}" "$BASE/api/health")"
|
||||
check "old password no longer signs in → 401" 401 \
|
||||
"$(status -X POST "$BASE/login" -d "password=${PASSWORD}")"
|
||||
check "new password signs in → 302" 302 \
|
||||
"$(status -X POST "$BASE/login" -d "password=rotated-password")"
|
||||
|
||||
echo
|
||||
printf '%s\n' "-----"
|
||||
printf '%d passed, %d failed\n' "$PASS" "$FAIL"
|
||||
[[ "$FAIL" -eq 0 ]]
|
||||
@@ -0,0 +1,37 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Copies third-party browser libraries out of node_modules into public/vendor/.
|
||||
*
|
||||
* Workers Static Assets are served verbatim — nothing in public/ goes through a
|
||||
* bundler — so a library from npm has to be physically present there. Keeping
|
||||
* it a copy step (rather than a checked-in blob or a CDN <script>) means the
|
||||
* version is pinned by package.json, there is no third-party origin at runtime,
|
||||
* and the CSP can stay `script-src 'self'`.
|
||||
*
|
||||
* public/vendor/ is gitignored; `npm run dev` and `npm run deploy` both run this
|
||||
* first, so it is always present and always matches the lockfile.
|
||||
*/
|
||||
import { copyFileSync, mkdirSync, existsSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const root = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const vendorDir = join(root, 'public', 'vendor');
|
||||
|
||||
const FILES = [
|
||||
['node_modules/chart.js/dist/chart.umd.js', 'chart.umd.js'],
|
||||
['node_modules/chart.js/LICENSE.md', 'chart.js-LICENSE.md'],
|
||||
];
|
||||
|
||||
mkdirSync(vendorDir, { recursive: true });
|
||||
|
||||
for (const [from, to] of FILES) {
|
||||
const source = join(root, from);
|
||||
if (!existsSync(source)) {
|
||||
console.error(`vendor-assets: missing ${from} — run \`npm install\` first`);
|
||||
process.exit(1);
|
||||
}
|
||||
copyFileSync(source, join(vendorDir, to));
|
||||
}
|
||||
|
||||
console.log(`vendor-assets: copied ${FILES.length} file(s) into public/vendor/`);
|
||||
Reference in New Issue
Block a user