feat(ui): classify code from the engine's own tree-sitter parse, retiring Shiki (CG-57)

The viewer ran a second highlighter over source the engine had already parsed
with a real grammar: Shiki, plus 56 pruned TextMate grammars shipped in
dist/textmate/. The classification now comes off that tree instead, so a file is
read by exactly the grammar that decided what its symbols are.

The swap is complete rather than flagged: @shikijs/core, @shikijs/engine-javascript
and @shikijs/langs are off the dependency list, scripts/prune-grammars.mjs and
`npm run build:textmate` are deleted, and check-ui-build.mjs asserts the
tree-sitter grammars in dist/extraction/wasm instead of dist/textmate.

The wire contract is unchanged — `[classId, text]` pairs with the class names
alongside — so the viewer's decoder and code blocks did not have to be rewritten.
Two classes are added to the six: `type` (a named type reference, painted at
plain ink) and `def` (the name a definition declares, weight 600), the latter
taken from the extractors' own definition tables so it cannot drift from what
indexing calls a definition.

Three differences are not cosmetic:

* Interpolations (`${…}`, `#{…}`, `$"{…}"`, f-strings) are classified as code,
  not as string. The call-site overlay refuses to claim a token classed string,
  so calls written inside interpolated strings now link.
* Built-in type words are emitted whole and classed `type` in every language.
  The grammars disagree about whether `string` is a type_identifier or an
  anonymous token inside a predefined_type, and TextMate scoped them
  inconsistently too.
* 3 000 lines of TypeScript cost 24-41 ms instead of ~700 ms.

Given up deliberately: Liquid, Razor, YAML, Twig, XML and .properties render
plain. .svelte/.vue/.astro are classified through their <script> blocks, the same
delegation the SFC extractors do. Pulling html/css/vue out of tree-sitter-wasms
would cover them, but those ABI-13 builds are the known cause of shared-WASM-heap
corruption for every other language in the same process.

Measured parity, per-language before/after screenshots and the reproduction
recipe: docs/design/cg57-highlighting-parity.md.

ui/src/components/flow/FlowCard.svelte

@@ -8,7 +8,7 @@
   pinned here so the arrows land where the arithmetic said they would.
 
   The source window is the Symbol view's code block with the noise removed. It
-  keeps the two things that make the code readable: the server's TextMate
+  keeps the two things that make the code readable: the server's classified
   classification, and one accent link on the identifier the graph resolved. It
   drops gutter ports and multi-window folding, because a seven-line card has
   neither a gutter worth reading nor anything to fold.
@@ -213,7 +213,7 @@
   }
 
   /* Token classes — the same near-monochrome ramp the Symbol view paints
-     (design spec §2.2); the class names come from the server's theme. */
+     (design spec §2.2); the class names come from the server's classifier. */
   .t-c {
     color: var(--code-comment);
   }
@@ -227,6 +227,11 @@
     color: var(--ink-2);
   }
 
+  /* A definition's own name, from the extractor's tables. */
+  .t-def {
+    font-weight: 600;
+  }
+
   /* The only colour in the window: the call this card is opened at. */
   .ref {
     padding: 0;

ui/src/components/symbol/SourceBlock.svelte

@@ -4,10 +4,11 @@
 
   Two things make this more than a <pre>:
 
-  * Syntax classification arrives already done, from `/api/source` — real
-    TextMate grammars, run server-side, indexed by file line. The whole slice
-    is tokenised in one pass there, so a window that starts 200 lines into a
-    body still knows it is inside a block comment; nothing is re-lexed here.
+  * Syntax classification arrives already done, from `/api/source` — taken off
+    the engine's own tree-sitter parse, server-side, indexed by file line. The
+    whole slice is classified in one pass there, so a window that starts 200
+    lines into a body still knows it is inside a block comment; nothing is
+    re-lexed here.
   * Each ref is matched to an actual token rather than to a column, because the
     recorded column points at the start of the calling expression — see
     `assignRefs`. The overlay CLAIMS a token the highlighter produced; it never

ui/src/lib/api.ts

@@ -163,7 +163,7 @@ export interface WireSource {
   truncated?: boolean;
   reason?: string;
   /**
-   * The same lines, classified by the server's TextMate grammars — one entry
+   * The same lines, classified by the server's tree-sitter parse — one entry
    * per line, each a list of `[classId, text]` pairs indexed into `classes`.
    * Absent whenever `lines` is, and `engine: 'plain'` whenever no grammar
    * covers the file. See `lib/highlight.ts`.

ui/src/lib/filecode-model.ts

@@ -60,12 +60,13 @@ export const OVERSCAN_LINES = 24;
 /**
  * Source lines fetched in one page.
  *
- * Measured on this repo's own TypeScript with the shipped Shiki setup: a warm
- * grammar tokenises ~7 000 lines/second, so a page plus its lead-in is ~130 ms
- * of single-threaded server. Bigger pages mean fewer, longer stalls; smaller
- * ones mean the lead-in dominates. The scroll itself never waits on this —
- * ports, arcs and rail rows are already drawn from the graph, and the text
- * arrives behind them.
+ * Measured on this repo's own TypeScript with the shipped classifier: a loaded
+ * grammar classifies ~50 000 lines/second (CG-57 replaced the TextMate path,
+ * which managed ~4 000), so a page plus its lead-in is ~20 ms of
+ * single-threaded server. Bigger pages mean fewer, longer stalls; smaller ones
+ * mean the lead-in dominates. The scroll itself never waits on this — ports,
+ * arcs and rail rows are already drawn from the graph, and the text arrives
+ * behind them.
  */
 export const PAGE_LINES = 800;
 
@@ -73,7 +74,7 @@ export const PAGE_LINES = 800;
  * Lines fetched BEFORE a page and thrown away.
  *
  * A page that starts in the middle of a block comment, a template literal or a
- * JSX block does not know it: TextMate state is built by scanning from the top.
+ * JSX block does not know it: a parse starts from the top of what it is given.
  * Tokenising a run-up and discarding it is what keeps page 6 from rendering a
  * doc comment as code. The same trick the Flow strip's source windows use, at a
  * different scale — 150 lines covers every real comment block; a 3 000-line

ui/src/lib/highlight.ts

@@ -2,7 +2,8 @@
  * Turning the server's classified source into tokens the code block can draw.
  *
  * The classification itself happens on the server (`src/ui-server/highlight/`),
- * with real TextMate grammars via Shiki. What arrives is deliberately small:
+ * off the engine's own tree-sitter parse — the same grammar that decided what
+ * the file's symbols are. What arrives is deliberately small:
  * one array per line, each entry a `[classId, text]` pair, with the class names
  * carried alongside so the payload is self-describing. This module does two
  * things to it and nothing else — resolve the class ids to names, and compute
@@ -23,7 +24,18 @@
  * underline land on the callee's own name whatever boundaries a grammar chose.
  */
 
-export type TokenClass = 'other' | 'ident' | 'comment' | 'string' | 'keyword' | 'number';
+export type TokenClass =
+  | 'other'
+  | 'ident'
+  | 'comment'
+  | 'string'
+  | 'keyword'
+  | 'number'
+  /** A named type reference. Plain ink today — the class is here so a consumer
+   *  of this payload can style it without a second round of server work. */
+  | 'type'
+  /** The name a definition declares, from the extractor's own tables. */
+  | 'def';
 
 export interface Token {
   cls: TokenClass;
@@ -36,7 +48,7 @@ export interface Token {
 export type WireToken = [number, string];
 
 export interface WireHighlight {
-  engine: 'shiki' | 'plain';
+  engine: 'tree-sitter' | 'plain';
   grammar: string | null;
   classes: string[];
   lines: WireToken[][];
@@ -51,6 +63,8 @@ const CLASS_NAMES: ReadonlySet<string> = new Set<TokenClass>([
   'string',
   'keyword',
   'number',
+  'type',
+  'def',
 ]);
 
 /**
@@ -116,7 +130,15 @@ export function tokensByLine(
   return byLine;
 }
 
-/** The CSS class for a token, or null where the default ink is right. */
+/**
+ * The CSS class for a token, or null where the default ink is right.
+ *
+ * `type` deliberately returns null: the design's code colouring is
+ * near-monochrome and a type name is not one of the four things it moves off
+ * plain ink. It stays a distinct class on the wire because the classification
+ * is free once the tree has been walked, and re-deriving it in a consumer would
+ * not be.
+ */
 export function tokenClass(cls: TokenClass): string | null {
   switch (cls) {
     case 'comment':
@@ -127,6 +149,8 @@ export function tokenClass(cls: TokenClass): string | null {
       return 't-k';
     case 'number':
       return 't-n';
+    case 'def':
+      return 't-def';
     default:
       return null;
   }