feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43)
The viewer's code block stops lexing with a hand-rolled dialect table and reads real TextMate grammars instead, run once in `/api/source`. Three things make that safe to depend on: * Highlighting never fails a request. A missing grammar, an oversized slice, an ESM import that did not resolve — every one of them answers `engine: 'plain'` with a reason and the source still goes out. * Identifiers survive whatever token boundaries a grammar chose. Every code token is split into identifier runs before it goes on the wire, so the graph's call-site overlay claims a token the highlighter produced rather than re-cutting the line. `assignRefs` now matches on a token's text rather than on the class a grammar gave it, so a language that scopes type names as `storage.type` still links. * The theme classifies rather than colours: its foregrounds are sentinels the server maps back to class names, and the viewer paints them from CSS custom properties — one token stream serves light and dark with no refetch, and the ramp lives only in app.css. Comments move from --ink-3 to a new --code-comment. --ink-3 measures 3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px text; --code-comment is the smallest step along the same ramp that clears 4.5:1 on every background a code line can have, and stays quieter than the strings and numbers above it. Shipping: @shikijs/core and @shikijs/engine-javascript are runtime dependencies (no wasm, no native module); @shikijs/langs stays a devDependency and `npm run build:textmate` writes only the closure the engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for all 722. check-ui-build.mjs asserts the tree after every build and inside every release archive.
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
/**
|
||||
* Finding and reading the pruned TextMate grammars on disk.
|
||||
*
|
||||
* Shiki ships 700-odd grammars; the engine indexes 40-odd languages. The build
|
||||
* writes only the closure those 40 need — including the grammars they embed, so
|
||||
* a `.vue` file still gets its `<script lang="ts">` — into `dist/textmate/`,
|
||||
* and `@shikijs/langs` stays a devDependency that never reaches a user's disk.
|
||||
* See `scripts/prune-grammars.mjs`.
|
||||
*
|
||||
* They are located the way `db/index.ts` finds `schema.sql` and `assets.ts`
|
||||
* finds the viewer: relative to `__dirname`, never to `process.cwd()`, which is
|
||||
* whatever directory the user happened to be standing in.
|
||||
*
|
||||
* `dist/textmate`, not `dist/highlight` — `src/ui-server/highlight/` is this
|
||||
* module and tsc already owns `dist/ui-server/highlight/`. The same collision
|
||||
* that put the viewer in `dist/viewer` rather than `dist/ui`.
|
||||
*/
|
||||
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
|
||||
/** Overrides where the grammars are read from. For tests and for packagers. */
|
||||
export const TEXTMATE_PATH_ENV = 'CODEGRAPH_TEXTMATE_PATH';
|
||||
|
||||
/** What `scripts/prune-grammars.mjs` writes beside the grammar files. */
|
||||
export interface GrammarManifest {
|
||||
/** Shiki version the grammars were pruned from — surfaced when one fails. */
|
||||
shikiVersion: string;
|
||||
/** Grammar id → the files to load, dependencies first, the grammar itself last. */
|
||||
languages: Record<string, string[]>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Candidate locations, most-specific first.
|
||||
*
|
||||
* 1. The `CODEGRAPH_TEXTMATE_PATH` override.
|
||||
* 2. `<__dirname>/../../textmate` — the shipped layout
|
||||
* (`dist/ui-server/highlight/` → `dist/textmate/`).
|
||||
* 3. `<__dirname>/../../../dist/textmate` — running the TypeScript straight out
|
||||
* of `src/` (vitest, tsx), where `__dirname` is `src/ui-server/highlight/`.
|
||||
*/
|
||||
export function grammarDirCandidates(): string[] {
|
||||
const override = process.env[TEXTMATE_PATH_ENV]?.trim();
|
||||
const candidates = [
|
||||
path.join(__dirname, '..', '..', 'textmate'),
|
||||
path.join(__dirname, '..', '..', '..', 'dist', 'textmate'),
|
||||
];
|
||||
return override ? [path.resolve(override), ...candidates] : candidates;
|
||||
}
|
||||
|
||||
/**
|
||||
* The grammar directory and its manifest, or null when the build did not run.
|
||||
*
|
||||
* Null is a normal outcome, not an error: a source checkout that has only had
|
||||
* `tsc` run against it has no `dist/textmate`, and the right answer there is
|
||||
* plain text, not a 500 on a request for source.
|
||||
*/
|
||||
export function loadManifest(): { dir: string; manifest: GrammarManifest } | null {
|
||||
for (const dir of grammarDirCandidates()) {
|
||||
try {
|
||||
const raw = fs.readFileSync(path.join(dir, 'manifest.json'), 'utf-8');
|
||||
const manifest = JSON.parse(raw) as GrammarManifest;
|
||||
if (manifest && typeof manifest === 'object' && manifest.languages) return { dir, manifest };
|
||||
} catch {
|
||||
// Not here — try the next candidate.
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the grammar registrations one language needs, dependencies first.
|
||||
*
|
||||
* Shiki resolves a grammar's `embeddedLangs` against what is already in its
|
||||
* registry, so the order the manifest records matters: `vue` must arrive after
|
||||
* the `html`, `css` and `typescript` it embeds, or the embedded blocks come
|
||||
* back unhighlighted.
|
||||
*/
|
||||
export function readGrammarChain(dir: string, manifest: GrammarManifest, id: string): unknown[] {
|
||||
const files = manifest.languages[id];
|
||||
if (!files) return [];
|
||||
return files.map((file) => {
|
||||
const full = path.join(dir, `${file}.json`);
|
||||
return JSON.parse(fs.readFileSync(full, 'utf-8')) as unknown;
|
||||
});
|
||||
}
|
||||
Reference in New Issue
Block a user