feat(ui): highlight source server-side with a near-monochrome Shiki theme (CG-43)
The viewer's code block stops lexing with a hand-rolled dialect table and reads real TextMate grammars instead, run once in `/api/source`. Three things make that safe to depend on: * Highlighting never fails a request. A missing grammar, an oversized slice, an ESM import that did not resolve — every one of them answers `engine: 'plain'` with a reason and the source still goes out. * Identifiers survive whatever token boundaries a grammar chose. Every code token is split into identifier runs before it goes on the wire, so the graph's call-site overlay claims a token the highlighter produced rather than re-cutting the line. `assignRefs` now matches on a token's text rather than on the class a grammar gave it, so a language that scopes type names as `storage.type` still links. * The theme classifies rather than colours: its foregrounds are sentinels the server maps back to class names, and the viewer paints them from CSS custom properties — one token stream serves light and dark with no refetch, and the ramp lives only in app.css. Comments move from --ink-3 to a new --code-comment. --ink-3 measures 3.46:1 on paper and 3.00:1 on the hot-line tint, both under AA for 12.5px text; --code-comment is the smallest step along the same ramp that clears 4.5:1 on every background a code line can have, and stays quieter than the strings and numbers above it. Shipping: @shikijs/core and @shikijs/engine-javascript are runtime dependencies (no wasm, no native module); @shikijs/langs stays a devDependency and `npm run build:textmate` writes only the closure the engine's 40-odd languages reach — 56 grammars, 2.6 MB, against 11 MB for all 722. check-ui-build.mjs asserts the tree after every build and inside every release archive.
This commit is contained in:
Colby McHenry
@@ -30,6 +30,18 @@
|
||||
--amber: #8a5a0b;
|
||||
--amber-soft: #f3e9d2;
|
||||
|
||||
/* The one code colour that is not a plain re-use of the ink ramp.
|
||||
The spec asks for comments at --ink-3; measured against --paper that
|
||||
is 3.46:1 and against the hot-line tint --accent-soft it is 3.00:1,
|
||||
both under the 4.5:1 an AA reading of 12.5px body text needs. This is
|
||||
the smallest step DOWN the same warm-grey ramp that clears 4.5:1 on
|
||||
all three backgrounds a code line can have (paper 5.23, paper-2 4.92,
|
||||
accent-soft 4.53) while staying quieter than --ink-2, which strings
|
||||
and numbers use — so the recession order the spec describes is
|
||||
unchanged, only legible. Dark needed the mirror step UP (4.51 on
|
||||
accent-soft, where --ink-3 was 4.10). */
|
||||
--code-comment: #6a675d;
|
||||
|
||||
--sans: 'Archivo Variable', 'Archivo', -apple-system, BlinkMacSystemFont, 'Helvetica Neue', Arial, sans-serif;
|
||||
--mono: 'IBM Plex Mono', ui-monospace, 'SF Mono', Menlo, Consolas, monospace;
|
||||
--code-size: 12.5px;
|
||||
@@ -66,6 +78,7 @@
|
||||
--accent-line: #6b3a42;
|
||||
--amber: #d9a94a;
|
||||
--amber-soft: #2e2716;
|
||||
--code-comment: #8e8b81;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -87,6 +100,7 @@
|
||||
--accent-line: #6b3a42;
|
||||
--amber: #d9a94a;
|
||||
--amber-soft: #2e2716;
|
||||
--code-comment: #8e8b81;
|
||||
}
|
||||
|
||||
/* ---------- reset ---------- */
|
||||
|
||||
@@ -4,21 +4,25 @@
|
||||
|
||||
Two things make this more than a <pre>:
|
||||
|
||||
* The lexer state is threaded across lines AND across the gaps between
|
||||
windows, so the first line after a skipped block is not mis-read as the
|
||||
inside of a comment that closed 200 lines ago.
|
||||
* Syntax classification arrives already done, from `/api/source` — real
|
||||
TextMate grammars, run server-side, indexed by file line. The whole slice
|
||||
is tokenised in one pass there, so a window that starts 200 lines into a
|
||||
body still knows it is inside a block comment; nothing is re-lexed here.
|
||||
* Each ref is matched to an actual token rather than to a column, because the
|
||||
recorded column points at the start of the calling expression — see
|
||||
`assignRefs`.
|
||||
`assignRefs`. The overlay CLAIMS a token the highlighter produced; it never
|
||||
re-cuts one, which is what keeps the accent underline landing on the
|
||||
callee's own name whatever boundaries a grammar chose.
|
||||
-->
|
||||
<script lang="ts">
|
||||
import { newLexState, tokenClass, tokenize, type Token } from '../../lib/highlight';
|
||||
import { tokenClass, type Token } from '../../lib/highlight';
|
||||
import { assignRefs, type CodeBlock, type LineRef } from '../../lib/symbol-model';
|
||||
import { hot } from '../../lib/focus.svelte';
|
||||
|
||||
interface Props {
|
||||
block: CodeBlock;
|
||||
language: string;
|
||||
/** Classified source by 1-based file line — see `tokensByLine`. */
|
||||
tokens: Map<number, Token[]>;
|
||||
refs: Map<number, LineRef[]>;
|
||||
/** The line the definition's own name sits on — it is set in bold there. */
|
||||
defLine: number;
|
||||
@@ -28,7 +32,7 @@
|
||||
onfollow: (ref: LineRef) => void;
|
||||
}
|
||||
|
||||
let { block, language, refs, defLine, defName, highlight, onfollow }: Props = $props();
|
||||
let { block, tokens, refs, defLine, defName, highlight, onfollow }: Props = $props();
|
||||
|
||||
interface Part {
|
||||
text: string;
|
||||
@@ -52,33 +56,37 @@
|
||||
lines: RenderedLine[];
|
||||
}
|
||||
|
||||
let chunks = $derived.by<Chunk[]>(() => {
|
||||
const state = newLexState();
|
||||
return block.windows.map((window, windowIndex) => ({
|
||||
let chunks = $derived.by<Chunk[]>(() =>
|
||||
block.windows.map((window, windowIndex) => ({
|
||||
gapBefore: windowIndex === 0 ? 0 : (block.gapsAfter[windowIndex - 1] ?? 0),
|
||||
lines: window.lines.map((text, offset) => {
|
||||
const n = window.start + offset;
|
||||
const tokens = tokenize(text, state, language);
|
||||
const lineTokens = tokens.get(n) ?? [{ cls: 'other' as const, text, col: 0 }];
|
||||
const lineRefs = refs.get(n) ?? [];
|
||||
const claimed = assignRefs(tokens, lineRefs);
|
||||
const claimed = assignRefs(lineTokens, lineRefs);
|
||||
return {
|
||||
n,
|
||||
parts: toParts(tokens, claimed, n === defLine ? defName : null),
|
||||
parts: toParts(lineTokens, claimed, n === defLine ? defName : null),
|
||||
port: portFor(lineRefs),
|
||||
targets: [...new Set(lineRefs.map((r) => r.targetId).filter((id): id is string => !!id))],
|
||||
};
|
||||
}),
|
||||
}));
|
||||
});
|
||||
}))
|
||||
);
|
||||
|
||||
function toParts(tokens: Token[], claimed: Map<number, LineRef>, definition: string | null): Part[] {
|
||||
return tokens.map((token, index) => {
|
||||
function toParts(line: Token[], claimed: Map<number, LineRef>, definition: string | null): Part[] {
|
||||
return line.map((token, index) => {
|
||||
const ref = claimed.get(index) ?? null;
|
||||
return {
|
||||
text: token.text,
|
||||
cls: ref ? null : tokenClass(token.cls),
|
||||
ref,
|
||||
def: !ref && definition !== null && token.cls === 'ident' && token.text === definition,
|
||||
def:
|
||||
!ref &&
|
||||
definition !== null &&
|
||||
token.text === definition &&
|
||||
token.cls !== 'comment' &&
|
||||
token.cls !== 'string',
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -215,9 +223,11 @@
|
||||
font-size: 11px;
|
||||
}
|
||||
|
||||
/* ---- token classes (near-monochrome by design, spec §2.2) ---- */
|
||||
/* ---- token classes (near-monochrome by design, spec §2.2) ----
|
||||
Comments use --code-comment rather than --ink-3: the spec's colour reads
|
||||
at 3.46:1 on paper, under AA for 12.5px text. See app.css. */
|
||||
.t-c {
|
||||
color: var(--ink-3);
|
||||
color: var(--code-comment);
|
||||
}
|
||||
|
||||
.t-s {
|
||||
|
||||
@@ -11,6 +11,8 @@
|
||||
* carries the server's own sentence instead of "Failed to fetch".
|
||||
*/
|
||||
|
||||
import type { WireHighlight } from './highlight';
|
||||
|
||||
/* ---------------------------------------------------------------- shapes -- */
|
||||
|
||||
export type NodeKind = string;
|
||||
@@ -153,6 +155,13 @@ export interface WireSource {
|
||||
lines?: string[];
|
||||
truncated?: boolean;
|
||||
reason?: string;
|
||||
/**
|
||||
* The same lines, classified by the server's TextMate grammars — one entry
|
||||
* per line, each a list of `[classId, text]` pairs indexed into `classes`.
|
||||
* Absent whenever `lines` is, and `engine: 'plain'` whenever no grammar
|
||||
* covers the file. See `lib/highlight.ts`.
|
||||
*/
|
||||
highlight?: WireHighlight;
|
||||
}
|
||||
|
||||
export interface WireBlastScale {
|
||||
|
||||
+92
-299
@@ -1,326 +1,119 @@
|
||||
/**
|
||||
* Near-monochrome tokenising for the code block (design spec §2.2).
|
||||
* Turning the server's classified source into tokens the code block can draw.
|
||||
*
|
||||
* The colouring is deliberately almost absent: comments and strings recede,
|
||||
* keywords carry weight rather than hue, and the ONLY colour in the body is a
|
||||
* resolved call site. That is the point of the screen — the graph's edges are
|
||||
* what the eye should find, and a six-colour syntax theme buries them.
|
||||
* The classification itself happens on the server (`src/ui-server/highlight/`),
|
||||
* with real TextMate grammars via Shiki. What arrives is deliberately small:
|
||||
* one array per line, each entry a `[classId, text]` pair, with the class names
|
||||
* carried alongside so the payload is self-describing. This module does two
|
||||
* things to it and nothing else — resolve the class ids to names, and compute
|
||||
* each token's column, which is what the graph's call-site overlay matches
|
||||
* against.
|
||||
*
|
||||
* A hand-rolled lexer, not a highlighter library. It has one job — separate
|
||||
* comments, strings, numbers and keywords from everything else, well enough to
|
||||
* be honest across the 30-odd languages the engine indexes — and doing it here
|
||||
* keeps the viewer free of a runtime dependency and of a per-grammar download
|
||||
* on a machine that is reading its own source offline. CG-43 replaces this
|
||||
* with Shiki tokens produced server-side; `tokenize` is the seam.
|
||||
* ## Why the classes are names and not colours
|
||||
*
|
||||
* A theme that sent colours would have to send two of them, or force a refetch
|
||||
* every time `prefers-color-scheme` flipped. Class names let one token stream
|
||||
* serve light and dark and keep the design tokens in `app.css`, which is the
|
||||
* only place they should live. Design spec §2.2 — comments recede furthest,
|
||||
* strings and numbers one step in, keywords stay ink and gain weight, and the
|
||||
* only colour in the body is a call site the graph resolved.
|
||||
*
|
||||
* Nothing here re-tokenises. The overlay claims tokens the highlighter already
|
||||
* produced (`assignRefs` in `symbol-model.ts`), which is what makes the accent
|
||||
* underline land on the callee's own name whatever boundaries a grammar chose.
|
||||
*/
|
||||
|
||||
export type TokenClass =
|
||||
| 'comment'
|
||||
| 'string'
|
||||
| 'keyword'
|
||||
| 'number'
|
||||
| 'ident'
|
||||
| 'space'
|
||||
| 'punct';
|
||||
export type TokenClass = 'other' | 'ident' | 'comment' | 'string' | 'keyword' | 'number';
|
||||
|
||||
export interface Token {
|
||||
cls: TokenClass;
|
||||
text: string;
|
||||
/** Column of the token's first character, 0-based — how a ref finds its identifier. */
|
||||
/** Column of the token's first character, 0-based — how a ref finds it. */
|
||||
col: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lexer state that survives from one line to the next: a block comment or a
|
||||
* multi-line string opened on an earlier line. Rendering a window of a file
|
||||
* without this makes the first line after a `/*` look like code.
|
||||
*/
|
||||
export interface LexState {
|
||||
block: boolean;
|
||||
/** The delimiter that will close the open multi-line string (a backtick, `"""`, …). */
|
||||
stringEnd: string | null;
|
||||
/** `[classId, text]`, indexed into the payload's `classes` table. */
|
||||
export type WireToken = [number, string];
|
||||
|
||||
export interface WireHighlight {
|
||||
engine: 'shiki' | 'plain';
|
||||
grammar: string | null;
|
||||
classes: string[];
|
||||
lines: WireToken[][];
|
||||
/** Why the answer is unhighlighted, when it is. */
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
export function newLexState(): LexState {
|
||||
return { block: false, stringEnd: null };
|
||||
}
|
||||
|
||||
/* --------------------------------------------------------------- dialects -- */
|
||||
|
||||
interface Dialect {
|
||||
lineComment: string[];
|
||||
blockComment: [string, string] | null;
|
||||
/** Quote characters that never span lines. */
|
||||
quotes: string[];
|
||||
/** Delimiters that MAY span lines (template literals, triple quotes, heredoc-ish). */
|
||||
multiline: string[];
|
||||
keywords: ReadonlySet<string>;
|
||||
}
|
||||
|
||||
const kw = (words: string): ReadonlySet<string> => new Set(words.split(/\s+/).filter(Boolean));
|
||||
const CLASS_NAMES: ReadonlySet<string> = new Set<TokenClass>([
|
||||
'other',
|
||||
'ident',
|
||||
'comment',
|
||||
'string',
|
||||
'keyword',
|
||||
'number',
|
||||
]);
|
||||
|
||||
/**
|
||||
* Keywords shared widely enough across the C-family that listing them once is
|
||||
* both shorter and more accurate than a per-language table nobody maintains.
|
||||
*/
|
||||
const C_FAMILY = `
|
||||
abstract as async await break case catch class const constexpr continue default defer delete do
|
||||
else enum export extends extern false final finally for from func function go goto if impl implements
|
||||
import in instanceof interface internal is let match mod module mut namespace new nil null object
|
||||
operator out override package private protected public readonly record ref return sealed select self
|
||||
static struct super switch this throw throws trait true try type typedef typeof union unsafe use using
|
||||
var virtual void when where while with yield
|
||||
`;
|
||||
|
||||
const DIALECTS: Record<string, Dialect> = {
|
||||
c: {
|
||||
lineComment: ['//'],
|
||||
blockComment: ['/*', '*/'],
|
||||
quotes: ['"', "'"],
|
||||
multiline: [],
|
||||
keywords: kw(C_FAMILY),
|
||||
},
|
||||
ts: {
|
||||
lineComment: ['//'],
|
||||
blockComment: ['/*', '*/'],
|
||||
quotes: ['"', "'"],
|
||||
multiline: ['`'],
|
||||
keywords: kw(
|
||||
`${C_FAMILY} any asserts bigint boolean declare infer keyof never number readonly satisfies
|
||||
string symbol undefined unknown`
|
||||
),
|
||||
},
|
||||
hash: {
|
||||
// Python, Ruby, shell, YAML, Nix, Terraform, Perl, R, Elixir…
|
||||
lineComment: ['#'],
|
||||
blockComment: null,
|
||||
quotes: ['"', "'"],
|
||||
multiline: ['"""', "'''"],
|
||||
keywords: kw(
|
||||
`and as assert async await begin break case class def defp defmodule del do elif else elsif end
|
||||
ensure except exec finally for from global if import in is lambda let module next nil none not
|
||||
or pass raise require rescue return self struct then trait true false try unless until use when
|
||||
while with yield`
|
||||
),
|
||||
},
|
||||
sql: {
|
||||
lineComment: ['--'],
|
||||
blockComment: ['/*', '*/'],
|
||||
quotes: ["'", '"'],
|
||||
multiline: [],
|
||||
keywords: kw(
|
||||
`select insert update delete from where group by order having join left right inner outer on as
|
||||
and or not null create table index view primary key foreign references into values set limit`
|
||||
),
|
||||
},
|
||||
lisp: {
|
||||
lineComment: [';'],
|
||||
blockComment: null,
|
||||
quotes: ['"'],
|
||||
multiline: [],
|
||||
keywords: kw('def defn defmacro let fn if cond do loop recur ns require import when case'),
|
||||
},
|
||||
};
|
||||
|
||||
/** Engine `Language` values → the lexer that reads them closely enough. */
|
||||
const LANGUAGE_DIALECT: Record<string, keyof typeof DIALECTS> = {
|
||||
typescript: 'ts',
|
||||
tsx: 'ts',
|
||||
javascript: 'ts',
|
||||
jsx: 'ts',
|
||||
svelte: 'ts',
|
||||
vue: 'ts',
|
||||
astro: 'ts',
|
||||
dart: 'c',
|
||||
java: 'c',
|
||||
kotlin: 'c',
|
||||
scala: 'c',
|
||||
csharp: 'c',
|
||||
vbnet: 'hash',
|
||||
go: 'c',
|
||||
rust: 'c',
|
||||
swift: 'c',
|
||||
objc: 'c',
|
||||
c: 'c',
|
||||
cpp: 'c',
|
||||
cuda: 'c',
|
||||
metal: 'c',
|
||||
php: 'c',
|
||||
zig: 'c',
|
||||
solidity: 'c',
|
||||
glsl: 'c',
|
||||
python: 'hash',
|
||||
ruby: 'hash',
|
||||
crystal: 'hash',
|
||||
elixir: 'hash',
|
||||
perl: 'hash',
|
||||
r: 'hash',
|
||||
shell: 'hash',
|
||||
bash: 'hash',
|
||||
powershell: 'hash',
|
||||
yaml: 'hash',
|
||||
toml: 'hash',
|
||||
nix: 'hash',
|
||||
terraform: 'hash',
|
||||
hcl: 'hash',
|
||||
dockerfile: 'hash',
|
||||
makefile: 'hash',
|
||||
sql: 'sql',
|
||||
clojure: 'lisp',
|
||||
lisp: 'lisp',
|
||||
scheme: 'lisp',
|
||||
elm: 'ts',
|
||||
haskell: 'ts',
|
||||
lua: 'hash',
|
||||
erlang: 'hash',
|
||||
cobol: 'hash',
|
||||
};
|
||||
|
||||
function dialectFor(language: string | undefined): Dialect {
|
||||
const key = LANGUAGE_DIALECT[(language ?? '').toLowerCase()] ?? 'ts';
|
||||
return DIALECTS[key] as Dialect;
|
||||
}
|
||||
|
||||
/* ----------------------------------------------------------------- lexer -- */
|
||||
|
||||
const IDENT_START = /[A-Za-z_$@]/;
|
||||
const IDENT_BODY = /[\w$]/;
|
||||
|
||||
/**
|
||||
* Split one line into tokens, carrying `state` across lines.
|
||||
* Decode one line's tokens, filling in columns.
|
||||
*
|
||||
* Mutates `state` — a window of source is tokenised line by line in order, and
|
||||
* threading the block-comment flag through a return value would make every
|
||||
* caller responsible for a detail only this function understands.
|
||||
* Columns are derived rather than sent: they are the running sum of the token
|
||||
* texts, so putting them on the wire would be duplicating a fact the payload
|
||||
* already determines — and a wire column that disagreed with the text would be
|
||||
* a silent mis-underline rather than a visible error.
|
||||
*/
|
||||
export function tokenize(line: string, state: LexState, language?: string): Token[] {
|
||||
const d = dialectFor(language);
|
||||
export function decodeLine(wire: readonly WireToken[], classes: readonly string[]): Token[] {
|
||||
const out: Token[] = [];
|
||||
const len = line.length;
|
||||
let i = 0;
|
||||
|
||||
const push = (cls: TokenClass, from: number, to: number): void => {
|
||||
if (to > from) out.push({ cls, text: line.slice(from, to), col: from });
|
||||
};
|
||||
|
||||
while (i < len) {
|
||||
// --- continuations of something opened on an earlier line ---------------
|
||||
if (state.block && d.blockComment) {
|
||||
const close = line.indexOf(d.blockComment[1], i);
|
||||
if (close < 0) {
|
||||
push('comment', i, len);
|
||||
i = len;
|
||||
} else {
|
||||
push('comment', i, close + d.blockComment[1].length);
|
||||
i = close + d.blockComment[1].length;
|
||||
state.block = false;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (state.stringEnd) {
|
||||
const end = findUnescaped(line, state.stringEnd, i);
|
||||
if (end < 0) {
|
||||
push('string', i, len);
|
||||
i = len;
|
||||
} else {
|
||||
push('string', i, end + state.stringEnd.length);
|
||||
i = end + state.stringEnd.length;
|
||||
state.stringEnd = null;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
const rest = line.slice(i);
|
||||
|
||||
// --- comments -----------------------------------------------------------
|
||||
const lineMarker = d.lineComment.find((m) => rest.startsWith(m));
|
||||
if (lineMarker) {
|
||||
push('comment', i, len);
|
||||
i = len;
|
||||
continue;
|
||||
}
|
||||
if (d.blockComment && rest.startsWith(d.blockComment[0])) {
|
||||
const close = line.indexOf(d.blockComment[1], i + d.blockComment[0].length);
|
||||
if (close < 0) {
|
||||
push('comment', i, len);
|
||||
i = len;
|
||||
state.block = true;
|
||||
} else {
|
||||
push('comment', i, close + d.blockComment[1].length);
|
||||
i = close + d.blockComment[1].length;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// --- strings ------------------------------------------------------------
|
||||
// Longest delimiter first, so `"""` never matches as `"`.
|
||||
const multi = [...d.multiline].sort((a, b) => b.length - a.length).find((m) => rest.startsWith(m));
|
||||
if (multi) {
|
||||
const end = findUnescaped(line, multi, i + multi.length);
|
||||
if (end < 0) {
|
||||
push('string', i, len);
|
||||
i = len;
|
||||
state.stringEnd = multi;
|
||||
} else {
|
||||
push('string', i, end + multi.length);
|
||||
i = end + multi.length;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
const quote = d.quotes.find((q) => rest.startsWith(q));
|
||||
if (quote) {
|
||||
const end = findUnescaped(line, quote, i + quote.length);
|
||||
// An unterminated single-line quote is an apostrophe in prose far more
|
||||
// often than a real string, so it stops at the line rather than eating
|
||||
// the rest of the window.
|
||||
push('string', i, end < 0 ? len : end + quote.length);
|
||||
i = end < 0 ? len : end + quote.length;
|
||||
continue;
|
||||
}
|
||||
|
||||
// --- words, numbers, space, everything else -----------------------------
|
||||
const ch = line[i] as string;
|
||||
if (IDENT_START.test(ch)) {
|
||||
let j = i + 1;
|
||||
while (j < len && IDENT_BODY.test(line[j] as string)) j++;
|
||||
const word = line.slice(i, j);
|
||||
push(d.keywords.has(word) ? 'keyword' : 'ident', i, j);
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
if (ch >= '0' && ch <= '9') {
|
||||
let j = i + 1;
|
||||
while (j < len && /[\w.]/.test(line[j] as string)) j++;
|
||||
push('number', i, j);
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
if (/\s/.test(ch)) {
|
||||
let j = i + 1;
|
||||
while (j < len && /\s/.test(line[j] as string)) j++;
|
||||
push('space', i, j);
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
push('punct', i, i + 1);
|
||||
i++;
|
||||
let col = 0;
|
||||
for (const [id, text] of wire) {
|
||||
const name = classes[id];
|
||||
out.push({ cls: CLASS_NAMES.has(name as string) ? (name as TokenClass) : 'other', text, col });
|
||||
col += text.length;
|
||||
}
|
||||
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Index of `needle` at or after `from`, skipping backslash-escaped ones. */
|
||||
function findUnescaped(line: string, needle: string, from: number): number {
|
||||
let i = from;
|
||||
while (i < line.length) {
|
||||
if (line[i] === '\\') {
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
if (line.startsWith(needle, i)) return i;
|
||||
i++;
|
||||
/**
|
||||
* Split a line into identifier runs with no syntax classification at all.
|
||||
*
|
||||
* The fallback for the moment before a payload arrives, and for a payload that
|
||||
* carries no `highlight` block (an index served by an older build). It keeps
|
||||
* the call-site overlay working — the links come from the graph, never from the
|
||||
* grammar — so a line rendered this way loses only its colouring.
|
||||
*/
|
||||
export function plainLine(text: string): Token[] {
|
||||
const out: Token[] = [];
|
||||
const ident = /[A-Za-z_$À-][\w$À-]*/g;
|
||||
let at = 0;
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = ident.exec(text)) !== null) {
|
||||
if (match.index > at) out.push({ cls: 'other', text: text.slice(at, match.index), col: at });
|
||||
out.push({ cls: 'ident', text: match[0], col: match.index });
|
||||
at = match.index + match[0].length;
|
||||
}
|
||||
return -1;
|
||||
if (at < text.length) out.push({ cls: 'other', text: text.slice(at), col: at });
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* The tokens for a slice, by 1-based file line.
|
||||
*
|
||||
* `from` is the slice's first line, so the map is keyed the way every other
|
||||
* part of the Symbol view counts: real file lines, never offsets into a window.
|
||||
*/
|
||||
export function tokensByLine(
|
||||
lines: readonly string[],
|
||||
from: number,
|
||||
highlight: WireHighlight | undefined
|
||||
): Map<number, Token[]> {
|
||||
const byLine = new Map<number, Token[]>();
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const wire = highlight?.lines[i];
|
||||
byLine.set(
|
||||
from + i,
|
||||
wire ? decodeLine(wire, highlight.classes) : plainLine(lines[i] as string)
|
||||
);
|
||||
}
|
||||
return byLine;
|
||||
}
|
||||
|
||||
/** The CSS class for a token, or null where the default ink is right. */
|
||||
|
||||
@@ -286,6 +286,13 @@ function outsideRefs(outside: WireOutsideIndex): Array<{ line: number; ref: Line
|
||||
* unclaimed one, then the last — is what makes `this.mutex.withLock(…)` mark
|
||||
* `withLock` instead of `this`.
|
||||
*
|
||||
* A candidate is any token whose TEXT is the identifier and that is not inside
|
||||
* a comment or a string. Deliberately not "any token the highlighter called an
|
||||
* identifier": grammars disagree about that constantly — Go scopes `string` as
|
||||
* `storage.type`, Java scopes a declared type name the same way — and a link
|
||||
* that vanished because a grammar had an opinion about a scope name would be a
|
||||
* highlighting change silently breaking navigation.
|
||||
*
|
||||
* @returns token index → the ref that claimed it
|
||||
*/
|
||||
export function assignRefs(
|
||||
@@ -296,7 +303,8 @@ export function assignRefs(
|
||||
for (const ref of refs) {
|
||||
const candidates: number[] = [];
|
||||
tokens.forEach((token, index) => {
|
||||
if (token.cls === 'ident' && token.text === ref.ident) candidates.push(index);
|
||||
if (token.cls === 'comment' || token.cls === 'string') return;
|
||||
if (token.text === ref.ident) candidates.push(index);
|
||||
});
|
||||
if (candidates.length === 0) continue;
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
import SourceBlock from '../components/symbol/SourceBlock.svelte';
|
||||
import SymbolHeader from '../components/symbol/SymbolHeader.svelte';
|
||||
import { ApiFailure, fetchSource, fetchSymbol, type WireNodeRef, type WireSource, type WireSymbolPayload } from '../lib/api';
|
||||
import { tokensByLine, type Token } from '../lib/highlight';
|
||||
import { hot, railFocus } from '../lib/focus.svelte';
|
||||
import { project } from '../lib/project.svelte';
|
||||
import {
|
||||
@@ -148,6 +149,18 @@
|
||||
return buildCodeBlock(from, source.lines, graphCallLines(payload));
|
||||
});
|
||||
|
||||
/**
|
||||
* Classified source by file line, from `/api/source`.
|
||||
*
|
||||
* Keyed by real file line rather than by window offset, because a windowed
|
||||
* body renumbers nothing: the gaps are holes in the same numbering, and the
|
||||
* code block looks a line up by the number it prints in the gutter.
|
||||
*/
|
||||
let codeTokens = $derived.by(() => {
|
||||
if (!source?.lines) return new Map<number, Token[]>();
|
||||
return tokensByLine(source.lines, source.from ?? 1, source.highlight);
|
||||
});
|
||||
|
||||
let origin = $derived(arrivedFrom());
|
||||
let originLeft = $derived(origin?.rail === 'left' ? origin.id : null);
|
||||
let originRight = $derived(origin?.rail === 'right' ? origin.id : null);
|
||||
@@ -432,7 +445,7 @@
|
||||
{:else if codeBlock}
|
||||
<SourceBlock
|
||||
block={codeBlock}
|
||||
language={payload.node.language}
|
||||
tokens={codeTokens}
|
||||
{refs}
|
||||
defLine={payload.node.line}
|
||||
defName={payload.node.name}
|
||||
|
||||
Reference in New Issue
Block a user