hermes-agent/apps/desktop/scripts/profile-typing-lag.md

Profiling renderer typing lag

Workflow for empirically measuring (and fixing) typing/submit lag in the desktop chat composer.

Quick boot for profiling

Vite 8 + plugin-react 6 has a known issue where the React Fast Refresh preamble script isn't injected into index.html, so opening Electron at http://127.0.0.1:5174 throws $RefreshReg$ is not defined on every TSX module and the React tree never mounts. Workaround: run vite with HMR off.

# Terminal A — start dev server without HMR
cd apps/desktop
node scripts/dev-no-hmr.mjs

# Terminal B — start Electron with CDP exposed
cd apps/desktop
XCURSOR_SIZE=24 HERMES_DESKTOP_DEV_SERVER=http://127.0.0.1:5174 \
  ../../node_modules/.bin/electron --remote-debugging-port=9222 .

Terminal C is yours to run the harnesses.

Harnesses

All zero-dep — Node 24 built-in WebSocket + fetch.

Typing latency — measure-latency.mjs

Per-keystroke keypress → next paint latency, p50/p90/p99/max. Synthesizes keystrokes via Input.dispatchKeyEvent so the run is reproducible.

node apps/desktop/scripts/measure-latency.mjs --chars=120 --cps=20

Anything > 16ms is a dropped frame. On a freshly-loaded session (scripts/click-session.mjs 'Phaser particle') we currently see:

	unpatched	patched
p50 paint	1.9 ms	2.0 ms
p90 paint	3.3 ms	13.7 ms
p99 paint	16.7 ms	15.2 ms
max paint	20.5 ms	30.4 ms
>16ms drops	2/120	1/120

Roughly even on a quick session — patches don't fix typing latency under benign synthetic conditions because the existing baseline is already snappy on synthetic input. The real wins are in the leak counters (see below). If the user reports typing jank, capture a profile + heap diff during their actual usage and compare against the synthetic baseline to identify what condition (long thread, popover open, paste, etc.) makes the path slow.

Leak counters — leak-typing.mjs

Types N chars per round, clears, force-GCs, captures Performance.getMetrics deltas. Reveals leaked event listeners, heap drift, document node growth, and forced-layout counts.

# After clicking into a real session (e.g. via click-session.mjs):
node apps/desktop/scripts/leak-typing.mjs --rounds=8 --chars=200 --cps=50

Real-session numbers (Phaser thread, 8 rounds × 200 chars):

	unpatched (HEAD~2)	patched (HEAD)
jsListeners growth/round	+0	+0
DOM nodes growth/round	+0	+0
heap growth/round	~0 (V8 housekeeping)	~0
forced layouts/char	7.02	2.35 (3× fewer)

The forced-layout count is the load-bearing number — typing into a real session was triggering ~7 layouts per character on the unpatched build (scrollHeight reads + per-px CSS var writes + FadeText scrollWidth reads all stacking up). After the patches it's down to ~2.35/char, which is Blink's natural cost for a 1px/char-growing contentEditable and can't be lowered further without architectural changes.

The initial "+35 listeners/round leak" I called out on the first unpatched run turned out to be transient warm-up (popovers initializing, etc.); steady-state listener growth was 0 both before and after.

CPU profile + heap snapshot — profile-typing.mjs

Records a CPU profile while typing, plus before/after heap snapshots so you can do a comparison diff in Chrome DevTools Memory tab.

node apps/desktop/scripts/profile-typing.mjs \
  --chars=400 --cps=30 --out=/tmp/hermes-typing
# → /tmp/hermes-typing.cpuprofile  (open in Chrome DevTools Performance)
# → /tmp/hermes-typing.before.heapsnapshot
# → /tmp/hermes-typing.after.heapsnapshot

Loading the cpuprofile: Chrome DevTools → Performance tab → drag the file in, or VS Code → open the .cpuprofile directly.

For heap diff: Chrome DevTools → Memory → Load snapshot → load "before", then Comparison view → load "after". Sort by # Delta. Stay alert for detached DOM, FiberNodes (unmounted), and listener growth.

Helpers

• probe-renderer.mjs — dump page state (URL, composer mounted?, body text)
• click-session.mjs <title> — click a sidebar session by partial title match
• reload-renderer.mjs — force Page.reload via CDP (no HMR available)
• dump-state.mjs — richer state dump (thread message count, sticky session, etc.)
• probe-console.mjs — dump recent console errors / exceptions

Findings

See commit message for apps/desktop/src/app/chat/composer/index.tsx edits. Three changes:

1. Per-keystroke scrollHeight read removed. The expansion useEffect used to read editorRef.current.scrollHeight on every draft change (forces synchronous layout). Replaced with a draft.length > 60 heuristic; the ResizeObserver catches anything the heuristic misses.

2. Bucketed CSS custom-property writes. syncComposerMetrics used to setProperty('--composer-measured-height', height + 'px') on every observed resize, invalidating computed style for the whole tree. Now writes only when the height crosses an 8 px bucket, so typing in a fixed-height row produces no style invalidation at all.

3. Removed dead $composerDraft → aui.composer().setText round-trip. Nothing outside the composer subscribed to $composerDraft (verified via grep). The two useEffects that pushed draft → store and store → composer were pure overhead per keystroke. reconcileComposerTerminalSelections was also called per keystroke; can be deferred to submit time (it's a stale-pruning step, not a correctness one — terminalContextBlocksFromDraft walks the current text directly at submit and ignores stale labels).

4. refreshTrigger fast-bails when no @// in draft. Previously textBeforeCaret() did range.toString() (O(n)) on every keystroke even when no trigger char was present.

The biggest win is the listener leak in (3) — without it, each round of typing leaked ~35 event listeners until a steady state.

Submit / TTFT stall (open)

User reports a perceived stall after Enter, before the assistant starts streaming. scripts/measure-submit.mjs measures enter → composer-cleared → user-message-rendered → first-paint. The script triggers a real prompt submission, so use it on a throwaway session. Not enabled in CI.

Streaming "5fps" investigation (May 21, 2026)

User complaint: "the streaming must bring fps to like 5? lol" — felt hitches during assistant streaming on long threads.

Tooling added

• src/app/chat/perf-probe.tsx — dev-only side-effect import (guarded by import.meta.env.MODE !== 'production' in main.tsx). Attaches two helpers to window:
  • __PERF_PROBE__ — React <Profiler> recorder. Currently inert because Vite is serving the production React build (see "Vite dev-build issue" below); kept for when that's fixed.
  • __PERF_DRIVE__ — synthetic stream driver. Pushes tokens through the live $messages atom at a fixed cadence, so the assistant-ui runtime, incremental repository, Streamdown markdown renderer, and React commit pipeline all see the same workload they'd see from a real LLM stream — but with no LLM call (and no credit cost).
• scripts/measure-synthetic-stream.mjs — drives __PERF_DRIVE__, records rAF frame intervals, PerformanceObserver({entryTypes:['longtask']}) entries, MutationObserver cadence on the live message, and optional type-while-streaming keystroke latency.
• scripts/profile-synth-stream.mjs — CPU profile during a synthetic stream; writes a .cpuprofile (open in Chrome DevTools Performance panel) and a top-30 self-time table.
• scripts/measure-real-stream.mjs — same harness as the synthetic but fires a real LLM prompt. Use when you have credits and want to confirm the synthetic predictions hold.
• scripts/profile-real-stream.mjs — CPU profile over the duration of a real LLM stream.

Helpers: scripts/eval.mjs (one-shot CDP eval), scripts/reload.mjs (hard reload renderer over CDP).

Findings

Measured on the Cloud Shadows session (7 turns, ~11k px scrollHeight) and the 34 MB session session_20260514_215353_fe0ac8.json (110 FadeText instances, lots of historical tool calls).

metric	Cloud Shadows	34 MB session
avgFps (60 tok/sec, 5s)	60.0	58.6
frame p50 / p95 / p99 (ms)	16.7 / 18.0 / 21.1	16.6 / 25.6 / 31.4
max frame (ms)	31.1	97-127 (varies)
longtasks per 5s window	0	1-2, 75-127 ms
type-while-stream p95 latency (ms)	17	—

A single real-LLM stream on Cloud Shadows (gpt-4o-mini, 39s window) saw 12 longtasks totalling 1.26 s — same cadence the synthetic predicted (~1 hitch per 3.25 s, max 123 ms). So the synthetic stream is a faithful proxy for the real one and is fine for iterating on fixes without paying for tokens.

CPU profile during streaming (synthetic, markdown content)

Top self-time costs (5 s window, 400 tokens at 125 tok/s, markdown chunks):

ms (self)	function	source
260	bn$1	chunk-BO2N…js:20003 (micromark tokenize)
249	m$1	chunk-BO2N…js:19949 (micromark)
128	compile	chunk-BO2N…js:21884 (mdast → hast compile)
73	FadeText body	components/ui/fade-text.tsx
62	parser	chunk-BO2N…js:22680
49	fromThreadMessageLike	@assistant-ui/internal

That chunk-BO2N2NFS is the vendored bundle containing micromark, mdast-util-from-markdown, mdast-util-to-hast, rehype-raw, hast-util-sanitize, etc. — i.e. Streamdown's markdown pipeline, re-parsing the entire growing assistant message on every token append. Cost scales linearly with message length.

Compare plain-text (no markdown) — the chunk-BO2N… entries drop out of the top 30 entirely; total work per 5 s window halves.

Fix landed: FadeText memo

FadeText is used in tool-fallback.tsx (110 instances on a tool-heavy thread). Before: each parent re-render during streaming triggered a useEffect([children]) that forced a scrollWidth layout read — even when the title text was unchanged. The useResizeObserver already covers the genuine resize case, so the effect was strictly redundant.

After: wrapped in React.memo with a custom comparator that compares children (scalar fast-path), className, fadeWidth, and style field-by-field. Verified via temporary render counter: 122 renders during a 2 s synthetic stream vs ~11 000 without memo (110 instances × ~100 stream updates). Doesn't move the longtask needle on its own — Streamdown dwarfs it — but eliminates a class of forced layouts and removes a steady CPU floor.

Not fixed: Streamdown markdown re-parse

This is the dominant cost and the cause of the user's perceived hitches. The renderer re-parses the entire message buffer on every stream update. At ~3-5 k chars, each parse costs ~30 ms; when several pile into one frame the result is a 75-125 ms longtask = the "5 fps moment".

Possible approaches (none implemented here):

1. Coalesce/throttle Streamdown updates — render at most every 32 ms instead of every set-state. Reduces parses but doesn't reduce per-parse cost; trades latency for smoothness.
2. Memoize per-prefix — diff the new text against the prior parsed version; only re-parse the changed suffix.
3. Render in stable segments — close-form historical paragraphs as immutable React nodes; only the live tail goes through markdown each token. Probably the highest-impact change but requires forking or patching @assistant-ui/react-streamdown.
4. Move parsing to a Web Worker — main thread no longer blocks on markdown. Largest surgery; requires double-buffered hast.

Vite dev-build issue (separate)

http://127.0.0.1:5174/node_modules/.vite/deps/react.js resolves to react/cjs/react.production.js, and react-dom_client.js → react-dom-client.production.js. As a result:

• <React.Profiler> onRender is never called (production build is a no-op).
• import.meta.env.DEV is false, PROD is true even under vite dev (hence MODE !== 'production' as the workaround in main.tsx).
• All the React 19 dev-only warnings/devtools backend hooks are absent.

Root cause likely sits in vite.config.ts aliasing + dedupe + Vite 8's new optimizeDeps defaults. Worth a separate fix pass — when it's resolved, the <PerfProbe> blocks in perf-probe.tsx become useful (per-id commit timings) instead of inert.