Part 3 — renderer landscape and canonical AO terminal direction

Date: 2026年08月17日

This is the canonical continuation checkpoint for discussion #3991. It builds on the original proposal, WebGL addendum, and terminal-authority correction. It independently compares open-source terminal and agent-orchestration projects with AO and focuses on four boundaries that are often conflated: PTY/process ownership, authoritative terminal state, transport/replay, and final painting.

Executive answer

Verified: xterm.js integration is a real source of bugs and complexity in AO and in mature peer applications. AO, VS Code, Wave, and Tabby all carry renderer fallback, WebGL context-loss recovery, resize/fit coordination, and host-specific input workarounds. AO and Tabby also reach into xterm private internals. These are not hypothetical costs.

Correction: xterm.js is not the source of most terminal-system failure modes, and replacing it with a native embedded renderer would not eliminate PTY ownership, process lifetime, reconnect ordering, backpressure, resize authority, or terminal-query routing problems. A native renderer replaces part of the VT/input/rendering stack and introduces native-view, GPU, FFI, packaging, signing, accessibility, and per-platform integration work.

Recommendation for AO: do not make a big-bang xterm-to-native rewrite. First separate a durable terminal authority from interchangeable terminal painters:

agent process <-> persistent PTY owner
 |
 +-> canonical VT state and sole query responder
 +-> atomic snapshot at sequence N + live events after N
 |
 terminal-view contract
 / | \
 xterm.js painter Ghostty web native experiment
 (baseline) DOM/canvas (macOS, later)

The first renderer bake-off should stay inside AO's existing Electron/web boundary: compare a current xterm baseline with wterm plus its Ghostty core and with ghostty-web. That isolates VT/painter behavior from native-overlay plumbing and preserves a path to Windows, Linux, and mobile. Test a full native painter later against the same recorded byte corpus, authority, input contract, and lifecycle stress suite. A native renderer should replace xterm only where it demonstrates measured wins large enough to justify platform-specific ownership. A separate semantic overview should serve agent monitoring; an exact interactive terminal should remain available for arbitrary TUIs.

What "embedded terminal" can mean

The phrase covers three materially different products:

1. A complete embedded widget/painter. xterm.js provides browser-side VT parsing, buffer state, selection/input behavior, accessibility hooks, and WebGL/canvas/DOM painting. cmux embeds a full Ghostty surface in an AppKit NSView; SwiftTerm provides AppKit/UIKit terminal views.
2. A headless parser and state engine. @xterm/headless, libghostty-vt, and libvterm maintain a grid, modes, cursor, and protocol state but do not by themselves give an Electron application a complete native UI surface. The host must still render cells and implement input, selection, IME, clipboard, accessibility, links, and graphics.
3. An externally painted terminal. Zellij, Herdr, Agent Deck, and NTM ultimately write ANSI to the user's existing terminal emulator. They can avoid shipping an embedded GPU painter because their product UI itself runs inside a terminal. AO cannot get an in-app desktop terminal from this approach without either launching an external terminal or adding a painter at the desktop boundary.

This distinction is the key to interpreting the projects below.

Revisions and method

All claims below are based on source or first-party documentation at these exact revisions. "Verified" means directly supported by those sources. "Inference for AO" is architectural analysis, not a claim made by the upstream project.

Architecture matrix

AO baseline: renderer and authority are already separate

Verified

On Unix, AO starts a fresh tmux -u -T RGB attach-session inside a real PTY for every viewer; tmux owns the durable pane and screen state, while the attach process emits a repaint (tmux attach, attach PTY). The terminal layer intentionally does not replay state because the runtime is expected to generate a full repaint for each attachment (attachment contract).

On Windows, a detached host owns ConPTY, keeps a bounded output ring, broadcasts live bytes, and atomically attaches a client to a ring snapshot before live fan-out (host ownership and sizing, snapshot boundary). This is durable process/transport ownership, but a suffix of raw output is not the same as a canonical screen and mode snapshot.

The desktop sends JSON/base64 output frames over a shared WebSocket mux (wire and socket pool, pooled connection). xterm then parses and paints those bytes.

AO's xterm component documents and implements concrete integration work:

• WebGL is preferred, canvas is the fallback, and renderer failure/context loss is handled explicitly (renderer selection).
• Font metrics, zero-size mounts, fit/resize ordering, hidden scrollbars, viewport selection, and disposal all need coordination; some workarounds read private _core state (private metric access, selection/scrollbar workarounds, fit and resize path).
• AO forwards physical key events rather than all onData output because xterm may generate terminal-query responses while replaying an attach repaint; forwarding those to the real PTY can corrupt the TUI (input boundary).

Inference for AO

The last point is not merely a rendering bug. It is an ownership ambiguity: which emulator is authorized to answer DA/DSR and related terminal queries, and whether a reply came from current output or historical repaint. A native painter would still need that contract. Swapping only xterm leaves tmux/ConPTY lifetime, raw-stream replay, backpressure, and resize arbitration unchanged.

AO is also testing an old xterm baseline

Verified: AO pins @xterm/xterm 5.5.0, canvas 0.7.0, and WebGL 0.19.0 (package manifest). On 2026年08月17日, the npm registry's stable xterm tag is 6.0.0 and its beta tag is 6.1.0-beta.302 (registry). xterm 6.0 added DEC synchronized-output mode 2026 (implementation commit), which is specifically valuable for flicker-free full-screen TUI updates. Further viewport synchronization and concrete WebGL corruption/atlas fixes landed after the 6.0 tag (viewport follow-up, atlas-merge corruption, atlas overflow, stale shared atlas).

Inference for AO: the fair baseline is not today's 5.5 stack. Before concluding that xterm must go, AO should run the same corpus against stable 6.0 and a pinned 6.1 beta, with a non-WebGL fallback. That upgrade can plausibly fix flicker and corruption cases; it cannot fix AO's transport, PTY, replay, or resize-authority defects. A beta should be shipped only after AO's own trace and lifecycle gates, not because VS Code happens to track beta releases.

Full native embedding: cmux and SwiftTerm

cmux

Verified: cmux is a macOS-only Swift/AppKit application. Its README says it is not Electron and uses libghostty for GPU rendering (implementation summary, scope, platform). It creates a Ghostty surface by passing an AppKit NSView in ghostty_surface_config, and its terminal object explicitly owns the native surface lifecycle (surface creation, surface lifecycle).

The most relevant capability for AO is Ghostty manual I/O. In that mode Ghostty spawns no child process; input is delivered to a host callback, and externally owned output is injected with ghostty_surface_process_output (manual-I/O configuration). cmux also explicitly records whether the surface or a manual transport owns the process, PTY, and terminal protocol (I/O ownership). This is evidence that a native Ghostty painter can be separated from PTY ownership.

cmux does not prove transparent persistence of arbitrary child processes. Its restore contract saves layouts, directories, and best-effort scrollback but says arbitrary live process state is not checkpointed (restore contract).

Inference for AO: cmux is the strongest evidence that a libghostty painter could improve a native macOS terminal surface. It is not evidence of a drop-in cross-platform Electron replacement. Hosting an AppKit/Metal surface beside Chromium requires native view and lifecycle integration; Linux and Windows would need another complete surface or a separate AO renderer. cmux's source itself contains substantial native lifecycle, focus, input, resize, and renderer-recovery code, so the complexity moves rather than disappears.

electron-libghostty

Verified: electron-libghostty is the closest direct experiment to AO's desktop host. It creates a Ghostty CAMetalLayer surface in an AppKit NSView, places that native view above Electron's Chromium content, and synchronizes its rectangle, focus, occlusion, and lifecycle through the native module. Its documented integration measures a DOM element with getBoundingClientRect, multiplies by device scale, and sends every resize/scroll change over IPC (preview status and platform limits, DOM-to-overlay synchronization, surface and Metal layer). The project labels itself preview, macOS-only, and plans to move to Electron shared textures.

Inference for AO: this validates feasibility but also exposes the cost hidden by the phrase "embed Ghostty": DOM/native z-order, clipping, scroll synchronization, focus, DPI, IPC, N-API ABI, signing, and crash ownership all become product code. It is a useful later macOS prototype, not the cleanest first test of whether xterm's painter is AO's problem.

Ghostty API scope

Verified: Ghostty describes libghostty-vt as the first cross-platform library layer: it parses sequences and maintains state on macOS, Linux, Windows, and WebAssembly, but its API signatures remain in flux (libghostty roadmap and current status). Separately, Ghostty's full C embedding entry point says it is currently not a general-purpose embedding API and is used to embed Ghostty within a macOS app (full embedding limitation).

Inference for AO: "use libghostty" currently presents two different risk profiles:

• use the complete macOS surface and accept a platform-specific native painter;
• use cross-platform libghostty-vt, then build and maintain AO's own painter and input/accessibility layer.

The latter can replace terminal-state parsing but does not replace xterm as a complete UI widget.

SwiftTerm

Verified: SwiftTerm offers a headless engine plus AppKit/UIKit terminal views and a convenience local-process view; its macOS PTY helper uses forkpty and resizes with ioctl (library surfaces, frontends, PTY implementation). The project advertises CoreText/Metal rendering and notes incomplete accessibility in its feature list (features).

Inference for AO: SwiftTerm is another credible Apple-native prototype candidate. AO should feed it externally owned terminal data instead of using a convenience view that owns the process; otherwise a view's lifetime would become the agent's lifetime. It does not solve AO's Windows/Linux painter requirement.

Ghostty-backed web painters: wterm, ghostty-web, and Restty

These projects matter more to AO's first renderer experiment than a native surface because they keep Electron, browser/mobile delivery, and the deployment boundary constant.

wterm

Verified: wterm separates a pluggable terminal core from a DOM painter. It offers a lightweight Zig/WASM core or an opt-in libghostty-vt core, renders only dirty rows on animation frames, implements synchronized-output mode 2026, and uses ordinary DOM text for selection, browser find, and screen-reader access (packages and features).

It is not renderer-bug-free. Its DOM painter manually handles wide cells and block glyphs; one source-level workaround pixel-snaps CSS gradients specifically because Claude Code's horizontal rule exposed inconsistent browser rounding (block-glyph workaround).

Inference for AO: wterm plus @wterm/ghostty is the strongest first alternative to benchmark. It removes WebGL contexts and xterm's JS parser while retaining AO's web/Electron/mobile shape, and it improves native selection/accessibility. The tradeoff is likely lower peak throughput and a different set of DOM layout/glyph bugs; neither claim should be accepted without AO's sustained-output and 20–50-session tests.

ghostty-web

Verified: ghostty-web exposes an xterm-compatible API around a Ghostty WASM state engine and a custom Canvas2D renderer, explicitly advertising migration by swapping the import (scope and API, usage). It implements its own selection, scrolling, link detection, input, dirty-row painting, and disposal above the VT core.

The repository contains a regression test for stale cells becoming visible after scroll growth under reset-heavy output (viewport-corruption regression). That is useful counter-evidence to "Ghostty core means rendering bugs disappear": the core can be correct while the viewport/cache/painter around it is not.

Inference for AO: this is the lowest-friction parser-and-painter swap because its API resembles xterm. It avoids WebGL context and glyph-atlas failure classes, but Canvas2D still owns DPI, font metrics, selection hit testing, and redraw scheduling. It is a good bake-off candidate, not yet evidence for a production migration.

Restty

Verified: Restty combines libghostty-vt in WASM with TypeScript shaping and WebGPU, falling back to WebGL2. It exposes both an xterm-style wrapper and a DOM-free headless entry point, while describing itself as early-release software with APIs that may change (renderer and maturity, entry points).

Inference for AO: Restty is the most ambitious web-native candidate, but it rebuilds the same difficult GPU resource surface—glyph atlases, context/device lifecycle, shaders, fallbacks—that makes xterm WebGL costly. It may ultimately outperform both xterm and DOM painting, but its present maturity and GPU complexity make it a later benchmark rather than AO's first replacement candidate.

Browser-widget lineage: VS Code, Wave, and Tabby

VS Code

Verified: VS Code uses both node-pty and xterm packages (dependencies). PTYs live in a separate pty-host service with heartbeat/restart and attach/input/resize APIs (host lifecycle, process operations).

For persistent terminals, the host feeds output into @xterm/headless and @xterm/addon-serialize; a later client gets ANSI serialized from that headless state before live output (persistent process and serializer, serializer implementation). Its code also handles ConPTY DSR/CPR hangs when no renderer is attached, showing that terminal replies and attachment state remain explicit concerns (ConPTY query handling).

The client creates xterm and dynamically falls back from WebGL to DOM on context loss; it notes WebGL and DOM renderers can calculate different cell dimensions and therefore requests another resize (client construction, renderer fallback).

Inference for AO: VS Code is strong precedent for a durable authority plus disposable xterm painter. It also demonstrates that introducing a headless emulator does not remove mode/query/replay complexity. Using the same emulator family for headless snapshots and client rendering reduces semantic drift, but it couples authority behavior to xterm's parser and serializer coverage.

Wave Terminal

Verified: Wave's Go shell controller owns the PTY, continuously appends raw bytes to a circular terminal file, and separately handles input/resize (circular output file, PTY pump). The browser owns xterm, its serializer, and WebGL/DOM switching with context-loss fallback (xterm setup, renderer fallback).

For restore, the frontend loads a previously serialized xterm cache at its recorded byte offset, then consumes the remaining raw output tail. During idle periods it serializes current xterm state back to storage (cache plus tail, idle serialization).

Inference for AO: this is a practical cache design, not a single server-side terminal authority. If no renderer has consumed recent output, server-side semantic state and terminal replies are not inherently current. It illustrates how split ownership can work for an interactive desktop, but it is weaker for multi-viewer, remote, or crash-consistent AO attachments.

Tabby

Verified: Tabby's Electron main process wraps node-pty and uses an explicit acknowledgement queue to pause and resume PTY flow under renderer backpressure (PTY and acknowledgement queue, PTY ownership and IPC operations). A renderer proxy can reconnect to a surviving PTY by ID (renderer proxy).

Its xterm frontend implements flow control, WebGL recovery, input/resize, and private _core workarounds for scroll pinning, blank resize frames, and texture-upload flicker (frontend flow control and setup, renderer workarounds, WebGL recovery).

Inference for AO: Tabby directly validates the user's concern: browser-terminal integration can require brittle internals. It also validates a separate point: backpressure and PTY lifetime live outside the renderer and remain necessary after a renderer swap.

Server-owned semantic state: Zellij and Herdr

Zellij

Verified: a Zellij terminal pane owns a server-side grid/VTE parser and feeds every PTY byte into it. It pauses or intercepts some host queries and tracks pending input (pane state, PTY-byte handling). Rendering produces semantic character chunks from the grid; the output layer builds per-client changes and encodes them to ANSI (pane rendering, per-client output, ANSI serialization). Its server-to-client render payload is an ANSI string, which the client writes to stdout (IPC message, client output).

Herdr

Verified: Herdr owns its PTYs, feeds output into vendored libghostty-vt, composes terminal cells and application chrome into a Ratatui buffer, and transports either semantic cells or server-diffed ANSI to thin clients. Its external terminal performs final glyph rasterization (server workspace, semantic client renderer, ANSI client renderer).

Inference for AO

These designs move reconnect, inspection, and slow-client policy onto authoritative visual state. A later frame can safely supersede an unsent earlier frame because it represents a complete current result; arbitrary raw VT chunks cannot be dropped this way.

They do not show that AO can eliminate all painters. Zellij and Herdr can rely on an outer terminal because their primary clients are terminal applications. For AO, one of these must still be true:

• translate the server grid into ANSI and feed xterm/native terminal again;
• define a structured cell-diff protocol and build an AO cell painter;
• open the session in an external terminal and accept a split desktop experience.

A custom cell painter is a major product in its own right: Unicode width and shaping, bidi policy, cursor and selection, IME, keyboard protocols, hyperlinks, clipboard, accessibility, images, ligatures, font fallback, and GPU lifetime all become AO's responsibility.

Native semantic models: WezTerm and Arbor

WezTerm

Verified: WezTerm's native mux does not treat a renderer as the sole copy of terminal state. Its Pane interface exposes the current sequence number, the rows changed since an earlier sequence, stable row indices, cursor state, logical lines, and semantic zones (pane state contract). The GUI consumes this semantic state and paints it with WezTerm's native frontend.

Inference for AO: WezTerm is the strongest architectural reference for a state-authoritative mux with incremental painting. It is not a drop-in embeddable AO widget, but its sequence/change seam is closer to the correct long-term contract than forwarding an unbounded historical byte suffix to every painter.

Arbor

Verified: Arbor, another agent/worktree desktop, keeps daemon-backed terminal sessions, uses an Alacritty-derived emulator by default, offers experimental libghostty-vt, and gives its native GPUI view a semantic TerminalSnapshot. The inspected snapshot includes styled cell text, cursor, and a deliberately small mode set (app_cursor and alt_screen) (snapshot schema and engines, product behavior).

Inference for AO: Arbor demonstrates that a custom native cell view is viable when the product defines the supported terminal semantics. Its narrow snapshot is also a warning: a screen suitable for common agent output is not automatically full fidelity for arbitrary TUIs, graphics protocols, query replies, and accessibility. AO should state that product choice explicitly if it ever narrows the contract.

Agent orchestrators and terminal managers

Superset: durable PTY plus headless xterm authority

Verified: Superset's session object says it owns a PTY subprocess, a headless emulator, attached clients, and disk output. It feeds PTY output through the headless emulator before broadcasting and queues emulator work to prevent blocking the I/O path (session ownership and construction, headless setup and query responses, output pipeline, emulator queue).

The authority waits for a consistent emulator boundary, serializes a snapshot, and attaches live output after that boundary (snapshot boundary, attach). Its headless wrapper uses @xterm/headless, sends terminal-generated replies back to the PTY, serializes state, and separately parses selected DEC modes and OSC-7 (headless wrapper, query output, snapshot, extra mode parser).

The PTY daemon can hand inherited PTY master descriptors and snapshots to a replacement process (daemon handoff).

Inference for AO: this is the closest evidence for the proposed terminal-authority shape, not necessarily a code target. It shows both its value and its cost: consistent snapshots and a sole query responder become possible, but headless write scheduling, mode completeness, handoff, and renderer rehydration are substantial systems work. The extra hand-maintained mode parser also shows a semantic-drift risk when the underlying emulator does not expose every state needed by the product.

Vibe Kanban: simple PTY-per-WebSocket baseline

Verified: Vibe Kanban creates a portable-pty shell, stores it in a session map, and exposes write/resize/close methods (PTY service, operations). Its terminal WebSocket creates a new PTY, exchanges base64 raw output plus input/resize messages, and closes that PTY with the socket (WebSocket lifecycle). The web component is a small xterm/Fit/WebLinks wrapper (xterm view).

Inference for AO: this is an intentionally simpler endpoint, useful as a complexity floor. It does not provide AO's persistent agent terminal, canonical replay, multi-viewer, or daemon-restart requirements. A renderer that reconnects cannot restore a session when the backend creates a new shell per connection.

Agent Deck: tmux is the terminal authority

Verified: Agent Deck describes itself as a tmux-backed terminal session manager (README). It creates detached tmux sessions, configures them, and uses pipe-pane logs and capture-pane for observation (session creation, pipe log, capture). Interactive attach runs tmux attach-session under a PTY, copies it directly to the raw outer terminal, and forwards resize/input (interactive attach).

Inference for AO: this avoids embedded renderer bugs only because the user's existing terminal paints. It retains tmux as a dependency and gives the application less control over exact visual composition and accessibility. It is a credible optional "Open externally" path, not a full AO desktop replacement.

NTM: exact tmux session plus monitoring-grade event stream

Verified: NTM explicitly uses tmux as its execution control plane and exposes REST/SSE/WebSocket surfaces (README, capabilities). Its pane streamer prefers a tmux pipe-pane FIFO, batches output into line events, and falls back to polling capture-pane and hashing the full text (stream setup, FIFO streaming, capture fallback). The event hub adds sequence numbers, an in-memory ring, and optional SQLite persistence (event store, store and resume).

Inference for AO: NTM's events are monitoring-friendly lines, not a faithful interactive terminal frame. That is a useful product separation: AO's overview can use structured agent status/transcripts and avoid continuously painting every hidden terminal, while an exact terminal remains available on demand.

Parser/state library without a painter: libvterm

Verified: libvterm exposes terminal state properties, input/output and key/mouse APIs, state callbacks, and a screen-cell/callback API (state and input API, screen API). The inspected Neovim fork explicitly says Neovim currently bundles its own vterm rather than using this fork (fork status).

Inference for AO: like libghostty-vt, libvterm can underpin an authoritative grid or custom renderer, but it is not an embedded visual terminal. Adopting it means AO owns all painting and UI behavior above the cell API and must assess modern protocol coverage separately.

Which bugs does replacing xterm actually address?

The user's diagnosis is therefore directionally correct at the painter/host-integration layer. It becomes incorrect if "xterm bug" is used as a catch-all for stream corruption, stale replay, terminal replies, process lifetime, or sizing races.

Strategy comparison

Keep xterm as both state engine and painter

Advantages:

• one cross-platform Electron implementation;
• mature VT/input/accessibility surface and existing AO integration;
• same family can run headlessly for server snapshots, as VS Code and Superset demonstrate;
• easiest path to browser/mobile viewers.

Costs:

• Chromium/WebGL/canvas lifetime and measurement bugs;
• hidden terminals consume parser and GPU resources unless parked or evicted;
• private APIs and browser event translation are fragile;
• server authority can drift from client state if only the painter parses output.

Ghostty VT core plus a web painter (wterm or ghostty-web)

Advantages:

• preserves AO's Electron, browser/mobile, and cross-platform delivery model;
• removes xterm's parser and WebGL-specific failure modes;
• wterm's DOM path gains native text selection/find/accessibility, while ghostty-web offers a low-friction xterm-like API;
• substantially cheaper to A/B than a native Electron overlay.

Costs:

• both projects are younger than xterm and own their own selection, input, sizing, viewport, and redraw bugs;
• DOM can cost more CPU/layout under output flood; Canvas2D makes selection and accessibility custom work;
• libghostty-vt API/build integration is still evolving;
• renderer replacement still does nothing to PTY, transport, replay, or resize authority.

Native embedded painter (libghostty surface or SwiftTerm)

Advantages:

• native text/GPU pipeline independent of Chromium;
• likely better local latency, font fidelity, and resource behavior if measured;
• Ghostty manual I/O preserves AO's external PTY authority.

Costs:

• complete Ghostty surface embedding is presently macOS-oriented; SwiftTerm's complete widgets are Apple-oriented;
• Electron/native child-view integration, focus, clipping, z-order, DPI, IME, accessibility, crashes, ABI, signing, and updater packaging become AO concerns;
• different platform painters can render or encode input differently;
• remote/mobile still need a web or semantic view.

Headless native VT engine plus AO-owned painter

Advantages:

• server can own canonical state, exact attach boundaries, inspection, and state-based backpressure;
• libghostty-vt is explicitly cross-platform.

Costs:

• this is the largest scope: AO becomes a terminal-renderer project;
• Unicode, graphics protocols, input, selection, accessibility, and GPU work are not supplied by the parser;
• engine API churn and FFI still apply.

External terminal delegation

Advantages:

• eliminates AO's embedded rasterizer and lets users choose Ghostty, Kitty, WezTerm, iTerm2, or another terminal;
• Agent Deck, NTM, Zellij, and Herdr show mature forms of this model.

Costs:

• breaks the integrated supervisor experience;
• window focus, navigation, previews, mobile access, and exact UI composition become harder;
• tmux or another persistent authority is still needed unless AO owns PTYs itself.

Semantic frames and structured agent views

Advantages:

• reconnect and slow-client coalescing operate on current state rather than an indispensable byte history;
• hidden sessions can show cheap status/transcript summaries instead of live GPU terminals;
• server-side search, observation, and mobile rendering improve.

Costs:

• a faithful cell protocol must cover width, styles, cursor, hyperlinks, graphics, modes, and versioning;
• a semantic agent transcript is not equivalent to an arbitrary terminal/TUI;
• if frames are re-encoded as ANSI into xterm, there are still two emulation/presentation stages.

Recommended AO direction

1. Measure and classify before replacing

For two weeks, tag every reproducible terminal failure by boundary:

• PTY/process lifecycle;
• authority/state/query response;
• transport/replay/backpressure;
• xterm parser/input;
• xterm WebGL/canvas/DOM painter;
• Electron host integration.

Record renderer, platform, GPU/context loss, terminal count, visibility/park state, bytes and writes per frame, attach sequence, resize epochs, and whether canvas fallback changes the result. This converts "xterm feels buggy" into a replacement budget and prevents a native rewrite from being judged against unrelated failures.

2. Define a deep terminal-authority interface

The daemon-side owner should eventually guarantee:

• one continuously drained PTY and launch identity;
• canonical normal/alternate buffers, scrollback, cursor, modes, title, and protocol state;
• one and only one responder for terminal-generated queries;
• atomic snapshot(sequence=N) followed by live changes strictly after N;
• explicit resize ownership and epoch;
• state-safe slow-client coalescing, or lossless ordered raw delivery;
• painter detach/crash without process loss.

The painter contract should receive state/output, report geometry, return user input, and expose capabilities. It should not own the agent process.

3. Build bounded prototypes, not one migration

Prototype A: headless authority with the existing xterm painter. On one opt-in runtime path, feed PTY output into a headless VT engine, handle query responses there, and attach with serialized state plus a sequence boundary. This tests the hardest correctness change while keeping the current visual surface. VS Code and Superset are relevant precedents.

Prototype B: web-painter bake-off. Behind one renderer adapter, feed identical recorded and live sessions into (1) current xterm 6.0 or a pinned 6.1 beta, (2) wterm's DOM painter with @wterm/ghostty, and (3) ghostty-web's Canvas2D painter. This isolates parser/painter correctness, resource use, accessibility, and latency without changing Electron/native integration or abandoning browser/mobile. Restty can join later when its API stabilizes.

Prototype C: macOS Ghostty manual-I/O painter. Only after the authority and adapter exist, feed that same contract into a native Ghostty surface and route Ghostty input callbacks back through the AO input path. Do not let the surface spawn or own the agent. Keep the winning web painter as the Windows/Linux/mobile and recovery path.

Do not combine authority replacement, binary transport, and native painter in one experiment; otherwise failures cannot be localized.

4. Add a semantic overview beside, not instead of, the exact terminal

Use agent hooks, structured events, process facts, and optionally a server-side screen model to render cheap waiting/running/error summaries. Park or evict hidden exact painters under a measured budget, and reconstruct them from authoritative snapshots when selected. NTM's separation between line events and exact tmux interaction is the useful idea here.

Acceptance tests and decision gates

Shared replay corpus

Record real PTY byte streams and replay them into xterm, a headless authority, and the native candidate. Compare final grid, scrollback, cursor, modes, title, and generated replies—not byte-for-byte renderer output. Include:

• normal and alternate screens;
• bracketed paste, focus reporting, mouse modes, application cursor/keypad;
• DA/DSR/CPR and replies during fresh output versus historical replay;
• Kitty keyboard protocol and common agent TUIs;
• split escape sequences and sustained output flood;
• combining marks, wide glyphs, emoji, ligatures, font fallback, and ambiguous-width characters;
• OSC titles, hyperlinks, clipboard policy, and supported graphics.

Lifecycle stress

• 20–50 sessions with rapid mount, park, unpark, and disposal;
• app/daemon restart, painter crash, stale registry, sidecar crash, and child exit;
• sleep/wake, display hot-plug, DPI and font changes;
• forced WebGL context loss and native GPU-surface loss;
• reconnect at arbitrary sequence boundaries and slow-client recovery;
• simultaneous desktop/mobile viewers with resize contention.

Input/accessibility matrix

• IME composition, dead keys, non-US layouts, Option/Alt/Meta, function keys;
• bracketed and large paste, clipboard permissions, mouse protocols, drag selection;
• screen reader navigation, contrast, cursor announcements, and keyboard-only use.

Resource and product gates

Measure per-visible and per-parked-session RSS, idle CPU, GPU contexts/memory, output-flood latency, first-paint/reconnect latency, binary size, crash isolation, and packaging complexity. Review licenses and API/version stability before selecting an engine.

A native renderer should ship broadly only if it materially beats the stabilized xterm path on reproducible reliability, fidelity, and resource tests and AO accepts the per-platform maintenance surface. Otherwise, keep xterm, fix authority/transport bugs at their actual layer, cap or park hidden painters, and offer native/external modes as optional paths.

Bottom line

There is no single project whose terminal architecture AO should copy:

• cmux is the best evidence for high-quality native Ghostty painting and a useful manual-I/O seam, but it is macOS-specific;
• wterm and ghostty-web are the cleanest first tests of a Ghostty-backed renderer inside AO's existing web boundary; each already demonstrates that custom DOM/canvas painting has its own correctness work;
• Restty is a promising but early Ghostty/WebGPU stack whose GPU-resource complexity overlaps the class AO is trying to escape;
• WezTerm is the strongest reference for a sequence-aware semantic pane contract, while Arbor shows both the viability and fidelity limits of a custom native cell painter;
• VS Code and Superset are the strongest evidence for headless canonical state plus a disposable xterm painter;
• Zellij and Herdr show the benefits of server-owned semantic state, but rely on an external terminal for final painting;
• Wave and Tabby expose both the practicality and the integration costs of xterm in Electron-style applications;
• Agent Deck and NTM show that tmux plus an outer terminal remains a valid control-plane choice, especially when monitoring and exact interaction are separated;
• Vibe Kanban shows the low-complexity raw PTY/WebSocket/xterm baseline, but not AO-grade persistence.

The globally stronger architecture is a hybrid: persistent PTY authority, canonical state and sequence-safe attach, a painter-independent contract, an upgraded xterm baseline, a Ghostty-backed DOM/Canvas bake-off, a later measured native macOS experiment, and structured agent views that avoid painting hidden terminals unnecessarily.

Continuation checkpoint

Future work on AO's terminal stack should start from these carried-forward conclusions unless new measurements invalidate one of them:

1. Do not equate a raw replay ring with terminal state. The correctness boundary is an authoritative VT state plus an atomic snapshot/sequence contract.
2. Do not let viewers answer child terminal queries. Exactly one authority answers DA/DSR and related queries, including when no viewer is attached.
3. Do not couple process lifetime to a painter. The PTY owner survives renderer mount, disposal, crash, park, and replacement.
4. Do not attribute every visible defect to xterm. Classify it as PTY/process, authority/query, transport/replay, parser/input, painter, or Electron integration first.
5. Do not compare a new renderer only against AO's xterm 5.5 stack. The control is a tested current xterm line with synchronized output and relevant fixes.
6. Test Ghostty in the web boundary before adopting a native overlay. wterm plus @wterm/ghostty and ghostty-web isolate the renderer hypothesis without adding platform divergence; native Ghostty manual-I/O is a later macOS experiment.
7. Keep exact terminals and agent overview views separate. Structured status/transcript views should avoid painting hidden terminals, while arbitrary TUIs remain available through an exact terminal.

To reopen one of these decisions, add a comment that names the numbered conclusion, supplies a reproducer or measurement, identifies the affected layer, and records the AO revision and renderer/runtime versions. That keeps the discussion cumulative rather than restarting from product analogies or symptoms.