Lesson 6: Terminal Protocol

Terminal applications don't write "move cursor to row 5, column 10, print 'hello' in bold red." They write escape sequences: byte strings that the terminal emulator interprets. After this lesson, you understand the VT100 state machine, can parse any CSI sequence by hand, and can build a VT parser that correctly handles the bytestream Ghostty produces.

Why a State Machine? #

The terminal protocol is in-band. The same byte stream contains printable text AND control commands. A byte A (0x41) could be a literal 'A' or part of a cursor movement sequence (\e[10A = move cursor up 10 lines). The parser must track which context it's in.

The core idea: the parser is always in exactly one state. The next byte determines the transition. This is a deterministic finite automaton (DFA).

The State Machine States #

C0 and C1 Control Characters #

C0 (0x00-0x1F): single-byte controls embedded in the text stream.

C1 (0x80-0x9F): 8-bit equivalents of escape sequences. Rarely used; 7-bit ESC-prefixed forms are standard.

CSI Sequences (Control Sequence Introducer) #

Format: ESC [ followed by parameter bytes (0x30-0x3F), intermediate bytes (0x20-0x2F), and a final byte (0x40-0x7E).

\e[<params><intermediates><final>

Cursor movement #

SGR: Select Graphic Rendition #

\e[<params>m

Each parameter sets a display attribute. Parameters are applied in sequence. An empty parameter means 0 (reset).

DEC Private Modes #

CSI sequences with ? prefix: \e[?<params><final>.

The DEC private modes are critical for terminal applications. vim uses ?1049h to enter the alternate screen. tmux uses ?1000h for mouse support. Ghostty must handle every one of these correctly.

OSC Sequences (Operating System Command) #

Format: ESC ] followed by the command and body, terminated by ST (\e\) or BEL (\a).

\e]<command>;<body>\e\    (ST-terminated)
\e]<command>;<body>\a     (BEL-terminated)

DCS Sequences (Device Control String) #

Format: ESC P followed by the command, parameter bytes, data, terminated by ST or BEL.

Ghostty implements sixel support via DCS sequences. Each sixel is a bitmap that occupies grid cells. This is how terminal image protocols work: not as a separate channel, but as specially-interpreted escape sequences in the text stream.

The Kitty Keyboard Protocol #

The standard terminal protocol encodes keyboard input poorly. Modifiers (Ctrl, Alt, Super) are hacky (Ctrl+A = byte 0x01, but Ctrl+Shift+A = ?). The kitty keyboard protocol fixes this with explicit key event encoding:

\e[<key>;<modifiers>u    // Key press
\e[<key>;<modifiers>:<action>u   // Key event with action (press/repeat/release)

Each physical key has a Unicode codepoint (e.g., 'a' = 97). The kitty protocol sends the base key plus modifier flags. Ghostty can distinguish Ctrl+Shift+A from Ctrl+A from plain A, which the traditional protocol cannot do.

Project: vt-parse #

Build a VT parser from scratch:

1. State machine: implement the full DFA (Ground, Escape, CSI Entry/Param/Intermediate/Ignore, OSC, DCS, SOS/PM/APC)
2. CSI dispatch: parse parameters (numbers separated by ;, with defaults for omitted params), intermediate bytes (private mode markers), and call the appropriate action
3. SGR state: maintain attribute state (fg, bg, bold, italic, underline, inverse). Update on SGR sequences. Apply to characters written to the grid.
4. Terminal grid: fixed-size 2D array of cells. Each cell: codepoint, fg color, bg color, attribute flags
5. Test it: pipe ls --color=auto, vim, htop, cat of a file with ANSI art through your parser and verify the grid state matches the expected output

// Minimal interface
typedef enum { GROUND, ESCAPE, CSI_ENTRY, CSI_PARAM, CSI_INTERMEDIATE, CSI_IGNORE,
               OSC_STRING, DCS_ENTRY, DCS_PARAM, DCS_DATA, DCS_PASSTHROUGH } vt_state_t;

void vt_parse_byte(vt_t *vt, uint8_t byte);
// Called for each byte from the PTY master. Updates vt->state, accumulates params,
// and dispatches actions (print_char, cursor_up, erase_display, set_sgr, etc.)

Ghostty Source to Study #

Self-Check #

Can you:

• Draw the VT100 state machine (all 10+ states and transitions) from memory
• Parse \e[38;2;255;128;0;1;4mHello\e[0m by hand: what colors is "Hello" rendered in, and with what attributes?
• Explain the difference between CSI, OSC, and DCS: prefix, purpose, and termination
• Why does \e[?1049h cause "the screen to clear" when entering vim? (It switches to the alternate screen buffer.)
• Write the CSI sequence to move the cursor to row 10, column 20
• Explain why the kitty keyboard protocol exists and what problem it solves
• Trace the byte sequence of a key press through the parser: from keyboard input to grid update