one-os-dev-wayland-html5-compositor-greenfield-analysis/02-wayland-protocol.md

02 — Wayland Protocol Layer

This chapter covers Greenfield's protocol layer in depth: how Wayland XML protocol definitions get parsed, how typed TypeScript stubs get generated for each of the three sides (browser client, browser compositor, native proxy), how wire messages are marshalled/unmarshalled at runtime, how objects lifecycle is managed, and how the FD-over-WebSocket problem is solved.

The three generators

Greenfield ships three sibling generator CLIs that consume the same Wayland XML files but emit slightly different TypeScript flavors:

	client-generator	compositor-generator	compositor-proxy-generator
Output	TS *Proxy classes (browser)	TS *Resource + *Requests (Node)	CommonJS *_interface.js + *_interceptor.js (Node + N-API)
Listener pattern	listener?: <Itf>Events (optional, duck-typed, async)	implementation: <Itf>Requests (required)	requestHandlers / eventHandlers maps
Skips	none	wl_display, wl_registry (hand-written in runtime)	none — proxy must intercept display/registry/bind
Arg-type table	ProtocolArguments.mjs returns {signature, jsType, marshallGen}	Same shape, uses Westfield.${resourceName} for objects	EndpointProtocolArguments.mjs returns plain wayland sig letters (u, ?s, n, h, ...)
XMLs consumed	wayland.xml + xdg-shell.xml + web-bitmapbuf-unstable-v1.xml	same 3	wayland.xml + xdg-shell.xml + wayland-drm.xml

All three share xml2js + camelcase + uppercamelcase and a near-identical ProtocolParser.mjs. The XML → JSON → walk interface[]/request[]/ event[]/enum[] → per-arg type-dispatch pipeline is identical.

Why three generators?

Because each side has a fundamentally different job:

1. client-generator → browser apps that want to be Wayland clients of the browser compositor (via MessagePort). These are the same-origin iframe JS/WASM apps that ship in examples/webapps/.
2. compositor-generator → the browser-resident compositor itself (@gfld/compositor). It receives requests, dispatches them to *Requests implementation classes, and emits events back.
3. compositor-proxy-generator → the Node.js native proxy bridging real libwayland-client apps to the browser. Emits wl_interface struct definitions for the N-API addon + per-opcode interceptors that decide routing.

Wire format (libs/common/src/Connection.ts)

Standard upstream Wayland wire format, byte-for-byte. No custom binary encoding at the protocol layer.

┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
│  object id  │  opcode  │   size   │  args (variable per arg type, see §4)       │
│   uint32    │  uint16  │  uint16  │  u,i,f,n,o = 4 bytes; s,a = 4+len+pad;     │
│   (4 bytes) │  (2 B)   │  (2 B)   │  h = 0 bytes, FD goes in side-band fds[]    │
└─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
                            ▲
                            └── size includes this 8-byte header.
                                Total message size = size field.

Per-arg marshalling table

Arg signature letter	Type	Size on wire	Notes
u	uint32	4 bytes	unsigned integer
i	int32	4 bytes	signed integer
f	fixed	4 bytes	wl_fixed_t (24.8 signed)
s	string	4 + len + pad	length-prefixed u32, padded to 4-byte alignment
a	array	4 + len + pad	length-prefixed u32, padded to 4-byte alignment
o	object	4 bytes	object id (0 means null)
n	new_id	4 bytes	newly-allocated object id
h	file descriptor	0 bytes	FD goes in side-band wireMsg.fds[]

Reading

Connection.message dispatches by literal wlObject[opcode](message). The generated classes use computed-property-key methods: [0](message) { ... }, [1](message) { ... } so opcode is the array index. Each handler returns void or a Promise; Connection awaits the promise before reading the next message — this preserves wire-order while still allowing async implementations. Per-client Connections mean a slow client doesn't block others.

Object lifecycle & new-id tracking

Client side

Proxy.marshallConstructor calls display.generateNextId() (1-indexed, recycles ids from wl_display.delete_id), constructs the new Proxy (self-registers in constructor), patches the new_id arg's value (it was 0 placeholder from newObject()), then flushes.

Server side

Client.marshallConstructor allocates from SERVER_OBJECT_ID_BASE = 0xff000000 (or recycles locally). When a client-allocated id is destroyed, the server sends wl_display.delete_id so the client can recycle. On Client.close(), objects are destroyed in descending id order (children first).

Connection registry

wlObjects: { [id]: WlObject }; registerWlObject throws on collision.

Listener dispatch

Connection.message dispatches by literal wlObject[opcode](message). Each handler returns void or a Promise; Connection awaits the promise before reading the next message — this preserves wire-order while still allowing async implementations. Per-client Connections mean a slow client doesn't block others.

File descriptor handling — the Greenfield innovation

type FD = unknown — it's intentionally opaque. The actual FD representation depends on which link it's traversing.

Browser ↔ Browser (in-iframe, via MessagePort)

FDs are transferable JS objects:

• ArrayBufferView for wl_shm.create_pool's fd (SHM backing memory lives in browser, zero-copy transferred via MessagePort.postMessage(..., transferList)).
• ImageBitmap for web_bitmapbuf_factory.create_buffer's bitmap arg (GPU-backed texture, zero-copy).
• MessagePort for wl_data_offer.receive's fd (in-browser pipe).

Browser ↔ Proxy (over WebSocket/KCP)

FDs are virtualized as JSON ProxyFD descriptors:

ProxyFD = { handle: number, type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown', host: string }

Where:

• handle is the proxy's internal numeric fd id
• type is the kind of fd (so the browser knows what to do with it)
• host is the proxy's HTTP base URL (for proxy-to-proxy FD relay)

NativeWaylandClientSession.onWireMessageEnd converts each POSIX fd to a ProxyFD JSON, length-prefixed + 4-byte aligned, prepended to the wire message before sending over WebSocket/KCP. The browser's webFS resolves them by fetching data over a separate ad-hoc WebSocket endpoint (/read-fd, /write-fd-as-stream, etc.).

Native App ↔ Proxy (libwayland socket)

FDs are real POSIX ints, sent via SCM_RIGHTS by libwayland-server.

Most wl_buffer FDs never need to be read

Because the proxy intercepts wl_surface.commit, reads the buffer locally via DRM/SHM, encodes to H.264, and ships only the encoded bytes — the browser never needs to read the original FD.

How the proxy translates between native and browser

It doesn't translate wire bytes — both sides speak identical Wayland wire format. What it translates is:

• FDs: POSIX int ↔ ProxyFD JSON.
• Object-id space: native libwayland's id range vs browser's 0xff000000+ server range.
• Routing per message via MessageInterceptor.interceptRequest / interceptEvent. Special cases:
  • wl_display.sync → both native and browser (must complete before done event)
  • wl_display.get_registry → both
  • wl_registry.bind → native if id is in nativeGlobalNames[] (e.g. wl_drm, wl_shm), else browser
  • Fast-sync optimization skips browser round trip when no prior browser-bound request occurred

The four protocol XMLs

protocol/wayland.xml

Standard upstream Wayland core. 22 interfaces, wl_compositor v4, wl_seat v6, wl_surface v4 (roughly Wayland 1.20-ish). Kristian Høgsberg copyright. Used by all 3 generators.

protocol/xdg-shell.xml

Stable xdg_shell v1, 5 interfaces: xdg_wm_base, xdg_surface, xdg_toplevel, xdg_popup, xdg_positioner. Used by all 3 generators.

protocol/wayland-drm.xml

Legacy wl_drm v2, used by native apps (Mesa, XWayland) to advertise dmabuf buffers. Only the proxy generator processes this file — so the proxy can intercept create_buffer / create_planar_buffer / create_prime_buffer and turn dmabuf fds into ProxyFDs for the browser's GStreamer encoding pipeline. Browser-side libs do not generate wl_drm stubs.

protocol/web-bitmapbuf-unstable-v1.xml — Greenfield's custom extension

This is Greenfield's own Wayland protocol extension for in-browser JS/WASM clients. It defines a single singleton interface:

• web_bitmapbuf_factory (v1) with one request create_buffer(new_id wl_buffer id, fd bitmap) — "Create a wl_buffer object by wrapping an HTML bitmap, so it can be used with a surface."

The fd argument is an HTML bitmap file descriptor (a Greenfield-specific construct: an ImageBitmap-like object exposed as an FD-like handle). This lets JavaScript or WebAssembly Wayland clients back a surface with a native browser bitmap (ImageBitmap, OffscreenCanvas transferable) and zero-copy it into the compositor's WebGL texture pipeline — bypassing the SHM pixel-copy path.

The browser-side factory lives in packages/compositor/src/web/WebBuffersFactory.ts. Not processed by the proxy generator — native apps never use it.

This is a trade-off worth noting: the fast path is Greenfield-specific. To get zero-copy ImageBitmap upload to a GPU texture, apps must be Greenfield-aware (i.e. built against @gfld/client-protocol). Standard Wayland apps fall back to the H.264-encoded path even when running in the browser compositor.

ASCII pipeline diagram

                  ┌──────────────────────────────────────────────────────────────────────────────┐
                  │                          PROTOCOL XML SOURCES                                 │
                  │   wayland.xml   xdg-shell.xml   wayland-drm.xml   web-bitmapbuf-unstable-v1   │
                  └──────────┬──────────────┬──────────────┬─────────────────┬─────────────────────┘
                             │              │              │                 │
                             ▼              ▼              ▼                 ▼
                  ┌────────────────┐ ┌────────────────┐ ┌───────────────────┐ ┌────────────────────┐
                  │ client-        │ │ compositor-    │ │ compositor-proxy- │ │ (no proxy gen for  │
                  │ generator      │ │ generator      │ │ generator         │ │  web-bitmapbuf)    │
                  │ ProtocolParser │ │ ProtocolParser │ │ EndpointProtocol │ │                    │
                  │  .mjs          │ │  .mjs          │ │  Parser.mjs       │ │                    │
                  └───────┬────────┘ └───────┬────────┘ └─────────┬─────────┘ └─────────┬──────────┘
                          │ xml2js            │ xml2js             │ xml2js              │
                          ▼                   ▼                    ▼                    ▼
                  ┌────────────────┐ ┌────────────────┐  ┌─────────────────────┐  ┌────────────────────┐
                  │ wayland.ts      │ │ wayland.ts      │  │ *_interface.js      │  │ wayland.ts          │
                  │ xdg_shell.ts    │ │ xdg_shell.ts    │  │ *_interceptor.js    │  │ xdg_shell.ts        │
                  │ web_bitmapbuf.ts│ │ web_bitmapbuf.ts│  │ (wayland,xdg,drm)   │  │ web_bitmapbuf.ts    │
                  │  *Proxy classes │ │  *Resource +    │  │ wl_interface structs│  │  *Proxy classes     │
                  │  *Events ifaces │ │   *Requests     │  │ + R*/E* handlers    │  │  *Events interfaces │
                  │  *ProtocolName  │ │  *ProtocolName  │  │ + requestHandlers  │  │  *ProtocolName      │
                  │  *Enum          │ │  *Enum          │  │   /eventHandlers   │  │  *Enum              │
                  └───────┬─────────┘ └───────┬─────────┘  └──────────┬──────────┘  └─────────┬──────────┘
                          │ imports           │ imports               │ require()'d           │ imports
                          ▼                   ▼                       ▼                       ▼
                  ┌──────────────────────────────────────────┐  ┌──────────────────────────────────────────────┐
                  │           @gfld/common                    │  │ packages/compositor-proxy/                    │
                  │  Connection.marshallMsg() ── writes      │  │  src/wayland-server.ts                        │
                  │   u32 id | u16 opcode | u16 size | args  │  │   ↳ re-exports wayland-server-addon           │
                  │  Connection.message() ── parses header,  │  │   ↳ MessageInterceptor (R*/E* dispatch)      │
                  │   dispatches wlObject[opcode](message)   │  │   ↳ unmarshallArgs(message, 'usun') etc     │
                  │  Per-arg marshallers: uint/int/fixed/    │  │  NativeWaylandClientSession.ts                │
                  │   string/array/object/objectOptional/   │  │   ↳ onWireMessageRequest → interceptRequest │
                  │   newObject/fileDescriptor              │  │   ↳ onWireMessageEnd → pack ProxyFD JSON    │
                  │  Per-arg unmarshallers: u/i/f/s/a/o/n/h │  │   ↳ onWireMessageEvents → sendEvents (C)    │
                  └──────────────┬───────────────────────────┘  └─────────────────────┬────────────────────────┘
                                 │                                                     │
                                 ▼                                                     ▼
            ┌────────────────────────────────────────┐         ┌────────────────────────────────────────────────────┐
            │ Browser (client iframe)                │         │ Node (compositor-proxy)                            │
            │                                        │         │                                                    │
            │  DisplayImpl.messagePort.onmessage ←───┼─────────┼─ protocolChannel (WS/KCP) sends/receives          │
            │  connection.message({buffer, fds})      │         │  [u32 oob-op][u32 lastSerial][u32 fdCount] ...     │
            │  wlObject[opcode](message)              │         │  [u32 len][ProxyFD JSON, padded][wayland msg bytes]│
            │  listener?.<eventName>(...unmarshalled) │         │                                                    │
            │                                        │         │  ← from native: wl_client wire callback (C addon) │
            │  proxy.marshall(id, opcode, [args])    │         │  → to native: sendEvents(wlClient, buf, fds) (C)   │
            │  connection.marshallMsg writes Uint32   │         │                                                    │
            │  onFlush → postMessage(buf, transfer)   │         │  Native libwayland-server                          │
            │   ↳ buffer + ArrayBufferView/ImageBitmap│        │   accepts socket from native wayland apps          │
            │      /MessagePort as transferables      │         │   parses wire bytes, dispatches to interceptors    │
            └────────────────────────────────────────┘         └────────────────────────────────────────────────────┘

   WIRE FORMAT (identical on all links — standard Wayland):
   ┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
   │  object id  │  opcode  │   size   │  args (variable per arg type, see §4)       │
   │   uint32    │  uint16  │  uint16  │  u,i,f,n,o = 4 bytes; s,a = 4+len+pad;     │
   │   (4 bytes) │  (2 B)   │  (2 B)   │  h = 0 bytes, FD goes in side-band fds[]    │
   └─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
                            ▲
                            └── size includes this 8-byte header. Total message size = size field.

   FD transport varies per link:
   Browser↔Browser (MessagePort): FD is a transferable JS object (ArrayBufferView, ImageBitmap, MessagePort).
   Browser↔Proxy (WebSocket/KCP): FD is a JSON ProxyFD {handle, type, host}, length-prefixed + 4-byte aligned, prepended before the message header.
   Native App↔Proxy (libwayland socket): FD is a real POSIX fd, sent via SCM_RIGHTS by libwayland-server.

Build-time workflow

If you edit a protocol XML in your fork, you regenerate stubs by running the matching generate script:

• yarn workspace @gfld/client-protocol generate — regenerates browser client stubs.
• yarn workspace @gfld/compositor-protocol generate — regenerates browser compositor stubs.
• yarn workspace @gfld/compositor-proxy generate — regenerates proxy interceptor modules, then build:native (because *_interface.js declares wl_interface structs the N-API addon consumes).

To add a new Greenfield-only extension: drop XML in /protocol/, add a generate:<name> script to client-protocol / compositor-protocol package.json, regenerate. Only add to compositor-proxy if native apps will speak it.

Notable file references

• libs/client-generator/{ProtocolParser, ProtocolArguments, index}.mjs
• libs/compositor-generator/{ProtocolParser, ProtocolArguments, index}.mjs
• libs/compositor-proxy-generator/src/{EndpointProtocolParser, EndpointProtocolArguments, index}.mjs
• libs/common/src/Connection.ts — the runtime wire codec (shared)
• libs/compositor-protocol/src/westfield-runtime-server.ts — server-side runtime
• libs/client-protocol/src/westfield-runtime-client.ts — client-side runtime
• packages/compositor-proxy/src/wayland-server.ts — proxy's TS wrapper around the N-API addon
• packages/compositor-proxy/src/NativeWaylandClientSession.ts — per-client message interceptor dispatch
• packages/compositor-proxy/src/protocol/wl_*_interceptor.js — generated per-opcode routing decisions
• packages/compositor/src/web/WebBuffersFactory.ts — browser-side implementation of the custom web_bitmapbuf_factory global

One stale file spotted

libs/compositor-protocol/index.js (CommonJS, references non-existent ./src/Resource / ./src/Client / ./src/Display / ./src/Global). The actual entry point is src/index.ts. Safe to delete in a fork.