326 lines
21 KiB
Markdown
326 lines
21 KiB
Markdown
# 02 — Wayland Protocol Layer
|
|
|
|
This chapter covers Greenfield's protocol layer in depth: how Wayland XML
|
|
protocol definitions get parsed, how typed TypeScript stubs get generated for
|
|
each of the three sides (browser client, browser compositor, native proxy),
|
|
how wire messages are marshalled/unmarshalled at runtime, how objects
|
|
lifecycle is managed, and how the FD-over-WebSocket problem is solved.
|
|
|
|
## The three generators
|
|
|
|
Greenfield ships **three sibling generator CLIs** that consume the same
|
|
Wayland XML files but emit slightly different TypeScript flavors:
|
|
|
|
| | `client-generator` | `compositor-generator` | `compositor-proxy-generator` |
|
|
|---|---|---|---|
|
|
| Output | TS `*Proxy` classes (browser) | TS `*Resource` + `*Requests` (Node) | CommonJS `*_interface.js` + `*_interceptor.js` (Node + N-API) |
|
|
| Listener pattern | `listener?: <Itf>Events` (optional, duck-typed, async) | `implementation: <Itf>Requests` (required) | `requestHandlers` / `eventHandlers` maps |
|
|
| Skips | none | `wl_display`, `wl_registry` (hand-written in runtime) | none — proxy must intercept display/registry/bind |
|
|
| Arg-type table | `ProtocolArguments.mjs` returns `{signature, jsType, marshallGen}` | Same shape, uses `Westfield.${resourceName}` for objects | `EndpointProtocolArguments.mjs` returns plain wayland sig letters (`u`, `?s`, `n`, `h`, ...) |
|
|
| XMLs consumed | `wayland.xml` + `xdg-shell.xml` + `web-bitmapbuf-unstable-v1.xml` | same 3 | `wayland.xml` + `xdg-shell.xml` + `wayland-drm.xml` |
|
|
|
|
All three share `xml2js` + `camelcase` + `uppercamelcase` and a near-identical
|
|
`ProtocolParser.mjs`. The XML → JSON → walk `interface[]`/`request[]`/
|
|
`event[]`/`enum[]` → per-arg type-dispatch pipeline is identical.
|
|
|
|
### Why three generators?
|
|
|
|
Because each side has a fundamentally different job:
|
|
|
|
1. **`client-generator`** → browser apps that want to be Wayland clients of
|
|
the browser compositor (via `MessagePort`). These are the same-origin
|
|
iframe JS/WASM apps that ship in `examples/webapps/`.
|
|
2. **`compositor-generator`** → the browser-resident compositor itself
|
|
(`@gfld/compositor`). It receives requests, dispatches them to
|
|
`*Requests` implementation classes, and emits events back.
|
|
3. **`compositor-proxy-generator`** → the Node.js native proxy bridging real
|
|
`libwayland-client` apps to the browser. Emits `wl_interface` struct
|
|
definitions for the N-API addon + per-opcode interceptors that decide
|
|
routing.
|
|
|
|
## Wire format (`libs/common/src/Connection.ts`)
|
|
|
|
**Standard upstream Wayland wire format, byte-for-byte.** No custom binary
|
|
encoding at the protocol layer.
|
|
|
|
```
|
|
┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
|
|
│ object id │ opcode │ size │ args (variable per arg type, see §4) │
|
|
│ uint32 │ uint16 │ uint16 │ u,i,f,n,o = 4 bytes; s,a = 4+len+pad; │
|
|
│ (4 bytes) │ (2 B) │ (2 B) │ h = 0 bytes, FD goes in side-band fds[] │
|
|
└─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
|
|
▲
|
|
└── size includes this 8-byte header.
|
|
Total message size = size field.
|
|
```
|
|
|
|
### Per-arg marshalling table
|
|
|
|
| Arg signature letter | Type | Size on wire | Notes |
|
|
|---|---|---|---|
|
|
| `u` | uint32 | 4 bytes | unsigned integer |
|
|
| `i` | int32 | 4 bytes | signed integer |
|
|
| `f` | fixed | 4 bytes | wl_fixed_t (24.8 signed) |
|
|
| `s` | string | 4 + len + pad | length-prefixed u32, padded to 4-byte alignment |
|
|
| `a` | array | 4 + len + pad | length-prefixed u32, padded to 4-byte alignment |
|
|
| `o` | object | 4 bytes | object id (0 means null) |
|
|
| `n` | new_id | 4 bytes | newly-allocated object id |
|
|
| `h` | file descriptor | 0 bytes | FD goes in side-band `wireMsg.fds[]` |
|
|
|
|
### Reading
|
|
|
|
`Connection.message` dispatches by literal `wlObject[opcode](message)`. The
|
|
generated classes use computed-property-key methods:
|
|
`[0](message) { ... }`, `[1](message) { ... }` so opcode is the array index.
|
|
Each handler returns void or a Promise; `Connection` awaits the promise
|
|
before reading the next message — this preserves wire-order while still
|
|
allowing async implementations. Per-client Connections mean a slow client
|
|
doesn't block others.
|
|
|
|
## Object lifecycle & new-id tracking
|
|
|
|
### Client side
|
|
|
|
`Proxy.marshallConstructor` calls `display.generateNextId()` (1-indexed,
|
|
recycles ids from `wl_display.delete_id`), constructs the new Proxy
|
|
(self-registers in constructor), patches the `new_id` arg's `value` (it was
|
|
`0` placeholder from `newObject()`), then flushes.
|
|
|
|
### Server side
|
|
|
|
`Client.marshallConstructor` allocates from `SERVER_OBJECT_ID_BASE =
|
|
0xff000000` (or recycles locally). When a client-allocated id is destroyed,
|
|
the server sends `wl_display.delete_id` so the client can recycle. On
|
|
`Client.close()`, objects are destroyed in **descending id order** (children
|
|
first).
|
|
|
|
### Connection registry
|
|
|
|
`wlObjects: { [id]: WlObject }`; `registerWlObject` throws on collision.
|
|
|
|
## Listener dispatch
|
|
|
|
`Connection.message` dispatches by literal `wlObject[opcode](message)`. Each
|
|
handler returns void or a Promise; `Connection` awaits the promise before
|
|
reading the next message — this preserves wire-order while still allowing
|
|
async implementations. Per-client Connections mean a slow client doesn't
|
|
block others.
|
|
|
|
## File descriptor handling — the Greenfield innovation
|
|
|
|
`type FD = unknown` — it's intentionally opaque. The actual FD
|
|
representation depends on which link it's traversing.
|
|
|
|
### Browser ↔ Browser (in-iframe, via `MessagePort`)
|
|
|
|
FDs are transferable JS objects:
|
|
|
|
- `ArrayBufferView` for `wl_shm.create_pool`'s `fd` (SHM backing memory lives
|
|
in browser, zero-copy transferred via `MessagePort.postMessage(...,
|
|
transferList)`).
|
|
- `ImageBitmap` for `web_bitmapbuf_factory.create_buffer`'s `bitmap` arg
|
|
(GPU-backed texture, zero-copy).
|
|
- `MessagePort` for `wl_data_offer.receive`'s `fd` (in-browser pipe).
|
|
|
|
### Browser ↔ Proxy (over WebSocket/KCP)
|
|
|
|
FDs are virtualized as JSON `ProxyFD` descriptors:
|
|
|
|
```
|
|
ProxyFD = { handle: number, type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown', host: string }
|
|
```
|
|
|
|
Where:
|
|
- `handle` is the proxy's internal numeric fd id
|
|
- `type` is the kind of fd (so the browser knows what to do with it)
|
|
- `host` is the proxy's HTTP base URL (for proxy-to-proxy FD relay)
|
|
|
|
`NativeWaylandClientSession.onWireMessageEnd` converts each POSIX fd to a
|
|
ProxyFD JSON, length-prefixed + 4-byte aligned, prepended to the wire
|
|
message before sending over WebSocket/KCP. The browser's `webFS` resolves
|
|
them by fetching data over a separate ad-hoc WebSocket endpoint
|
|
(`/read-fd`, `/write-fd-as-stream`, etc.).
|
|
|
|
### Native App ↔ Proxy (libwayland socket)
|
|
|
|
FDs are real POSIX ints, sent via `SCM_RIGHTS` by libwayland-server.
|
|
|
|
### Most wl_buffer FDs never need to be read
|
|
|
|
Because the proxy intercepts `wl_surface.commit`, reads the buffer locally
|
|
via DRM/SHM, encodes to H.264, and ships only the encoded bytes — the
|
|
browser never needs to read the original FD.
|
|
|
|
## How the proxy translates between native and browser
|
|
|
|
**It doesn't translate wire bytes** — both sides speak identical Wayland
|
|
wire format. What it translates is:
|
|
|
|
- **FDs**: POSIX int ↔ `ProxyFD` JSON.
|
|
- **Object-id space**: native libwayland's id range vs browser's
|
|
`0xff000000`+ server range.
|
|
- **Routing per message** via `MessageInterceptor.interceptRequest` /
|
|
`interceptEvent`. Special cases:
|
|
- `wl_display.sync` → both native and browser (must complete before `done`
|
|
event)
|
|
- `wl_display.get_registry` → both
|
|
- `wl_registry.bind` → native if id is in `nativeGlobalNames[]` (e.g.
|
|
`wl_drm`, `wl_shm`), else browser
|
|
- Fast-sync optimization skips browser round trip when no prior
|
|
browser-bound request occurred
|
|
|
|
## The four protocol XMLs
|
|
|
|
### `protocol/wayland.xml`
|
|
|
|
Standard upstream Wayland core. 22 interfaces, `wl_compositor` v4, `wl_seat`
|
|
v6, `wl_surface` v4 (roughly Wayland 1.20-ish). Kristian Høgsberg copyright.
|
|
Used by all 3 generators.
|
|
|
|
### `protocol/xdg-shell.xml`
|
|
|
|
Stable `xdg_shell` v1, 5 interfaces: `xdg_wm_base`, `xdg_surface`,
|
|
`xdg_toplevel`, `xdg_popup`, `xdg_positioner`. Used by all 3 generators.
|
|
|
|
### `protocol/wayland-drm.xml`
|
|
|
|
Legacy `wl_drm` v2, used by native apps (Mesa, XWayland) to advertise dmabuf
|
|
buffers. **Only the proxy generator processes this file** — so the proxy can
|
|
intercept `create_buffer` / `create_planar_buffer` / `create_prime_buffer`
|
|
and turn dmabuf fds into ProxyFDs for the browser's GStreamer encoding
|
|
pipeline. Browser-side libs do not generate `wl_drm` stubs.
|
|
|
|
### `protocol/web-bitmapbuf-unstable-v1.xml` — Greenfield's custom extension
|
|
|
|
This is **Greenfield's own Wayland protocol extension** for in-browser
|
|
JS/WASM clients. It defines a single singleton interface:
|
|
|
|
- **`web_bitmapbuf_factory` (v1)** with one request
|
|
`create_buffer(new_id wl_buffer id, fd bitmap)` — "Create a wl_buffer
|
|
object by wrapping an HTML bitmap, so it can be used with a surface."
|
|
|
|
The `fd` argument is an HTML bitmap file descriptor (a Greenfield-specific
|
|
construct: an `ImageBitmap`-like object exposed as an FD-like handle). This
|
|
lets JavaScript or WebAssembly Wayland clients back a surface with a native
|
|
browser bitmap (`ImageBitmap`, `OffscreenCanvas` transferable) and zero-copy
|
|
it into the compositor's WebGL texture pipeline — bypassing the SHM
|
|
pixel-copy path.
|
|
|
|
The browser-side factory lives in
|
|
`packages/compositor/src/web/WebBuffersFactory.ts`. **Not processed by the
|
|
proxy generator** — native apps never use it.
|
|
|
|
This is a trade-off worth noting: the fast path is Greenfield-specific. To
|
|
get zero-copy `ImageBitmap` upload to a GPU texture, apps must be
|
|
Greenfield-aware (i.e. built against `@gfld/client-protocol`). Standard
|
|
Wayland apps fall back to the H.264-encoded path even when running in the
|
|
browser compositor.
|
|
|
|
## ASCII pipeline diagram
|
|
|
|
```
|
|
┌──────────────────────────────────────────────────────────────────────────────┐
|
|
│ PROTOCOL XML SOURCES │
|
|
│ wayland.xml xdg-shell.xml wayland-drm.xml web-bitmapbuf-unstable-v1 │
|
|
└──────────┬──────────────┬──────────────┬─────────────────┬─────────────────────┘
|
|
│ │ │ │
|
|
▼ ▼ ▼ ▼
|
|
┌────────────────┐ ┌────────────────┐ ┌───────────────────┐ ┌────────────────────┐
|
|
│ client- │ │ compositor- │ │ compositor-proxy- │ │ (no proxy gen for │
|
|
│ generator │ │ generator │ │ generator │ │ web-bitmapbuf) │
|
|
│ ProtocolParser │ │ ProtocolParser │ │ EndpointProtocol │ │ │
|
|
│ .mjs │ │ .mjs │ │ Parser.mjs │ │ │
|
|
└───────┬────────┘ └───────┬────────┘ └─────────┬─────────┘ └─────────┬──────────┘
|
|
│ xml2js │ xml2js │ xml2js │
|
|
▼ ▼ ▼ ▼
|
|
┌────────────────┐ ┌────────────────┐ ┌─────────────────────┐ ┌────────────────────┐
|
|
│ wayland.ts │ │ wayland.ts │ │ *_interface.js │ │ wayland.ts │
|
|
│ xdg_shell.ts │ │ xdg_shell.ts │ │ *_interceptor.js │ │ xdg_shell.ts │
|
|
│ web_bitmapbuf.ts│ │ web_bitmapbuf.ts│ │ (wayland,xdg,drm) │ │ web_bitmapbuf.ts │
|
|
│ *Proxy classes │ │ *Resource + │ │ wl_interface structs│ │ *Proxy classes │
|
|
│ *Events ifaces │ │ *Requests │ │ + R*/E* handlers │ │ *Events interfaces │
|
|
│ *ProtocolName │ │ *ProtocolName │ │ + requestHandlers │ │ *ProtocolName │
|
|
│ *Enum │ │ *Enum │ │ /eventHandlers │ │ *Enum │
|
|
└───────┬─────────┘ └───────┬─────────┘ └──────────┬──────────┘ └─────────┬──────────┘
|
|
│ imports │ imports │ require()'d │ imports
|
|
▼ ▼ ▼ ▼
|
|
┌──────────────────────────────────────────┐ ┌──────────────────────────────────────────────┐
|
|
│ @gfld/common │ │ packages/compositor-proxy/ │
|
|
│ Connection.marshallMsg() ── writes │ │ src/wayland-server.ts │
|
|
│ u32 id | u16 opcode | u16 size | args │ │ ↳ re-exports wayland-server-addon │
|
|
│ Connection.message() ── parses header, │ │ ↳ MessageInterceptor (R*/E* dispatch) │
|
|
│ dispatches wlObject[opcode](message) │ │ ↳ unmarshallArgs(message, 'usun') etc │
|
|
│ Per-arg marshallers: uint/int/fixed/ │ │ NativeWaylandClientSession.ts │
|
|
│ string/array/object/objectOptional/ │ │ ↳ onWireMessageRequest → interceptRequest │
|
|
│ newObject/fileDescriptor │ │ ↳ onWireMessageEnd → pack ProxyFD JSON │
|
|
│ Per-arg unmarshallers: u/i/f/s/a/o/n/h │ │ ↳ onWireMessageEvents → sendEvents (C) │
|
|
└──────────────┬───────────────────────────┘ └─────────────────────┬────────────────────────┘
|
|
│ │
|
|
▼ ▼
|
|
┌────────────────────────────────────────┐ ┌────────────────────────────────────────────────────┐
|
|
│ Browser (client iframe) │ │ Node (compositor-proxy) │
|
|
│ │ │ │
|
|
│ DisplayImpl.messagePort.onmessage ←───┼─────────┼─ protocolChannel (WS/KCP) sends/receives │
|
|
│ connection.message({buffer, fds}) │ │ [u32 oob-op][u32 lastSerial][u32 fdCount] ... │
|
|
│ wlObject[opcode](message) │ │ [u32 len][ProxyFD JSON, padded][wayland msg bytes]│
|
|
│ listener?.<eventName>(...unmarshalled) │ │ │
|
|
│ │ │ ← from native: wl_client wire callback (C addon) │
|
|
│ proxy.marshall(id, opcode, [args]) │ │ → to native: sendEvents(wlClient, buf, fds) (C) │
|
|
│ connection.marshallMsg writes Uint32 │ │ │
|
|
│ onFlush → postMessage(buf, transfer) │ │ Native libwayland-server │
|
|
│ ↳ buffer + ArrayBufferView/ImageBitmap│ │ accepts socket from native wayland apps │
|
|
│ /MessagePort as transferables │ │ parses wire bytes, dispatches to interceptors │
|
|
└────────────────────────────────────────┘ └────────────────────────────────────────────────────┘
|
|
|
|
WIRE FORMAT (identical on all links — standard Wayland):
|
|
┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
|
|
│ object id │ opcode │ size │ args (variable per arg type, see §4) │
|
|
│ uint32 │ uint16 │ uint16 │ u,i,f,n,o = 4 bytes; s,a = 4+len+pad; │
|
|
│ (4 bytes) │ (2 B) │ (2 B) │ h = 0 bytes, FD goes in side-band fds[] │
|
|
└─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
|
|
▲
|
|
└── size includes this 8-byte header. Total message size = size field.
|
|
|
|
FD transport varies per link:
|
|
Browser↔Browser (MessagePort): FD is a transferable JS object (ArrayBufferView, ImageBitmap, MessagePort).
|
|
Browser↔Proxy (WebSocket/KCP): FD is a JSON ProxyFD {handle, type, host}, length-prefixed + 4-byte aligned, prepended before the message header.
|
|
Native App↔Proxy (libwayland socket): FD is a real POSIX fd, sent via SCM_RIGHTS by libwayland-server.
|
|
```
|
|
|
|
## Build-time workflow
|
|
|
|
If you edit a protocol XML in your fork, you regenerate stubs by running
|
|
the matching `generate` script:
|
|
|
|
- `yarn workspace @gfld/client-protocol generate` — regenerates browser
|
|
client stubs.
|
|
- `yarn workspace @gfld/compositor-protocol generate` — regenerates browser
|
|
compositor stubs.
|
|
- `yarn workspace @gfld/compositor-proxy generate` — regenerates proxy
|
|
interceptor modules, then `build:native` (because `*_interface.js`
|
|
declares `wl_interface` structs the N-API addon consumes).
|
|
|
|
To add a new Greenfield-only extension: drop XML in `/protocol/`, add a
|
|
`generate:<name>` script to `client-protocol` / `compositor-protocol`
|
|
`package.json`, regenerate. Only add to `compositor-proxy` if native apps
|
|
will speak it.
|
|
|
|
## Notable file references
|
|
|
|
- `libs/client-generator/{ProtocolParser, ProtocolArguments, index}.mjs`
|
|
- `libs/compositor-generator/{ProtocolParser, ProtocolArguments, index}.mjs`
|
|
- `libs/compositor-proxy-generator/src/{EndpointProtocolParser, EndpointProtocolArguments, index}.mjs`
|
|
- `libs/common/src/Connection.ts` — the runtime wire codec (shared)
|
|
- `libs/compositor-protocol/src/westfield-runtime-server.ts` — server-side runtime
|
|
- `libs/client-protocol/src/westfield-runtime-client.ts` — client-side runtime
|
|
- `packages/compositor-proxy/src/wayland-server.ts` — proxy's TS wrapper around the N-API addon
|
|
- `packages/compositor-proxy/src/NativeWaylandClientSession.ts` — per-client message interceptor dispatch
|
|
- `packages/compositor-proxy/src/protocol/wl_*_interceptor.js` — generated per-opcode routing decisions
|
|
- `packages/compositor/src/web/WebBuffersFactory.ts` — browser-side implementation of the custom `web_bitmapbuf_factory` global
|
|
|
|
## One stale file spotted
|
|
|
|
`libs/compositor-protocol/index.js` (CommonJS, references non-existent
|
|
`./src/Resource` / `./src/Client` / `./src/Display` / `./src/Global`). The
|
|
actual entry point is `src/index.ts`. Safe to delete in a fork.
|