1b-agent/one-os-dev-wayland-html5-compositor-greenfield-analysis
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: Greenfield architectural analysis (14 files, ~80 pages, 10 parallel research agents)

Browse Source
This commit is contained in:
1b-agent committed 2026-10-08 08:48:25 +00:00
commit c4b45cf218
15 files changed
+5444
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

02-wayland-protocol.md
+325
View File
@@ -0,0 +1,325 @@
# 02 — Wayland Protocol Layer
This chapter covers Greenfield's protocol layer in depth: how Wayland XML
protocol definitions get parsed, how typed TypeScript stubs get generated for
each of the three sides (browser client, browser compositor, native proxy),
how wire messages are marshalled/unmarshalled at runtime, how objects
lifecycle is managed, and how the FD-over-WebSocket problem is solved.
## The three generators
Greenfield ships **three sibling generator CLIs** that consume the same
Wayland XML files but emit slightly different TypeScript flavors:
| | `client-generator` | `compositor-generator` | `compositor-proxy-generator` |
|---|---|---|---|
| Output | TS `*Proxy` classes (browser) | TS `*Resource` + `*Requests` (Node) | CommonJS `*_interface.js` + `*_interceptor.js` (Node + N-API) |
| Listener pattern | `listener?: <Itf>Events` (optional, duck-typed, async) | `implementation: <Itf>Requests` (required) | `requestHandlers` / `eventHandlers` maps |
| Skips | none | `wl_display`, `wl_registry` (hand-written in runtime) | none — proxy must intercept display/registry/bind |
| Arg-type table | `ProtocolArguments.mjs` returns `{signature, jsType, marshallGen}` | Same shape, uses `Westfield.${resourceName}` for objects | `EndpointProtocolArguments.mjs` returns plain wayland sig letters (`u`, `?s`, `n`, `h`, ...) |
| XMLs consumed | `wayland.xml` + `xdg-shell.xml` + `web-bitmapbuf-unstable-v1.xml` | same 3 | `wayland.xml` + `xdg-shell.xml` + `wayland-drm.xml` |
All three share `xml2js` + `camelcase` + `uppercamelcase` and a near-identical
`ProtocolParser.mjs`. The XML → JSON → walk `interface[]`/`request[]`/
`event[]`/`enum[]` → per-arg type-dispatch pipeline is identical.
### Why three generators?
Because each side has a fundamentally different job:
1. **`client-generator`** → browser apps that want to be Wayland clients of
the browser compositor (via `MessagePort`). These are the same-origin
iframe JS/WASM apps that ship in `examples/webapps/`.
2. **`compositor-generator`** → the browser-resident compositor itself
(`@gfld/compositor`). It receives requests, dispatches them to
`*Requests` implementation classes, and emits events back.
3. **`compositor-proxy-generator`** → the Node.js native proxy bridging real
`libwayland-client` apps to the browser. Emits `wl_interface` struct
definitions for the N-API addon + per-opcode interceptors that decide
routing.
## Wire format (`libs/common/src/Connection.ts`)
**Standard upstream Wayland wire format, byte-for-byte.** No custom binary
encoding at the protocol layer.
```
┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
│ object id │ opcode │ size │ args (variable per arg type, see §4) │
│ uint32 │ uint16 │ uint16 │ u,i,f,n,o = 4 bytes; s,a = 4+len+pad; │
│ (4 bytes) │ (2 B) │ (2 B) │ h = 0 bytes, FD goes in side-band fds[] │
└─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
▲
└── size includes this 8-byte header.
Total message size = size field.
```
### Per-arg marshalling table
| Arg signature letter | Type | Size on wire | Notes |
|---|---|---|---|
| `u` | uint32 | 4 bytes | unsigned integer |
| `i` | int32 | 4 bytes | signed integer |
| `f` | fixed | 4 bytes | wl_fixed_t (24.8 signed) |
| `s` | string | 4 + len + pad | length-prefixed u32, padded to 4-byte alignment |
| `a` | array | 4 + len + pad | length-prefixed u32, padded to 4-byte alignment |
| `o` | object | 4 bytes | object id (0 means null) |
| `n` | new_id | 4 bytes | newly-allocated object id |
| `h` | file descriptor | 0 bytes | FD goes in side-band `wireMsg.fds[]` |
### Reading
`Connection.message` dispatches by literal `wlObject[opcode](message)`. The
generated classes use computed-property-key methods:
`[0](message) { ... }`, `[1](message) { ... }` so opcode is the array index.
Each handler returns void or a Promise; `Connection` awaits the promise
before reading the next message — this preserves wire-order while still
allowing async implementations. Per-client Connections mean a slow client
doesn't block others.
## Object lifecycle & new-id tracking
### Client side
`Proxy.marshallConstructor` calls `display.generateNextId()` (1-indexed,
recycles ids from `wl_display.delete_id`), constructs the new Proxy
(self-registers in constructor), patches the `new_id` arg's `value` (it was
`0` placeholder from `newObject()`), then flushes.
### Server side
`Client.marshallConstructor` allocates from `SERVER_OBJECT_ID_BASE =
0xff000000` (or recycles locally). When a client-allocated id is destroyed,
the server sends `wl_display.delete_id` so the client can recycle. On
`Client.close()`, objects are destroyed in **descending id order** (children
first).
### Connection registry
`wlObjects: { [id]: WlObject }`; `registerWlObject` throws on collision.
## Listener dispatch
`Connection.message` dispatches by literal `wlObject[opcode](message)`. Each
handler returns void or a Promise; `Connection` awaits the promise before
reading the next message — this preserves wire-order while still allowing
async implementations. Per-client Connections mean a slow client doesn't
block others.
## File descriptor handling — the Greenfield innovation
`type FD = unknown` — it's intentionally opaque. The actual FD
representation depends on which link it's traversing.
### Browser ↔ Browser (in-iframe, via `MessagePort`)
FDs are transferable JS objects:
- `ArrayBufferView` for `wl_shm.create_pool`'s `fd` (SHM backing memory lives
in browser, zero-copy transferred via `MessagePort.postMessage(...,
transferList)`).
- `ImageBitmap` for `web_bitmapbuf_factory.create_buffer`'s `bitmap` arg
(GPU-backed texture, zero-copy).
- `MessagePort` for `wl_data_offer.receive`'s `fd` (in-browser pipe).
### Browser ↔ Proxy (over WebSocket/KCP)
FDs are virtualized as JSON `ProxyFD` descriptors:
```
ProxyFD = { handle: number, type: 'pipe-read' | 'pipe-write' | 'shm' | 'unknown', host: string }
```
Where:
- `handle` is the proxy's internal numeric fd id
- `type` is the kind of fd (so the browser knows what to do with it)
- `host` is the proxy's HTTP base URL (for proxy-to-proxy FD relay)
`NativeWaylandClientSession.onWireMessageEnd` converts each POSIX fd to a
ProxyFD JSON, length-prefixed + 4-byte aligned, prepended to the wire
message before sending over WebSocket/KCP. The browser's `webFS` resolves
them by fetching data over a separate ad-hoc WebSocket endpoint
(`/read-fd`, `/write-fd-as-stream`, etc.).
### Native App ↔ Proxy (libwayland socket)
FDs are real POSIX ints, sent via `SCM_RIGHTS` by libwayland-server.
### Most wl_buffer FDs never need to be read
Because the proxy intercepts `wl_surface.commit`, reads the buffer locally
via DRM/SHM, encodes to H.264, and ships only the encoded bytes — the
browser never needs to read the original FD.
## How the proxy translates between native and browser
**It doesn't translate wire bytes** — both sides speak identical Wayland
wire format. What it translates is:
- **FDs**: POSIX int ↔ `ProxyFD` JSON.
- **Object-id space**: native libwayland's id range vs browser's
`0xff000000`+ server range.
- **Routing per message** via `MessageInterceptor.interceptRequest` /
`interceptEvent`. Special cases:
- `wl_display.sync` → both native and browser (must complete before `done`
event)
- `wl_display.get_registry` → both
- `wl_registry.bind` → native if id is in `nativeGlobalNames[]` (e.g.
`wl_drm`, `wl_shm`), else browser
- Fast-sync optimization skips browser round trip when no prior
browser-bound request occurred
## The four protocol XMLs
### `protocol/wayland.xml`
Standard upstream Wayland core. 22 interfaces, `wl_compositor` v4, `wl_seat`
v6, `wl_surface` v4 (roughly Wayland 1.20-ish). Kristian Høgsberg copyright.
Used by all 3 generators.
### `protocol/xdg-shell.xml`
Stable `xdg_shell` v1, 5 interfaces: `xdg_wm_base`, `xdg_surface`,
`xdg_toplevel`, `xdg_popup`, `xdg_positioner`. Used by all 3 generators.
### `protocol/wayland-drm.xml`
Legacy `wl_drm` v2, used by native apps (Mesa, XWayland) to advertise dmabuf
buffers. **Only the proxy generator processes this file** — so the proxy can
intercept `create_buffer` / `create_planar_buffer` / `create_prime_buffer`
and turn dmabuf fds into ProxyFDs for the browser's GStreamer encoding
pipeline. Browser-side libs do not generate `wl_drm` stubs.
### `protocol/web-bitmapbuf-unstable-v1.xml` — Greenfield's custom extension
This is **Greenfield's own Wayland protocol extension** for in-browser
JS/WASM clients. It defines a single singleton interface:
- **`web_bitmapbuf_factory` (v1)** with one request
`create_buffer(new_id wl_buffer id, fd bitmap)` — "Create a wl_buffer
object by wrapping an HTML bitmap, so it can be used with a surface."
The `fd` argument is an HTML bitmap file descriptor (a Greenfield-specific
construct: an `ImageBitmap`-like object exposed as an FD-like handle). This
lets JavaScript or WebAssembly Wayland clients back a surface with a native
browser bitmap (`ImageBitmap`, `OffscreenCanvas` transferable) and zero-copy
it into the compositor's WebGL texture pipeline — bypassing the SHM
pixel-copy path.
The browser-side factory lives in
`packages/compositor/src/web/WebBuffersFactory.ts`. **Not processed by the
proxy generator** — native apps never use it.
This is a trade-off worth noting: the fast path is Greenfield-specific. To
get zero-copy `ImageBitmap` upload to a GPU texture, apps must be
Greenfield-aware (i.e. built against `@gfld/client-protocol`). Standard
Wayland apps fall back to the H.264-encoded path even when running in the
browser compositor.
## ASCII pipeline diagram
```
┌──────────────────────────────────────────────────────────────────────────────┐
│ PROTOCOL XML SOURCES │
│ wayland.xml xdg-shell.xml wayland-drm.xml web-bitmapbuf-unstable-v1 │
└──────────┬──────────────┬──────────────┬─────────────────┬─────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌───────────────────┐ ┌────────────────────┐
│ client- │ │ compositor- │ │ compositor-proxy- │ │ (no proxy gen for │
│ generator │ │ generator │ │ generator │ │ web-bitmapbuf) │
│ ProtocolParser │ │ ProtocolParser │ │ EndpointProtocol │ │ │
│ .mjs │ │ .mjs │ │ Parser.mjs │ │ │
└───────┬────────┘ └───────┬────────┘ └─────────┬─────────┘ └─────────┬──────────┘
│ xml2js │ xml2js │ xml2js │
▼ ▼ ▼ ▼
┌────────────────┐ ┌────────────────┐ ┌─────────────────────┐ ┌────────────────────┐
│ wayland.ts │ │ wayland.ts │ │ *_interface.js │ │ wayland.ts │
│ xdg_shell.ts │ │ xdg_shell.ts │ │ *_interceptor.js │ │ xdg_shell.ts │
│ web_bitmapbuf.ts│ │ web_bitmapbuf.ts│ │ (wayland,xdg,drm) │ │ web_bitmapbuf.ts │
│ *Proxy classes │ │ *Resource + │ │ wl_interface structs│ │ *Proxy classes │
│ *Events ifaces │ │ *Requests │ │ + R*/E* handlers │ │ *Events interfaces │
│ *ProtocolName │ │ *ProtocolName │ │ + requestHandlers │ │ *ProtocolName │
│ *Enum │ │ *Enum │ │ /eventHandlers │ │ *Enum │
└───────┬─────────┘ └───────┬─────────┘ └──────────┬──────────┘ └─────────┬──────────┘
│ imports │ imports │ require()'d │ imports
▼ ▼ ▼ ▼
┌──────────────────────────────────────────┐ ┌──────────────────────────────────────────────┐
│ @gfld/common │ │ packages/compositor-proxy/ │
│ Connection.marshallMsg() ── writes │ │ src/wayland-server.ts │
│ u32 id | u16 opcode | u16 size | args │ │ ↳ re-exports wayland-server-addon │
│ Connection.message() ── parses header, │ │ ↳ MessageInterceptor (R*/E* dispatch) │
│ dispatches wlObject[opcode](message) │ │ ↳ unmarshallArgs(message, 'usun') etc │
│ Per-arg marshallers: uint/int/fixed/ │ │ NativeWaylandClientSession.ts │
│ string/array/object/objectOptional/ │ │ ↳ onWireMessageRequest → interceptRequest │
│ newObject/fileDescriptor │ │ ↳ onWireMessageEnd → pack ProxyFD JSON │
│ Per-arg unmarshallers: u/i/f/s/a/o/n/h │ │ ↳ onWireMessageEvents → sendEvents (C) │
└──────────────┬───────────────────────────┘ └─────────────────────┬────────────────────────┘
│ │
▼ ▼
┌────────────────────────────────────────┐ ┌────────────────────────────────────────────────────┐
│ Browser (client iframe) │ │ Node (compositor-proxy) │
│ │ │ │
│ DisplayImpl.messagePort.onmessage ←───┼─────────┼─ protocolChannel (WS/KCP) sends/receives │
│ connection.message({buffer, fds}) │ │ [u32 oob-op][u32 lastSerial][u32 fdCount] ... │
│ wlObject[opcode](message) │ │ [u32 len][ProxyFD JSON, padded][wayland msg bytes]│
│ listener?.<eventName>(...unmarshalled) │ │ │
│ │ │ ← from native: wl_client wire callback (C addon) │
│ proxy.marshall(id, opcode, [args]) │ │ → to native: sendEvents(wlClient, buf, fds) (C) │
│ connection.marshallMsg writes Uint32 │ │ │
│ onFlush → postMessage(buf, transfer) │ │ Native libwayland-server │
│ ↳ buffer + ArrayBufferView/ImageBitmap│ │ accepts socket from native wayland apps │
│ /MessagePort as transferables │ │ parses wire bytes, dispatches to interceptors │
└────────────────────────────────────────┘ └────────────────────────────────────────────────────┘
WIRE FORMAT (identical on all links — standard Wayland):
┌─────────────┬──────────┬──────────┬─────────────────────────────────────────────┐
│ object id │ opcode │ size │ args (variable per arg type, see §4) │
│ uint32 │ uint16 │ uint16 │ u,i,f,n,o = 4 bytes; s,a = 4+len+pad; │
│ (4 bytes) │ (2 B) │ (2 B) │ h = 0 bytes, FD goes in side-band fds[] │
└─────────────┴──────────┴──────────┴─────────────────────────────────────────────┘
▲
└── size includes this 8-byte header. Total message size = size field.
FD transport varies per link:
Browser↔Browser (MessagePort): FD is a transferable JS object (ArrayBufferView, ImageBitmap, MessagePort).
Browser↔Proxy (WebSocket/KCP): FD is a JSON ProxyFD {handle, type, host}, length-prefixed + 4-byte aligned, prepended before the message header.
Native App↔Proxy (libwayland socket): FD is a real POSIX fd, sent via SCM_RIGHTS by libwayland-server.
```
## Build-time workflow
If you edit a protocol XML in your fork, you regenerate stubs by running
the matching `generate` script:
- `yarn workspace @gfld/client-protocol generate` — regenerates browser
client stubs.
- `yarn workspace @gfld/compositor-protocol generate` — regenerates browser
compositor stubs.
- `yarn workspace @gfld/compositor-proxy generate` — regenerates proxy
interceptor modules, then `build:native` (because `*_interface.js`
declares `wl_interface` structs the N-API addon consumes).
To add a new Greenfield-only extension: drop XML in `/protocol/`, add a
`generate:<name>` script to `client-protocol` / `compositor-protocol`
`package.json`, regenerate. Only add to `compositor-proxy` if native apps
will speak it.
## Notable file references
- `libs/client-generator/{ProtocolParser, ProtocolArguments, index}.mjs`
- `libs/compositor-generator/{ProtocolParser, ProtocolArguments, index}.mjs`
- `libs/compositor-proxy-generator/src/{EndpointProtocolParser, EndpointProtocolArguments, index}.mjs`
- `libs/common/src/Connection.ts` — the runtime wire codec (shared)
- `libs/compositor-protocol/src/westfield-runtime-server.ts` — server-side runtime
- `libs/client-protocol/src/westfield-runtime-client.ts` — client-side runtime
- `packages/compositor-proxy/src/wayland-server.ts` — proxy's TS wrapper around the N-API addon
- `packages/compositor-proxy/src/NativeWaylandClientSession.ts` — per-client message interceptor dispatch
- `packages/compositor-proxy/src/protocol/wl_*_interceptor.js` — generated per-opcode routing decisions
- `packages/compositor/src/web/WebBuffersFactory.ts` — browser-side implementation of the custom `web_bitmapbuf_factory` global
## One stale file spotted
`libs/compositor-protocol/index.js` (CommonJS, references non-existent
`./src/Resource` / `./src/Client` / `./src/Display` / `./src/Global`). The
actual entry point is `src/index.ts`. Safe to delete in a fork.