commit 582e61e942e4b3abd0a9f4d72da12f591ed54733 Author: 1b-agent Date: Thu Oct 8 08:48:28 2026 +0000 Initial commit: MIT-licensed compositor architecture & build plan (14 files, ~80 pages, 10 parallel research agents) diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4d5081d --- /dev/null +++ b/.gitignore @@ -0,0 +1,24 @@ +# Python +__pycache__/ +*.pyc +*.pyo +.venv/ +venv/ + +# OS +.DS_Store +Thumbs.db + +# Editor +.vscode/ +.idea/ +*.swp + +# Logs +*.log + +# Build +node_modules/ +dist/ +build/ +*.egg-info/ diff --git a/00-overview.md b/00-overview.md new file mode 100644 index 0000000..e270357 --- /dev/null +++ b/00-overview.md @@ -0,0 +1,312 @@ +# 00 — The MIT Approach at a Glance + +## The one-paragraph summary + +Build a fully proprietary Wayland-to-HTML5 compositor for the Odoo 20 fork +by **reusing Greenfield's 7 MIT-licensed libraries** as browser-side +building blocks (the protocol runtime, the generators, the WASM modules +for pixman+xkbcommon, the X11 bindings if XWayland is needed) and +**replacing the 4 AGPL packages** (`@gfld/compositor`, +`@gfld/compositor-proxy`, `@gfld/compositor-shell`, +`@gfld/compositor-proxy-cli`) with **clean-room reimplementations** in +TypeScript (browser) and Python (Linux host). The result integrates with +Odoo 20 as a custom OWL shell + backend addon, with JWT-based auth and +bubblewrap/seccomp app sandboxing — features Greenfield lacks entirely. + +## What's reused (MIT/Apache/BSD — no AGPL contamination) + +### Browser side — reused from Greenfield + +| Library | License | Purpose | +|---|---|---| +| `@gfld/common` | MIT | Wayland wire format codec (Connection, WlObject, Fixed, marshallers) | +| `@gfld/client-protocol` | MIT | Browser-side Wayland client bindings (for browser-resident JS/WASM apps that want to be Wayland clients of OUR compositor) | +| `@gfld/compositor-protocol` | MIT | Browser-side Wayland server bindings — the **strategic keystone lib**. Verified: imports ZERO code from `@gfld/compositor` (AGPL). | +| `@gfld/client-generator` | MIT | Build-time CLI: XML → TS browser client bindings | +| `@gfld/compositor-generator` | MIT | Build-time CLI: XML → TS browser server bindings | +| `@gfld/compositor-wasm` | MIT | pixman 0.42.2 + xkbcommon 1.5.0 compiled to WASM | +| `@gfld/xtsb` | MIT | X11 TypeScript bindings (only if XWayland support is needed) | + +### Browser side — native APIs (W3C standards, MIT-equivalent) + +| API | Standard | Purpose | +|---|---|---| +| WebGL2 | W3C | GPU-accelerated 2D/3D rendering (replaces Greenfield's WebGL1) | +| WebCodecs `VideoDecoder` | W3C | Hardware H.264/VP9/AV1 decode (replaces Greenfield's GPL FFmpeg WASM) | +| OffscreenCanvas | W3C | Move WebGL to a Web Worker (Greenfield doesn't use this) | +| Web Workers | W3C | Parallel decode + render (Greenfield uses only 2 workers for the fallback path) | +| WebSocket | W3C | Binary transport (same as Greenfield) | +| SharedArrayBuffer + Atomics | W3C | Zero-copy between workers (needs COOP/COEP headers) | +| ImageBitmap | W3C | GPU-backed bitmap for the `web-bitmapbuf` fast path | +| WebTransport | W3C (future) | UDP-over-QUIC transport (when browser support is widespread) | + +### Linux side — Python ecosystem + +| Component | License | Purpose | +|---|---|---| +| pywayland | Apache-2.0 | Python bindings to libwayland-server (replaces Greenfield's `wayland-server-addon.node` + `wayland-server.ts`) | +| python-xkbcommon (`xkbcommon` on PyPI) | MIT | Python bindings to libxkbcommon | +| PyGObject (`gi.repository.Gst`) | LGPL-2.1+ | Python bindings to GStreamer (replaces Greenfield's `proxy-encoding-addon.node` + `libproxy-encoding.so`) | +| GLib | LGPL-2.1+ | GStreamer runtime dep | +| GStreamer core | LGPL-2.0+ | Media pipeline framework | +| gst-plugins-base, gst-plugins-good | LGPL | `pngenc`, `appsrc`, `appsink`, `videoconvert` | +| gst-vaapi | LGPL + MIT (libva) | `vaapih264enc` for Intel/AMD GPU encoding | +| gst-plugins-bad (BSD subset) | BSD | `nvh264enc` for NVIDIA NVENC, `openh264enc` for software fallback | +| libwayland | MIT | The Wayland protocol C library | +| libdrm | MIT | Direct Rendering Manager | +| mesa (libGL, libEGL, libgbm) | MIT | OpenGL/EGL/GBM for GPU buffer management | +| aiohttp | Apache-2.0 | Async HTTP + WebSocket server | +| kcp (PyPI, RealistikDash) | MIT | Cython wrapper over upstream `ikcp.c` | +| PyJWT | MIT | JWT issuance + validation | +| bubblewrap (bwrap) | LGPL-2.0+ | Unprivileged sandboxing | +| libseccomp | LGPL-2.1 | Seccomp BPF filter compilation | + +### Linux side — vendored Greenfield C files (MIT) + +These are the C files in `packages/compositor-proxy/native/wayland/src/` +that are MIT-licensed (Greenfield's own work, NOT wlroots-derived unless +noted) and can be **reused verbatim** in our Python proxy: + +| File | Lines | Purpose | +|---|---|---| +| `westfield-buffer.c` | ~150 | Buffer helpers | +| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM | +| `westfield-egl.c` | ~900 | EGL zero-copy setup (creates EGLDisplay, EGLContext, wraps dmabuf as EGLImage) | +| `westfield-xwayland.c` | ~465 | XWayland spawn + bridge | +| `wlr_drm.c` | ~244 | `wl_drm` global impl (vendored from wlroots, MIT) | +| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots, MIT) | +| `drm_format_set.c` | ~239 | DRM format helpers (vendored from wlroots, MIT) | +| `addon.c` | ~73 | wlroots addon system helper | +| `changes.diff` | ~593 | Patches to libwayland-server adding wire-message interception hooks | + +**Total**: ~3,974 lines of MIT-licensed C that can be reused as-is. + +## What's replaced (custom code, written clean-room) + +### Browser side — custom TypeScript compositor library + +**Replaces**: `@gfld/compositor` (AGPL, ~21,101 LOC) + +| Component | Lines | Weeks | Notes | +|---|---|---|---| +| `surface/` (Surface, SurfaceRole, Subsurface, Subcompositor, Region, FrameCallback) | ~1,200 | 3 | Double-buffered state, commit semantics | +| `render/` (Renderer, Scene, View, Output, RenderState, Texture, shaders) | ~1,800 | 3 | WebGL2 + OffscreenCanvas + Web Workers | +| `buffer/` (Shm, BufferContents, ImageBitmapBuffer, EncodedFrameBuffer) | ~600 | 1.5 | SHM with BMP-header hack | +| `transport/` (Channel, KCP, ReconnectingWebSocket, BufferStream, decoders) | ~3,500 | 4 | KCP ported fresh from MIT C ref | +| `input/` (Seat, Keyboard, Pointer, Touch, Xkb, InputQueue, InputOutput) | ~2,500 | 3 | Uses @gfld/compositor-wasm | +| `roles/` (XdgWmBase, XdgSurface, XdgToplevel, XdgPopup, XdgPositioner, Shell) | ~1,800 | 3 | xdg-shell state machine, biggest risk | +| `data/` (DataDeviceManager, DataSource, DataOffer, WaylandDataSource) | ~600 | 1.5 | Clipboard + DnD + primary selection (Greenfield lacks primary) | +| `desktop/` (Desktop, FloatingDesktopSurface, FullscreenDesktopSurface) | ~800 | 1.5 | Layout policy | +| `math/` (Mat4, Vec4, Point, Size, Rect, Transformations) | ~500 | 0.5 | | +| `userapi/` (UserShellApi) | ~120 | 0.5 | Odoo OWL shell integration seam | +| `browser/` (input.ts, pointer.ts, dnd.ts, selection.ts) | ~750 | 2 | DOM glue | +| `decoders/` (WebCodecsFrameDecoder, OpenH264FrameDecoder) | ~500 | 1 | NO FFmpeg Wasm (GPL) | +| `core/` (Session, Globals) | ~200 | 0.5 | | +| **MVP TOTAL** | **~13,400** | **~25 weeks** | | +| `xwayland/` (XWindowManager, XWindow, XWindowFrame) | ~5,000 | 8 | Optional, only for X11 apps | +| Tests + hardening | — | 3 | | +| **GRAND TOTAL** | **~18,900** | **~30-36 weeks** | | + +### Linux side — custom Python proxy + +**Replaces**: `@gfld/compositor-proxy` + `@gfld/compositor-proxy-cli` (AGPL, ~5,000 LOC TS + ~3,974 LOC C) + +| Component | Lines | Dev-days | Notes | +|---|---|---|---| +| aiohttp WS server (9 endpoints) | 600 | 3 | /signal, /channel, /mkfifo, /mkstemp-mmap, /read-fd, /read-fd-as-stream, /write-fd-as-stream, /close-fd, /request-keyframe | +| `Channel` / `ARQChannel` / `SimpleChannel` port | 250 | 1.5 | | +| `kcp` PyPI package integration | 100 | 0.5 | | +| `NativeAppContext` (signaling, kill timers) | 350 | 2 | 10-min SIGHUP grace, SIGTERM escalation | +| `ProxyInputOutput` (FD workaround over WS) | 300 | 2 | memfd_create, mkfifo via os.pipe | +| libwayland event-loop integration | 80 | 1 | 5-line cffi shim for `wl_event_loop_get_fd` | +| Multiprocessing scaffolding + FD passing | 200 | 1.5 | `multiprocessing.reduction.send_handle` | +| GStreamer encoder thread + marshalling | 250 | 2 | `asyncio.run_coroutine_threadsafe` | +| Reconnection & lifecycle | 150 | 1 | | +| Tests (pytest + pytest-asyncio) | 600 | 3 | | +| **TOTAL** | **~2,880** | **~17 days** | | + +### Odoo 20 integration — custom addon + +| Component | Dev-days | Notes | +|---|---|---| +| Odoo addon skeleton (models + controllers + views + security) | 2 | compositor.app, compositor.session, compositor.audit.log | +| JWT issuance + audit log model | 1.5 | HS256, 15-min expiry | +| OWL 3 fullscreen client action + canvas + WS wiring | 2.5 | `ir.actions.client` with `target=fullscreen` | +| Browser compositor library integration | 2 | `userShell.actions.initScene(canvasCreator)` | +| Compositor proxy JWT middleware + WS handshake | 1.5 | | +| Audit POSTback + Nginx + Docker Compose + GPU passthrough | 2 | | +| Multi-tenant: per-DB proxy spawning + db claim scoping | 1.5 | | +| Tests (HttpCase + TransactionCase) | 2 | | +| Documentation + buffer | 3 | | +| **TOTAL** | **~18 days** | | + +### Sandboxing — NEW (Greenfield has none) + +| Component | Dev-days | Notes | +|---|---|---| +| bubblewrap integration | 2-3 | | +| seccomp profile + compile | 1-2 | Docker default profile + CLONE_NEWUSER allow | +| systemd-run cgroup wrapper | 1 | MemoryMax=1G, CPUQuota=200%, TasksMax=256 | +| Wayland socket ownership fix | 0.5 | chmod 0660 + add sandbox group | +| GPU passthrough testing | 1 | /dev/dri/renderD128 only | +| Schema + docs | 1 | | +| **v1 TOTAL** | **~7-9 days** | shippable, XWayland unsandboxed | +| XWayland sandbox (v2) | +4-9 | | +| veth + nftables networking (v2) | +2 | | +| Per-tenant hierarchical cgroups (v2) | +2 | | +| XDG Desktop Portal (FileChooser/Secret) (v2) | +3 | | + +## Architecture at a glance + +``` +┌──────────────────────────────────────────────────────────────────────────────────┐ +│ BROWSER (Odoo 20 web client) │ +│ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Odoo 20 OWL Shell addon (PROPRIETARY) │ │ +│ │ - ir.actions.client target=fullscreen │ │ +│ │ - owned by OWL component │ │ +│ │ - Implements userShell.events.* callbacks │ │ +│ │ - Calls userShell.actions.initScene, activateSurface, ... │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ userShell.events + actions (clean-room spec) │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Custom Browser Compositor Library (PROPRIETARY, clean-room TS) │ │ +│ │ - Session, Compositor, Globals, Surface, View, Output, Seat, ... │ │ +│ │ - render/ (WebGL2 + OffscreenCanvas + Web Workers) │ │ +│ │ - transport/ (WebSocket + KCP + ReconnectingWebSocket) │ │ +│ │ - roles/ (xdg-shell state machine, clean-room from spec) │ │ +│ │ - desktop/, input/, data/, math/ │ │ +│ │ - userapi/ (UserShellApi — the OWL shell integration seam) │ │ +│ │ ~13,400 LOC, ~25 dev-weeks for MVP │ │ +│ │ │ │ +│ │ Reuses MIT libs as npm deps: │ │ +│ │ - @gfld/common, @gfld/compositor-protocol, @gfld/client-protocol (MIT) │ │ +│ │ - @gfld/compositor-wasm (pixman + xkbcommon WASM, MIT) │ │ +│ │ - @gfld/xtsb (MIT, X11 — only if XWayland needed) │ │ +│ │ - WebCodecs API (W3C standard, replaces GPL FFmpeg WASM) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────────────────┘ + │ + │ WebSocket (TCP) + KCP-over-WS + │ 9 endpoint paths per session + │ JWT (HS256) auth, 15-min expiry + │ Same-origin via Nginx reverse proxy + ▼ +┌──────────────────────────────────────────────────────────────────────────────────┐ +│ PYTHON HOST (Linux server) │ +│ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Nginx (TLS termination, BSD-2-Clause) │ │ +│ │ / → Odoo :8069 (HTTP workers) │ │ +│ │ /websocket → Odoo :8072 (gevent, Discuss/chat) │ │ +│ │ /compositor/ → Compositor proxy :8081 (aiohttp WS, proxy_buffering off) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Odoo 20 (LGPL-3.0, separate process — no contamination) │ │ +│ │ - HTTP workers (preforked, N processes) │ │ +│ │ - Controllers: /web/compositor/{token, apps, launch, terminate, audit} │ │ +│ │ - Models: compositor.app, compositor.session, compositor.audit.log │ │ +│ │ - JWT issuer (HS256 secret in ir.config_parameter) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ subprocess + IPC │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Python Compositor Proxy (PROPRIETARY, ~2880 LOC) │ │ +│ │ - aiohttp WebSocket server on :8081, 9 endpoints │ │ +│ │ - asyncio + 5-line cffi shim for libwayland event loop │ │ +│ │ - multiprocessing.Process per Odoo session (fork) │ │ +│ │ - JWT validation (pyjwt) │ │ +│ │ - pywayland (Apache-2.0) for Wayland protocol dispatch │ │ +│ │ - python-xkbcommon (MIT) for keymap compilation │ │ +│ │ - PyGObject+GStreamer (LGPL) for H.264/PNG encoding │ │ +│ │ - Uses vaapih264enc / nvh264enc / openh264enc (NOT x264enc — GPL) │ │ +│ │ - kcp (PyPI, MIT, Cython wrapper over ikcp.c) for ARQ channels │ │ +│ │ - Vendored Greenfield C files (MIT) for dmabuf/egl/xwayland helpers │ │ +│ │ - Greenfield's changes.diff applied to libwayland-server (MIT patches) │ │ +│ │ - bubblewrap + seccomp + systemd-run --scope for app sandboxing │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ WAYLAND_DISPLAY=wayland-N │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Native Linux apps (gtk4-demo, xterm, kwrite, gnome-calculator, ...) │ │ +│ │ - Spawned via: systemd-run --user --scope -p MemoryMax=1G -p CPUQuota=200% │ │ +│ │ → bwrap --unshare-all --clearenv --seccomp --ro-bind /usr /usr │ │ +│ │ --dev-bind /dev/dri/renderD128 /dev/dri/renderD128 │ │ +│ │ --ro-bind $HOST_XDG/wayland-N /run/user//wayland-N │ │ +│ │ --uid <100000-165535 per-app> --gid │ │ +│ │ --new-session --die-with-parent │ │ +│ │ /usr/bin/ │ │ +│ │ - Filesystem: /usr ro-bind, minimal /etc whitelist, $HOME overlay │ │ +│ │ - Network: --unshare-net by default (most apps don't need it) │ │ +│ │ - GPU: /dev/dri/renderD128 only (kernel render-node access control) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────────────────┘ +``` + +## What we GAIN over Greenfield + +1. **Full proprietary control** — no AGPL contamination, ship as a closed-source + Odoo addon. +2. **Modern browser stack** — WebGL2 (vs Greenfield's WebGL1), OffscreenCanvas + + Web Workers (vs Greenfield's main-thread WebGL), WebCodecs with hardware + decode enabled (Greenfield disables it with a FIXME). +3. **Sandboxing** — bubblewrap + seccomp + systemd-run + cgroups. Greenfield + has zero sandboxing. +4. **Multi-tenancy** — per-Odoo-database proxy isolation, JWT-scoped sessions. + Greenfield is single-user single-session. +5. **Odoo integration** — auth via Odoo session, audit log, app launcher from + Odoo's app catalog. Greenfield has its own Preact shell. +6. **Adaptive bitrate** — consume the `EncoderFeedback` data Greenfield produces + but doesn't use. +7. **Primary selection** — middle-click paste, which Greenfield lacks. +8. **Real touch input** — Greenfield has a 46-line stub that never fires. +9. **Modern codecs** — VP9/AV1 support via WebCodecs (Greenfield is H.264 only). +10. **Clean upgrade path** — no native Node addons (no `.node` files), no + Node.js ABI lock-in, no CMake/Ninja build chain. + +## What we LOSE vs Greenfield + +1. **~25 dev-weeks of engineering** — the browser compositor is the + biggest cost. Clean-room rewrite is mandatory for legal safety. +2. **Greenfield's battle-tested codebase** — we'd be writing fresh Wayland + state machines, which is subtle (xdg-shell configure/ack round-trips, + subsurface sync/desync cascades, popup positioner math). +3. **Time-to-market** — Greenfield works today; our MIT approach takes + ~6-8 months to MVP. +4. **Greenfield's WASM app SDK** — the `sdk/` directory is for compiling + native Wayland apps to WASM and running them in the browser. If you + need this feature, it's a separate ~4-8 week project to port. + +## Total cost summary (single engineer, sequential) + +| Phase | Weeks | Cumulative | +|---|---|---| +| Phase 0: Spike (validate libwayland ↔ asyncio) | 1 | 1 | +| Phase 1: Browser compositor MVP (clean-room TS) | 25 | 26 | +| Phase 2: Python proxy MVP (aiohttp + pywayland + GStreamer) | 3.5 | 29.5 | +| Phase 3: Odoo integration (OWL shell + backend addon + JWT) | 3.5 | 33 | +| Phase 4: Sandboxing v1 (bubblewrap + seccomp) | 1.5 | 34.5 | +| Phase 5: Polish & testing | 3 | 37.5 | +| **MVP TOTAL** | **~37.5 weeks** | (~9 months) | +| Phase 6 (optional): XWayland + multi-tenant + portal | +9 | ~46.5 weeks | + +With 2 senior engineers in parallel (one on browser compositor, one on +Python proxy + Odoo integration): + +| Phase | Weeks | Cumulative | +|---|---|---| +| Phase 0: Spike | 1 | 1 | +| Phase 1+2 (parallel) | 25 (browser is critical path) | 26 | +| Phase 3+4 (parallel) | 3.5 | 29.5 | +| Phase 5: Polish & testing | 3 | 32.5 | +| **MVP TOTAL** | **~32.5 weeks** | (~8 months) | +| Phase 6 (optional): XWayland + multi-tenant + portal | +6 | ~38.5 weeks | + +See `../Greenfield-vs-MIT-Comparison.md` for the head-to-head comparison. diff --git a/01-architecture.md b/01-architecture.md new file mode 100644 index 0000000..20ee213 --- /dev/null +++ b/01-architecture.md @@ -0,0 +1,194 @@ +# 01 — Layered Architecture Map + +This chapter walks the proposed MIT-licensed architecture layer by layer, +mirroring the Greenfield analysis structure (see `../greenfield-analysis/01-architecture.md`) +so you can compare them side-by-side. + +## The eight layers (MIT approach) + +``` +┌──────────────────────────────────────────────────────────────────────────┐ +│ Layer 8: USER SHELL (Odoo 20 OWL addon, PROPRIETARY) │ +│ Hooks: UserShellApi (events + actions) — clean-room spec │ +│ See 10-odoo-integration.md │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 7: DESKTOP / LAYOUT POLICY (PROPRIETARY TS, clean-room) │ +│ Floating vs AlwaysFullScreen, view stacking, focus activation refcount │ +│ Custom: desktop/Desktop.ts (~800 LOC) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 6: SURFACE LIFECYCLE / ROLES (PROPRIETARY TS, clean-room) │ +│ XdgWmBase, XdgSurface, XdgToplevel, XdgPopup, XdgPositioner, Shell, │ +│ ShellSurface, Subcompositor, Subsurface, SurfaceRole │ +│ Custom: roles/*.ts (~1,800 LOC) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 5: RENDERING PIPELINE (PROPRIETARY TS, clean-room, modernized) │ +│ Surface (state stack), View, Output (WebGL2 canvas), Renderer, Scene, │ +│ Shaders (YUV→RGBA, YUVA→RGBA, RGB+A→RGBA), Loop (RAF), Callback │ +│ NEW: OffscreenCanvas + Web Workers + WebGL2 (Greenfield uses WebGL1) │ +│ Custom: render/*.ts (~1,800 LOC), surface/*.ts (~1,200 LOC) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 4: TRANSPORT (browser side, PROPRIETARY TS, clean-room) │ +│ Channel (ARQ + Simple), ReconnectingWebSocket, RemoteAppLauncher, │ +│ StreamingBuffer, BufferStream, EncodedFrame, EncoderFeedback, decoders │ +│ NEW: WebCodecs VideoDecoder replaces FFmpeg WASM (was GPL) │ +│ Custom: transport/*.ts (~3,500 LOC), decoders/*.ts (~500 LOC) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 4: TRANSPORT (Python side, PROPRIETARY, clean-room) │ +│ aiohttp WebSocket server (9 endpoints), asyncio + cffi shim, │ +│ kcp PyPI package, multiprocessing.Process per session, JWT auth │ +│ Custom: proxy/*.py (~2,880 LOC) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 3: WAYLAND RUNTIME (browser side) │ +│ REUSED: @gfld/common (Connection, WlObject, Fixed, marshallers) — MIT │ +│ REUSED: @gfld/compositor-protocol (Display, Client, Resource, Global) │ +│ — MIT, verified NO @gfld/compositor import │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 3: WAYLAND RUNTIME (Python side) │ +│ REUSED: pywayland (Apache-2.0) — Display, Client, Resource, Global │ +│ + 5-line cffi shim for wl_event_loop_get_fd (asyncio integration) │ +│ + Greenfield's changes.diff (MIT patches) applied to libwayland-server │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 2: PROTOCOL GENERATORS (REUSED) │ +│ REUSED: @gfld/client-generator, @gfld/compositor-generator (MIT) │ +│ REUSED: pywayland-protocols (auto-generated from wayland-protocols repo) │ +├──────────────────────────────────────────────────────────────────────────┤ +│ Layer 1: WAYLAND PROTOCOL XML (REUSED) │ +│ REUSED: wayland.xml, xdg-shell.xml, wayland-drm.xml (upstream MIT) │ +│ REUSED: web-bitmapbuf-unstable-v1.xml (Greenfield's custom extension) │ +└──────────────────────────────────────────────────────────────────────────┘ +``` + +## Side-by-side comparison with Greenfield + +| Layer | Greenfield | MIT approach | +|---|---|---| +| Layer 8: User Shell | `@gfld/compositor-shell` (AGPL, Preact) | Custom Odoo 20 OWL addon (proprietary) | +| Layer 7: Desktop policy | `desktop/Desktop.ts` (AGPL) | Clean-room rewrite (~800 LOC TS) | +| Layer 6: Surface roles | `Xdg*.ts`, `Shell*.ts`, `Sub*.ts` (AGPL) | Clean-room rewrite (~1,800 LOC TS) | +| Layer 5: Rendering | `render/*.ts`, `Surface.ts`, etc. (AGPL, WebGL1, main-thread) | Clean-room rewrite (~3,000 LOC TS, WebGL2 + OffscreenCanvas + Workers) | +| Layer 4: Transport (browser) | `remote/*.ts` (AGPL) + FFmpeg WASM (GPL) | Clean-room rewrite (~3,500 LOC TS) + WebCodecs (W3C standard) | +| Layer 4: Transport (host) | `compositor-proxy/src/*.ts` (AGPL) + 3 N-API addons | Clean-room Python rewrite (~2,880 LOC Python, NO native addons) | +| Layer 3: Wayland runtime (browser) | `@gfld/common` + `@gfld/compositor-protocol` (MIT) | **REUSED AS-IS** | +| Layer 3: Wayland runtime (host) | `wayland-server-addon.node` (AGPL, 1231 lines C, patched libwayland) | **REPLACED** with pywayland (Apache-2.0) + 5-line cffi shim + Greenfield's `changes.diff` (MIT patches) | +| Layer 2: Generators | `@gfld/*-generator` (MIT) | **REUSED AS-IS** + pywayland-protocols (auto-generated) | +| Layer 1: Protocol XML | `protocol/*.xml` (upstream MIT + Greenfield's web-bitmapbuf) | **REUSED AS-IS** | + +## How data flows across the layers + +``` + NATIVE APP (e.g. gtk4-demo, X11 via XWayland) + │ + │ libwayland-client → UNIX socket + SCM_RIGHTS FDs + ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ PYTHON PROXY (aiohttp + pywayland + GStreamer + bubblewrap) │ +│ │ +│ Layer 3 (Python): pywayland + 5-line cffi shim │ +│ ↳ VANILLA libwayland-server (NO patches needed for basic use) │ +│ ↳ Apply Greenfield's changes.diff (MIT) IF we want wire-message │ +│ interception hooks (recommended for the fast-sync optimization) │ +│ ↳ Per-resource dispatchers (auto-generated by pywayland scanner) │ +│ ↳ FDs wrapped as ProxyFD JSON │ +│ │ +│ Layer 4 (Python): aiohttp WebSocket + kcp PyPI package │ +│ ↳ 9 endpoints: /signal, /channel, /mkfifo, /mkstemp-mmap, │ +│ /read-fd, /read-fd-as-stream, /write-fd-as-stream, │ +│ /close-fd, /request-keyframe │ +│ ↳ asyncio integrates libwayland FD via loop.add_reader() │ +│ ↳ multiprocessing.Process per Odoo session (fork) │ +│ ↳ JWT validation (pyjwt) │ +│ │ +│ Layer 5 (encoding): PyGObject + GStreamer │ +│ ↳ gst_parse_launch: same pipeline strings as Greenfield │ +│ ↳ x264enc REPLACED with vaapih264enc / nvh264enc / openh264enc │ +│ (x264enc is GPL — see 11-pitfalls.md) │ +│ ↳ Zero-copy dmabuf via EGLImage → GstGLMemoryEGL (same as Greenfield)│ +│ ↳ GLib MainLoop in separate Python thread │ +│ ↳ asyncio.run_coroutine_threadsafe for cross-thread marshalling │ +│ │ +│ Layer 7 (sandboxing): bubblewrap + seccomp + systemd-run --scope │ +│ ↳ Per-app: MemoryMax=1G, CPUQuota=200%, TasksMax=256 │ +│ ↳ --unshare-all, --clearenv, --ro-bind /usr /usr │ +│ ↳ --dev-bind /dev/dri/renderD128 /dev/dri/renderD128 (single GPU) │ +│ ↳ --ro-bind $HOST_XDG/wayland-N /run/user//wayland-N │ +│ ↳ --uid <100000-165535 per-app> --gid │ +│ ↳ --unshare-net by default │ +│ │ +│ Layer 1 (XML) → Layer 2 (pywayland scanner) feeds Layer 3 dispatchers │ +└─────────────────────────────────────────────────────────────────────────┘ + │ + │ WebSocket (TCP) + KCP-over-WS, JWT auth + │ Same-origin via Nginx reverse proxy + ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ BROWSER COMPOSITOR (Custom TypeScript + WebGL2 + WebCodecs + WASM) │ +│ │ +│ Layer 4 (browser): Custom Channel + KCP + ReconnectingWebSocket │ +│ ↳ ARQChannel + SimpleChannel (clean-room, ~250 LOC) │ +│ ↳ KCP ported fresh from MIT C reference (skywind3000/kcp) │ +│ ↳ ReconnectingWebSocket (vendored from pladaria, MIT) │ +│ ↳ WebCodecs VideoDecoder (primary, hardware-accelerated) │ +│ ↳ OpenH264 BSD Wasm (fallback, NOT FFmpeg — FFmpeg is GPL) │ +│ │ +│ Layer 3 (browser): REUSED @gfld/common + @gfld/compositor-protocol │ +│ ↳ Standard Wayland wire format, byte-for-byte │ +│ ↳ WlObject registry, opcode dispatch via wlObject[opcode](msg) │ +│ ↳ Per-client Connections (slow client never blocks another) │ +│ │ +│ Layer 5: Custom Surface.ts + View.ts + Output.ts + render/ │ +│ ↳ Double-buffered state (state vs pendingState) │ +│ ↳ Damage tracking via @gfld/compositor-wasm pixman regions │ +│ ↳ WebGL2 with FBOs for YUV→RGBA, RGB+Alpha→RGBA conversion │ +│ ↳ OffscreenCanvas in Web Worker (NEW vs Greenfield) │ +│ ↳ Frame callbacks fired on requestAnimationFrame vsync │ +│ │ +│ Layer 6: Custom XdgWmBase + XdgSurface + XdgToplevel + XdgPopup + ... │ +│ ↳ Configure/ack-configure round-trips │ +│ ↳ Popup positioner math (anchor + gravity + constraints) │ +│ ↳ Subsurface sync/desync mode cascading │ +│ ↳ Primary selection (NEW — Greenfield lacks this) │ +│ │ +│ Layer 7: Custom desktop/Desktop.ts │ +│ ↳ Floating vs AlwaysFullScreen mode │ +│ ↳ View stacking, focus refcount │ +│ ↳ Adaptive bitrate (NEW — consumes EncoderFeedback data) │ +│ │ +│ Layer 8: UserShellApi (clean-room spec) + Odoo 20 OWL shell │ +│ ↳ Events: clientCreated, surfaceCreated, surfaceTitleUpdated, │ +│ surfaceActivationUpdated, ... │ +│ ↳ Actions: initScene, refreshScene, destroyScene, │ +│ closeClient, activateSurface, setUserConfiguration │ +│ ↳ OWL shell wires events to reactive state │ +└─────────────────────────────────────────────────────────────────────────┘ + │ + │ userShell.events.* fired by compositor + │ userShell.actions.* called by OWL shell + ▼ +┌─────────────────────────────────────────────────────────────────────────┐ +│ USER SHELL (Odoo 20 OWL addon — PROPRIETARY) │ +│ │ +│ - ir.actions.client with target=fullscreen │ +│ - Application launcher (lists apps from Odoo /web/compositor/apps) │ +│ - Window management UI (taskbar, decorations, minimize/maximize) │ +│ - Keyboard layout picker (RMLVO) │ +│ - Per-output element (OWL component owns the lifecycle) │ +│ - Triggers app launches via /web/compositor/launch │ +│ - Receives JWT from /web/compositor/token │ +│ │ +│ Not a Greenfield component — entirely your code. │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +## Reading guide for the rest of the bundle + +- `02-wayland-protocol.md` — Layer 1, 2, 3 (XML → generators → runtime, both sides) +- `03-html5-rendering.md` — Layer 5 (custom WebGL2 + Workers + OffscreenCanvas) +- `04-transport.md` — Layer 4 (Python aiohttp + kcp, browser side clean-room) +- `05-input-handling.md` — Layer 5 input subsystem (Python xkbcommon, browser side) +- `06-data-clipboard.md` — Layer 5 data device (with primary selection, Greenfield lacks) +- `07-native-app-bridge.md` — Layer 4 Python + Layer 3 Python + sandboxing +- `08-app-lifecycle.md` — Layer 6 + Layer 8 (UserShellApi clean-room spec) +- `09-hardware-accel.md` — Layer 5 encoding + Layer 4 frame channel + WebCodecs +- `10-odoo-integration.md` — Layer 8 + Odoo 20 backend addon + JWT + Nginx + Docker +- `11-pitfalls.md` — 3 GPL contaminants found + clean-room legal requirements +- `12-appendix.md` — glossary, file index, references diff --git a/02-wayland-protocol.md b/02-wayland-protocol.md new file mode 100644 index 0000000..01529b3 --- /dev/null +++ b/02-wayland-protocol.md @@ -0,0 +1,291 @@ +# 02 — Wayland Protocol Layer (MIT-licensed) + +This chapter covers how the Wayland protocol layer works in the MIT +approach: protocol XML → generators → runtime, for both the browser side +and the Python host side. + +## Layer 1 — Protocol XML (REUSED AS-IS) + +Same 4 XML files as Greenfield, all upstream MIT or Greenfield's own MIT +extension: + +- `protocol/wayland.xml` — upstream Wayland core (MIT) +- `protocol/xdg-shell.xml` — stable xdg-shell (MIT) +- `protocol/wayland-drm.xml` — legacy wl_drm (MIT, host-side only) +- `protocol/web-bitmapbuf-unstable-v1.xml` — Greenfield's custom extension + for browser JS apps (MIT) + +**No changes needed.** These files are the source of truth. + +## Layer 2 — Protocol Generators (REUSED AS-IS + pywayland scanner) + +### Browser side — REUSED from Greenfield (MIT) + +- `@gfld/client-generator` (MIT) — generates TS `*Proxy` classes for browser + apps that want to be Wayland clients of OUR compositor. +- `@gfld/compositor-generator` (MIT) — generates TS `*Resource` + `*Requests` + classes for our custom browser compositor. +- `@gfld/compositor-proxy-generator` — **NOT NEEDED** (this generates + interceptor files for the Node N-API addon; we don't have one). + +### Python side — pywayland's built-in scanner + +pywayland ships its own scanner that parses the same `wayland.xml` + +`wayland-protocols/*.xml` and emits Python bindings +(`pywayland/protocol/*.py`). It uses the upstream `wayland-scanner` +C tool under the hood. + +**No new generator code to write.** The scanner produces: +- `pywayland/protocol/wayland.py` — wl_display, wl_registry, wl_compositor, + wl_surface, wl_buffer, wl_seat, wl_keyboard, wl_pointer, wl_touch, + wl_output, wl_shm, wl_data_device_manager, wl_subcompositor +- `pywayland/protocol/xdg_shell.py` — xdg_wm_base, xdg_surface, + xdg_toplevel, xdg_popup, xdg_positioner +- 60+ additional files for stable and unstable Wayland protocols + +For Greenfield's custom `web-bitmapbuf-unstable-v1.xml`: **not needed on +the Python side** (native apps don't use it; it's browser-only). If you +need to expose it to native apps later, pywayland supports custom +protocol XMLs via `pywayland.scanner`. + +## Layer 3 — Wayland Runtime + +### Browser side — REUSED from Greenfield (MIT, verified clean) + +`@gfld/common` (`libs/common/src/Connection.ts`) — the shared wire format +codec. Standard upstream Wayland wire format, byte-for-byte. 8-byte +header (`u32 object_id | u16 opcode | u16 size`), per-arg bytes. + +`@gfld/compositor-protocol` (`libs/compositor-protocol/src/`) — the +server-side runtime. Provides `Display`, `Client`, `Resource`, `Global`, +`Registry`, plus generated `*Resource` classes for wayland, xdg_shell, +web_bitmapbuf. + +**CRITICAL VERIFICATION** (from Agent M's audit): `@gfld/compositor-protocol` +imports ZERO code from `@gfld/compositor` (AGPL). It only depends on +`@gfld/common` (MIT). So it's safe to reuse in a proprietary codebase. + +### Python side — pywayland (Apache-2.0) + +pywayland is the Python bindings to libwayland-server. License: +**Apache-2.0** (not MIT as the task brief assumed, but Apache-2.0 is +permissive — allows proprietary use, modification, distribution). + +#### API surface + +**`Display`** — wraps `struct wl_display*`: +- `Display()` — create +- `.add_socket(name=None)` → returns `"wayland-N"` (becomes WAYLAND_DISPLAY) +- `.get_event_loop()` → `EventLoop` +- `.run()` — block (we don't call this — we use asyncio integration instead) +- `.terminate()` — break the run loop +- `.init_shm()` — register wl_shm global +- `.add_shm_format(fmt)` — add SHM format (YUV420, NV12, etc.) +- `.flush_clients()` — send pending events +- `.get_serial()` → next serial number +- `.next_serial()` — allocate a serial +- `.destroy()` +- `.add_destroy_listener(listener)` + +**`EventLoop`** — wraps `struct wl_event_loop*`: +- `.add_fd(fd, cb, mask, data)` — register an FD for polling +- `.add_signal(signum, cb, data)` +- `.add_timer(cb, data)` → `Timer` +- `.add_idle(cb, data)` +- `.dispatch(timeout_ms)` — process pending events (we call this from asyncio) +- `.dispatch_idle()` +- `.add_destroy_listener(listener)` +- **MISSING: `.get_fd()`** — needed for asyncio integration, see §asyncio below + +**`Client`** — wraps `struct wl_client*`: +- `Client(display, fd)` — create from inbound socketpair +- `Client(ptr=existing_ptr)` — wrap existing +- `.destroy()` +- `.flush()` +- `.get_credentials()` → `(pid, uid, gid)` — used to identify which app spawned this client +- `.add_destroy_listener(listener)` +- `.get_object(id)` → `Resource` +- `Client.from_resource(res)` — static method +- **MISSING: `wl_display_add_client_created_listener`** — for new-client notifications. Available in upstream libwayland, exposed via 5-line cffi shim. + +**`Resource`** (base class for generated `Wl*` types): +- `Resource(client, version, id=0)` +- `.destroy()` +- `.get_id()` → int +- `.add_destroy_listener(listener)` +- `._post_event(opcode, *args)` — emit event to client +- `._post_error(code, msg)` — send protocol error +- `.dispatcher[opcode_or_name] = func` — per-resource dispatch table (KEY mechanism) +- `.dispatcher.destructor = func` + +**`Global`** (base class for generated `Wl*.global_class`): +- `WlFoo.global_class(display)` — instantiate +- `.bind_func = handler` — callback when a client binds +- `.destroy()` + +**`Listener`** + **`Signal`** — standard observer pattern: +- `Listener(callable)` +- `Signal.add(listener)` +- `Signal.emit(data)` +- `Listener.remove()` + +#### asyncio integration (verified working) + +pywayland bundles vanilla upstream libwayland 1.26.0 (verified by `objdump` +of the bundled `.so` — only standard `wl_*` exports, NO Greenfield patch +additions). To integrate with asyncio: + +```python +import asyncio, cffi, os, pywayland +from pywayland.server import Display + +# 5-line cffi shim for wl_event_loop_get_fd (pywayland doesn't expose it) +extras_ffi = cffi.FFI() +extras_ffi.cdef(""" + int wl_event_loop_get_fd(void *loop); + void *wl_display_get_event_loop(void *display); + int wl_event_loop_dispatch(void *loop, int timeout); + void wl_display_flush_clients(void *display); +""") +_pywl_libs = os.path.join( + os.path.dirname(os.path.dirname(pywayland.__file__)), + 'pywayland.libs' +) +_lib = extras_ffi.dlopen([ + os.path.join(_pywl_libs, f) for f in os.listdir(_pywl_libs) + if 'libwayland-server' in f +][0]) + +display = Display() +display.add_socket() # returns "wayland-N" +display.init_shm() + +_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr)) +_display_voidp = extras_ffi.cast('void *', _ptr_int) +_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp) +_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp) + +async def main(): + loop = asyncio.get_running_loop() + def on_ready(): + _lib.wl_event_loop_dispatch(_loop_voidp, 0) + display.flush_clients() + loop.add_reader(_wl_fd, on_ready) + await asyncio.Future() # run forever + +asyncio.run(main()) +``` + +This pattern is used in production by **qtile's Wayland backend** +(`libqtile/backend/wayland/core.py`). The qtile code is GPL-2.0 so +**don't copy** — but the architectural pattern is well-documented and +reusable. + +#### Does pywayland need a patched libwayland-server? + +**NO** — this is one of pywayland's biggest advantages over Greenfield. + +- pywayland bundles and links against **vanilla upstream libwayland 1.26.0** + (no patches needed for basic use). +- Greenfield patches upstream libwayland-server with 9+ additions (in + `native/wayland/src/wayland-server/changes.diff`, MIT-licensed) to add + wire-message interception hooks: `wl_client_set_wire_message_cb`, + `wl_client_set_registry_created_cb`, `wl_client_set_sync_done_cb`, etc. + +**Should we apply Greenfield's `changes.diff` to our libwayland-server?** + +- **Optional, recommended** for the fast-sync optimization (skip browser + round-trip when no prior browser-bound request occurred). +- The patches are MIT-licensed, so applying them to a vendored + libwayland-server is fine. +- Without the patches: per-resource dispatchers work fine but with slightly + more overhead (every request goes through the full Python dispatch path). +- With the patches: wire-message interception hook lets us route requests + to `{native: true}` (handle in proxy) or `{browser: true}` (forward to + browser compositor) before they hit the per-resource dispatcher. + +#### Comparison to Greenfield's `wayland-server.ts` + `wayland-server-addon.node` + +| Greenfield function | pywayland equivalent | Status | +|---|---|---| +| `createDisplay`, `destroyDisplay`, `addSocketAuto` | `Display()`, `Display.destroy()`, `Display.add_socket()` | ✅ | +| `flush`, `getCredentials`, `initShm` | `Display.flush_clients()`, `Client.get_credentials()`, `Display.init_shm()` | ✅ | +| `getFd`, `dispatchRequests` (libuv integration) | NOT exposed — 5-line cffi shim | ✅ (shimmable) | +| `setWireMessageCallback`, `setWireMessageEndCallback` | NO EQUIVALENT in upstream — Greenfield patch | Use per-resource dispatchers + apply changes.diff if you want the hook | +| `setRegistryCreatedCallback`, `setSyncDoneCallback`, `setBufferCreatedCallback` | NO EQUIVALENT in upstream — Greenfield patch | Reimplement by intercepting corresponding requests via per-resource dispatchers | +| `setClientDestroyedCallback` | `Client.add_destroy_listener(Listener(cb))` | ✅ (different style) | +| `destroyClient`, `sendEvents`, `createWlResource` | `Client.destroy()`, `_post_event()`, `WlFoo.resource_class(client, version, id)` | ✅ | +| `initDrm`, `setupXWayland` | NO EQUIVALENT — use vendored Greenfield C files (MIT) | ✅ (vendored) | +| `createMemoryMappedFile`, `makePipe` | `pywayland.utils.AnonymousFile(size)`, `os.pipe()` | ✅ | + +## Wire format (REUSED, same as Greenfield) + +Standard upstream Wayland, byte-for-byte. See +`../greenfield-analysis/02-wayland-protocol.md` for the full byte layout. + +## Object lifecycle (REUSED semantics, different API) + +pywayland uses standard libwayland object lifecycle: +- Client-allocated IDs (1-indexed, recycled via `wl_display.delete_id`) +- Server-allocated IDs (from `0xff000000`) +- `Resource.add_destroy_listener(Listener(cb))` for cleanup +- `Client.add_destroy_listener(Listener(cb))` for client disconnect + +## File descriptor handling (REUSED approach) + +Same ProxyFD JSON descriptor as Greenfield: +``` +ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string } +``` + +In Python: +- `os.pipe()` for pipes +- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon) +- Subprocess + WebSocket endpoints for the FD-over-WebSocket workaround (same 9 endpoints) + +## Why this works + +The protocol layer is fundamentally just: +1. XML describing the protocol surface (upstream MIT) +2. Code generators that produce typed bindings (MIT CLIs from Greenfield + pywayland's scanner) +3. A runtime that marshalls/unmarshals wire bytes (MIT @gfld/common + Apache-2.0 pywayland) + +None of these layers require AGPL code. The AGPL parts of Greenfield +(`@gfld/compositor`, `@gfld/compositor-proxy`) are the **consumers** of +these protocol layers — they implement the actual Wayland semantics +(Surface state, Seat, Keyboard, etc.). We replace those consumers with +custom clean-room implementations. + +## File reference + +### Reused from Greenfield (MIT) + +| Path | Role | +|---|---| +| `protocol/wayland.xml` | Upstream Wayland core protocol | +| `protocol/xdg-shell.xml` | Stable xdg-shell protocol | +| `protocol/wayland-drm.xml` | Legacy wl_drm for DMA-BUF | +| `protocol/web-bitmapbuf-unstable-v1.xml` | Greenfield's custom extension for browser JS apps | +| `libs/common/src/Connection.ts` | Wire format codec (REUSED) | +| `libs/common/src/index.ts` | Public exports | +| `libs/compositor-protocol/src/westfield-runtime-server.ts` | Server-side runtime (REUSED) | +| `libs/compositor-protocol/src/protocol/` | Generated server bindings (REUSED) | +| `libs/client-protocol/src/westfield-runtime-client.ts` | Client-side runtime (REUSED) | +| `libs/client-protocol/src/protocol/` | Generated client bindings (REUSED) | +| `libs/client-generator/{ProtocolParser,ProtocolArguments,index}.mjs` | Browser client bindings generator (REUSED) | +| `libs/compositor-generator/{ProtocolParser,ProtocolArguments,index}.mjs` | Browser server bindings generator (REUSED) | +| `greenfield/packages/compositor-proxy/native/wayland/src/wayland-server/changes.diff` | Patches to libwayland-server (MIT, optional but recommended) | + +### Python equivalents (Apache-2.0 / MIT) + +| Component | Role | +|---|---| +| pywayland | Python bindings to libwayland-server | +| `pywayland.server.Display` | wl_display wrapper | +| `pywayland.server.EventLoop` | wl_event_loop wrapper | +| `pywayland.server.Client` | wl_client wrapper | +| `pywayland.server.Resource` | wl_resource base class | +| `pywayland.server.Global` | wl_global base class | +| `pywayland.server.Listener`, `Signal` | Observer pattern | +| `pywayland.protocol.*` | Auto-generated protocol bindings (60+ interfaces) | +| `pywayland.utils.AnonymousFile` | memfd_create wrapper (replaces Greenfield's createMemoryMappedFile) | diff --git a/03-html5-rendering.md b/03-html5-rendering.md new file mode 100644 index 0000000..02a5b4b --- /dev/null +++ b/03-html5-rendering.md @@ -0,0 +1,290 @@ +# 03 — HTML5 Rendering Pipeline (Custom WebGL2 + Workers + OffscreenCanvas) + +This chapter describes the **custom browser-side rendering pipeline** that +replaces Greenfield's AGPL `@gfld/compositor`. It's the largest single +component of the MIT approach (~3,000 LOC TypeScript, ~5 dev-weeks). + +## Critical: clean-room requirement + +**You CANNOT read Greenfield's `@gfld/compositor` source and reimplement it.** +Even "inspired-by" code is risky under the *Computer Associates v. Altai* +Abstraction-Filtration-Comparison test. AGPL §13 network-use clause would +force you to open-source your entire Odoo fork if a court finds your +compositor is an AGPL derivative. + +**Right approach**: True clean-room. +- Person A (spec author, may read AGPL source) writes a behavioral spec + (inputs/outputs/edge cases, NO code). +- Person B (implementer, never opens `packages/compositor/src/`) implements + from spec + MIT references only: + - Wayland protocol spec (https://wayland.freedesktop.org/docs/html/) + - wlroots C source (MIT, https://gitlab.freedesktop.org/wlroots/wlroots) + - Weston C source (MIT, https://gitlab.freedesktop.org/wayland/weston) + - Smithay Rust source (MIT, https://github.com/Smithay/smithay) + - WebGL2/WebCodecs/KCP MDN docs + +Maintain `SPEC.md` and `CLEAN_ROOM_LOG.md` as legal-defensibility artifacts. + +## Modernization wins over Greenfield + +Greenfield's browser compositor uses: +- WebGL1 (no OffscreenCanvas, no WebGL2) +- Main-thread WebGL (jank on heavy frames) +- WebCodecs with hardware decode explicitly disabled (`// FIXME hardware decoding frame offset is wrong`) +- FFmpeg Wasm fallback (GPL — we can't use it) + +Our MIT approach modernizes to: +- **WebGL2** — uniform buffer objects, transform feedback, simplified YUV→RGB shaders +- **OffscreenCanvas + Web Workers** — render in a worker, never block main thread +- **WebCodecs with hardware decode enabled** — fix Greenfield's "frame offset is wrong" bug +- **OpenH264 BSD Wasm fallback** — replaces Greenfield's GPL FFmpeg Wasm +- **Adaptive bitrate** — consume `EncoderFeedback` data Greenfield produces but doesn't use +- **Real touch input** — Greenfield has a 46-line stub that never fires + +## Proposed file structure + +``` +packages/compositor/ # PROPRIETARY, clean-room TS +├── SPEC.md # clean-room behavioral spec +├── CLEAN_ROOM_LOG.md # who read AGPL, who didn't, when +├── src/ +│ ├── index.ts # public API: createCompositorSession() +│ ├── core/ +│ │ ├── Session.ts # top-level session +│ │ ├── Compositor.ts # wl_compositor global +│ │ └── Globals.ts # Wayland globals registry +│ ├── surface/ # ~1,200 LOC, 3 weeks +│ │ ├── Surface.ts # wl_surface, double-buffered state +│ │ ├── SurfaceState.ts # state struct +│ │ ├── SurfaceRole.ts # role abstraction +│ │ ├── SurfaceTree.ts # parent/child tree +│ │ ├── Subsurface.ts # wl_subsurface +│ │ ├── Subcompositor.ts # wl_subcompositor global +│ │ ├── Region.ts # wl_region (pixman-WASM) +│ │ └── FrameCallback.ts # wl_callback +│ ├── render/ # ~1,800 LOC, 3 weeks +│ │ ├── Renderer.ts # 4-phase render pipeline +│ │ ├── Scene.ts # one per output (canvas) +│ │ ├── View.ts # positioned surface for compositing +│ │ ├── Output.ts # wl_output, OffscreenCanvas binding +│ │ ├── RenderState.ts # per-view-per-scene texture + transform +│ │ ├── Texture.ts # WebGL2 texture wrapper +│ │ ├── ShaderProgram.ts # WebGL2 program wrapper +│ │ ├── Loop.ts # requestAnimationFrame integration +│ │ └── shaders/ +│ │ ├── YUVA2RGBA.glsl # BT.601 YUVA → RGBA +│ │ ├── YUV2RGB.glsl # YUV → RGB (no alpha) +│ │ ├── RGBandA2RGBA.glsl # RGB+Alpha → RGBA +│ │ └── SceneShader.glsl # final pass: textured quads +│ ├── buffer/ # ~600 LOC, 1.5 weeks +│ │ ├── BufferContents.ts # buffer abstraction (mimeType) +│ │ ├── BufferImplementation.ts # interface +│ │ ├── Shm.ts # wl_shm, BMP-header hack +│ │ ├── ImageBitmapBuffer.ts # web-bitmapbuf fast path +│ │ └── EncodedFrameBuffer.ts # H.264/PNG from proxy +│ ├── transport/ # ~3,500 LOC, 4 weeks +│ │ ├── Channel.ts # ARQChannel + SimpleChannel +│ │ ├── kcp.ts # ported fresh from MIT C ref +│ │ ├── ReconnectingWebSocket.ts # vendored from pladaria (MIT) +│ │ ├── BufferStream.ts # serial → frame promise resolution +│ │ ├── StreamingBuffer.ts # per-surface encoded-frame buffer +│ │ ├── EncodedFrame.ts # wire format parser +│ │ ├── EncoderFeedback.ts # 4-byte decode feedback, 1 Hz +│ │ ├── RemoteInputOutput.ts # FD-over-WebSocket for native apps +│ │ ├── RemoteAppLauncher.ts # channel setup per app +│ │ ├── RemoteOutOfBandChannel.ts # OOB opcode dispatch +│ │ └── decoders/ +│ │ ├── FrameDecoder.ts # interface +│ │ ├── WebCodecsFrameDecoder.ts # primary: VideoDecoder (HW-accelerated) +│ │ └── OpenH264FrameDecoder.ts # fallback: OpenH264 BSD Wasm +│ ├── input/ # ~2,500 LOC, 3 weeks +│ │ ├── Seat.ts # wl_seat global +│ │ ├── Keyboard.ts # wl_keyboard, keymap, modifiers +│ │ ├── Pointer.ts # wl_pointer, motion, buttons, axis +│ │ ├── Touch.ts # wl_touch (REAL implementation, not stub) +│ │ ├── Xkb.ts # xkbcommon WASM wrapper +│ │ ├── InputQueue.ts # event coalescing + RAF dispatch +│ │ ├── InputOutput.ts # FD abstraction +│ │ └── events.ts # KeyEvent, ButtonEvent, AxisEvent +│ ├── roles/ # ~1,800 LOC, 3 weeks (HIGH RISK) +│ │ ├── XdgWmBase.ts # xdg_wm_base global, ping +│ │ ├── XdgSurface.ts # xdg_surface, configure/ack +│ │ ├── XdgToplevel.ts # xdg_toplevel, window state +│ │ ├── XdgPopup.ts # xdg_popup, transient windows, grabs +│ │ ├── XdgPositioner.ts # popup positioning math +│ │ ├── Shell.ts # legacy wl_shell (optional) +│ │ └── ShellSurface.ts # legacy wl_shell_surface +│ ├── data/ # ~600 LOC, 1.5 weeks +│ │ ├── DataDeviceManager.ts # wl_data_device_manager +│ │ ├── DataSource.ts # wl_data_source +│ │ ├── DataOffer.ts # wl_data_offer +│ │ ├── WaylandDataSource.ts # native source wrapper +│ │ ├── PrimarySelectionDeviceManager.ts # NEW: zwp_primary_selection_v1 +│ │ └── PrimarySelectionSource.ts # NEW: middle-click paste +│ ├── desktop/ # ~800 LOC, 1.5 weeks +│ │ ├── Desktop.ts # Floating vs AlwaysFullScreen +│ │ ├── DesktopSurface.ts # abstraction +│ │ ├── FloatingDesktopSurface.ts # default mode +│ │ └── FullscreenDesktopSurface.ts # kiosk mode +│ ├── math/ # ~500 LOC, 0.5 weeks +│ │ ├── Mat4.ts # 4×4 matrix +│ │ ├── Vec4.ts # 4-vector +│ │ ├── Point.ts # 2D point +│ │ ├── Size.ts # 2D size +│ │ ├── Rect.ts # 2D rectangle +│ │ └── Transformations.ts # affine transforms +│ ├── userapi/ # ~120 LOC, 0.5 weeks +│ │ └── UserShellApi.ts # OWL shell integration seam +│ ├── browser/ # ~750 LOC, 2 weeks +│ │ ├── input.ts # DOM keyboard events +│ │ ├── pointer.ts # DOM pointer events +│ │ ├── dnd.ts # DnD bridge +│ │ ├── selection.ts # clipboard bridge (native → DOM) +│ │ ├── text-selection.ts # Firefox fallback (text only) +│ │ ├── attributes.ts # canvas attributes +│ │ └── capabilities.ts # browser feature detection +│ └── (optional, Phase 6) +│ ├── webapp/ # ~500 LOC, 1 week (WASM web apps) +│ │ ├── WebAppLauncher.ts +│ │ ├── WebConnectionHandler.ts +│ │ ├── WebInputOutput.ts +│ │ └── WebBuffersFactory.ts +│ └── xwayland/ # ~5,000 LOC, 8 weeks (X11 apps) +│ ├── XWindowManager.ts +│ ├── XWindow.ts +│ ├── XWindowFrame.ts +│ └── ... (uses @gfld/xtsb MIT) +└── test/ + ├── unit/ # ~1,000 LOC, Jest unit tests + ├── protocol/ # ~2,000 LOC, Weston test clients ported to TS + └── e2e/ # ~500 LOC, Playwright E2E with real Linux apps +``` + +## Modern rendering pipeline (WebGL2 + OffscreenCanvas + Workers) + +``` +┌─────────────────────────────────────────────────────────────────────────┐ +│ MAIN THREAD (Odoo OWL shell + DOM events) │ +│ │ +│ element owned by OWL component │ +│ │ │ +│ │ canvas.transferControlToOffscreen() → OffscreenCanvas │ +│ │ │ +│ │ postMessage({canvas: offscreenCanvas}, [offscreenCanvas]) │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────┐ │ +│ │ Render Worker (WebGL2 + OffscreenCanvas) │ │ +│ │ │ │ +│ │ Receives: │ │ +│ │ - OffscreenCanvas (transferred) │ │ +│ │ - EncodedVideoChunk / ImageBitmap (transferred from decode) │ │ +│ │ - Surface state updates (postMessage) │ │ +│ │ - Frame callback requests (postMessage) │ │ +│ │ │ │ +│ │ Runs the 4-phase render pipeline: │ │ +│ │ PHASE 1 — pixel upload (per view per scene) │ │ +│ │ PHASE 2 — requestAnimationFrame │ │ +│ │ PHASE 3 — draw (per scene: viewport + clear + drawArrays) │ │ +│ │ PHASE 4 — fire frame callbacks │ │ +│ │ │ │ +│ │ Sends: │ │ +│ │ - Frame callback timestamps (postMessage to main thread) │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────────┘ │ +│ │ +│ ┌────────────────────────────────────────────────────────────────┐ │ +│ │ Decode Worker (WebCodecs VideoDecoder, hardware-accelerated) │ │ +│ │ │ │ +│ │ Receives: │ │ +│ │ - EncodedVideoChunk (transferred from transport) │ │ +│ │ │ │ +│ │ Decodes: │ │ +│ │ - VideoDecoder.decode(chunk) │ │ +│ │ - VideoFrame output (zero-copy GPU texture source) │ │ +│ │ │ │ +│ │ Sends: │ │ +│ │ - VideoFrame (transferred to render worker) │ │ +│ │ - Decode duration telemetry (for adaptive bitrate) │ │ +│ │ │ │ +│ └────────────────────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +## Why WebGL2 over WebGL1 + +- **Uniform Buffer Objects** — share state between shaders efficiently +- **Transform feedback** — could be used for vertex skinning (not needed for compositor) +- **Simplified YUV→RGB shaders** — `texelFetch` for direct texel access +- **3D textures** — could simplify YUV planar upload (Y, U, V as 3 slices) +- **Better browser support** — 97%+ global browser support per caniuse + +## Why OffscreenCanvas + Web Workers + +- **Never block main thread** — heavy decode + render happens off-thread +- **Smooth UI** — Odoo shell stays responsive during video playback +- **Multi-output scaling** — one worker per output for parallel rendering +- **Recovery** — if render worker crashes, restart without losing Odoo session + +## WebCodecs vs FFmpeg Wasm + +| Aspect | WebCodecs (primary) | OpenH264 Wasm (fallback) | +|---|---|---| +| License | W3C standard (MIT-equivalent) | BSD-2-Clause | +| Hardware decode | Yes (Intel/AMD/NVIDIA) | No (CPU only) | +| Latency | ~5-10ms per frame | ~30-50ms per frame | +| CPU usage | Low (offloaded to GPU) | High (full decode on CPU) | +| Browser support | Chrome 94+, Edge 94+, Safari 16+, Firefox 130+ | All browsers with Wasm | +| Codecs | H.264, VP9, AV1, H.265 (where supported) | H.264 only | + +**Recommended approach**: Probe `VideoDecoder.isConfigSupported()` at session +startup. If supported, use WebCodecs. Otherwise, fall back to OpenH264 Wasm +(loaded lazily, only when needed). + +## Wire format for encoded frames (REUSED from Greenfield) + +Same wire format as Greenfield (see `../greenfield-analysis/09-hardware-accel.md` +for byte layout). The browser side parses `{contentSerial, mimeType, size, +encodedSize, opaque, alpha?}` and dispatches to the right decoder. + +## Engineering observations + +1. **Vestigial damage tracking in Greenfield** — `damageRects` are accumulated + but not consumed by the renderer. Our MIT approach should actually use + them for scissoring (per-rect partial repaints). +2. **Multi-output texture cost** — each view has one texture per scene. Our + approach should share textures across scenes where possible. +3. **SHM xrgb8888 BMP-header hack** — cleverly offloads pixel conversion + to the browser's BMP decoder. Worth re-implementing. +4. **Encoder feedback telemetry** — Greenfield produces it but doesn't use + it. Our approach should consume it for adaptive bitrate. +5. **Cursor bypass** — cursor is rendered as CSS `cursor: url(dataURL)`, + not via WebGL. Same approach in our MIT version. +6. **Frame-callback coalescing** — multiple `wl_surface.frame` requests + within one render pass fire on the same RAF tick. Same in our version. +7. **Subsurface sync cascade** — three-layer mechanism (cachedState, + commitCache, effectiveSync). Hard to get right; clean-room from spec. + +## Effort estimate + +| Component | LOC | Weeks (1 dev) | Risk | +|---|---|---|---| +| Surface state stack | 1,200 | 3 | Med | +| Rendering pipeline (WebGL2 + OffscreenCanvas) | 1,800 | 3 | Med | +| Buffers (SHM + BMP hack) | 600 | 1.5 | Low | +| Transport (incl. fresh KCP port from MIT C ref) | 3,500 | 4 | **High** | +| Input (seat/kbd/ptr/touch/xkb) | 2,500 | 3 | Med | +| Roles (xdg-shell state machine) | 1,800 | 3 | **High** | +| Data device / clipboard / DnD + primary selection | 600 | 1.5 | Low | +| Desktop policy | 800 | 1.5 | Low | +| Math | 500 | 0.5 | Low | +| User shell API | 120 | 0.5 | Low | +| Browser glue | 750 | 2 | Med | +| H.264 decoder (WebCodecs + OpenH264 BSD) | 500 | 1 | Low | +| Core (Session/Globals) | 200 | 0.5 | Low | +| **MVP TOTAL** | **~13,400** | **~25 weeks** | | +| XWayland (optional Phase 6) | 5,000 | 8 | High | +| WASM web app (optional) | 500 | 1 | Low | +| Test suite + hardening | — | 3 | — | +| **GRAND TOTAL** | **~18,900** | **~30-36 weeks** | | + +With 2 senior devs in parallel: **~15-18 weeks for MVP**. diff --git a/04-transport.md b/04-transport.md new file mode 100644 index 0000000..7baa7d6 --- /dev/null +++ b/04-transport.md @@ -0,0 +1,332 @@ +# 04 — Transport Layer (Python aiohttp + KCP, browser side clean-room) + +This chapter covers the transport layer for the MIT approach: how Wayland +wire traffic flows between the Python proxy and the browser compositor. + +## Recommended Python transport stack + +| Layer | Choice | Why | +|---|---|---| +| **WebSocket framework** | **aiohttp.web** (v3.10+) | Best binary perf (300-600 Mbps on CPython, 2× on PyPy), native backpressure via `await ws.drain()`, single dependency, asyncio-native — eliminates the ASGI indirection tax of Starlette/uvicorn and the framework overhead of FastAPI | +| **KCP library** | **`kcp` PyPI package** (RealistikDash/kcp.py, v0.1.6, MIT, Cython wrapper over upstream `ikcp.c`) | API (`enqueue`/`receive`/`get_all_received`/`update`/`outbound_handler`) maps cleanly to Greenfield's `Kcp` class. ~1500 LoC port of Greenfield's TS KCP would buy nothing — the C-backed package is faster and battle-tested | +| **libwayland binding** | **pywayland 0.4.19** (Apache-2.0, CFFI) + **cffi escape hatch for `wl_event_loop_get_fd`** | pywayland's `EventLoop` wrapper omits `wl_event_loop_get_fd()`. Fix: call `pywayland.lib.wl_event_loop_get_fd(loop._ptr)` directly via cffi, then `asyncio.get_running_loop().add_reader(wl_fd, dispatch_wl)`. Replaces Greenfield's entire `proxy-poll-addon.node` C addon with ~10 lines of Python | +| **Multi-process model** | **`multiprocessing.Process` (fork start method) per Odoo session** | Matches Greenfield's per-`compositorSessionId` forking. FD passing of WS upgrade socket via `multiprocessing.reduction.send_handle`/`recv_handle` (uses SCM_RIGHTS on Linux). Preserves the SIGHUP/SIGKILL escalation semantics | +| **Encoder threading** | **`threading.Thread` + GMainLoop + `loop.call_soon_threadsafe`** | Mirrors Greenfield's `napi_threadsafe_function` pattern. GIL released during C encoding calls (GStreamer). Reuse Greenfield's MIT C files for dmabuf/egl via cffi — preserves all EGL/GBM/GStreamer pipelines | +| **Binary message format** | **Keep Greenfield's verbatim** | Browser-side TS unchanged; no need for a version field until v2. KCP gives message boundaries | +| **ReconnectingWebSocket** | **Reuse Greenfield's vendored copy** (Pedro Ladaria, MIT, 514 lines TS) | Browser-side; copy with attribution into Odoo OWL shell assets | + +## Channel types (same as Greenfield) + +| Type | Class | KCP? | Purpose | +|------|-------|------|---------| +| PROTOCOL | ARQChannel | Yes | Wayland wire protocol, per `wl_client` | +| FRAME | ARQChannel | Yes | H.264/PNG encoded surface content, per `wl_surface` | +| XWM | ARQChannel | Yes | XWayland window manager protocol | +| FEEDBACK | SimpleChannel | No | 4-byte telemetry (refreshInterval + decode avg), 1 Hz | + +## WebSocket endpoints (9 paths, same as Greenfield) + +``` +/signal # JSON control plane (per-app) +/channel # binary data channels (multiplexed) +/mkfifo # create a POSIX pipe, return 2 ProxyFDs +/mkstemp-mmap # create a memfd-backed shm, return 1 ProxyFD +/read-fd # read bytes from a ProxyFD (one-shot) +/read-fd-as-stream # read bytes from a ProxyFD (streaming) +/write-fd-as-stream # write bytes to a ProxyFD (streaming) +/close-fd # close a ProxyFD +/request-keyframe # ask the encoder for a keyframe +``` + +All endpoints require JWT auth (HS256, validated by `pyjwt`). + +## asyncio + libwayland integration (the riskiest piece) + +**Verified working** in the sandbox: + +```python +import asyncio, cffi +from pywayland.server import Display + +extras_ffi = cffi.FFI() +extras_ffi.cdef(""" + int wl_event_loop_get_fd(void *loop); + void *wl_display_get_event_loop(void *display); + int wl_event_loop_dispatch(void *loop, int timeout); + void wl_display_flush_clients(void *display); +""") +_lib = extras_ffi.dlopen("libwayland-server.so.0") # or pywayland's bundled + +display = Display() +display.add_socket() +display.init_shm() + +_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr)) +_display_voidp = extras_ffi.cast('void *', _ptr_int) +_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp) +_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp) + +async def main(): + loop = asyncio.get_running_loop() + def on_ready(): + _lib.wl_event_loop_dispatch(_loop_voidp, 0) + display.flush_clients() + loop.add_reader(_wl_fd, on_ready) + await asyncio.Future() # run forever + +asyncio.run(main()) +``` + +This pattern is used in production by **qtile's Wayland backend** +(`libqtile/backend/wayland/core.py`). qtile is GPL-2.0 so don't copy code, +but the architectural pattern is documented and reusable. + +## Process model — three layers (same as Greenfield) + +### Layer 1: Main process (HTTP + WS hub) + +Single Python process listens on `:8081`. Per Odoo session, it forks +one `SessionProcess` child — one child per `compositor_session_id`, **not** +per app. Multiple apps in the same browser tab share one child. + +WebSocket upgrades are handed off to the matching child via +`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux). + +### Layer 2: Session process + +The forked child. Receives IPC messages: + +- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher +- `launch_app` — spawns the actual app binary via `subprocess.Popen` + bubblewrap +- `ws_upgrade` — hands WS to the dispatcher + +On `session.close()` the process exits. + +### Layer 3: Per-client session + +For each new `wl_client` (one per Wayland connection), creates a +`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD +JSON. Routes requests via per-resource dispatchers. + +## FD virtualization (same approach as Greenfield) + +`ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }` + +In Python: +- `os.pipe()` for pipes +- `os.memfd_create(name, flags=0)` for shared memory (Python 3.8+ builtin, + replaces Greenfield's `createMemoryMappedFile` native addon) +- `asyncio.StreamReader` / `StreamWriter` for streaming reads/writes over + separate WebSocket endpoints + +## Reconnection & lifecycle (same as Greenfield) + +### Browser side + +Vendored `ReconnectingWebSocket` (MIT): +- `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s` +- `minUptime=5s`, `connTimeout=4s`, `retries=∞` +- Close codes 4000-4999 tear down KCP; lower codes auto-reconnect + +### Python side + +No auto-reconnect (it's the server). Channel object lives on, waiting for +new WS upgrade with matching `id` + `key`. Outbound messages queue and +flush on reconnect. + +### App kill timers + +When browser disconnects from `/signal` WebSocket: +- 10-minute SIGHUP grace period +- After 10 min: SIGTERM, then SIGKILL after another 10s +- Close code 1001 → immediate SIGHUP (intentional tab close) +- Close code 4001 → immediate SIGTERM (force quit) + +## JWT auth (NEW vs Greenfield) + +Every WebSocket connection must include a valid JWT (HS256, 15-min expiry): + +``` +Authorization: Bearer +``` + +Or as a query param: `?token=` + +JWT payload: +```json +{ + "iss": "odoo.example.com", + "aud": "compositor-proxy", + "sub": 42, // Odoo user ID + "db": "production", // Odoo database name + "company_ids": [1, 3], + "scope": "compositor", + "compositor_session_id": "uuid-here", + "iat": 1736000000, + "exp": 1736000900 +} +``` + +Validated by the proxy using `compositor.jwt_secret` from Odoo's +`ir.config_parameter` (shared via env var at proxy startup). + +## ASCII diagram + +``` + PYTHON HOST (Linux) BROWSER (Odoo OWL shell) +========================== ======================== + +┌─────────────────────────┐ ┌──────────────────────────────┐ +│ Native app (gtk4-demo) │ │ Odoo web client │ +│ (sandboxed in bwrap) │ │ (Odoo 20 OWL shell addon) │ +└───────────┬─────────────┘ └──────────────┬───────────────┘ + │ WAYLAND_DISPLAY=wayland-0 │ + │ Unix socket + SCM_RIGHTS FD passing │ DOM events → input + ▼ │ WebSocket API +┌──────────────────────────────────────────┐ │ +│ SessionProcess (fork'd Python child) │ │ +│ ┌────────────────────────────────────┐ │ │ +│ │ pywayland (Apache-2.0) │ │ │ +│ │ + 5-line cffi shim │ │ │ +│ │ - wl_display + wl_client │ │ │ +│ │ - asyncio.add_reader for wl_fd │ │ │ +│ └─────────────┬──────────────────────┘ │ │ +│ │ wire msg + FDs │ │ +│ ▼ │ │ +│ ┌─────────────────────────────────────┐ │ │ +│ │ Per-resource dispatchers │ │ │ +│ │ (auto-generated by pywayland) │ │ │ +│ │ - route to {native, browser} │ │ │ +│ │ - serialize FDs as ProxyFD JSON │ │ │ +│ │ - fast-sync optimization │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +│ │ outbound buffer │ │ +│ ▼ │ │ +│ ┌─────────────────────────────────────┐ │ │ +│ │ ARQChannel (KCP-wrapped) │ │ │ +│ │ - kcp PyPI package (MIT) │ │ │ +│ │ - MTU=64000, snd_wnd=128, rcv=1024 │ │ │ +│ │ - kcp.update() every 100ms │ │ │ +│ │ - PROTOCOL (per wl_client) │ │ │ +│ │ - FRAME (per wl_surface) │ │ │ +│ │ - XWM (per XWayland) │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +│ ┌─────────────┴───────────────────────┐ │ │ +│ │ SimpleChannel (FEEDBACK, no KCP) │ │ │ +│ │ 4 bytes: refreshIntervalMs + │ │ │ +│ │ avgDecodeDurationMs │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +│ ┌─────────────▼───────────────────────┐ │ │ +│ │ NativeAppContext │ │ │ +│ │ - signaling WebSocket (JSON) │ │ │ +│ │ - kill timers: SIGHUP@10min, │ │ │ +│ │ SIGKILL@+10s │ │ │ +│ │ - bubblewrap + seccomp sandbox │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +│ ┌─────────────▼───────────────────────┐ │ │ +│ │ SessionController │ │ │ +│ │ - aiohttp.web.Application │ │ │ +│ │ - 9 endpoints │ │ │ +│ │ - JWT validation (pyjwt) │ │ │ +│ │ - perMessageDeflate: false │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +│ ┌─────────────▼───────────────────────┐ │ │ +│ │ Encoder thread (GStreamer + GLib │ │ │ +│ │ MainLoop in separate thread) │ │ │ +│ │ - vaapih264enc / nvh264enc / │ │ │ +│ │ openh264enc (NOT x264enc — GPL) │ │ │ +│ │ - asyncio.run_coroutine_threadsafe │ │ │ +│ │ for cross-thread marshalling │ │ │ +│ └─────────────┬───────────────────────┘ │ │ +└────────────────┼─────────────────────────────┘ │ + │ │ + │ ┌──────────────────────────┐ │ + │ │ main.py (HTTP+WS hub) │ │ + │ │ - socket.pause() + IPC │ │ + │ │ send to session proc │ │ + │ │ - multiprocessing.fork │ │ + │ │ - JWT issuance delegated │ │ + │ │ to Odoo backend │ │ + │ └────────────┬─────────────┘ │ + │ │ │ + │ ◄──── 1 WebSocket per channel ────► │ + │ ◄── 1 signaling WS per app ──► │ + │ ◄── ad-hoc WebFD streams ─► │ + │ │ │ + │ ▼ │ + │ ┌────────────────────────────────────────┐ │ + │ │ ReconnectingWebSocket (vendored MIT) │ │ + │ │ min 1-5s, factor 1.3, max 10s │ │ + │ │ minUptime 5s, connTimeout 4s │ │ + │ │ + Authorization: Bearer │ │ + │ └────────────┬───────────────────────────┘ │ + │ ▼ │ + │ ┌────────────────────────────────────────────┐ │ + │ │ ARQChannel / SimpleChannel (clean-room TS) │ │ + │ │ snd_wnd=1024, rcv_wnd=128 │ │ + │ │ kcp.input → peekSize/recv → onMessage │ │ + │ └────────────┬───────────────────────────────┘ │ + │ ▼ │ + │ ┌────────────────────────────────────────────┐ │ + │ │ Custom Browser Compositor (clean-room TS) │ │ + │ │ - WebCodecs VideoDecoder (HW-accelerated)│ │ + │ │ - WebGL2 + OffscreenCanvas in Worker │ │ + │ │ - Surface/View/Output/Seat/Keyboard/... │ │ + │ └────────────┬───────────────────────────────┘ │ + │ │ input events flow back via the │ + │ │ PROTOCOL channel as ordinary │ + │ │ Wayland requests (same framing) │ + └──────────────────┘ │ + Browser DOM ◄─┘ +``` + +## Effort estimate (Python transport) + +| Component | LoC | Dev-days | Risk | +|---|---|---|---| +| aiohttp WS server (9 endpoints + HTTP launcher) | 600 | 3 | Low | +| `Channel`/`ARQChannel`/`SimpleChannel` port | 250 | 1.5 | Low | +| `kcp.py` integration adapter | 100 | 0.5 | Low | +| `NativeAppContext` port (signaling, escalation) | 350 | 2 | Low | +| `ProxyInputOutput` port (FD workaround) | 300 | 2 | Medium | +| libwayland event-loop integration | 80 | 1 | **Medium-High** (under-documented) | +| Multiprocessing scaffolding + FD passing | 200 | 1.5 | Medium | +| GStreamer encoder thread + marshalling | 250 | 2 | Medium | +| Reconnection & lifecycle | 150 | 1 | Low | +| Tests (pytest + pytest-asyncio) | 600 | 3 | Low | +| **Total** | **2880** | **~17** | | + +## Top risks & mitigations + +1. **Highest risk**: libwayland ↔ asyncio event-loop integration (under-documented). + **Mitigation**: Spend 1 day on a standalone prototype (the 10-line script + above) before committing. +2. **Medium risk**: FD passing via `multiprocessing.reduction`. Fallback: + single-process with asyncio concurrency. +3. **Medium risk**: GStreamer threading. Mitigation: reuse Greenfield's + MIT C files via cffi — only the marshalling needs Python. + +## Dependency manifest + +``` +# requirements.txt +aiohttp>=3.10,<4 # WebSocket server + HTTP +kcp>=0.1.6 # Cython KCP bindings (RealistikDash) +pywayland>=0.4.19 # libwayland CFFI wrapper (Apache-2.0) +cffi>=1.16 # for wl_event_loop_get_fd escape hatch +pygobject>=3.48 # GStreamer Python bindings +pyjwt>=2.8 # JWT validation +``` + +System packages (Debian): +``` +libwayland-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev +libgstreamer-plugins-bad1.0-dev libgbm-dev libdrm-dev libegl-dev +gstreamer1.0-vaapi # Intel/AMD hardware encoding +libnvidia-encode1 # NVIDIA hardware encoding (if applicable) +bubblewrap libseccomp-dev +python3-dev build-essential pkg-config +``` + +**DO NOT INSTALL**: `gstreamer1.0-plugins-ugly` (provides x264enc, GPL) or +`gstreamer1.0-libav` (LGPL source but GPL binary on Debian). diff --git a/05-input-handling.md b/05-input-handling.md new file mode 100644 index 0000000..4c511f8 --- /dev/null +++ b/05-input-handling.md @@ -0,0 +1,347 @@ +# 05 — Input Handling (Python xkbcommon + Browser WASM) + +This chapter covers input handling in the MIT approach: keyboard (with +xkbcommon), pointer, touch, focus management, and how input events travel +between browser and native apps. + +## Architecture (same as Greenfield, but with improvements) + +The browser is the source of truth for the keymap (same as Greenfield). +The keymap is compiled in the browser using @gfld/compositor-wasm's xkbcommon +WASM module, then shipped to the Python proxy as a real native shm fd via +the `/mkstemp-mmap` WebSocket endpoint. The proxy then sends it to native +apps via libwayland-server's SCM_RIGHTS FD passing. + +**Improvements over Greenfield**: +1. **Real touch input** — Greenfield has a 46-line stub that never fires +2. **Primary selection** — middle-click paste, which Greenfield lacks +3. **Adaptive keyboard layout** — easier to switch layouts at runtime + +## Browser side — Custom TypeScript (clean-room) + +### Seat abstraction + +Same architecture as Greenfield (clean-room from spec + Wayland docs): +- `Seat.ts` — `wl_seat` global, multi-seat support, focus management +- `Keyboard.ts` — `wl_keyboard` impl, keymap, modifiers, key events +- `Pointer.ts` — `wl_pointer` impl, motion, buttons, axis +- `Touch.ts` — `wl_touch` (REAL implementation, not stub) +- `Xkb.ts` — xkbcommon WASM wrapper (REUSED @gfld/compositor-wasm) +- `InputQueue.ts` — event coalescing + RAF dispatch +- `InputOutput.ts` — FD abstraction +- `events.ts` — KeyEvent, ButtonEvent, AxisEvent wrappers + +### xkbcommon WASM (REUSED) + +`@gfld/compositor-wasm` (MIT) bundles: +- libxkbcommon 1.5.0 compiled to WASM via emsdk 3.1.46 +- Embedded xkeyboard-config database +- Exports `init()` + `lib.xkbcommon` singleton with keymap/state functions + +This is REUSED AS-IS. No custom code needed. + +### DOM → Wayland key event translation (clean-room) + +Same approach as Greenfield (verified correct Wayland behavior): +1. `event.code` (e.g. `"KeyA"`) looked up in EvDevKeyCode reverse-name-map → evdev keycode (e.g. 30) +2. `x11KeyCode = evdevKeyCode + 8` (Linux input protocol offset) +3. `capsLock`/`numLock` read from `event.getModifierState(...)` +4. DOM `.key` glyph IGNORED — only physical `.code` used (mirrors evdev) + +### Modifier tracking (REUSED WASM) + +Via @gfld/compositor-wasm's xkbcommon state machine: +1. `Seat.updateModifierState` calls `xkb.keyDown(x11)` or `xkb.keyUp(x11)` +2. `_xkb_state_update_key` invoked internally +3. `notifyModifiers` reads `xkb.modsDepressed` / `modsLatched` / `modsLocked` / `group` +4. `Keyboard.sendModifiers(modsDepressed, modsLatched, modsLocked, group)` sends to native client + +### Touch input (NEW vs Greenfield) + +Greenfield has a 46-line stub for `Touch.ts`. Our MIT approach implements +real touch: + +```typescript +// Pseudocode for Touch.ts (clean-room) +export class Touch { + private touchPoints: Map = new Map(); + + constructor(public seat: Seat) { + // Listen to DOM pointer events with pointerType === 'touch' + document.addEventListener('pointerdown', this.onPointerDown.bind(this)); + document.addEventListener('pointermove', this.onPointerMove.bind(this)); + document.addEventListener('pointerup', this.onPointerUp.bind(this)); + document.addEventListener('pointercancel', this.onPointerCancel.bind(this)); + } + + private onPointerDown(ev: PointerEvent) { + if (ev.pointerType !== 'touch') return; + // Map to wl_touch.down event + this.touchPoints.set(ev.pointerId, { x: ev.offsetX, y: ev.offsetY }); + this.sendDown(ev.pointerId, ev.offsetX, ev.offsetY); + } + + // ... etc +} +``` + +### Keymap FD workaround (REUSED approach) + +Same as Greenfield (see `../greenfield-analysis/05-input-handling.md`): + +1. Browser compiles keymap via @gfld/compositor-wasm's xkbcommon +2. `inputOutput.mkstempMmap(new Blob([utf8(keymapString)]))` opens a + `/mkstemp-mmap` WebSocket, sends the blob +3. Python proxy creates a real native shm fd via `os.memfd_create()` (replaces Greenfield's `createMemoryMappedFile` native addon) +4. Returns `ProxyFD { handle: , type: 'shm', host: baseURL }` +5. Browser sends `wl_keyboard.keymap(format, ProxyFD, size)` +6. Proxy deserializes, looks up native fd via `proxyFDtoNativeFD`, ships via + libwayland-server's SCM_RIGHTS + +## Python side — pywayland + python-xkbcommon + +### python-xkbcommon (MIT) + +The `xkbcommon` PyPI package (by Stephen Early, MIT-licensed) provides +Python bindings to libxkbcommon. Useful if the proxy needs to compile +keymaps (e.g., for XWayland apps that don't speak Wayland natively). + +For most use cases, the proxy doesn't need xkbcommon — it just forwards +the keymap from browser to native apps. The browser does the compilation. + +### Pointer (clean-room) + +Same approach as Greenfield: +- DOM `pointermove` / `pointerdown` / `pointerup` / `wheel` events +- Mapped to `ButtonEvent` / `AxisEvent` wrappers +- `InputQueue` coalesces and dispatches on next RAF +- `DefaultPointerGrab.focus()` runs on every motion: `Renderer.pickView` + walks the view stack front-to-back, returns first view whose + `Surface.isWithinInputRegion(point)` (pixman-WASM region) is true +- On change, `Pointer.setFocus` emits `wl_pointer.leave` + `frame` to old + surface, then `wl_pointer.enter` + `frame` (+ modifiers if keyboard is + grabbed) to new surface + +### Pointer capture / window leave (REUSED approach) + +- `canvas.setPointerCapture` on `pointerdown` +- `pointerleave` calls `notifyKeyboardFocusOut` (saves keyboard focus, + releases depressed keys, sets `needFocusInit=true` for re-sync), + resets `buttonCount=0` +- DnD suppresses this handler mid-drag + +### Cursor images (REUSED approach) + +CSS `cursor: url(data:...) x y, auto`: +1. Native client calls `wl_pointer.set_cursor` +2. Cursor surface commits a buffer +3. `CursorRole` draws the bitmap to a hidden canvas +4. `toDataURL()` produces a data URL +5. Set on `document.body.style.cursor` + +OS composites the cursor. No rotation/scale transforms. + +**NEW**: Use SVG cursor sprites (e.g., from the Phosphor icon set, MIT) +instead of bundled PNGs (Greenfield's bundled PNGs are AGPL-contaminated +since they're inside `@gfld/compositor`). + +### Scroll wheel / touchpad (REUSED approach) + +`WheelEvent.deltaMode`: +- `DOM_DELTA_LINE × 12` — discrete wheel clicks +- `DOM_DELTA_PAGE × surface size` — page-sized scrolls +- `DOM_DELTA_PIXEL ÷ 3` — fine-grained touchpad + +Each non-zero axis → `wl_pointer.axis(time, axis, Fixed)`. Wayland version +≥5 also gets `axis_source(wheel)` + `frame()` (groups events). + +### Linux button codes (REUSED approach) + +DOM `event.button` (0=left, 1=middle, 2=right, 3=Back, 4=forward) mapped +to Linux input protocol button codes via the `linuxInput` table. + +## Focus management (REUSED approach) + +### DOM focus + +The `` owns keyboard focus permanently (`tabIndex=1`). +`Keyboard.setFocus(surface)` emits `wl_keyboard.leave` / `enter` between +native surfaces. Activation is driven by `XdgToplevel` / `ShellSurface` +activation handlers. + +### DOM → Wayland focus + +- Browser DOM `focus` event → `seat.notifyKeyboardFocusIn()` (restores + saved focus, re-syncs modifiers) +- Browser DOM `blur` event → `seat.notifyKeyboardFocusOut()` (saves + focus, releases depressed keys, sets `needFocusInit=true`) + +### Surface-level focus + +- Click on a surface → `XdgToplevel` activation request → + `desktopSurface.activate()` → `gainFocus()` (refcounted via + `focusCount`, fires `surfaceActivationUpdated` event to the user shell + on 0↔1 transitions) +- Popup grabs take keyboard + pointer focus via `seat.popupGrabStart`, + validating the serial against the keyboard/pointer `grabSerial` + +## Input queue (REUSED approach) + +`InputQueue.ts` coalesces events and dispatches them on the next RAF +(`InputQueue.drainQueues`). Batches bursts of input events (e.g. fast +mouse motion) into a single dispatch, avoiding redundant work. + +FIFO per event type (motion, axis, key, button). Each dispatch calls the +appropriate `seat.notify*` method. + +## ASCII event-flow diagram + +``` +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ BROWSER (DOM) │ +│ │ +│ ─┐ │ +│ pointermove/dn/up ───┤── createButtonEventFromMouseEvent() ── ButtonEvent │ +│ wheel ────┤── createAxisEventFromWheelEvent() ── AxisEvent │ +│ keydown / keyup ────┤── createKeyEventFromKeyboardEvent() ── KeyEvent │ +│ pointerdown (touch) ─┤── createTouchEventFromPointerEvent() ── TouchEvent (NEW) │ +│ │ • evdevKeyCode = EvDevKeyCode[event.code] │ +│ │ • x11KeyCode = evdevKeyCode + 8 │ +│ │ • capsLock/numLock = event.getModifierState(...) │ +│ pointerleave / blur ──┤── seat.notifyKeyboardFocusOut() (saves focus, resets mods) │ +│ focus ────┴── seat.notifyKeyboardFocusIn() (restores saved focus) │ +└──────────────────────────────┬──────────────────────────────────────────────────────┘ + │ session.inputQueue.queue{Motion,Axis,Key,Button,Touch} + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ BROWSER COMPOSITOR (InputQueue.drainQueues, scheduled on next animation frame) │ +│ │ +│ seat.notifyMotion / notifyAxis / notifyButton / notifyKey / notifyFrame / notifyTouch│ +│ │ │ +│ ├─► Seat.notifyKey ─► Keyboard.grab.key(event) │ +│ │ └─ Keyboard.sendKey(event) │ +│ │ └─ keyboardResource.key(serial,time,evdev,st) │ +│ │ │ +│ ├─► Seat.notifyButton ─► runButtonBindings + grab.button(event) │ +│ │ └─ Pointer.sendButton │ +│ │ └─ pointerResource.button(serial,time, │ +│ │ linuxInput[btn], state) │ +│ │ │ +│ ├─► Seat.notifyMotion ─► grab.motion ─► Pointer.moveTo / sendMotion │ +│ │ └─ pointerResource.motion(time, sx_f, sy_f) │ +│ │ │ +│ ├─► Seat.notifyAxis ─► grab.axis ─► Pointer.sendAxis │ +│ │ deltaMode→scale: line=×12, page=×surface-size, pixel=÷3 │ +│ │ └─ pointerResource.axis(time, axis, Fixed) │ +│ │ + axisSource(wheel) + frame() if v>=5 │ +│ │ │ +│ ├─► Seat.notifyTouch (NEW) ─► Touch.sendDown/Motion/Up │ +│ │ └─ touchResource.down/motion/up/cancel │ +│ │ │ +│ ├─► Seat.notifyFrame ─► pointerResource.frame() (groups events) │ +│ │ │ +│ └─► Seat.notifyModifiers ─ xkb.keyDown/Up(x11) ─► xkb_state_serialize_mods │ +│ └─ Keyboard.sendModifiers(modsDepressed,Latched, │ +│ Locked,group) │ +│ │ +│ Keymap path (once per WlKeyboardResource): │ +│ Seat.notifyUpdateKeymap(nrmlvo) → createFromNames(rules,model,layout,variant,opt) │ +│ → xkb_context_new + xkb_keymap_new_from_names (@gfld/compositor-wasm, MIT) │ +│ → xkb_keymap_get_as_string ──► keymapString │ +│ Keyboard.sendKeymap(resource): │ +│ inputOutput.mkstempMmap(new Blob([utf8(keymapString)])) ──► ProxyFD │ +│ (Python proxy creates memfd via os.memfd_create()) │ +│ keyboardResource.keymap(xkbV1, ProxyFD, size) │ +└──────────────────────────────┬──────────────────────────────────────────────────────┘ + │ client.connection.onFlush(wireMessages) + │ ─ one big Uint32Array, FDs serialized as ProxyFD JSON + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ WIRE (ARQChannel = KCP-over-WebSocket, custom TS, clean-room) │ +│ Browser ⇄ ReconnectingWebSocket ⇄ Internet ⇄ aiohttp on proxy :8081 │ +│ Message layout (Uint32Array): │ +│ [0] outOfBandOpcode (0 = in-band) │ +│ [1] eventSerial │ +│ [2] = nFds, then for each fd: [len][JSON bytes, 4-byte aligned] │ +│ then concatenated Wayland wire messages │ +│ + Authorization: Bearer │ +└──────────────────────────────┬──────────────────────────────────────────────────────┘ + │ NativeWaylandClientSession.onMessage (Python) + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ PYTHON PROXY (aiohttp + pywayland) │ +│ │ +│ SessionController endpoints: /mkfifo /mkstemp-mmap /read-fd /close-fd │ +│ /read-fd-as-stream /write-fd-as-stream │ +│ /signal /channel /request-keyframe │ +│ │ +│ mkstempMmap(ws): │ +│ receives Blob → os.memfd_create() → write bytes → return fd │ +│ returns ProxyFD { handle: , type: 'shm', host: baseURL } │ +│ │ +│ onWireMessageEvents: │ +│ for each ProxyFD in message: │ +│ nativeFd = webFS.proxyFDtoNativeFD(proxyFD) # identity when host matches │ +│ └─ if foreign host → spawn bridging pipe + WebSocket │ +│ pywayland per-resource dispatcher fires │ +│ # For seat events, dispatcher is a no-op (events just pass through) │ +│ # libwayland-server sends via wl_resource_post_event_array │ +│ flush_clients() │ +└──────────────────────────────┬──────────────────────────────────────────────────────┘ + │ libwayland-server sends wire msg + SCM_RIGHTS FDs + ▼ +┌─────────────────────────────────────────────────────────────────────────────────────┐ +│ NATIVE (Wayland client, sandboxed in bubblewrap) │ +│ wl_display.fd readable → libwayland-client demarshals │ +│ wl_keyboard.keymap(fd) → mmap(fd) → xkb_keymap_new_from_string (native xkbcommon) │ +│ wl_keyboard.key / enter / leave / modifiers → client app │ +│ wl_pointer.enter / motion / button / axis / frame → client app │ +│ wl_touch.down / motion / up / cancel → client app (NEW: actually works) │ +└──────────────────────────────────────────────────────────────────────────────────────┘ +``` + +## Key answers (quick reference) + +| Question | Answer | +|---|---| +| DOM→Wayland key event translation | `event.code` → EvDevKeyCode → evdev keycode → +8 = x11KeyCode. DOM `.key` glyph ignored. | +| Modifier tracking | @gfld/compositor-wasm xkbcommon state machine (REUSED MIT) | +| Focus: DOM ↔ native surfaces | `` owns keyboard focus. `Keyboard.setFocus(surface)` emits leave/enter between native surfaces. | +| Pointer capture / window leave | `canvas.setPointerCapture` on pointerdown. `pointerleave` saves focus + releases keys. | +| Touch event mapping | **NEW: REAL implementation** via `pointerdown`/`pointermove`/`pointerup` with `pointerType === 'touch'`. Greenfield has only a stub. | +| Pointer enter/leave between surfaces | `DefaultPointerGrab.focus()` runs on every motion. `Renderer.pickView` walks view stack. | +| Pointer acceleration / cursor images | No acceleration. CSS `cursor: url(data:...) x y, auto`. NEW: SVG sprites (MIT) instead of bundled PNGs (AGPL). | +| Scroll wheel / touchpad → `wl_pointer.axis` | `WheelEvent.deltaMode`: LINE×12, PAGE×size, PIXEL÷3. v≥5 gets `axis_source(wheel)` + `frame()`. | +| WASM xkbcommon? | Yes — @gfld/compositor-wasm (MIT, REUSED). libxkbcommon 1.5.0 + embedded xkeyboard-config. | +| Keymap FD workaround for WebSocket | Browser → `/mkstemp-mmap` WebSocket → Python proxy `os.memfd_create()` → ProxyFD JSON → libwayland SCM_RIGHTS to native app. | +| Primary selection (middle-click paste)? | **NEW: implemented** via `zwp_primary_selection_device_manager_v1` (Greenfield lacks). | + +## File reference + +| File (custom, clean-room) | Role | +|---|---| +| `packages/compositor/src/input/Seat.ts` | `wl_seat` global | +| `packages/compositor/src/input/Keyboard.ts` | `wl_keyboard` impl | +| `packages/compositor/src/input/Pointer.ts` | `wl_pointer` impl | +| `packages/compositor/src/input/Touch.ts` | `wl_touch` impl (REAL, not stub) | +| `packages/compositor/src/input/Xkb.ts` | @gfld/compositor-wasm wrapper (REUSED MIT) | +| `packages/compositor/src/input/InputQueue.ts` | Event coalescing | +| `packages/compositor/src/input/InputOutput.ts` | FD abstraction | +| `packages/compositor/src/input/events.ts` | Event wrappers | +| `packages/compositor/src/browser/input.ts` | DOM keyboard events | +| `packages/compositor/src/browser/pointer.ts` | DOM pointer events | +| `packages/compositor/src/browser/touch.ts` | DOM touch events (NEW) | + +### Reused (MIT) + +| File | Role | +|---|---| +| `libs/compositor-wasm/src/index.ts` | xkbcommon + pixman WASM (REUSED) | + +### Python side + +| Component | Role | +|---|---| +| `pywayland.server.Client` | wl_client wrapper | +| `pywayland.server.Resource` | wl_resource base (per-resource dispatchers) | +| `xkbcommon` PyPI package | Optional, only needed for XWayland keymap compilation | diff --git a/06-data-clipboard.md b/06-data-clipboard.md new file mode 100644 index 0000000..0596a59 --- /dev/null +++ b/06-data-clipboard.md @@ -0,0 +1,276 @@ +# 06 — Data Device, Selection, DnD (`wl_data_device`) + +This chapter covers the data device / clipboard / drag-and-drop layer in +the MIT approach. Same architecture as Greenfield (clean-room), but with +**primary selection** added (which Greenfield lacks). + +## Faithful Wayland model + primary selection (NEW) + +`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per +seat-per-client and the implementation lives directly on `Seat`. +`WaylandDataSource` / `DataOffer` are the standard source/offer pair. + +**NEW**: Implement `zwp_primary_selection_device_manager_v1` for middle-click +paste, which Greenfield lacks entirely (grep returns zero hits). + +## FD-over-WebSocket workaround (REUSED approach) + +Same `ProxyFD` JSON descriptor as Greenfield: +``` +ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string } +``` + +Three concrete `InputOutput` / `InputOutputFD` implementations: + +### `webInputOutput` — for in-browser web apps + +Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s. + +### `RemoteInputOutput` — for native apps + +All fd operations go over a secondary WebSocket to endpoints: `/mkfifo`, +`/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`, `/read-fd`, +`/close-fd`. + +### `ProxyInputOutput` (Python side) — owns the real OS pipes and memfds + +In Python: +- `os.pipe()` for pipes +- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon) + +`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD +has `host === proxy.baseURL`. + +**Crucial consequence**: For native→native clipboard transfers (user +copies in one Linux app, pastes in another), the browser compositor never +sees a single byte of the data — it only relays the fd reference. The +actual content stays inside the proxy host's OS kernel pipe. + +## MIME-type negotiation (IMPROVED over Greenfield) + +Same hard-coded allow-list initially: `['text/plain', 'text/html', +'image/png']`. **IMPROVEMENT**: use exact prefix matching, not substring +matching (Greenfield's substring match can collide — +`text/plain;charset=iso-8859-1` silently becomes `text/plain`). + +### Native → browser + +`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is +matched against the allow-list (exact prefix match), survivors pulled as +separate blobs via `mkfifo + send + readBlob`, then +`navigator.clipboard.write([new ClipboardItem(...)])`. + +### Browser → native + +`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types, +plus a synthetic `text/plain;charset=utf-8` alias appended (Linux apps +expect the charset form). On `send`, the alias is normalised back to +`text/plain` for the lookup. + +## Clipboard mirroring (REUSED approach) + +The browser has no `clipboardchange` event (Chromium issue 933608 cited in +Greenfield's code). MIT approach polls `navigator.clipboard.read()`: + +- on init +- on every `window.focus` event + +De-duplicates offers by mime-type set + total blob size, then calls +`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`. + +The feedback loop is broken (same as Greenfield) because +`handleWaylandDataSourceUpdate` returns early when `dataSource instanceof +BrowserDataSource`. Watch for the `clipboardchange` event in modern +browsers (currently behind flags in Chrome) and switch to event-driven +when available. + +### Firefox fallback (REUSED approach) + +`text-selection.ts` uses only `readText`/`writeText` and a +`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images +or HTML. + +## Drag-and-drop (REUSED approach + NEW) + +### Wayland-side (Seat.startDrag) + +1. Validates the implicit pointer grab. +2. Attaches `DragIconRole` to the optional icon surface. +3. Installs a `PointerDrag` grab on both pointer and keyboard. + +`PointerDrag` tracks the focused data device, sends +`wl_data_device.enter`/`leave`/`motion`/`drop`, creates fresh +`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation. + +### Browser-side (`BrowserDnD` class) + +Creates a hidden full-canvas `dragHelper` div with `draggable=true` when +the Wayland drag-icon surface commits, so the browser shows the Wayland +icon via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the +user drags, `dragover` events on the dragHelper are converted back to +`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`, +keeping `PointerDrag` fed. + +**NEW: Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD` +so a Wayland drag can deliver bytes to a real DOM drop target. Greenfield +has a FIXME admitting this isn't implemented. + +## XWayland bridge (REUSED approach, IF XWayland enabled) + +`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and +builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the +receiver pulls data: + +1. `XDataSource.send` calls `XConvertSelection` to have the X server push + the bytes into a property on `selectionWindow`. +2. `writeProperty` copies them into the receiver's `InputOutputFD`. +3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`). + +Uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings. + +## Primary selection (NEW — Greenfield lacks this) + +Implement `zwp_primary_selection_device_manager_v1` (or the older +`gtk_primary_selection`) on the compositor side. Forward `PRIMARY` +notifications in `XWindowManager`. This unlocks middle-click paste from +native apps. + +## ASCII diagram — clipboard flow in both directions + +``` + CLIPBOARD / SELECTION FLOW + +================================================================= +A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app) +================================================================= + +┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐ +│ Native app │ ─────────────────────────────────────────────► │ compositor │ +│ (sandboxed │ │ (browser, in DOM) │ +│ in bwrap) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │ +└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘ + │ selectionListeners fire + ▼ + ┌──────────────────────────┐ + │ handleWaylandDataSource │ + │ Update() (selection.ts)│ + └──────────┬──────────────┘ + mkfifo() ──────────────────────────► │ Python /mkfifo + ProxyFD read+write (os.pipe) ◄───── returns 2 ProxyFDs + ▼ + dataSource.send("text/plain", writeFD) + │ + ▼ + wl_data_source.send("text/plain", writeFDProxyFD) + ─────────────────────────────────────────► │ Native source + │ receives real write fd + │ (proxyFDtoNativeFD + │ is a passthrough) + ▼ + Native app writes bytes + ──────────────► OS pipe ───┐ + │ + readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘ + ▼ (proxy streams over WS) + Blob {type:"text/plain"} + ▼ + navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ]) + ▼ + ┌────────────────────────────┐ + │ DOM clipboard populated │ + │ (Ctrl+V anywhere now works) │ + └────────────────────────────┘ + + +================================================================= +B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM) +================================================================= + +┌─────────────────────────────┐ +│ User copies in DOM │ +│ (browser tab, textarea...) │ +└──────────────┬──────────────┘ + │ on window 'focus' event (or initial load) + ▼ + navigator.clipboard.read() ──► ClipboardItems[] + ▼ + handleBrowserDataSourceUpdate() — de-dup vs previous offers + ▼ + createBrowserDataSource(offers) + .mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias + ▼ + seat.setSelectionInternal(browserDataSource, serial) + │ + │ 1) selectionListeners fire — handleWaylandDataSourceUpdate + │ returns early (instanceof BrowserDataSource), breaking loop + │ + │ 2) sendSelection(focusedClient) + ▼ +┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐ +│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │ +│ (browser, in DOM) │ │ (focused) │ +└───────────┬────────────┘ └──────┬───────┘ + │ │ + │ When the native app calls wl_data_offer.receive("text/plain", fd): + │ │ + │ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe + ▼ + BrowserDataSource.send(mimeType, ioFD) + │ + ▼ + matchingOffer.getType(mimeType) ─► Promise (from ClipboardItem) + │ + ▼ + ioFD.write(blob) ─► POST /write-fd-as-stream?fd= ─► Python proxy + │ + ▼ + asyncio.StreamWriter ──► native write fd + │ + ▼ + Bytes flow through OS pipe into native app + │ + ▼ + ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0 + + +================================================================= +C) PRIMARY SELECTION (NEW — middle-click paste) ──► same flow but via + zwp_primary_selection_device_manager_v1 instead of wl_data_device +================================================================= + + Native app selects text ──► zwp_primary_selection_source.set_selection + │ + ▼ + Browser receives primary selection offer ──► stores in seat.primarySelectionDataSource + │ + ▼ + User middle-clicks on a surface ──► seat.notifyPrimarySelectionRequest + │ + ▼ + primarySelectionSource.send(mimeType, writeFD) ──► same FD-over-WS flow as A/B +``` + +## File reference + +### Custom (clean-room TS) + +| File | Role | +|---|---| +| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` global | +| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` impl | +| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` impl | +| `packages/compositor/src/data/WaylandDataSource.ts` | Native source wrapper | +| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** | +| `packages/compositor/src/data/PrimarySelectionSource.ts` | **NEW: middle-click paste source** | +| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge | +| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback | +| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD implemented) | +| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps | + +### Python side + +| Component | Role | +|---|---| +| `os.pipe()` | Create pipes (replaces Greenfield's `makePipe` native addon) | +| `os.memfd_create()` | Create shared memory (replaces Greenfield's `createMemoryMappedFile`) | +| `asyncio.StreamReader` / `StreamWriter` | Streaming reads/writes over WebSocket endpoints | diff --git a/07-native-app-bridge.md b/07-native-app-bridge.md new file mode 100644 index 0000000..1efe709 --- /dev/null +++ b/07-native-app-bridge.md @@ -0,0 +1,423 @@ +# 07 — Native App Bridge (pywayland + bubblewrap + seccomp + systemd-run) + +This chapter covers the native app bridge in the MIT approach: how real +Linux applications get launched, sandboxed, connected to the Python proxy, +and routed to the browser. **This is the most important chapter for the +user** — they want to run real Linux apps in their Odoo 20 web app. + +## Process model — three layers (same as Greenfield, but in Python) + +### Layer 1: Main process (HTTP + WS hub) + +Single Python process (aiohttp) listens on `:8081`. Per Odoo session, it +forks one `SessionProcess` child — one child per `compositor_session_id`, +**not** per app. Multiple apps in the same browser tab share one child. + +WebSocket upgrades are handed off to the matching child via +`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux). + +### Layer 2: Session process + +The forked child. Receives IPC messages: + +- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher +- `launch_app` — spawns the actual app binary via `subprocess.Popen` + + `bubblewrap` + `seccomp` + `systemd-run --scope` +- `ws_upgrade` — hands WS to the dispatcher + +On `session.close()` the process exits. + +### Layer 3: Per-client session + +For each new `wl_client` (one per Wayland connection), creates a +`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD +JSON. Routes requests via per-resource dispatchers. + +## App launch flow (definitive answer) + +1. Browser sends `GET http://proxy:8081/` with + `Authorization: Bearer ` header. +2. Main process validates JWT (HS256, via `pyjwt`). +3. Main process forks a `SessionProcess` if none exists for that + `compositor_session_id`, sends `start` IPC. +4. Main sends `launch_app` IPC with `{name, executable, args, env, serial}`. +5. `SessionProcess` calls `launch_application()`: + - `appEnv = {...os.environ, ...env, WAYLAND_DISPLAY: session.wayland_display}` + - `subprocess.Popen(['systemd-run', '--user', '--scope', '-p', + 'MemoryMax=1G', '-p', 'CPUQuota=200%', '-p', 'TasksMax=256', '--', + 'bwrap', '--unshare-all', '--clearenv', '--seccomp', '', + '--ro-bind', '/usr', '/usr', '--dev-bind', '/dev/dri/renderD128', + '/dev/dri/renderD128', '--ro-bind', + f'{host_xdg}/wayland-N', f'/run/user/{uid}/wayland-N', + '--uid', str(uid), '--gid', str(gid), + '--new-session', '--die-with-parent', executable, *args], + env=appEnv)` +6. On `spawn` event, captures `pid`, creates `NativeAppContext`, registers + `exit` listener. +7. Replies to main with `{pid, key}` → main returns HTTP 201 with + `{baseURL, signalURL, key, pid, name}`. +8. Browser opens `signalURL` as WebSocket → control channel established. +9. App connects to `$XDG_RUNTIME_DIR/wayland-N` → libwayland fires + `wl_client` creation → pywayland's per-resource dispatchers route + requests to `{native, browser}`. +10. For each new `wl_client`, creates a `PROTOCOL` ARQ channel and sends + `CONNECT_CHANNEL` over the signaling WS; browser opens a new WS at + `/channel?id=…`. +11. Wayland wire messages now flow: app → libwayland → pywayland + dispatcher → routed to `{native, browser}`. + +## Sandboxing (NEW — Greenfield has none) + +This is the **critical gap** in Greenfield (pitfall M). The MIT approach +ships with proper sandboxing from day one. + +### Recommended stack + +- **bubblewrap (bwrap)** — LGPL-2.0+, unprivileged, setuid-free, Flatpak-grade +- **seccomp BPF** — libseccomp LGPL-2.1, Docker's default profile + allow `CLONE_NEWUSER` +- **systemd-run --scope** — native cgroup v2 integration (MemoryMax, CPUQuota, TasksMax) +- **Linux user namespaces** — UID remap to 100000-165535 via `/etc/subuid`/`/etc/subgid` +- **cgroups v2** — memory, CPU, PID limits per app + +### Reference bwrap command line (GTK4 app, GPU, no network) + +```bash +systemd-run --user --scope --unit=app-odoo-calculator-$(uuid) \ + -p MemoryMax=256M -p MemorySwapMax=0 -p CPUQuota=50% -p TasksMax=32 -p Delegate=yes -- \ + bwrap \ + --clearenv --unshare-all --hostname sandbox --new-session --die-with-parent \ + --symlink usr/lib /lib --symlink usr/lib64 /lib64 \ + --symlink usr/bin /bin --symlink usr/bin /sbin \ + --ro-bind /usr /usr --ro-bind /opt /opt \ + --ro-bind /etc/passwd /etc/passwd --ro-bind /etc/group /etc/group \ + --ro-bind /etc/hosts /etc/hosts --ro-bind /etc/localtime /etc/localtime \ + --ro-bind /etc/locale.conf /etc/locale.conf --ro-bind /etc/machine-id /etc/machine-id \ + --ro-bind /etc/fonts /etc/fonts --ro-bind /etc/ssl /etc/ssl \ + --ro-bind /etc/ca-certificates /etc/ca-certificates \ + --proc /proc --dev /dev \ + --dev-bind-try /dev/dri/renderD128 /dev/dri/renderD128 \ + --tmpfs /tmp --tmpfs /var/tmp --tmpfs /run \ + --dir /run/user/100000 \ + --ro-bind "$HOST_XDG/wayland-0" /run/user/100000/wayland-0 \ + --overlay /usr/share/compositor-proxy/skel/gnome-calculator:/var/lib/compositor-proxy/tenants/$TENANT/appdata/gnome-calculator-upper /home/appuser \ + --setenv HOME /home/appuser \ + --setenv XDG_RUNTIME_DIR /run/user/100000 \ + --setenv WAYLAND_DISPLAY wayland-0 \ + --setenv XDG_DATA_DIRS /usr/share --setenv PATH /usr/bin --setenv LANG en_US.UTF-8 \ + --uid 100000 --gid 100000 \ + --seccomp /proc/self/fd/3 \ + /usr/bin/gnome-calculator +``` + +### Key design decisions + +| Concern | Decision | Rationale | +|---|---|---| +| Sandbox runner | **bubblewrap** | MIT license, unprivileged, setuid-free, Flatpak-grade | +| cgroup limits | **`systemd-run --user --scope` wrapping bwrap** | Native cgroup v2 integration; no manual cgroupfs writes | +| Wayland socket | **Single-file `--ro-bind` of `wayland-N`** into fresh `/run/user/` tmpfs | Critical: prevents access to PulseAudio, pipewire, D-Bus, keyring, and sibling tenants' sockets | +| GPU | **`--dev-bind /dev/dri/renderD128`** (single render node, NOT `card0`) | Kernel render-node access control already restricts to non-display ioctls | +| Network | **`--unshare-net` by default** (loopback-down only); per-app opt-in to veth+nftables filter | Most GTK apps need no network. Firefox/Odoo clients opt in. | +| Seccomp | **Docker's default profile** (allowlist of ~250 syscalls, blocking ~44) + allow `CLONE_NEWUSER` for nested sandboxes | Compile to BPF at proxy startup; pass via `bwrap --seccomp ` | +| Filesystem | **`/usr` ro-bind, minimal `/etc` whitelist, `/tmp` tmpfs, `$HOME` overlay (changes discarded)** | No host write access; per-app `$HOME` reset on each launch kills malware persistence | +| User ns | **UID remap to 100000–165535 via `/etc/subuid`/`/etc/subgid`** | Even on kernel escape the app is unprivileged on host; per-app UID enables nftables filtering and audit | +| XWayland | **v1: unsandboxed (documented limitation); v2: per-Xwayland bwrap sandbox** | Hardest piece (3–4 days). v1 ships like Flatpak's old `socket=x11`. | +| `$XDG_RUNTIME_DIR` | **`--clearenv` then explicit `--setenv` whitelist** | No host env leak (DB password, Odoo session cookies, AWS keys, etc.) | + +## Native addon architecture (NO native Node addons — pure Python) + +**No N-API addons, no `.node` files, no CMake/Ninja build chain.** The +entire proxy is pure Python. + +### Replaces Greenfield's 3 native C addons + +| Greenfield native | Python replacement | Notes | +|---|---|---| +| `wayland-server-addon.node` (1231 lines C, patched libwayland) | **pywayland** (Apache-2.0) + 5-line cffi shim | Vanilla upstream libwayland 1.26.0 bundled in pywayland's wheel. Apply Greenfield's `changes.diff` (MIT patches) optionally for fast-sync optimization. | +| `proxy-poll-addon.node` (127 lines C, libuv `uv_poll_t`) | `asyncio.get_running_loop().add_reader(wl_fd, dispatch_fn)` | Built-in asyncio. NO addon needed. | +| `proxy-encoding-addon.node` (290 lines C) + `libproxy-encoding.so` (~1400 lines C, GStreamer) | **PyGObject + GStreamer Python bindings** (LGPL) | Same GStreamer, Python wrapper. ~700 LOC Python replaces ~1700 LOC C. | +| `westfield-*.c` (EGL, dmabuf, xwayland helpers, ~2000 lines C) | **Vendored Greenfield C files via cffi** (MIT) | Reuse verbatim. ~3000 LOC of MIT C cherry-picked from Greenfield. | + +### Vendored Greenfield C files (MIT, reused as-is) + +These files from `greenfield/packages/compositor-proxy/native/wayland/src/` +are MIT-licensed (Greenfield's own work or vendored from wlroots which is +also MIT) and can be **reused verbatim** in our Python proxy: + +| File | Lines | Purpose | +|---|---|---| +| `westfield-buffer.c` | ~150 | Buffer helpers | +| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM | +| `westfield-egl.c` | ~900 | EGL zero-copy setup | +| `westfield-xwayland.c` | ~465 | XWayland spawn + bridge | +| `wlr_drm.c` | ~244 | `wl_drm` global impl (vendored from wlroots) | +| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) | +| `drm_format_set.c` | ~239 | DRM format helpers (vendored from wlroots) | +| `addon.c` | ~73 | wlroots addon system helper | +| **TOTAL** | **~3,374 lines C** | All MIT, reused verbatim | + +These compile into a single `libwestfield.so` shared library, loaded by +the Python proxy via cffi at runtime. + +## XWayland bridge + +### Native side (Python equivalent of westfield-xwayland.c) + +In Python: +```python +import subprocess, os, socket + +def spawn_xwayland(display_num, wl_client_fd, wm_fd, display_fd): + # Create X11 socket + x11_socket = f"/tmp/.X11-unix/X{display_num}" + listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + listener.bind(x11_socket) + listener.listen(1) + + # Set DISPLAY env + os.environ["DISPLAY"] = f":{display_num}" + + # Fork + exec Xwayland + subprocess.Popen([ + "Xwayland", f":{display_num}", "-ac", "-rootless", + "-listen", str(listener.fileno()), + "-displayfd", str(display_fd), + "-wm", str(wm_fd), + ], pass_fds=[listener.fileno(), display_fd, wm_fd]) + + # Xwayland itself becomes a Wayland client of the proxy + # (via wl_client_create(wl_display, wl_client_fd)) +``` + +### xtsb (REUSED MIT, browser side) + +`@gfld/xtsb` (MIT) — pure-TS X11 client, no libxcb. Dual-environment +(Node `net.Socket` + browser WebSocket). Auto-generated from XCB XML +protocol files via `xcbgen/ts_client.py`. The browser-side xtsb +implements the actual XWM logic (translates X `CreateNotify` events → +`xdg_toplevel` Wayland surfaces). + +**Optional**: skip XWayland in MVP (8 weeks of effort). Add only if you +need to run X11-only apps (most modern GTK/Qt apps are Wayland-native). + +## SDK misconception corrected (same as Greenfield) + +The `sdk/` directory is **NOT for native apps**. It's exclusively for +**WASM apps** that run inside the browser via Emscripten. The +`sysrootlibs/` cross-compiles wayland/gtk/cairo/etc. to wasm32. Native +Linux apps on the proxy use the host's own system libraries. + +## ASCII diagram — native app launch & connection + +``` +Browser (Odoo OWL shell) Python proxy host (Linux) App (sandboxed in bwrap) +========================== ==================== ============================== + +[1] User clicks app in shell $ python -m compositor_proxy.cli \ + │ └─── main.py listens on :8081 + │ sessionProcesses: Dict[id, Process] + │ + │ HTTP GET /gtk4-demo + │ Authorization: Bearer + ├──────────────────────────────► │ + │ │ validate JWT (HS256, pyjwt) + │ │ if no child for : + │ │ multiprocessing.Process(target=session_main) + │ │ IPC: {type:'start', payload:{config, session_id}} + │ │ └── create_session() → Display + │ │ ├── Display() + │ │ ├── display.add_socket() → "wayland-N" + │ │ │ (becomes WAYLAND_DISPLAY) + │ │ ├── display.init_shm() + YUV420/NV12/... + │ │ ├── init_drm("/dev/dri/renderD128") + │ │ │ (uses vendored westfield-egl.c via cffi) + │ │ │ (registers zwp_linux_dmabuf_v1 + wl_drm globals) + │ │ └── asyncio.add_reader(wl_fd, dispatch_wl) + │ │ (5-line cffi shim for wl_event_loop_get_fd) + │ │ + │ │ IPC: {type:'launch_app', payload:{name, executable, args, env, serial}} + │ │ └── launch_application() (NativeAppContext.py) + │ │ ├── appEnv = {...os.environ, ...env, + │ │ │ WAYLAND_DISPLAY: "wayland-N"} + │ │ ├── subprocess.Popen([ + │ │ │ 'systemd-run', '--user', '--scope', + │ │ │ '-p', 'MemoryMax=1G', + │ │ │ '-p', 'CPUQuota=200%', + │ │ │ '-p', 'TasksMax=256', '--', + │ │ │ 'bwrap', + │ │ │ '--unshare-all', '--clearenv', + │ │ │ '--seccomp', '', + │ │ │ '--ro-bind', '/usr', '/usr', + │ │ │ '--dev-bind', '/dev/dri/renderD128', '/dev/dri/renderD128', + │ │ │ '--ro-bind', '$XDG/wayland-N', '/run/user/100000/wayland-N', + │ │ │ '--uid', '100000', '--gid', '100000', + │ │ │ '--new-session', '--die-with-parent', + │ │ │ '/usr/bin/gtk4-demo' + │ │ │ ], env=appEnv) + │ │ │ ◄── systemd-run + bwrap, NO chroot, NO container + │ │ │ ──────────────────────────────► [2] gtk4-demo (sandboxed) + │ │ │ │ + │ │ │ │ libwayland-client.so + │ │ │ │ connects to + │ │ │ │ $XDG_RUNTIME_DIR/wayland-N + │ │ │ │ + │ │ │ ▼ + │ │ │ ┌──────────────────────────┐ + │ │ │ │ wl_display event loop │ + │ │ │ │ (in proxy, libwayland- │ + │ │ │ │ server.so.0 — vanilla, │ + │ │ │ │ NO patches needed) │ + │ │ │ │ │ + │ │ │ ◄── asyncio fires, │ wl_client created → │ + │ │ │ dispatch_wl() called │ Client.get_credentials │ + │ │ │ wl_event_loop_dispatch() │ → (pid, uid, gid) │ + │ │ │ │ find_matching_app_ctx │ + │ │ │ │ (pid) by PPID walk │ + │ │ │ │ │ + │ │ │ │ Per-resource dispatchers│ + │ │ │ │ (auto-generated by │ + │ │ │ │ pywayland scanner) │ + │ │ │ │ route to {native, │ + │ │ │ │ browser} │ + │ │ │ └──────────────────────────┘ + │ │ │ + │ │ ├── child.on('spawn') → capture pid + │ │ │ create_native_app_context(pid, name, sandbox=True) + │ │ │ native_app_context.on_disconnect() # start 10-min SIGHUP timer + │ │ │ + │ │ └── child.on('exit') → native_app_context.on_exit({exit_code|signal}) + │ │ + │ │ IPC reply: {type:'launch_app_success', payload:{pid, key}} + │ + │ HTTP 201 Created + │ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, sandboxed:true } + ◄────────────────────────────── │ + │ │ + │ Open WS: ws://…/signal?key=<16-hex> + │ Authorization: Bearer + ├──────────────────────────────► │ + │ │ aiohttp demux /signal → native_app_context.on_connect(ws) + │ │ clears 10-min SIGHUP timer + │ │ + │ │ For each new wl_client (one per wayland connection): + │ │ create_protocol_channel() → ARQChannel (kcp over WS) + │ │ sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc} + │ ◄──── CONNECT_CHANNEL JSON ───┤ + │ │ + │ Open WS: ws://…/channel?id=…&key=… + │ Authorization: Bearer + ├──────────────────────────────► │ binds WS to the ARQChannel + │ │ + │ Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval): + │ + │ App → wl_surface.commit ──── proxy intercepts via dispatcher ────► browser compositor + │ App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄── FrameFeedback paces callback + │ App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected) + │ App ← wl_keyboard.enter ─── browser → proxy → libwayland-server sends via SCM_RIGHTS + │ + │ Buffer path (per wl_surface): + │ - App submits wl_buffer (dmabuf or shm) + │ - setBufferCreatedCallback → ProxyBuffer with creationSerial + │ - On commit: dispatcher → Encoder.encodeFrame() + │ └── gst_parse_launch("appsrc ! glupload ! glcolorconvert ! glshader ! " + │ "glcolorconvert ! gldownload ! queue ! vaapih264enc ! appsink") + │ (vaapih264enc — LGPL, NOT x264enc which is GPL) + │ └── GLib MainLoop in separate thread + │ └── encoded H.264 sample → asyncio.run_coroutine_threadsafe → main thread + │ └── sent over the surface's FRAME data channel (separate WS) + │ + │ X11 path (xterm etc., IF XWayland enabled): + │ - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by spawn_xwayland()) + │ - First X connect → spawn_xwayland() forks Xwayland: execlp("Xwayland", …, "-rootless", "-wm", wm_fd) + │ - Xwayland itself becomes a Wayland client of the proxy (wl_client_create) + │ - When Xwayland ready (displayFd readable): + │ └── xtsb (REUSED MIT, browser side) speaks X11 → translates X windows → xdg_toplevel surfaces + │ + │ Lifecycle: + │ - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP') + │ - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM') + │ └── if not exited in 10s → SIGKILL + │ - App crashes → child 'exit' event → native_app_context.on_exit() → APP_TERMINATED msg → browser UI updates + │ - All clients gone for >1s → session.close() → process.exit() in SessionProcess.py → main deletes entry + │ + │ Sandbox (NEW vs Greenfield): + │ - bwrap --unshare-all --clearenv --seccomp + │ - --ro-bind /usr /usr, minimal /etc whitelist + │ - --dev-bind /dev/dri/renderD128 (single GPU render node, no card0) + │ - --ro-bind $HOST_XDG/wayland-N /run/user/100000/wayland-N (single socket only) + │ - --uid 100000-165535 per-app (--gid same) + │ - --unshare-net by default + │ - cgroup: MemoryMax=1G, CPUQuota=200%, TasksMax=256 + │ - $HOME overlay (changes discarded on app exit) +``` + +## Effort estimates + +| Phase | Days | Deliverable | +|---|---|---| +| **v1 shippable** | **7-9 days** | bubblewrap integration + seccomp + cgroup limits + Wayland socket isolation + GPU render-node passthrough + default-deny net. XWayland unsandboxed (documented). | +| v2 full | +4-9 days | + XWayland sandbox + veth+nftables networking + per-tenant hierarchical cgroups + XDG Desktop Portal | + +### v1 breakdown + +| Component | Days | +|---|---| +| bwrap integration | 2-3 | +| seccomp profile + compile | 1-2 | +| systemd-run cgroup wrapper | 1 | +| Wayland socket ownership fix | 0.5 | +| GPU passthrough testing | 1 | +| Schema + docs | 1 | +| **TOTAL** | **7-9** | + +## File reference + +### Python proxy (custom, ~2880 LOC) + +| File | Role | +|---|---| +| `proxy/main.py` | HTTP + WS hub, forks per session | +| `proxy/session.py` | Top-level proxy session | +| `proxy/session_controller.py` | aiohttp WS server with 9 endpoint paths | +| `proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn | +| `proxy/native_wayland_client_session.py` | Per-client message dispatcher | +| `proxy/native_wayland_compositor_session.py` | Display creation, initShm, initDrm | +| `proxy/channel.py` | ARQChannel + SimpleChannel (clean-room) | +| `proxy/kcp_adapter.py` | kcp PyPI package integration | +| `proxy/frame_feedback.py` | Receives 4-byte decode feedback, paces wl_frame.done | +| `proxy/proxy_buffer.py` | Proxy-side buffer representation | +| `proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration | +| `proxy/xwayland_session.py` | XWayland spawn + bridge (Python equivalent of westfield-xwayland.c) | +| `proxy/wayland_server.py` | pywayland wrapper + cffi shim | +| `proxy/io/` | WebSocket server glue | +| `proxy/encoding/` | GStreamer encoder orchestration via PyGObject | +| `proxy/protocol/` | pywayland-generated dispatchers (auto) | +| `proxy/jwt_auth.py` | JWT validation (pyjwt) | +| `proxy/sandbox.py` | bubblewrap + seccomp + systemd-run wrappers | + +### Vendored Greenfield C files (MIT, reused as-is) + +| Path | Role | +|---|---| +| `native/westfield-buffer.c` | Buffer helpers | +| `native/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM | +| `native/westfield-egl.c` | EGL zero-copy setup | +| `native/westfield-xwayland.c` | XWayland spawn + bridge (used as reference, ported to Python) | +| `native/wlr_drm.c` | `wl_drm` global impl (vendored from wlroots) | +| `native/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) | +| `native/drm_format_set.c` | DRM format helpers (vendored from wlroots) | +| `native/addon.c` | wlroots addon system helper | +| `native/changes.diff` | Patches to libwayland-server (optional, for fast-sync) | + +### Reused (MIT/Apache) + +| Component | Role | +|---|---| +| pywayland | Python bindings to libwayland-server | +| `xkbcommon` PyPI package | Python bindings to libxkbcommon (optional, for XWayland) | +| PyGObject (`gi.repository.Gst`) | GStreamer Python bindings | +| `kcp` PyPI package | Cython wrapper over upstream `ikcp.c` | +| aiohttp | Async HTTP + WebSocket server | +| PyJWT | JWT validation | +| bubblewrap | Unprivileged sandboxing | +| libseccomp | Seccomp BPF filter compilation | diff --git a/08-app-lifecycle.md b/08-app-lifecycle.md new file mode 100644 index 0000000..1aa815a --- /dev/null +++ b/08-app-lifecycle.md @@ -0,0 +1,287 @@ +# 08 — App Lifecycle (Xdg Shell, Toplevels, Popups, Subsurfaces) + +This chapter covers the application lifecycle layer in the MIT approach. +Same architecture as Greenfield (clean-room from spec), with the +`UserShellApi` integration seam documented in detail since that's where +the Odoo 20 OWL shell hooks in. + +## Critical: clean-room requirement for roles + +The xdg-shell state machine is the **highest-risk component** of the MIT +approach (3 dev-weeks, high risk per Agent P's estimate). Subtle bugs in +configure/ack-configure round-trips, popup positioner math, or subsurface +sync/desync cascades can cause apps to hang or crash. + +**Approach**: +1. Person A writes a behavioral spec from: + - Wayland protocol spec (https://wayland.freedesktop.org/docs/html/) + - xdg-shell.xml protocol description + - wlroots C source (MIT) — `types/xdg_shell/wlr_xdg_*.c` + - Weston C source (MIT) — `libweston/desktop/xdg-shell.c` + - Smithay Rust source (MIT) — `src/wayland/xdg_shell/` +2. Person B implements from spec — never opens `@gfld/compositor` source. + +## Three role families, one shared abstraction (same as Greenfield) + +All surface roles implement `SurfaceRole` (`{ view, optional +desktopSurface, onCommit(surface) }`). `XdgToplevel`, `XdgPopup`, and +`ShellSurface` additionally implement `DesktopSurfaceRole`, which bridges +them to a `DesktopSurface` (`FloatingDesktopSurface` or +`AlwaysFullScreenDesktopSurface`, chosen by `session.config.mode`). The +`DesktopSurface` is where **every** `userShell.events.*` emission +originates — making it the true integration boundary. + +## XdgToplevel lifecycle — the headline diagram (same as Greenfield) + +``` + ┌───────────────────────────────────────────────┐ + │ wl_surface created (no role, no buffer yet) │ + └─────────────────────┬─────────────────────────┘ + │ client: xdg_wm_base.get_xdg_surface + │ (errors if surface already has buffer) + ▼ + ┌───────────────────────────────────────────────┐ + │ XdgSurface created │ + │ • surface.hasKeyboard/Pointer/Touch = true │ + │ • configured=false, configureList=[] │ + └─────────────────────┬─────────────────────────┘ + │ client: xdg_surface.get_toplevel + │ (errors if surface has another role) + ▼ + ┌───────────────────────────────────────────────┐ + │ XdgToplevel created │ + │ • surface.role = xdgToplevel │ + │ • View created │ + │ • DesktopSurface created + init() │ + │ • ackConfigure override installed │ + └─────────────────────┬─────────────────────────┘ + │ client: xdg_toplevel.set_title / + │ set_app_id / set_min/max_size / + │ set_parent (stored, propagated) + ▼ + ┌───────────────────────────────────────────────┐ + │ First wl_surface.commit (buffer === undefined) │ + │ XdgToplevel.onCommit() → ensureAdded() │ + │ • desktopSurface.add() │ + │ → userShell.events.surfaceCreated(...) │ + │ • scheduleConfigure(first) │ + │ → microtask → sendConfigure() │ + │ → resource.configure(w,h,states) │ + │ • added=true │ + └─────────────────────┬─────────────────────────┘ + │ client receives configure + │ client: xdg_surface.ack_configure(serial) + ▼ + ┌───────────────────────────────────────────────┐ + │ ackConfigure(serial, configure) │ + │ • handleAckConfigure trims older serials │ + │ • configured=true │ + │ • next.state = configure.state │ + │ • next.size = configure.size │ + └─────────────────────┬─────────────────────────┘ + │ client: wl_surface.commit (with buffer) + ▼ + ┌───────────────────────────────────────────────┐ + │ ★ MAPPED (steady state) ★ │ + │ onCommit(): │ + │ • surface.commitPending() (state latch) │ + │ • xdgSurface.commit() (geometry latch) │ + │ • current.state ← next.state │ + │ • current.min/max ← next.min/max │ + │ • desktopSurface.commit() → render() │ + └─────────────────────┬─────────────────────────┘ + │ + ┌────────────────────────────────────────┼────────────────────────────┐ + │ │ │ + user/shell requests client requests destruction + │ │ │ + ▼ ▼ ▼ + ┌──────────────────────────┐ ┌─────────────────────────────┐ ┌──────────────────────┐ + │ configureMaximized(b) │ │ setMaximized / unsetMaximized│ │ wl_surface.destroy │ + │ configureFullscreen(b) │ │ setFullscreen/unsetFullscreen│ │ or xdg_toplevel. │ + │ configureActivated(b) │ │ setMinimized │ │ destroy │ + │ configureResizing(b) │ │ setParent / setTitle / │ ├──────────────────────┤ + │ configureSize(w,h) │ │ setAppId │ │ destroy listener: │ + └────────────┬─────────────┘ │ move / resize / showWindowMenu│ │ • surface.unmap() │ + │ └──────────────┬──────────────┘ │ • desktopSurface. │ + │ each mutates │ │ removed() │ + │ pending.state / pending.size │ │ → surfaceDestroyed │ + ▼ │ │ • view.destroy() │ + ┌──────────────────────────┐ │ └──────────────────────┘ + │ scheduleConfigure( │ │ + │ stateCompareEquals(), │ ┌─────────────────────────────┐ + │ sendConfigure) │ │ setMinimized → │ + │ • coalesce via µtask │ │ desktopSurface.minimize() │ + │ • if pending==current → │ │ • view.mapped=false │ + │ skip │ │ • seat drops keyboard focus │ + │ • else: push serial, │ │ if this was the focused │ + │ emit configure event │ │ surface │ + │ │ └─────────────────────────────┘ + └────────────┬─────────────┘ + │ + │ cycle repeats: ack_configure → commit → render + └────────────────────────────────────► back to ★ MAPPED ★ +``` + +## Configure / ack-configure round-trip (REUSED semantics) + +The most important mechanism in xdg-shell. Async protocol-level sync: + +1. Compositor mutates `pending.{state,size}`. +2. Coalesces via `queueCancellableMicrotask` with a `stateCompareEquals` + short-circuit (if pending == current, skip). +3. Emits `resource.configure(w,h,states)` with a fresh serial. +4. Client acks: `xdg_surface.ack_configure(serial)`. +5. `handleAckConfigure` trims older serials, sets `configured=true`, + copies `configure.{state,size}` into `next`. +6. Next commit latches `next → current`. + +The `configured` flag gates buffer attachment: an `unconfiguredBuffer` +error is raised if the client tries to attach a buffer before acking. + +## XdgPositioner math (REUSED semantics) + +`anchor` selects a point on the anchor rect (in parent window-geometry +space); `gravity` selects which corner/edge of the popup is placed at +that point; `offset` is applied last. + +### Constraint adjustment cascade (X-then-Y) + +1. Try **flip** — swap anchor+gravity via `inverseX` / `inverseY` tables, + revert if still violating. +2. Try **slide** — shift offset by violation delta. +3. Try **resize** — shrink `sizeRect`. + +Slides/flips are skipped if popup size on that axis exceeds viewport. + +## Popup grabs (REUSED semantics) + +`XdgPopup.grab` rejects if: +- Surface is already mapped, OR +- Parent isn't either: + - a toplevel (root of new chain), OR + - the topmost popup of an existing grab (`notTheTopmostPopup` error) + +`seat.popupGrabStart` validates the serial against the keyboard/pointer +`grabSerial`, then starts a `PopupGrab` on both devices. `popupGrabEnd` +walks the chain in reverse calling `requestClose()` on each (sends +`popup_done` / `xdg_popup.popup_done`). + +## Subsurfaces sync/desync (REUSED semantics) + +Three-layer mechanism: +1. **`cachedState`** — full pending-state shadow on the subsurface role. +2. **`commitCache`** — flush on parent commit. +3. **`effectiveSync`** getter — cascades: if any ancestor is sync, this + subsurface is effectively sync. + +### Decision matrix + +| Sync mode | Dirty | Action | +|---|---|---| +| Sync | yes | Cache to `cachedState`, flush when parent commits | +| Sync | no | No-op (already cached) | +| Desync | yes | Flush immediately to parent's pending state | +| Desync | no | Acts like a normal surface (commit directly to its own pendingState) | + +`setDesync` immediately calls `onCommit` to flush any pending cache. + +## wl_shell (legacy path, OPTIONAL) + +Legacy path. Reduced feature set. State machine via `SurfaceStates` enum +(MAXIMIZED/FULLSCREEN/POPUP/TRANSIENT/TOP_LEVEL) and `changeState()`. +Per-surface ping (separate from `XdgWmBase`'s per-client ping). + +**Recommendation**: skip `wl_shell` in MVP — modern GTK4/Qt6 apps use +xdg-shell exclusively. Only needed for very old apps. + +## The UserShellApi seam — clean-room spec for Odoo 20 OWL shell + +This is the integration point where the Odoo 20 OWL shell hooks in. +Designed to match Greenfield's API for compatibility (since it's a +reasonable API), but written from spec. + +### Events (compositor → shell) — all optional + +| Event | Payload | Fired from | When | +|---|---|---|---| +| `clientCreated` | `CompositorClient` | `WebConnectionHandler`, `RemoteAppLauncher` | A new app client connects. | +| `clientDestroyed` | `CompositorClient` | same | A client disconnects. | +| `clientUnresponsiveUpdated` | `(client, unresponsive: boolean)` | `XdgWmBase`, `ShellSurface` | Ping (2s timeout, every 5s) — `true` = hung, `false` = recovered. | +| `surfaceCreated` | `CompositorSurface` | `FloatingDesktopSurface.add()`, `AlwaysFullScreenDesktopSurface` | Toplevel first added to desktop. | +| `surfaceDestroyed` | `CompositorSurface` | `.removed()` | Window destroyed. | +| `surfaceTitleUpdated` | `(surface, title)` | `.setTitle()` | `xdg_toplevel.set_title`. | +| `surfaceAppIdUpdated` | `(surface, appId)` | `.setAppId()` | `xdg_toplevel.set_app_id`. | +| `surfaceActivationUpdated` | `(surface, active: boolean)` | `gainFocus()` / `loseFocus()` | Refcounted via `focusCount` — only fires on 0↔1 transitions. | +| `notify` | `(variant, message)` | (declared, free-form) | Free-form notification channel. | +| `sceneRefreshed` | `sceneId: string` | `Scene` | Render finished. | +| **NEW: `surfaceStateChanged`** | `(surface, state)` | `setMaximized`/`setFullscreen`/etc. | Window state transitions (maximized/fullscreen/minimized/resizing). Greenfield lacks this — Odoo shell needs it for window decorations. | + +### Actions (shell → compositor) — provided by default + +| Action | Effect | +|---|---| +| `initScene(canvasCreator)` | Bootstraps a WebGL2 canvas + input/output. One per output. | +| `refreshScene()` | `session.renderer.render()` — manual redraw. | +| `destroyScene(sceneId)` | Destroys a specific scene. | +| `setUserConfiguration(config)` | Updates `scrollFactor`, `keyboardLayoutName`. | +| `closeClient({ id })` | Closes a client connection. | +| `activateSurface(compositorSurface)` | **Click-to-focus entry point from the shell's taskbar.** | +| **NEW: `querySurfaceState(compositorSurface)`** | Returns current surface state (maximized/fullscreen/minimized/resizing/activated). Greenfield lacks this. | +| **NEW: `requestKeyframe(compositorSurface)`** | Triggers an immediate keyframe request for the encoder. Greenfield has plumbing but doesn't call it. | + +### Opaque handle types + +```typescript +interface CompositorSurface { id: number; client: CompositorClient } +interface CompositorClient { id: string } +``` + +## Notable gaps addressed (vs Greenfield) + +1. **No maximize/fullscreen/resize/minimize events in Greenfield's + UserShellApi** — the shell can't directly observe window-state + transitions through this API. **Our MIT approach adds + `surfaceStateChanged` event** so the Odoo shell can render window + decorations correctly. +2. **No geometry/position event in Greenfield** — the shell can't render + a frame that hugs surface bounds. **Our MIT approach can extend the API + later** if needed (currently out of MVP scope). +3. **`notify` declared but unused in Greenfield's compositor core** — dead + interface slot. **Our MIT approach implements it** as a free-form + notification channel (e.g. for "App not responding" warnings). +4. **`surfaceCreated` fires once on first commit; no separate map/unmap + signal in Greenfield** — minimization is internal (`view.mapped=false`) + and not propagated. **Our MIT approach adds `surfaceMapChanged` event** + if needed for taskbar indicators. + +## Risks flagged (same as Greenfield) + +- `XdgWmBase.doPing` should use a proper serial (Greenfield uses 0 with a + FIXME). Our clean-room implementation should generate proper serials. +- `XdgToplevel.onCommit` should validate buffer size vs configured state + (Greenfield has commented-out validation). Our implementation should + enforce this. +- `XdgSurface.scheduleConfigure`'s `pendingSame` argument inverts meaning + depending on whether a microtask is already queued — subtle. Our spec + should make this explicit and well-tested. + +## File reference + +### Custom (clean-room TS) + +| File | Role | +|---|---| +| `packages/compositor/src/roles/XdgWmBase.ts` | `xdg_wm_base` global, ping-based unresponsiveness detection | +| `packages/compositor/src/roles/XdgSurface.ts` | `xdg_surface` role, configure/ack round-trip | +| `packages/compositor/src/roles/XdgToplevel.ts` | `xdg_toplevel` role, window state | +| `packages/compositor/src/roles/XdgPopup.ts` | `xdg_popup` role, transient windows, grabs | +| `packages/compositor/src/roles/XdgPositioner.ts` | `xdg_positioner`, popup positioning math | +| `packages/compositor/src/roles/Shell.ts` | `wl_shell` global (legacy, optional) | +| `packages/compositor/src/roles/ShellSurface.ts` | `wl_shell_surface` role (legacy, optional) | +| `packages/compositor/src/subsurface/Subcompositor.ts` | `wl_subcompositor` global | +| `packages/compositor/src/subsurface/Subsurface.ts` | `wl_subsurface` role, sync/desync mode | +| `packages/compositor/src/subsurface/SurfaceRole.ts` | SurfaceRole interface | +| `packages/compositor/src/subsurface/SurfaceChild.ts` | Surface tree child abstraction | +| `packages/compositor/src/desktop/Desktop.ts` | Floating vs AlwaysFullScreen modes | +| `packages/compositor/src/userapi/UserShellApi.ts` | The integration seam — events + actions | diff --git a/09-hardware-accel.md b/09-hardware-accel.md new file mode 100644 index 0000000..162703d --- /dev/null +++ b/09-hardware-accel.md @@ -0,0 +1,522 @@ +# 09 — Hardware Acceleration, Encoding, WebCodecs, GPU Textures + +This chapter covers the encoding/decoding/WASM pipeline in the MIT +approach. **Critical changes from Greenfield**: +- **No x264enc** (GPL-2.0+) — replaced with `vaapih264enc`/`nvh264enc`/`openh264enc` +- **No FFmpeg Wasm** (GPL-3.0 in Greenfield's build) — replaced with WebCodecs primary + OpenH264 BSD fallback +- **Hardware decode enabled** (Greenfield disables it with a FIXME) +- **Adaptive bitrate** (consumes `EncoderFeedback` data Greenfield produces but doesn't use) + +## What's reused (MIT) + +### Browser side + +| Component | License | Role | +|---|---|---| +| `@gfld/compositor-wasm` | MIT | pixman 0.42.2 + xkbcommon 1.5.0 WASM (region math + keymap) | + +### Vendored C (MIT, used in Python proxy via cffi) + +| File | Lines | Role | +|---|---|---| +| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM | +| `westfield-egl.c` | ~900 | EGL zero-copy setup (EGLDisplay, EGLContext, EGLImage) | +| `wlr_drm.c` | ~244 | `wl_drm` global impl | +| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl | +| `drm_format_set.c` | ~239 | DRM format helpers | + +## What's replaced (proprietary, clean-room) + +### Browser side + +| Component | Replaces | Why | +|---|---|---| +| Custom `WebCodecsFrameDecoder.ts` | Greenfield's `webcodec-buffer-decoder.ts` (AGPL) | Clean-room; **enables hardware decode** (Greenfield disables with FIXME) | +| Custom `OpenH264FrameDecoder.ts` | Greenfield's `wasm-buffer-decoder.ts` + `H264NALDecoder.worker.ts` (AGPL) + `@gfld/compositor-ffmpeg-h264` (GPL) | Clean-room; uses **OpenH264 BSD Wasm** instead of FFmpeg GPL Wasm | +| Custom `YUVA2RGBA` / `RGBXandA2RGBA` shaders | Greenfield's (AGPL) | Clean-room; **WebGL2** (Greenfield uses WebGL1) | +| Custom `Renderer` / `Scene` / `SceneShader` | Greenfield's (AGPL) | Clean-room; **OffscreenCanvas + Web Workers** (Greenfield is main-thread) | + +### Python side + +| Component | Replaces | Why | +|---|---|---| +| Custom `gst_encoder.py` (~700 LOC) | Greenfield's `proxy-encoding-addon.node` (AGPL, 290 lines C) + `libproxy-encoding.so` (AGPL, ~1400 lines C) | Pure Python via PyGObject. Same GStreamer, no N-API addon. | +| `vaapih264enc` / `nvh264enc` / `openh264enc` | `x264enc` (GPL) | GPL-free H.264 encoding | + +## Encoder selection (per-frame, deterministic) + +Same logic as Greenfield, but **without x264enc**: + +| name | element | when used | split_alpha | coded-size multiple | License | +|---|---|---|---|---|---| +| `vaapih264` | `vaapih264enc` | >256×256 + ARGB/XRGB/dmabuf, Intel/AMD GPU | true (parallel alpha H.264 stream) | 128 | LGPL + MIT (libva) | +| `nvh264` | `nvh264enc` | same, NVIDIA GPU | true | 128 | BSD plugin + proprietary NVENC blob | +| `openh264` | `openh264enc` | same, no GPU (CPU fallback) | true | 128 | BSD-2-Clause (libopenh264) | +| `png` | `pngenc` | ≤256×256 + ARGB/XRGB/dmabuf | false (PNG has native alpha) | 16 | LGPL | + +**Rule**: pixel_count ≤ 256×256 → PNG; otherwise the configured H.264 +encoder (selected by `--encoder=vaapih264|nvh264|openh264`). + +**NOTE**: This means the **default encoder is hardware-accelerated** +(vaapih264 on Intel/AMD, nvh264 on NVIDIA). Greenfield defaults to x264 +(software, GPL). + +## Canonical H.264 pipeline (Python equivalent) + +Same GStreamer pipeline as Greenfield, ported to PyGObject: + +```python +# gst_encoder.py (clean-room, ~700 LOC) +import gi +gi.require_version("Gst", "1.0") +gi.require_version("GstGL", "1.0") +gi.require_version("GstEGL", "1.0") # if available +from gi.repository import Gst, GstGL, GLib + +class H264Encoder: + def __init__(self, egl_display_ptr, egl_context_ptr, encoder_type="vaapih264"): + # Create GLDisplay + GLContext from existing EGL context (zero-copy) + self.gl_display = GstGL.GLDisplayEGL.new_with_egl_display(egl_display_ptr) + self.gl_context = GstGL.GLContext.new_wrapped( + self.gl_display, + egl_context_ptr, + GstGL.GLPlatform.EGL, + GstGL.GLLAPI.GLES2, + ) + + # Encoder pipeline + encoder_element = { + "vaapih264": "vaapih264enc", + "nvh264": "nvh264enc", + "openh264": "openh264enc", + }[encoder_type] + + self.pipeline = Gst.parse_launch( + f"appsrc name=src ! " + f"glupload ! glcolorconvert ! " + f"glshader name=scale ! " # pads to macroblock multiple + f"glcolorconvert ! video/x-raw(memory:GLMemory),format=NV12 ! " + f"gldownload ! queue ! " + f"{encoder_element} tune=zerolatency bitrate=12000 vbv-buf-capacity=160 " + f"bframes=0 rc-lookahead=0 sliced-threads byte-stream=true ! " + f"video/x-h264,profile=high,stream-format=byte-stream,alignment=au ! " + f"appsink name=sink" + ) + + # Wire up appsink callback (zero-copy external Buffers) + sink = self.pipeline.get_by_name("sink") + sink.emit("set-property", "emit-signals", True) + sink.connect("new-sample", self._on_new_sample) + + # Alpha pipeline (separate, for transparent windows) + self.alpha_pipeline = self._build_alpha_pipeline(encoder_element) + + # GLib MainLoop in separate thread (GStreamer requires it) + self.glib_loop = GLib.MainLoop() + self.glib_context = GLib.MainContext.new() + GLib.MainContext.push_thread_default(self.glib_context) + import threading + self.glib_thread = threading.Thread(target=self.glib_loop.run, daemon=True) + self.glib_thread.start() + + def encode_frame(self, buffer_id, content_serial, creation_serial, dma_buf_fd, w, h): + # Push buffer into appsrc (zero-copy via GstBuffer wrapping the dmabuf) + # ... marshalled from asyncio via GLib.idle_add + + def _on_new_sample(self, appsink): + sample = appsink.emit("pull-sample") + buffer = sample.get_buffer() + # Marshal back to asyncio + asyncio.run_coroutine_threadsafe( + self._deliver_frame(buffer, self.current_buffer_id), + self.asyncio_loop, + ) +``` + +### Bitrates (same as Greenfield) + +- Opaque: 12000 (vaapih264 / nvh264 / openh264) +- Alpha: 1200 (10% of opaque) + +### Adaptive bitrate (NEW — Greenfield lacks) + +Consume `EncoderFeedback` data (refreshInterval + decode duration): + +```python +# adaptive_bitrate.py (clean-room, ~150 LOC) +class AdaptiveBitrate: + def __init__(self, encoder): + self.encoder = encoder + + def on_feedback(self, refresh_interval_ms, avg_decode_ms): + # If decode is taking too long, reduce bitrate + if avg_decode_ms > refresh_interval_ms * 0.8: + new_bitrate = max(self.current_bitrate * 0.7, MIN_BITRATE) + self.encoder.set_property("bitrate", new_bitrate) + # If decode is fast and we have headroom, increase bitrate + elif avg_decode_ms < refresh_interval_ms * 0.3: + new_bitrate = min(self.current_bitrate * 1.3, MAX_BITRATE) + self.encoder.set_property("bitrate", new_bitrate) +``` + +## Wire format (REUSED from Greenfield) + +Same binary format as Greenfield (see `../greenfield-analysis/09-hardware-accel.md`): + +``` +struct packed props { + u32 buffer_id, creation_serial, content_serial, + u32 frame_encoding_type (0=h264, 1=png), + u32 width, height, coded_width, coded_height +} +u32 opaque_length +u8 opaque[opaque_length] // H.264 AU or PNG blob +u32 alpha_length // 0 if no alpha split +u8 alpha[alpha_length] // H.264 AU of alpha-only stream +``` + +## Browser decode (clean-room TS) + +### Preferred path: WebCodecs `VideoDecoder` (with HW decode enabled) + +```typescript +// WebCodecsFrameDecoder.ts (clean-room, ~250 LOC) +export class WebCodecsFrameDecoder implements FrameDecoder { + private opaqueDecoders: Map = new Map(); + private alphaDecoders: Map = new Map(); + + async isSupported(): Promise { + const config = { + codec: 'avc1.64001f', + optimizeForLatency: true, + // NEW: enable hardware acceleration (Greenfield disables this) + hardwareAcceleration: 'prefer-hardware', + }; + const result = await VideoDecoder.isConfigSupported(config); + return result.supported; + } + + async decode(surface: Surface, encodedFrame: EncodedFrame): Promise { + let opaqueDecoder = this.opaqueDecoders.get(surface.id); + if (!opaqueDecoder) { + opaqueDecoder = new VideoDecoder({ + output: (frame) => this.onDecoded(surface, frame), + error: (e) => this.onError(surface, e), + }); + opaqueDecoder.configure({ + codec: 'avc1.64001f', + optimizeForLatency: true, + hardwareAcceleration: 'prefer-hardware', // NEW + }); + this.opaqueDecoders.set(surface.id, opaqueDecoder); + } + + const chunk = new EncodedVideoChunk({ + type: encodedFrame.isKeyframe ? 'key' : 'delta', + data: encodedFrame.opaque, + timestamp: encodedFrame.timestamp, + }); + opaqueDecoder.decode(chunk); + + // Alpha path (if present) + if (encodedFrame.alpha) { + // Same pattern for alphaDecoder + } + } +} +``` + +**Key difference from Greenfield**: We use `hardwareAcceleration: +'prefer-hardware'` (Greenfield uses `'prefer-software'` with a FIXME +about "frame offset is wrong"). This needs testing, but should give +significant CPU savings. + +### Fallback path: OpenH264 BSD Wasm (NOT FFmpeg — FFmpeg is GPL) + +```typescript +// OpenH264FrameDecoder.ts (clean-room, ~250 LOC) +// Uses OpenH264 BSD-2-Clause Wasm build (Cisco's binary release or self-built) +// Available from: https://github.com/cisco/openh264 (BSD-2-Clause) +``` + +OpenH264 is BSD-2-Clause — fully proprietary-compatible. Less featureful +than FFmpeg (no VP9/AV1), but sufficient for H.264 fallback. + +## WebGL2 upload (clean-room TS, modernized) + +WebGL2 with `desynchronized:true` + OffscreenCanvas in Web Worker. + +Per Scene (canvas/output), three GL helpers (same as Greenfield, but +WebGL2 instead of WebGL1): + +### `YUVA2RGBA` + +FBO pass: uploads Y/U/V/alphaY as **4× LUMINANCE textures** (WebGL1 +compatible) OR **1× R8UI texture array** (WebGL2, more efficient), +renders into the surface's RGBA texture via `YUVA2RGBAShader` (BT.601 +coefficients). + +### `RGBXandA2RGBA` + +FBO pass for the `DualPlaneRGBA*` WebCodecs outputs (opaque + alpha +planes composited via `RGBandA2RGBAShader`). + +### `SceneShader` + +Final pass: draws every view's RGBA texture to the canvas with +`gl.blendFunc(ONE, ONE_MINUS_SRC_ALPHA)`. + +**NEW**: WebGL2 `texStorage2D` for immutable textures (faster upload). +**NEW**: `VideoFrame` directly as a WebGL texture source (zero-copy from +WebCodecs to GPU, no `copyTo` needed). + +## End-to-end pipeline + +``` +┌──────────────────── NATIVE CLIENT PROCESS ────────────────────┐ +│ App renders via GTK/Qt/raw EGL into │ +│ ┌────────────────────────────────────┐ │ +│ │ wl_shm buffer (ARGB8888/XRGB8888)│ ─── or ─────────┐ │ +│ │ zwp_linux_dmabuf_v1 (DMA-BUF) │ │ │ +│ │ wl_drm create_prime_buffer │ │ │ +│ └────────────────────────────────────┘ │ │ +└────────────────────────┬──────────────────────────────────┼────┘ + │ Wayland socket (UNIX domain) │ + ▼ │ +┌────────────────────── PYTHON PROXY (aiohttp + pywayland + GStreamer)─┐ +│ pywayland (Apache-2.0) │ +│ └─ Display → Client → Resource (buffer) │ +│ └─ display.init_shm(), init_drm("/dev/dri/renderD128") │ +│ └─ Vendored westfield-egl.c (MIT, via cffi): │ +│ EGLDevice + EGLDisplay + EGLContext │ +│ │ +│ gst_encoder.py (~700 LOC Python, clean-room) │ +│ ┌─ frame_buffer {type: SHM|DMA, w, h, fmt} │ +│ │ │ +│ │ GLib MainLoop in separate thread (GLib.idle_add) │ +│ │ │ │ +│ │ ▼ │ +│ │ gst_parse_launch: │ +│ │ ┌─ Encoder selection (per-frame): │ +│ │ │ pixel_count ≤ 256×256 ─► PNG pipeline │ +│ │ │ > 256×256 + ARGB/XRGB/dmabuf ─► configured h264 │ +│ │ │ │ +│ │ │ H.264 pipelines (parallel, NO x264 — GPL): │ +│ │ │ opaque: appsrc → glupload → glcolorconvert → │ +│ │ │ glshader(scale→coded_size) → │ +│ │ │ glcolorconvert(NV12) → gldownload → │ +│ │ │ queue → vaapih264enc|nvh264enc|openh264enc │ +│ │ │ → byte-stream AU → appsink │ +│ │ │ alpha: same source, fragment shader │ +│ │ │ `vec4(pix.a,pix.a,pix.a,0.)` → │ +│ │ │ second H.264 stream │ +│ │ │ │ +│ │ │ PNG pipeline (single): │ +│ │ │ appsrc → glupload → glcolorconvert → glshader → │ +│ │ │ RGBA → gldownload → pngenc → appsink │ +│ │ │ │ +│ │ └─ asyncio.run_coroutine_threadsafe for cross-thread │ +│ │ │ +│ └─ surface_buffer_encoding.py │ +│ └─ frame_data_channel.send(buffer) │ +│ │ +│ channel.py: │ +│ ARQChannel (kcp PyPI package, MIT, over WebSocket) │ +│ SimpleChannel (raw WebSocket, for FEEDBACK only) │ +│ ▲ │ +│ │ frame_feedback.py: 16-bit rAF interval + 16-bit avg │ +│ │ decode duration → used to pace wl_frame.done AND │ +│ │ drive adaptive_bitrate.py (NEW vs Greenfield) │ +└───────┼───────────────────────────────────────────────────────┘ + │ 4 logical channels per client: PROTOCOL / FRAME / XWM / FEEDBACK + ▼ +┌────────────────────── BROWSER (Custom Compositor TS, clean-room) ──────┐ +│ WebSocket → ReconnectingWebSocket (vendored MIT) │ +│ ↓ │ +│ channel.ts: ARQChannel.input(kcp) → onMessage │ +│ ↓ │ +│ RemoteAppLauncher.setupFrameDataChannel → │ +│ BufferStream.deliverContentToBufferStream (parse serials) │ +│ ↓ │ +│ StreamingBuffer.getContents(surface, serial) │ +│ ↓ │ +│ Session.frameDecoder.decode(surface, encodedFrame) │ +│ │ │ +│ ├─ PNG path (both decoders): │ +│ │ new Blob([opaque], {type:'image/png'}) → │ +│ │ createImageBitmap(blob, sx, sy, w, h) // crop padding │ +│ │ → SinglePlane { bitmap: ImageBitmap } │ +│ │ │ +│ └─ H.264 path: │ +│ (preferred) WebCodecs VideoDecoder │ +│ config: avc1.64001f, prefer-hardware (NEW!), │ +│ optimizeForLatency │ +│ opaqueDecoder + alphaDecoder per surface │ +│ EncodedVideoChunk {type:key|delta, data:nal} │ +│ → VideoFrame.output → │ +│ if I420: copyTo → DualPlaneYUVAArrayBuffer │ +│ else: keep VideoFrame → DualPlaneRGBAVideoFrame │ +│ (fallback) OpenH264 BSD Wasm in 2 Web Workers │ +│ (NOT FFmpeg — GPL contamination) │ +│ worker.postMessage(nal, [nal.buffer]) // transferable │ +│ OpenH264._decode → I420 planes │ +│ postMessage back with [y.buffer, u.buffer, v.buffer] │ +│ → DualPlaneYUVASplitBuffer │ +│ │ +│ Renderer.render() (rAF-driven, in Web Worker via OffscreenCanvas)│ +│ ↓ │ +│ Scene.render(viewStack) │ +│ ├─ Scene['image/png'] → texImage2D(ImageBitmap) │ +│ ├─ Scene['image/bitmap'] (web-bitmapbuf path) → texImage2D │ +│ └─ Scene['video/h264']: │ +│ YUVA2RGBA.convertYUVASplitBufferInto / │ +│ convertYUVAArrayBufferInto: │ +│ Y/U/V/alphaY → 4× LUMINANCE textures (WebGL1 compat) │ +│ OR 1× R8UI texture array (WebGL2, more efficient) │ +│ bind FBO → renderState.texture (RGBA) as color attach │ +│ YUVA2RGBAShader (BT.601) → drawArrays → RGBA in tex │ +│ or RGBXandA2RGBA.convertInto (WebCodecs RGBA): │ +│ VideoFrame/ImageBitmap → texImage2D directly │ +│ ↓ │ +│ SceneShader.draw(): drawArrays with surface's RGBA texture, │ +│ blend gl.ONE / gl.ONE_MINUS_SRC_ALPHA, desynchronized canvas │ +│ ↓ │ +│ Browser composites canvas into the page │ +└─────────────────────────────────────────────────────────────────┘ + + +Same-frame web app fast path (web-bitmapbuf, REUSED): +┌───────────────┐ MessagePort.postMessage([view, ...fds], ┌──────────────┐ +│ Web app │ ──► [imageBitmap]) ──────────────────────────► │ Compositor │ +│ (iframe) │ web_bitmapbuf_factory.create_buffer(id, fd) │ WebConnHandl │ +└───────────────┘ → wl_buffer → ImageBitmapBuffer └──────────────┘ + (no encoding, no WebSocket, no GStreamer) +``` + +## Key answers (quick reference) + +### Why H.264 over PNG? + +H.264 at 12 Mbps × 1/60s ≈ 25 KB/frame for 1080p, vs PNG 100 KB – 3 MB. +PNG wins for ≤256×256 because H.264 I-frame overhead and alpha-split +doubling dominate at small sizes. + +### WebCodecs? + +Yes — preferred path. **Hardware decode enabled** (Greenfield disables +with FIXME). OpenH264 BSD Wasm is the fallback (NOT FFmpeg — GPL). + +### Trade-offs? + +- H.264 = small wire / decode cost +- PNG = medium wire / browser-native fast decode (`createImageBitmap`) +- raw SHM = **not used** between proxy and browser — every surface commit + is encoded. + +### Per-surface policy? + +No — per-frame deterministic. Only configurable knob is +`--encoder=vaapih264|nvh264|openh264`. PNG is always automatic fallback. + +### Cursors? + +Always PNG (32×32 ≤ 256×256). Decoded to ImageBitmap and set as the CSS +cursor via `setBrowserCursor` — never enters the WebGL compositor. + +### Alpha pre-multiplication? + +Implicit in shaders (same as Greenfield). Works acceptably for typical +Wayland content. + +### OffscreenCanvas / Web Workers? + +**Yes** — Render Worker (WebGL2 + OffscreenCanvas) + Decode Worker +(WebCodecs VideoDecoder). Greenfield uses only 2 workers for the WASM +fallback path; our MIT approach uses Workers for ALL rendering. + +### Adaptive bitrate (NEW)? + +**Yes** — `adaptive_bitrate.py` consumes `EncoderFeedback` data and +adjusts `vaapih264enc`/`nvh264enc`/`openh264enc` bitrate at runtime. +Greenfield produces the feedback data but doesn't use it. + +## Notable design trade-offs for the MIT approach + +1. **Hardware-accelerated encoding by default** (vaapih264 on Intel/AMD, + nvh264 on NVIDIA). Greenfield defaults to x264 (software, GPL). +2. **Hardware-accelerated decoding by default** (WebCodecs + `prefer-hardware`). Greenfield uses `prefer-software` with a FIXME. +3. **No GPL anywhere in the encoding/decoding pipeline** (no x264, no + FFmpeg Wasm). +4. **Adaptive bitrate** consumes the feedback data Greenfield ignores. +5. **OpenH264 BSD Wasm** as fallback (Cisco's binary release or + self-built). Sufficient for H.264 fallback when WebCodecs is + unavailable. +6. **OffscreenCanvas + Web Workers** for all rendering (Greenfield is + main-thread). +7. **WebGL2** instead of WebGL1 (uniform buffer objects, simplified + shaders). +8. **VideoFrame directly as WebGL texture source** (zero-copy from + WebCodecs to GPU, no `copyTo` needed). +9. **Same alpha-split H.264 approach** as Greenfield (doubles cost for + transparent windows) — could be replaced with AV1 with native alpha + support in the future. +10. **Future: AV1/VP9 support** via WebCodecs (when browser support is + widespread) — sidesteps H.264 MPEG-LA patent royalties entirely. + +## File reference + +### Browser side (custom, clean-room TS) + +| Path | Role | +|---|---| +| `packages/compositor/src/transport/decoders/WebCodecsFrameDecoder.ts` | WebCodecs VideoDecoder (HW-accelerated) | +| `packages/compositor/src/transport/decoders/OpenH264FrameDecoder.ts` | OpenH264 BSD Wasm fallback | +| `packages/compositor/src/transport/decoders/FrameDecoder.ts` | Decoder interface | +| `packages/compositor/src/transport/EncodedFrame.ts` | Wire format parser for encoded frames | +| `packages/compositor/src/transport/EncoderFeedback.ts` | Sends 4-byte decode feedback at 1 Hz | +| `packages/compositor/src/transport/StreamingBuffer.ts` | Per-surface encoded-frame buffer | +| `packages/compositor/src/transport/BufferStream.ts` | Manages serial → frame promise resolution | +| `packages/compositor/src/render/Renderer.ts` | 4-phase render pipeline (WebGL2 + OffscreenCanvas) | +| `packages/compositor/src/render/Scene.ts` | One per output, manages view stack + textures | +| `packages/compositor/src/render/YUVA2RGBA.ts` | YUVA→RGBA FBO pass | +| `packages/compositor/src/render/YUVA2RGBAShader.ts` | BT.601 YUVA→RGBA shader | +| `packages/compositor/src/render/RGBXandA2RGBA.ts` | RGB+Alpha → RGBA FBO pass (WebCodecs output) | + +### Browser side (REUSED MIT) + +| Path | Role | +|---|---| +| `libs/compositor-wasm/src/index.ts` | pixman + xkbcommon WASM (region math + keymap) | + +### Python side (custom, clean-room) + +| Path | Role | +|---|---| +| `proxy/encoding/gst_encoder.py` | GStreamer H.264/PNG encoder (~700 LOC) | +| `proxy/encoding/adaptive_bitrate.py` | Adaptive bitrate controller (NEW) | +| `proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration | +| `proxy/proxy_buffer.py` | Proxy-side buffer representation | +| `proxy/frame_feedback.py` | Receives 4-byte decode feedback | +| `proxy/encoding/glib_thread.py` | GLib MainLoop in separate thread + asyncio marshalling | + +### Vendored Greenfield C files (MIT, used via cffi) + +| Path | Role | +|---|---| +| `native/westfield-egl.c` | EGL zero-copy setup | +| `native/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM | +| `native/wlr_drm.c` | `wl_drm` global impl | +| `native/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl | +| `native/drm_format_set.c` | DRM format helpers | + +### Reused (LGPL/BSD/MIT) + +| Component | Role | +|---|---| +| PyGObject (`gi.repository.Gst`) | GStreamer Python bindings (LGPL-2.1+) | +| GStreamer core | Media pipeline framework (LGPL-2.0+) | +| gst-plugins-base, gst-plugins-good | PNG, appsrc, appsink, videoconvert (LGPL) | +| gst-vaapi | `vaapih264enc` for Intel/AMD (LGPL + MIT libva) | +| gst-plugins-bad (BSD subset) | `nvh264enc` for NVIDIA, `openh264enc` for CPU fallback | +| libopenh264 | OpenH264 BSD Wasm fallback decoder (BSD-2-Clause) | diff --git a/10-odoo-integration.md b/10-odoo-integration.md new file mode 100644 index 0000000..d8d6763 --- /dev/null +++ b/10-odoo-integration.md @@ -0,0 +1,659 @@ +# 10 — Odoo 20 Integration (OWL Shell + Backend Addon + JWT + Nginx + Docker) + +This chapter covers the Odoo 20 integration layer in the MIT approach. +**Critical finding**: Odoo's built-in WebSocket server is **unsuitable** +for compositor-scale traffic. We need a separate aiohttp Python process +for the compositor proxy. + +## Why Odoo's built-in WebSocket server is unsuitable + +Odoo does **NOT** use Werkzeug's WebSocket support (Werkzeug has no +first-party WS). Instead, Odoo ships its own pure-Python RFC 6455 +implementation in `addons/bus/websocket.py` (~900 LOC) running in a +**separate gevent worker process** on port 8072. + +### Hard limits that disqualify it for compositor use + +- **`MESSAGE_MAX_SIZE = 2**20 = 1 MB`** per frame (cannot be configured) + — too small for H.264 video frames + multiplexed channels +- **Shared PostgreSQL connection pool** — sustained WS streaming starves + other Odoo requests +- **Single gevent process** — one Python GIL for ALL Discuss/chat/presence + traffic +- `ir.websocket` only dispatches two events: `subscribe` and + `update_presence`. New event types require forking + `addons/bus/websocket.py`. +- Odoo.sh user confirmed edits to `ir.websocket` need gevent-worker restart + +### Conclusion + +Run a **separate aiohttp Python process** (port 8081) for the compositor +proxy. Route via Nginx for single-origin HTTPS. + +## Recommended architecture + +``` + ┌─────────────────────────────────────────────┐ + │ User's Browser │ + │ ┌─────────────────────────────────────────┐ │ + │ │ Odoo web client + OWL Compositor Shell │ │ + │ │ (ir.actions.client, target=fullscreen) │ │ + │ │ ┌──────────────────────────────────┐ │ │ + │ │ │ + WebGL compositor lib │ │ │ + │ │ │ (MIT-licensed browser compositor) │ │ │ + │ │ └──────────────────────────────────┘ │ │ + │ └─────────────────────────────────────────┘ │ + └────────────────────┬────────────────────────┘ + │ + HTTPS (single origin: https://your-odoo.example.com) + session_id cookie + JWT (in JS payload) + │ + ┌────────────────────▼────────────────────────┐ + │ Nginx (TLS termination) │ + │ / → Odoo :8069 (HTTP workers) │ + │ /websocket → Odoo :8072 (gevent worker) │ + │ /compositor/ → Proxy :8081 (aiohttp WS) │ + └─────────┬──────────────────┬─────────────────┘ + │ │ + ┌────────────────┴─────────┐ ┌────▼───────────────────┐ + │ Odoo 20 (HTTP) │ │ Odoo 20 (geevent) │ + │ N preforked workers │ │ 1 process │ + │ │ │ /websocket (Discuss, │ + │ Controllers: │ │ chatter, presence) │ + │ /web/compositor/token │ │ bus.bus notifications│ + │ /web/compositor/apps │ │ (NOT for compositor!)│ + │ /web/compositor/launch │ │ 1 MB cap, pgsql pool │ + │ /web/compositor/terminate│ └───────────────────────┘ + │ /web/compositor/audit │ + │ Models: │ + │ compositor.app │ + │ compositor.session │ + │ (mail.thread mixin) │ + │ compositor.audit.log │ + │ (append-only) │ + │ JWT issuer (HS256 secret│ + │ in ir.config_parameter)│ + └────────────┬─────────────┘ + │ + ▼ + ┌────────────────────────────────────────────────────┐ + │ PostgreSQL (Odoo schema) │ + └────────────────────────────────────────────────────┘ + ▲ + ┌────────────────────────────────────────────────┴──┐ + │ Compositor Proxy (aiohttp, :8081) │ + │ Async Python (asyncio + aiohttp) │ + │ ┌────────────────────────────────────────────┐ │ + │ │ WS server (aiohttp.web.Application) │ │ + │ │ /signal, /channel, /mkfifo, /mkstemp-mmap,│ │ + │ │ /read-fd, /write-fd-as-stream, │ │ + │ │ /request-keyframe │ │ + │ └────────────────────────────────────────────┘ │ + │ ┌────────────────────────────────────────────┐ │ + │ │ JWT validator (pyjwt, HS256, secret env) │ │ + │ └────────────────────────────────────────────┘ │ + │ ┌────────────────────────────────────────────┐ │ + │ │ Per-session fork: pywayland + bwrap + Gst │ │ + │ │ WAYLAND_DISPLAY=wayland-N, /dev/dri │ │ + │ └────────────────────────────────────────────┘ │ + │ ┌────────────────────────────────────────────┐ │ + │ │ Audit callback → Odoo /web/compositor/audit│ │ + │ └────────────────────────────────────────────┘ │ + └────────────────────────────────────────────────────┘ +``` + +## Auth bridging — JWT (HS256) is recommended + +### Why JWT (not session cookie or OAuth2) + +- Odoo's `session_id` cookie reuse pattern is too fragile and adds Odoo round-trips per WS message +- OAuth2 is overkill — we control both Odoo and the proxy +- JWT (HS256, symmetric) is stateless — no DB lookup per frame, fast validation + +### JWT payload + +```json +{ + "iss": "odoo.example.com", + "aud": "compositor-proxy", + "sub": 42, // Odoo user ID + "db": "production", // Odoo database name + "company_ids": [1, 3], + "scope": "compositor", + "compositor_session_id": "uuid-here", + "iat": 1736000000, + "exp": 1736000900 +} +``` + +- 15-minute expiry +- Signed with `compositor.jwt_secret` from Odoo's `ir.config_parameter` + (shared via env var at proxy startup) +- Validated by the proxy using `pyjwt` (MIT license) + +### Issuance flow + +1. OWL shell calls `/web/compositor/token` (Odoo controller) +2. Odoo backend signs JWT with the shared secret +3. Returns `{ jwt, proxy_url, expires_at }` to the OWL shell +4. OWL shell passes JWT to the compositor library +5. Compositor library includes `Authorization: Bearer ` on every WebSocket connection + +## Multi-tenancy + +### Recommended: one compositor proxy per Odoo database + +- `db` claim in JWT scopes every WS to one Odoo database +- Per-session wayland socket (`$XDG_RUNTIME_DIR/wayland-N`) isolates Wayland displays +- Per-session forked worker process (matches Greenfield's existing model) +- For SaaS with strict SLA: Docker container per tenant + bubblewrap per session + +### Optional: per-tenant Docker containers + +For strict isolation between tenants: +- One Docker container per Odoo database, each running its own compositor proxy +- Nginx routes based on the JWT's `db` claim +- `--device /dev/dri:/dev/dri` for GPU passthrough +- `--cpus` and `--memory` limits per container + +## OWL shell integration — `ir.actions.client` with `target=fullscreen` + +### Critical Odoo 20 / OWL 3.x gotcha + +`useState()` is **REMOVED** in Odoo 20. Must use `proxy()` from `@odoo/owl` +instead. This is a breaking change from Odoo 19, verified by Cybrosys +Sep 2026 blog + OWL migration guide gist. + +### OWL shell component (pseudocode) + +```typescript +// compositor_shell/static/src/components/compositor_shell.js +import { Component } from "@odoo/owl"; +import { useService } from "@web/core/utils/hooks"; +import { createCompositorSession } from "@odoo/compositor/compositor_lib"; + +export class CompositorShell extends Component { + setup() { + this.rpc = useService("rpc"); + this.notification = useService("notification"); + this.state = proxy({ + surfaces: new Map(), // id → { title, appId, active } + clients: new Map(), + }); + this.canvasRef = useRef("canvas"); + + onMounted(async () => { + // 1. Get JWT from Odoo backend + const { jwt, proxy_url, expires_at } = await this.rpc("/web/compositor/token"); + this.jwt = jwt; + this.proxyUrl = proxy_url; + + // 2. Initialize compositor session + const session = createCompositorSession({ + jwt, + proxyUrl: proxy_url, + }); + this.session = session; + + // 3. Wire UserShellApi events to OWL reactive state + session.userShell.events.clientCreated = (client) => { + this.state.clients.set(client.id, { id: client.id }); + }; + session.userShell.events.clientDestroyed = (client) => { + this.state.clients.delete(client.id); + }; + session.userShell.events.surfaceCreated = (surface) => { + this.state.surfaces.set(surface.id, { + title: "", appId: "", active: false, + }); + }; + session.userShell.events.surfaceDestroyed = (surface) => { + this.state.surfaces.delete(surface.id); + }; + session.userShell.events.surfaceTitleUpdated = (surface, title) => { + const s = this.state.surfaces.get(surface.id); + if (s) s.title = title; + }; + session.userShell.events.surfaceActivationUpdated = (surface, active) => { + // Deactivate all, then activate the new one + for (const s of this.state.surfaces.values()) { + s.active = false; + } + const s = this.state.surfaces.get(surface.id); + if (s) s.active = active; + }; + // NEW vs Greenfield: surface state changes + session.userShell.events.surfaceStateChanged = (surface, state) => { + const s = this.state.surfaces.get(surface.id); + if (s) Object.assign(s, state); + }; + + // 4. Initialize the canvas (OWL owns the lifecycle) + const canvas = this.canvasRef.el; + session.userShell.actions.initScene(() => ({ + canvas, + width: canvas.clientWidth, + height: canvas.clientHeight, + })); + }); + } + + // User clicks app in launcher + async launchApp(appId) { + try { + const response = await this.rpc("/web/compositor/launch", { + app_id: appId, + }); + // Backend returns { signal_url, pid, name } + // The compositor library handles the rest (opens WS, etc.) + } catch (e) { + this.notification.add("Failed to launch app: " + e.message, { type: "danger" }); + } + } + + // User clicks taskbar entry + activateSurface(surfaceId) { + this.session.userShell.actions.activateSurface({ id: surfaceId }); + } + + // User closes window + closeSurface(surfaceId) { + this.session.userShell.actions.closeSurface({ id: surfaceId }); + } +} + +CompositorShell.template = "compositor_shell.CompositorShell"; +CompositorShell.target = "fullscreen"; // Odoo ir.actions.client target +``` + +### Odoo action registration + +```xml + + + Compositor Shell + compositor_shell + fullscreen + +``` + +## Backend addon structure + +### Models + +```python +# compositor_shell/models/compositor_app.py +from odoo import models, fields, api + +class CompositorApp(models.Model): + _name = "compositor.app" + _description = "Compositor Application" + + name = fields.Char(required=True, translate=True) + app_id = fields.Char(required=True, help="Unique identifier, e.g. 'gnome-calculator'") + executable = fields.Char(required=True, help="Binary path, e.g. '/usr/bin/gnome-calculator'") + icon = fields.Image() + description = fields.Text() + + allowed_group_ids = fields.Many2many("res.groups", string="Allowed Groups") + memory_limit_mb = fields.Integer(default=1024, help="Max memory per app instance (MB)") + cpu_quota_percent = fields.Integer(default=200, help="CPU quota (100% = 1 core)") + + active = fields.Boolean(default=True) +``` + +```python +# compositor_shell/models/compositor_session.py +from odoo import models, fields, api + +class CompositorSession(models.Model): + _name = "compositor.session" + _description = "Compositor Session" + _inherit = ["mail.thread"] + + user_id = fields.Many2one("res.users", required=True, tracking=True) + compositor_session_id = fields.Char(required=True, help="UUID") + + app_ids = fields.One2many("compositor.app.instance", "session_id", string="Apps") + app_count = fields.Integer(compute="_compute_app_count") + + state = fields.Selection([ + ("active", "Active"), + ("disconnected", "Disconnected"), + ("closed", "Closed"), + ], default="active", tracking=True) + + started_at = fields.Datetime(default=fields.Datetime.now) + ended_at = fields.Datetime() + + @api.depends("app_ids") + def _compute_app_count(self): + for record in self: + record.app_count = len(record.app_ids) +``` + +```python +# compositor_shell/models/compositor_audit_log.py +from odoo import models, fields + +class CompositorAuditLog(models.Model): + _name = "compositor.audit.log" + _description = "Compositor Audit Log (append-only)" + _order = "create_date desc" + + user_id = fields.Many2one("res.users", required=True) + app_id = fields.Char(required=True) + action = fields.Selection([ + ("launch", "App Launched"), + ("terminate", "App Terminated"), + ("crash", "App Crashed"), + ], required=True) + pid = fields.Integer() + exit_code = fields.Integer() + timestamp = fields.Datetime(default=fields.Datetime.now) + + def write(self, vals): + raise UserError("Audit logs are append-only and cannot be modified.") + + def unlink(self): + raise UserError("Audit logs are append-only and cannot be deleted.") +``` + +### Controllers + +```python +# compositor_shell/controllers/main.py +from odoo import http +from odoo.http import request +import jwt, datetime, uuid + +class CompositorController(http.Controller): + + @http.route("/web/compositor/token", type="json", auth="user") + def get_token(self): + user = request.env.user + secret = request.env["ir.config_parameter"].sudo().get_param("compositor.jwt_secret") + if not secret: + raise UserError("Compositor JWT secret not configured") + + compositor_session_id = str(uuid.uuid4()) + payload = { + "iss": request.http.host, + "aud": "compositor-proxy", + "sub": user.id, + "db": request.db, + "company_ids": user.company_ids.ids, + "scope": "compositor", + "compositor_session_id": compositor_session_id, + "iat": datetime.datetime.utcnow(), + "exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=15), + } + token = jwt.encode(payload, secret, algorithm="HS256") + + # Create session record + request.env["compositor.session"].sudo().create({ + "user_id": user.id, + "compositor_session_id": compositor_session_id, + }) + + proxy_url = request.env["ir.config_parameter"].sudo().get_param( + "compositor.proxy_url", default="http://localhost:8081" + ) + return { + "jwt": token, + "proxy_url": proxy_url, + "expires_at": payload["exp"].isoformat(), + "compositor_session_id": compositor_session_id, + } + + @http.route("/web/compositor/apps", type="json", auth="user") + def list_apps(self): + user = request.env.user + apps = request.env["compositor.app"].search_read( + [("allowed_group_ids", "in", user.groups_id.ids), ("active", "=", True)], + ["name", "app_id", "icon", "description"], + ) + return apps + + @http.route("/web/compositor/launch", type="json", auth="user") + def launch_app(self, app_id): + user = request.env.user + app = request.env["compositor.app"].search([ + ("app_id", "=", app_id), + ("allowed_group_ids", "in", user.groups_id.ids), + ], limit=1) + if not app: + raise UserError("App not found or not allowed") + + # Get JWT (re-issued here for the proxy call) + # ... JWT issuance ... + + # Tell the compositor proxy to spawn the app + # ... HTTP call to proxy ... + + # Audit log + request.env["compositor.audit.log"].sudo().create({ + "user_id": user.id, + "app_id": app_id, + "action": "launch", + }) + + return {"signal_url": "...", "pid": 12345, "name": app.name} + + @http.route("/web/compositor/audit", type="json", auth="user") + def audit_callback(self, **kwargs): + """Called by the compositor proxy to report events.""" + # Verify this is from the proxy (shared secret or JWT) + # ... + request.env["compositor.audit.log"].sudo().create({ + "user_id": kwargs.get("user_id"), + "app_id": kwargs.get("app_id"), + "action": kwargs.get("action"), + "pid": kwargs.get("pid"), + "exit_code": kwargs.get("exit_code"), + }) + return {"ok": True} +``` + +### Security + +```xml + + + Compositor / User + + + + + Compositor / Manager + + + + + + Compositor App: User sees allowed apps only + + ["|", ("allowed_group_ids", "=", False), ("allowed_group_ids", "in", [g.id for g in user.groups_id])] + + +``` + +## Nginx configuration + +```nginx +# /etc/nginx/sites-available/odoo-compositor +server { + listen 443 ssl http2; + server_name your-odoo.example.com; + + ssl_certificate /etc/letsencrypt/live/your-odoo.example.com/fullchain.pem; + ssl_certificate_key /etc/letsencrypt/live/your-odoo.example.com/privkey.pem; + + # Odoo HTTP workers + location / { + proxy_pass http://127.0.0.1:8069; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + } + + # Odoo gevent worker (Discuss, chat, presence — NOT compositor!) + location /websocket { + proxy_pass http://127.0.0.1:8072; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + } + + # Compositor proxy (aiohttp, high-throughput WS) + location /compositor/ { + proxy_pass http://127.0.0.1:8081; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + + # CRITICAL: disable buffering for low-latency video streaming + proxy_buffering off; + proxy_request_buffering off; + + # WebSocket keepalive + proxy_read_timeout 86400s; + proxy_send_timeout 86400s; + } +} +``` + +## Docker Compose + +```yaml +# docker-compose.yml +version: "3.8" + +services: + odoo: + image: odoo:20 + ports: + - "8069:8069" + - "8072:8072" # gevent + environment: + - HOST=db + - USER=odoo + - PASSWORD=odoo + volumes: + - odoo-data:/var/lib/odoo + - ./addons:/mnt/extra-addons + depends_on: + - db + + db: + image: postgres:16 + environment: + - POSTGRES_DB=postgres + - POSTGRES_USER=odoo + - POSTGRES_PASSWORD=odoo + volumes: + - db-data:/var/lib/postgresql/data + + compositor-proxy: + build: ./compositor-proxy + ports: + - "8081:8081" + environment: + - COMPOSITOR_JWT_SECRET=${COMPOSITOR_JWT_SECRET} + - COMPOSITOR_BIND_IP=0.0.0.0 + - COMPOSITOR_BIND_PORT=8081 + - COMPOSITOR_ENCODER=vaapih264 # or nvh264 or openh264 + - COMPOSITOR_RENDER_DEVICE=/dev/dri/renderD128 + devices: + - /dev/dri:/dev/dri # GPU passthrough + volumes: + - /tmp/.X11-unix:/tmp/.X11-unix # XWayland (if enabled) + - compositor-runtime:/run/user/1000 # XDG_RUNTIME_DIR + depends_on: + - odoo + + nginx: + image: nginx:latest + ports: + - "443:443" + volumes: + - ./nginx.conf:/etc/nginx/nginx.conf:ro + - ./certs:/etc/nginx/certs:ro + depends_on: + - odoo + - compositor-proxy + +volumes: + odoo-data: + db-data: + compositor-runtime: +``` + +## Effort estimates + +| Component | Days | Notes | +|---|---|---| +| Odoo addon skeleton (models + controllers + views + security) | 2 | compositor.app, compositor.session, compositor.audit.log | +| JWT issuance + audit log model | 1.5 | HS256, 15-min expiry, pyjwt | +| OWL 3 fullscreen client action + canvas + WS wiring | 2.5 | ir.actions.client with target=fullscreen, **must use proxy() not useState()** | +| Browser compositor library integration | 2 | userShell.actions.initScene(canvasCreator) | +| Compositor proxy JWT middleware + WS handshake | 1.5 | pyjwt validation | +| Audit POSTback + Nginx + Docker Compose + GPU passthrough | 2 | | +| Multi-tenant: per-DB proxy spawning + db claim scoping | 1.5 | | +| Tests (HttpCase + TransactionCase) | 2 | | +| Documentation + buffer | 3 | | +| **Total** | **~18 days** | (parallelizable to ~10 with 2 engineers) | + +Critical path: **~13 working days** for single-tenant MVP, **+5 days** for +multi-tenant + bubblewrap isolation. + +## File reference + +### Odoo 20 backend addon + +| Path | Role | +|---|---| +| `compositor_shell/__manifest__.py` | Addon manifest | +| `compositor_shell/models/compositor_app.py` | App catalog model | +| `compositor_shell/models/compositor_session.py` | Session model (mail.thread) | +| `compositor_shell/models/compositor_audit_log.py` | Append-only audit log | +| `compositor_shell/controllers/main.py` | HTTP controllers (token, apps, launch, audit) | +| `compositor_shell/security/compositor_security.xml` | Groups + rules | +| `compositor_shell/security/ir.model.access.csv` | ACLs | +| `compositor_shell/views/compositor_app_views.xml` | App management views | +| `compositor_shell/views/compositor_session_views.xml` | Session views | +| `compositor_shell/views/ir_actions_client_views.xml` | OWL action registration | +| `compositor_shell/static/src/components/compositor_shell.js` | OWL shell component | +| `compositor_shell/static/src/components/compositor_shell.xml` | OWL template | +| `compositor_shell/static/src/components/compositor_shell.scss` | Styles | +| `compositor_shell/static/src/lib/compositor_lib/` | Custom browser compositor library (clean-room TS) | + +### Python compositor proxy + +| Path | Role | +|---|---| +| `compositor_proxy/main.py` | HTTP + WS hub, forks per session | +| `compositor_proxy/session.py` | Top-level proxy session | +| `compositor_proxy/session_controller.py` | aiohttp WS server with 9 endpoint paths | +| `compositor_proxy/jwt_auth.py` | JWT validation (pyjwt) | +| `compositor_proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn | +| ... | (see `07-native-app-bridge.md` for full file list) | + +### Nginx config + +| Path | Role | +|---|---| +| `nginx/sites-available/odoo-compositor` | Nginx reverse proxy with WS upgrade + proxy_buffering off | + +### Docker + +| Path | Role | +|---|---| +| `docker-compose.yml` | Odoo + PostgreSQL + compositor-proxy + Nginx | +| `compositor-proxy/Dockerfile` | Python proxy image with GPU passthrough | diff --git a/11-pitfalls.md b/11-pitfalls.md new file mode 100644 index 0000000..3a7b833 --- /dev/null +++ b/11-pitfalls.md @@ -0,0 +1,392 @@ +# 11 — Pitfalls, GPL Contaminants, Clean-Room Requirements + +This is the prominent pitfalls chapter for the MIT approach. It documents: + +1. **3 GPL contaminants found** (must be replaced) +2. **Clean-room legal requirements** (cannot read AGPL source and reimplement) +3. **Engineering pitfalls of building from scratch** +4. **Risk analysis** + +## Critical finding: 3 GPL contaminants + +The user's #1 priority is **license purity** — no GPL/AGPL contamination of +the shipping product. The license audit (Agent T) found **3 GPL +contaminants** that MUST be replaced: + +### Contaminant 1: `@gfld/compositor-ffmpeg-h264` is GPL-3.0, not Apache-2.0 + +**What Greenfield claims**: `package.json` declares `"license": +"Apache-2.0"`. + +**What the LICENSE file actually says**: Full GPL v3 text. + +**Why this happened**: The TypeScript wrapper (`H264Decoder.ts`, +`H264Worker.ts`, `index.ts`) is genuinely Apache-2.0. But the +`build_wasm.sh` script compiles FFmpeg with `--enable-gpl +--enable-version3 --enable-nonfree`, which makes the resulting WASM +binary GPL-3.0. The 1.8 MB compiled `libav-h264.js` contains embedded +GPL/LGPL license strings. + +**Impact on the user**: Using `@gfld/compositor-ffmpeg-h264` would force +the entire Odoo fork to be GPL-3.0. + +**Replacement**: WebCodecs `VideoDecoder` API (browser-native, +hardware-accelerated, W3C standard). Fallback: OpenH264 BSD Wasm (NOT +FFmpeg). + +### Contaminant 2: `gstreamer1.0-plugins-ugly` (provides `x264enc`) is GPL-2.0+ + +**What it provides**: The `x264enc` GStreamer element, which is the +software H.264 encoder Greenfield uses by default. + +**Why it's GPL**: `libx264.so.164` is GPL-2.0+. The plugin that wraps it +(`gst-plugins-ugly`) inherits the GPL. Loading this plugin into the +Python process = GPL contamination under FSF's strict interpretation. + +**Verified**: `ldd libgstx264.so` → links `libx264.so.164` (GPL-2.0+). +x264 has NO dynamic-linking exception. + +**Impact**: Using `x264enc` would force the entire Python proxy to be +GPL-2.0+. + +**Replacement**: Use `vaapih264enc` (Intel/AMD, LGPL plugin + MIT libva), +`nvh264enc` (NVIDIA, BSD plugin + proprietary NVENC blob), or +`openh264enc` (CPU fallback, BSD libopenh264). + +### Contaminant 3: `gstreamer1.0-libav` on Debian is GPL (HIDDEN TRAP) + +**What it provides**: The `gst-libav` plugin set, which wraps FFmpeg's +libavcodec for various codecs. + +**Why it's a trap**: The gst-libav plugin SOURCE CODE is LGPL-2.1+. +**BUT** Debian's `libavcodec.so.61` is built with `--enable-gpl` and +directly links `libx264.so.164` + `libx265.so.215`. + +**Verified**: `ldd libgstlibav.so` → `libavcodec.so.61` → `libx264.so.164` ++ `libx265.so.215`. + +**Impact**: Even if the user might never use `avenc_h264`, the mere +presence of `libgstlibav.so` triggers GStreamer's plugin auto-discovery, +which dlopens it, which loads the GPL'd FFmpeg into the Python process's +address space. + +**Replacement**: Don't install `gstreamer1.0-libav`. Use only +`gst-plugins-base` + `gst-plugins-good` + `gst-vaapi` + BSD subset of +`gst-plugins-bad`. + +## Clean-room legal requirements (CRITICAL) + +### The problem + +**You CANNOT read Greenfield's AGPL `@gfld/compositor` source and +reimplement it as "inspired-by" code.** Even non-literal similarity is +sufficient to find derivative-work infringement. + +### Reasons + +1. **AFC test risk**: Under *Computer Associates v. Altai*'s + Abstraction-Filtration-Comparison test, even non-literal similarity + (same class names, method names, code structure) is sufficient to + find derivative-work infringement. A single developer who reads AGPL + source then writes a TS replacement will almost certainly reproduce + the SSO (structure, sequence, organization) subconsciously. + +2. **AGPL §13 network-use clause**: This is the killer for an Odoo fork. + AGPL-3.0 §13 requires offering the source of any modified AGPL work + to all users interacting with it over a network. If a court finds the + custom compositor is an AGPL derivative, the entire Odoo fork (which + links to the compositor) must be open-sourced to all users — + destroying the proprietary value. + +3. **Westfield doesn't help**: Greenfield's predecessor repo at + https://github.com/udevbe/westfield IS MIT-licensed, but it contains + ONLY the protocol infrastructure (the same `@gfld/common` + + `@gfld/compositor-protocol` + `@gfld/compositor-wasm` we already have). + It NEVER had the in-browser compositor logic (Surface/Renderer/Seat/etc.). + The compositor code was always in the Greenfield monorepo, always AGPL. + +### Right approach: True clean-room + +1. **Person A** (spec author, MAY read AGPL source) writes a behavioral + spec (inputs/outputs/edge cases, NO code). +2. **Person B** (implementer, MUST NOT open `packages/compositor/src/`) + implements from spec + MIT references only: + - Wayland protocol spec (https://wayland.freedesktop.org/docs/html/) + - wlroots C source (MIT, https://gitlab.freedesktop.org/wlroots/wlroots) + - Weston C source (MIT, https://gitlab.freedesktop.org/wayland/weston) + - Smithay Rust source (MIT, https://github.com/Smithay/smithay) + - WebGL2/WebCodecs/KCP MDN docs + +### Legal-defensibility artifacts + +Maintain these as legal-defensibility artifacts: + +- `SPEC.md` — the behavioral spec (written by Person A) +- `CLEAN_ROOM_LOG.md` — who read AGPL, who didn't, when, what they referenced + +### What can Person B reference? + +| Reference | License | OK to read for implementation? | +|---|---|---| +| Greenfield's `packages/compositor/src/` | AGPL-3.0 | ❌ NO (Person B forbidden) | +| Greenfield's `libs/` (MIT) | MIT | ✅ Yes (already reused) | +| Greenfield's `protocol/` XML | MIT | ✅ Yes | +| Greenfield's `docs/` | MIT | ✅ Yes | +| Wayland protocol spec | MIT | ✅ Yes | +| wlroots C source | MIT | ✅ Yes | +| Weston C source | MIT | ✅ Yes | +| Smithay Rust source | MIT | ✅ Yes | +| MDN docs (WebGL2, WebCodecs, etc.) | CC-BY-SA | ✅ Yes | +| pywayland source | Apache-2.0 | ✅ Yes | +| qtile source | GPL-2.0 | ❌ NO (architectural patterns OK, code not) | + +## License audit table (full stack) + +### Browser side + +| Component | License | Compatible with proprietary? | +|---|---|---| +| @gfld/common | MIT | ✅ | +| @gfld/client-protocol | MIT | ✅ | +| @gfld/compositor-protocol | MIT | ✅ (verified NO @gfld/compositor import) | +| @gfld/client-generator | MIT | ✅ | +| @gfld/compositor-generator | MIT | ✅ | +| @gfld/compositor-wasm (pixman+xkbcommon) | MIT | ✅ | +| @gfld/xtsb (X11 bindings, optional) | MIT | ✅ (only if XWayland needed) | +| ~~@gfld/compositor-ffmpeg-h264~~ | ~~GPL-3.0~~ | ❌ **REPLACE with WebCodecs** | +| Custom browser compositor (your code) | PROPRIETARY | ✅ (clean-room) | +| WebCodecs API | W3C standard | ✅ | +| WebGL2 | W3C standard | ✅ | +| OffscreenCanvas | W3C standard | ✅ | +| OpenH264 BSD Wasm (fallback) | BSD-2-Clause | ✅ | + +### Python backend + +| Component | License | Compatible with proprietary? | +|---|---|---| +| pywayland | Apache-2.0 | ✅ | +| python-xkbcommon (`xkbcommon` PyPI) | MIT | ✅ | +| PyGObject (gi.repository.Gst) | LGPL-2.1+ | ✅ (dynamic linking) | +| GLib | LGPL-2.1+ | ✅ (dynamic linking) | +| GStreamer core | LGPL-2.0+ | ✅ (dynamic linking) | +| gst-plugins-base | LGPL | ✅ | +| gst-plugins-good (pngenc) | LGPL | ✅ | +| gst-vaapi (vaapih264enc) | LGPL + MIT (libva) | ✅ | +| gst-plugins-bad (nvh264enc, openh264enc) | BSD subset | ✅ | +| ~~gst-plugins-ugly (x264enc)~~ | ~~GPL-2.0+~~ | ❌ **DROP** | +| ~~gst-libav (libavcodec)~~ | ~~LGPL source, GPL binary on Debian~~ | ❌ **DROP** | +| libwayland | MIT | ✅ | +| libdrm | MIT | ✅ | +| mesa (libGL, libEGL, libgbm) | MIT | ✅ | +| aiohttp | Apache-2.0 | ✅ | +| kcp (PyPI, RealistikDash) | MIT | ✅ | +| PyJWT | MIT | ✅ | +| bubblewrap (bwrap) | LGPL-2.0+ | ✅ (dynamic linking) | +| libseccomp | LGPL-2.1 | ✅ | + +### Odoo integration + +| Component | License | Compatible? | +|---|---|---| +| Odoo 20 Community | LGPL-3.0 | ✅ (separate process, no contamination) | +| Nginx | BSD-2-Clause | ✅ | +| Docker | Apache-2.0 | ✅ | + +### Vendored Greenfield C files (MIT, reused verbatim) + +| File | License | Reuse? | +|---|---|---| +| westfield-buffer.c | MIT (Greenfield) | ✅ | +| westfield-dmabuf.c | MIT (Greenfield) | ✅ | +| westfield-egl.c | MIT (Greenfield) | ✅ | +| westfield-xwayland.c | MIT (Greenfield) | ✅ | +| wlr_drm.c | MIT (vendored from wlroots) | ✅ | +| wlr_linux_dmabuf_v1.c | MIT (vendored from wlroots) | ✅ | +| drm_format_set.c | MIT (vendored from wlroots) | ✅ | +| addon.c | MIT (vendored from wlroots) | ✅ | +| changes.diff | MIT (Greenfield patches to libwayland) | ✅ (optional, for fast-sync) | + +## CI/CD license hygiene gates (recommended) + +Add these to your CI pipeline to catch GPL contamination early: + +### Docker image audit + +```bash +# Must return zero matches +dpkg -l | grep -E "libx264|libx265|gstreamer1.0-plugins-ugly|gstreamer1.0-libav" +``` + +### npm license check + +```bash +# Must pass (no GPL/AGPL) +npx license-checker --production --failOn "GPL-3.0;AGPL-3.0;GPL-2.0;GPL-2.0+;GPL-3.0+" +``` + +### Python license check + +```bash +# Must pass (no GPL/AGPL) +pip-licenses --fail-on "GPL.*;AGPL.*" --ignore-builtin +``` + +### Runtime GStreamer plugin check + +```bash +# Must return zero matches +gst-inspect-1.0 --print-plugin-auto-install-info | grep -E "x264|x265|libav" +``` + +## Engineering pitfalls (beyond licensing) + +### 1. Browser compositor is the biggest effort + +The custom browser compositor is ~13,400 LOC TypeScript for MVP, ~25 +dev-weeks (1 senior dev). This is the critical path. + +**Mitigation**: Hire 2 senior engineers to parallelize. Use Smithay (Rust, +MIT) as a reference for Wayland semantics. Use wlroots (C, MIT) as a +reference for compositor patterns. + +### 2. xdg-shell state machine is the highest-risk component + +The configure/ack-configure round-trip, popup positioner math, and +subsurface sync/desync cascades are subtle. Bugs here can cause apps to +hang or crash. + +**Mitigation**: Write extensive unit tests using Weston's test clients +(ported to TypeScript). Test with real GTK4/Qt6 apps early and often. + +### 3. asyncio + libwayland integration is under-documented + +The 5-line cffi shim for `wl_event_loop_get_fd` is verified working, but +the pattern is not widely documented. Bugs here could cause Wayland +events to be dropped or duplicated. + +**Mitigation**: Spend 1 day on a standalone prototype before committing. +Reference qtile's Wayland backend (architectural patterns OK, code is +GPL — don't copy). + +### 4. GStreamer threading is non-trivial + +GLib MainLoop runs in a separate thread; results must be marshalled back +to asyncio via `asyncio.run_coroutine_threadsafe`. Bugs here can cause +race conditions or deadlocks. + +**Mitigation**: Reuse Greenfield's MIT C files via cffi — only the +marshalling needs Python. Test with high-frame-rate content (e.g. glxgears). + +### 5. XWayland is a major time sink (~8 weeks) + +If you need X11 app support (e.g. older apps that don't speak Wayland +natively), the XWayland bridge is ~5,000 LOC TypeScript + ~465 LOC C. +It uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings. + +**Mitigation**: Skip XWayland in MVP. Most modern GTK4/Qt6 apps are +Wayland-native. Add XWayland only if a specific X11-only app is needed. + +### 6. KCP Python port may have performance issues + +The `kcp` PyPI package (RealistikDash, MIT) is a Cython wrapper over +upstream `ikcp.c`. Performance should be adequate, but if it's not, +porting Greenfield's TypeScript KCP (~1100 lines) to Python would be +~1500 lines of pure Python (slower). + +**Mitigation**: Benchmark early. If Cython wrapper is too slow, write a +C extension via cffi. + +### 7. WebGL2 + OffscreenCanvas + Workers is complex + +Multi-threaded rendering with OffscreenCanvas requires careful message +passing between workers. Debugging is harder than main-thread code. + +**Mitigation**: Start with main-thread WebGL2 (simpler), then move to +OffscreenCanvas in Phase 5 (polish). + +### 8. WebCodecs hardware decode may have bugs + +Greenfield explicitly disables hardware decode with `// FIXME hardware +decoding frame offset is wrong`. We're enabling it (`prefer-hardware`), +but the bug Greenfield hit may still affect us. + +**Mitigation**: Test extensively on Intel/AMD/NVIDIA GPUs. Fall back to +`prefer-software` if hardware decode is broken on specific hardware. + +### 9. Bubblewrap + seccomp may break some apps + +Some GTK apps may need additional filesystem access (e.g. ~/.config for +settings, ~/.cache for thumbnails). The default-deny approach may break +them. + +**Mitigation**: Test with the apps you want to support. Add per-app +overrides in `compositor.app` model (e.g. extra filesystem binds). + +### 10. JWT secret rotation + +The JWT secret is shared between Odoo and the compositor proxy via env +var. If the secret leaks, attackers can forge JWTs and run apps on +behalf of any user. + +**Mitigation**: Rotate the secret periodically (e.g. monthly). Use a +secrets manager (Vault, AWS Secrets Manager). Log all JWT validations. + +## Risk analysis summary + +| Risk | Likelihood | Impact | Mitigation | +|---|---|---|---| +| GPL contamination in shipping product | Medium (3 contaminants found) | **CRITICAL** (forces open-source) | 3 substitutions + CI/CD license gates | +| AGPL derivative-work lawsuit | Low (clean-room approach) | **CRITICAL** (forces open-source) | True clean-room with SPEC.md + CLEAN_ROOM_LOG.md | +| Browser compositor takes >6 months | Medium | High (delays MVP) | 2 senior engineers in parallel | +| xdg-shell state machine bugs | High | Medium (app crashes) | Extensive testing with Weston test clients | +| asyncio + libwayland integration bugs | Medium | High (events dropped) | 1-day prototype before committing | +| GStreamer threading deadlocks | Medium | Medium (encoder stalls) | Reuse Greenfield's MIT C files via cffi | +| XWayland scope creep | High | High (+8 weeks) | Skip XWayland in MVP | +| KCP Python performance | Low | Medium (latency) | Benchmark early, fall back to cffi C extension | +| WebGL2 + Workers complexity | Medium | Medium (debugging pain) | Start with main-thread, move to Workers in Phase 5 | +| WebCodecs hardware decode bugs | Medium | Medium (CPU usage) | Test on Intel/AMD/NVIDIA, fall back to software | +| Bubblewrap breaks apps | Medium | Low (per-app overrides) | Test with target apps, add per-app overrides | +| JWT secret leak | Low | **CRITICAL** (auth bypass) | Rotate periodically, use secrets manager | + +## Design wins to copy from Greenfield + +These Greenfield design choices are worth copying in the MIT approach +(but implemented clean-room): + +1. **Generator-driven protocol stubs** — single XML source for both + sides. REUSED from Greenfield's MIT libs. +2. **Dual-buffer commit semantics** — Surface state vs pendingState, + atomic merge. Clean-room from spec. +3. **UserShellApi decoupling** — shell sees only `{events, actions}`, + never Wayland resources. Clean-room from spec. +4. **Asynchronous browser compositor** — per-client channels, never + blocks on a slow client. Clean-room from spec. +5. **KCP-over-WebSocket** for UDP-like latency over TCP. Use `kcp` PyPI + package (MIT). +6. **WebCodecs primary + fallback** — capability probe at session + startup. Clean-room, but enable hardware decode (Greenfield doesn't). +7. **Zero-copy dmabuf → encoder** via `linux-dmabuf-unstable-v1`. + REUSED vendored Greenfield C files (MIT). +8. **Per-surface encoder feedback channel** — Greenfield produces it + but doesn't use it. **Our MIT approach consumes it for adaptive + bitrate** (NEW). +9. **`ReconnectingWebSocket`** with exponential backoff. Vendored from + pladaria (MIT). +10. **Honest docs** — Greenfield's `docs/pages/design/` openly admits + what's wrong. Use as reference for trade-off analysis. + +## Top-5 recommendations + +1. **Replace the 3 GPL contaminants IMMEDIATELY**: WebCodecs (not FFmpeg + Wasm), vaapih264enc/nvh264enc/openh264enc (not x264enc), don't install + gst-libav. +2. **Adopt true clean-room discipline** for the browser compositor. One + spec author (may read AGPL), one implementer (must NOT). Maintain + `SPEC.md` + `CLEAN_ROOM_LOG.md` as legal-defensibility artifacts. +3. **Skip XWayland in MVP** — add 8 weeks only if legacy X11 apps are + required. +4. **Use the kcp PyPI package** — don't port Greenfield's TS KCP to + Python (would be slower). +5. **Add CI/CD license hygiene gates** — Docker image audit, npm + license-checker, pip-licenses, gst-inspect check. Catch GPL + contamination early. diff --git a/12-appendix.md b/12-appendix.md new file mode 100644 index 0000000..c970b2b --- /dev/null +++ b/12-appendix.md @@ -0,0 +1,339 @@ +# 12 — Appendix: Glossary, File Index, References + +## Glossary + +### Wayland concepts + +Same as Greenfield — see `../greenfield-analysis/12-appendix.md` for +the full Wayland glossary. Key terms: `wl_display`, `wl_registry`, +`wl_compositor`, `wl_surface`, `wl_seat`, `wl_keyboard`, `wl_pointer`, +`wl_touch`, `wl_output`, `wl_shm`, `wl_buffer`, `wl_callback`, +`wl_data_device_manager`, `wl_data_device`, `wl_data_source`, +`wl_data_offer`, `wl_drm`, `wl_subcompositor`, `wl_subsurface`, +`wl_region`. + +### xdg-shell concepts + +Same as Greenfield — `xdg_wm_base`, `xdg_surface`, `xdg_toplevel`, +`xdg_popup`, `xdg_positioner`, `configure`, `ack_configure`. + +### MIT-approach-specific terms + +| Term | Meaning | +|---|---| +| **Clean-room implementation** | Software written by an engineer who has NOT read the source code of the original (AGPL) implementation. The engineer writes from a behavioral spec produced by a separate person who MAY have read the AGPL source. This protects against derivative-work claims. | +| **Person A (spec author)** | The engineer who writes the behavioral spec. MAY read AGPL source for reference. Does NOT write implementation code. | +| **Person B (implementer)** | The engineer who writes the implementation. MUST NOT read AGPL source. Implements from spec + MIT references only. | +| **SPEC.md** | The behavioral spec artifact (inputs/outputs/edge cases, NO code). Written by Person A. | +| **CLEAN_ROOM_LOG.md** | Legal-defensibility artifact documenting who read AGPL source, who didn't, when, what they referenced. | +| **AFC test** | Abstraction-Filtration-Comparison test, from *Computer Associates v. Altai* (1992). The legal test for derivative-work infringement in software. Even non-literal similarity (SSO: structure, sequence, organization) can be infringing. | +| **AGPL §13** | The network-use clause of AGPL-3.0. Requires offering source to all users interacting with the modified work over a network. The killer clause for proprietary web services. | +| **GPL dynamic-linking exception** | GPL applies to the entire combined work when statically linked. Dynamic linking is more permissive (LGPL allows it from proprietary code). FSF's strict interpretation says loading a GPL .so into a proprietary process contaminates it. | +| **ProxyFD** | JSON descriptor for a file descriptor sent over WebSocket: `{handle, type, host}`. Same as Greenfield — see `../greenfield-analysis/04-transport.md`. | + +### Python-specific terms + +| Term | Meaning | +|---|---| +| **pywayland** | Python bindings to libwayland-server, via cffi. Apache-2.0 licensed. | +| **pywlroots** | Python bindings to libwlroots. NCSA/UIUC license (BSD-like). NOT RECOMMENDED — operates above wire-dispatch layer, incompatible with Greenfield's architecture. | +| **python-xkbcommon** | The `xkbcommon` PyPI package. Python bindings to libxkbcommon. MIT licensed. | +| **PyGObject** | Python bindings to GLib/GObject. LGPL-2.1+. Provides `gi.repository.Gst` for GStreamer. | +| **cffi** | C Foreign Function Interface for Python. Used to call C functions from Python without writing a C extension. | +| **asyncio** | Python's async I/O framework, built into the standard library. | +| **aiohttp** | Async HTTP + WebSocket server framework for Python. Apache-2.0. | +| **multiprocessing.Process** | Python's process-spawning primitive. Used to fork one child per Odoo session. | +| **multiprocessing.reduction.send_handle** | Function to pass an open file descriptor from one Python process to another (uses SCM_RIGHTS on Linux). | +| **GLib MainLoop** | GLib's event loop, required by GStreamer. Runs in a separate Python thread. | +| **asyncio.run_coroutine_threadsafe** | Function to schedule a coroutine on an asyncio event loop from a different thread. Used to marshal GStreamer results back to asyncio. | +| **bubblewrap (bwrap)** | Unprivileged sandboxing tool. LGPL-2.0+. Used by Flatpak. | +| **seccomp BPF** | Linux kernel feature for filtering syscalls. Compiled to BPF bytecode, loaded via `bwrap --seccomp `. | +| **systemd-run --scope** | systemd command to run a process in a transient cgroup. Used for memory/CPU/PID limits. | +| **memfd_create** | Linux syscall to create an anonymous in-memory file. Python 3.8+ has `os.memfd_create()`. Replaces Greenfield's `createMemoryMappedFile` native addon. | + +### Browser-specific terms + +| Term | Meaning | +|---|---| +| **WebGL2** | Next-gen WebGL API. Uniform buffer objects, transform feedback, simplified shaders. 97%+ browser support. | +| **WebCodecs** | W3C standard for native video decode/encode in the browser. `VideoDecoder`, `VideoEncoder`, `EncodedVideoChunk`, `VideoFrame`. Replaces FFmpeg Wasm. | +| **OffscreenCanvas** | W3C standard for canvas rendering in a Web Worker. Allows WebGL to run off the main thread. | +| **Web Workers** | W3C standard for running JavaScript in background threads. | +| **SharedArrayBuffer + Atomics** | W3C standard for zero-copy shared memory between workers. Needs COOP/COEP headers. | +| **ImageBitmap** | W3C standard for a GPU-backed bitmap. Transferable between workers. Used by the `web-bitmapbuf` fast path. | +| **WebTransport** | W3C standard (in progress) for UDP-over-QUIC transport. Future replacement for WebSocket+KCP. | +| **OpenH264** | BSD-2-Clause H.264 codec. Cisco's binary release. Used as fallback when WebCodecs is unavailable. | + +## File-to-feature index + +### Custom browser compositor (PROPRIETARY, clean-room TS) + +| File | Role | +|---|---| +| `packages/compositor/src/index.ts` | Public API: `createCompositorSession()` | +| `packages/compositor/src/core/Session.ts` | Top-level session | +| `packages/compositor/src/core/Compositor.ts` | `wl_compositor` global | +| `packages/compositor/src/core/Globals.ts` | Wayland globals registry | +| `packages/compositor/src/surface/Surface.ts` | `wl_surface`, double-buffered state | +| `packages/compositor/src/surface/SurfaceRole.ts` | Role abstraction | +| `packages/compositor/src/surface/Subsurface.ts` | `wl_subsurface` | +| `packages/compositor/src/surface/Subcompositor.ts` | `wl_subcompositor` global | +| `packages/compositor/src/surface/Region.ts` | `wl_region` (pixman-WASM) | +| `packages/compositor/src/surface/FrameCallback.ts` | `wl_callback` | +| `packages/compositor/src/render/Renderer.ts` | 4-phase render pipeline (WebGL2 + OffscreenCanvas) | +| `packages/compositor/src/render/Scene.ts` | One per output | +| `packages/compositor/src/render/View.ts` | Positioned surface | +| `packages/compositor/src/render/Output.ts` | `wl_output`, OffscreenCanvas binding | +| `packages/compositor/src/render/RenderState.ts` | Per-view-per-scene texture + transform | +| `packages/compositor/src/render/Texture.ts` | WebGL2 texture wrapper | +| `packages/compositor/src/render/Loop.ts` | requestAnimationFrame integration | +| `packages/compositor/src/render/shaders/YUVA2RGBA.glsl` | BT.601 YUVA → RGBA | +| `packages/compositor/src/render/shaders/RGBandA2RGBA.glsl` | RGB+Alpha → RGBA | +| `packages/compositor/src/render/shaders/SceneShader.glsl` | Final pass: textured quads | +| `packages/compositor/src/buffer/Shm.ts` | `wl_shm`, BMP-header hack | +| `packages/compositor/src/buffer/BufferContents.ts` | Buffer abstraction (mimeType) | +| `packages/compositor/src/buffer/ImageBitmapBuffer.ts` | web-bitmapbuf fast path | +| `packages/compositor/src/buffer/EncodedFrameBuffer.ts` | H.264/PNG from proxy | +| `packages/compositor/src/transport/Channel.ts` | ARQChannel + SimpleChannel | +| `packages/compositor/src/transport/kcp.ts` | Ported fresh from MIT C ref (skywind3000/kcp) | +| `packages/compositor/src/transport/ReconnectingWebSocket.ts` | Vendored from pladaria (MIT) | +| `packages/compositor/src/transport/BufferStream.ts` | Serial → frame promise resolution | +| `packages/compositor/src/transport/StreamingBuffer.ts` | Per-surface encoded-frame buffer | +| `packages/compositor/src/transport/EncodedFrame.ts` | Wire format parser | +| `packages/compositor/src/transport/EncoderFeedback.ts` | 4-byte decode feedback, 1 Hz | +| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps | +| `packages/compositor/src/transport/RemoteAppLauncher.ts` | Channel setup per app | +| `packages/compositor/src/transport/decoders/WebCodecsFrameDecoder.ts` | WebCodecs VideoDecoder (HW-accelerated) | +| `packages/compositor/src/transport/decoders/OpenH264FrameDecoder.ts` | OpenH264 BSD Wasm fallback | +| `packages/compositor/src/input/Seat.ts` | `wl_seat` global | +| `packages/compositor/src/input/Keyboard.ts` | `wl_keyboard` | +| `packages/compositor/src/input/Pointer.ts` | `wl_pointer` | +| `packages/compositor/src/input/Touch.ts` | `wl_touch` (REAL implementation, not stub) | +| `packages/compositor/src/input/Xkb.ts` | @gfld/compositor-wasm wrapper (REUSED MIT) | +| `packages/compositor/src/input/InputQueue.ts` | Event coalescing | +| `packages/compositor/src/input/InputOutput.ts` | FD abstraction | +| `packages/compositor/src/roles/XdgWmBase.ts` | `xdg_wm_base` global | +| `packages/compositor/src/roles/XdgSurface.ts` | `xdg_surface` role | +| `packages/compositor/src/roles/XdgToplevel.ts` | `xdg_toplevel` role | +| `packages/compositor/src/roles/XdgPopup.ts` | `xdg_popup` role | +| `packages/compositor/src/roles/XdgPositioner.ts` | `xdg_positioner` | +| `packages/compositor/src/roles/Shell.ts` | `wl_shell` (legacy, optional) | +| `packages/compositor/src/roles/ShellSurface.ts` | `wl_shell_surface` (legacy, optional) | +| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` | +| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` | +| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` | +| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** | +| `packages/compositor/src/desktop/Desktop.ts` | Floating vs AlwaysFullScreen | +| `packages/compositor/src/desktop/FloatingDesktopSurface.ts` | Default floating mode | +| `packages/compositor/src/userapi/UserShellApi.ts` | OWL shell integration seam | +| `packages/compositor/src/browser/input.ts` | DOM keyboard events | +| `packages/compositor/src/browser/pointer.ts` | DOM pointer events | +| `packages/compositor/src/browser/touch.ts` | DOM touch events (NEW) | +| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD) | +| `packages/compositor/src/browser/selection.ts` | Clipboard bridge | +| `packages/compositor/src/math/Mat4.ts` | 4×4 matrix | +| `packages/compositor/src/math/Rect.ts` | Rectangle math | + +### Custom Python proxy (PROPRIETARY, ~2880 LOC) + +| File | Role | +|---|---| +| `compositor_proxy/main.py` | HTTP + WS hub, forks per session | +| `compositor_proxy/session.py` | Top-level proxy session | +| `compositor_proxy/session_controller.py` | aiohttp WS server with 9 endpoints | +| `compositor_proxy/jwt_auth.py` | JWT validation (pyjwt) | +| `compositor_proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn | +| `compositor_proxy/native_wayland_client_session.py` | Per-client message dispatcher | +| `compositor_proxy/native_wayland_compositor_session.py` | Display creation, initShm, initDrm | +| `compositor_proxy/channel.py` | ARQChannel + SimpleChannel (clean-room) | +| `compositor_proxy/kcp_adapter.py` | kcp PyPI package integration | +| `compositor_proxy/frame_feedback.py` | Receives 4-byte decode feedback | +| `compositor_proxy/proxy_buffer.py` | Proxy-side buffer representation | +| `compositor_proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration | +| `compositor_proxy/xwayland_session.py` | XWayland spawn + bridge | +| `compositor_proxy/wayland_server.py` | pywayland wrapper + cffi shim | +| `compositor_proxy/io/` | WebSocket server glue | +| `compositor_proxy/encoding/gst_encoder.py` | GStreamer H.264/PNG encoder (~700 LOC) | +| `compositor_proxy/encoding/adaptive_bitrate.py` | Adaptive bitrate controller (NEW) | +| `compositor_proxy/encoding/glib_thread.py` | GLib MainLoop in separate thread | +| `compositor_proxy/sandbox.py` | bubblewrap + seccomp + systemd-run wrappers | + +### Reused from Greenfield (MIT/Apache) + +| Path | Role | +|---|---| +| `greenfield/protocol/wayland.xml` | Upstream Wayland core (MIT) | +| `greenfield/protocol/xdg-shell.xml` | Stable xdg-shell (MIT) | +| `greenfield/protocol/wayland-drm.xml` | Legacy wl_drm (MIT) | +| `greenfield/protocol/web-bitmapbuf-unstable-v1.xml` | Greenfield's custom extension (MIT) | +| `greenfield/libs/common/src/Connection.ts` | Wire format codec (REUSED) | +| `greenfield/libs/compositor-protocol/src/` | Server-side runtime + generated bindings (REUSED) | +| `greenfield/libs/client-protocol/src/` | Client-side runtime + generated bindings (REUSED) | +| `greenfield/libs/compositor-wasm/src/` | pixman + xkbcommon WASM (REUSED) | +| `greenfield/libs/xtsb/` | X11 TypeScript bindings (REUSED, only if XWayland) | +| `greenfield/libs/client-generator/` | Browser client bindings generator (REUSED) | +| `greenfield/libs/compositor-generator/` | Browser server bindings generator (REUSED) | +| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-buffer.c` | Buffer helpers (REUSED, MIT) | +| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-dmabuf.c` | DMA-BUF import (REUSED, MIT) | +| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-egl.c` | EGL zero-copy setup (REUSED, MIT) | +| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-xwayland.c` | XWayland bridge (REUSED as reference, ported to Python) | +| `greenfield/packages/compositor-proxy/native/wayland/src/wlr_drm.c` | `wl_drm` global impl (REUSED, MIT, vendored from wlroots) | +| `greenfield/packages/compositor-proxy/native/wayland/src/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl (REUSED, MIT, vendored from wlroots) | +| `greenfield/packages/compositor-proxy/native/wayland/src/drm_format_set.c` | DRM format helpers (REUSED, MIT, vendored from wlroots) | +| `greenfield/packages/compositor-proxy/native/wayland/src/wayland-server/changes.diff` | Patches to libwayland-server (REUSED, MIT, optional for fast-sync) | + +### Odoo 20 backend addon + +| Path | Role | +|---|---| +| `compositor_shell/__manifest__.py` | Addon manifest | +| `compositor_shell/models/compositor_app.py` | App catalog model | +| `compositor_shell/models/compositor_session.py` | Session model (mail.thread) | +| `compositor_shell/models/compositor_audit_log.py` | Append-only audit log | +| `compositor_shell/controllers/main.py` | HTTP controllers (token, apps, launch, audit) | +| `compositor_shell/security/compositor_security.xml` | Groups + rules | +| `compositor_shell/security/ir.model.access.csv` | ACLs | +| `compositor_shell/views/compositor_app_views.xml` | App management views | +| `compositor_shell/views/compositor_session_views.xml` | Session views | +| `compositor_shell/views/ir_actions_client_views.xml` | OWL action registration | +| `compositor_shell/static/src/components/compositor_shell.js` | OWL shell component | +| `compositor_shell/static/src/components/compositor_shell.xml` | OWL template | +| `compositor_shell/static/src/components/compositor_shell.scss` | Styles | +| `compositor_shell/static/src/lib/compositor_lib/` | Custom browser compositor library (clean-room TS) | + +## External references + +### Wayland protocol + +- **Wayland documentation**: https://wayland.freedesktop.org/docs/html/ +- **Wayland protocol XML**: https://gitlab.freedesktop.org/wayland/wayland/-/blob/main/protocol/wayland.xml +- **xdg-shell protocol**: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/stable/xdg-shell/xdg-shell.xml +- **wayland-drm protocol**: https://gitlab.freedesktop.org/wayland/wayland/-/blob/main/protocol/wayland.xml +- **linux-dmabuf-unstable-v1**: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/unstable/linux-dmabuf/linux-dmabuf-unstable-v1.xml + +### libwayland + pywayland + +- **libwayland source**: https://gitlab.freedesktop.org/wayland/wayland +- **Wayland book**: https://wayland-book.com/ +- **pywayland docs**: https://pywayland.readthedocs.io/ +- **pywayland source**: https://github.com/flacjacket/pywayland +- **pywayland on PyPI**: https://pypi.org/project/pywayland/ + +### wlroots + pywlroots + +- **wlroots source**: https://gitlab.freedesktop.org/wlroots/wlroots +- **wlroots mirror**: https://github.com/swaywm/wlroots +- **pywlroots source**: https://github.com/flacjacket/pywlroots +- **Smithay (Rust alternative)**: https://github.com/Smithay/smithay + +### xkbcommon + python-xkbcommon + +- **xkbcommon homepage**: https://xkbcommon.org/ +- **xkbcommon source**: https://github.com/xkbcommon/libxkbcommon +- **xkbcommon on PyPI**: https://pypi.org/project/xkbcommon/ + +### GStreamer + PyGObject + +- **GStreamer homepage**: https://gstreamer.freedesktop.org/ +- **GStreamer Python bindings (gst-python)**: https://github.com/GStreamer/gst-python +- **PyGObject docs**: https://pygobject.readthedocs.io/ +- **GStreamer tutorials**: https://gstreamer.freedesktop.org/documentation/tutorials/index.html +- **GStreamer GL plugins**: https://gstreamer.freedesktop.org/documentation/opengl/index.html +- **vaapih264enc docs**: https://gstreamer.freedesktop.org/documentation/vaapi/index.html +- **nvh264enc docs**: https://gstreamer.freedesktop.org/documentation/nvcodec/index.html +- **openh264enc docs**: https://gstreamer.freedesktop.org/documentation/openh264/index.html + +### OpenH264 (BSD fallback decoder) + +- **OpenH264 source**: https://github.com/cisco/openh264 +- **Cisco binary releases**: https://github.com/cisco/openh264/releases + +### WebCodecs + WebGL2 + OffscreenCanvas + +- **WebCodecs API (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API +- **VideoDecoder (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder +- **WebGL2 (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebGL2RenderingContext +- **OffscreenCanvas (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas +- **Web Workers (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API +- **SharedArrayBuffer (MDN)**: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer +- **WebTransport (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebTransport +- **WebCodecs samples**: https://w3c.github.io/webcodecs/samples/ + +### KCP + +- **KCP protocol (skywind3000)**: https://github.com/skywind3000/kcp +- **KCP documentation**: https://skywind3000.github.io/kcp/ +- **kcp on PyPI (RealistikDash)**: https://pypi.org/project/kcp/ + +### asyncio + aiohttp + +- **asyncio docs**: https://docs.python.org/3/library/asyncio.html +- **aiohttp docs**: https://docs.aiohttp.org/ +- **aiohttp WebSocket**: https://docs.aiohttp.org/en/stable/web_advanced.html#websockets + +### JWT + +- **PyJWT docs**: https://pyjwt.readthedocs.io/ +- **JWT.io**: https://jwt.io/ +- **RFC 7519 (JWT)**: https://datatracker.ietf.org/doc/html/rfc7519 + +### Sandboxing + +- **bubblewrap docs**: https://github.com/containers/bubblewrap +- **libseccomp**: https://github.com/seccomp/libseccomp +- **Flatpak sandboxing**: https://docs.flatpak.org/en/latest/sandbox-permissions.html +- **cgroups v2**: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html +- **systemd-run**: https://www.freedesktop.org/software/systemd/man/systemd-run.html + +### Odoo 20 + +- **Odoo 20 documentation**: https://www.odoo.com/documentation/20.0/ +- **Odoo OWL framework**: https://github.com/odoo/owl +- **Odoo OWL migration guide (Odoo 19→20)**: https://www.odoo.com/documentation/20.0/developer/howtos/owl_migration.html +- **Odoo controller patterns**: https://www.odoo.com/documentation/20.0/developer/howtos/routing.html +- **Odoo security**: https://www.odoo.com/documentation/20.0/developer/howtos/security.html + +### Greenfield (for reference, NOT for clean-room implementers) + +- **Greenfield homepage**: https://www.greenfield.app +- **Greenfield source**: https://github.com/udevbe/greenfield +- **Greenfield design docs**: https://github.com/udevbe/greenfield/blob/master/docs/pages/design/index.md +- **Greenfield MIT libs (REUSABLE)**: https://github.com/udevbe/greenfield/tree/master/libs +- **Greenfield AGPL packages (DO NOT READ for clean-room)**: https://github.com/udevbe/greenfield/tree/master/packages + +### Westfield (Greenfield's predecessor, MIT) + +- **Westfield source**: https://github.com/udevbe/westfield +- **Note**: Contains ONLY the protocol infrastructure (same as Greenfield's MIT libs). NEVER had the in-browser compositor logic. + +### Reference Wayland compositors (MIT, for clean-room spec authors) + +- **wlroots (C, MIT)**: https://gitlab.freedesktop.org/wlroots/wlroots — reference for compositor patterns +- **Weston (C, MIT)**: https://gitlab.freedesktop.org/wayland/weston — reference Wayland compositor +- **Smithay (Rust, MIT)**: https://github.com/Smithay/smithay — Rust compositor library +- **wprs (Rust, MIT)**: https://github.com/Smithay/wprs — Rust wayland-xpra, mirrors Greenfield's architecture almost exactly + +## Pointers to the user's next steps + +1. **Read the comparison document**: `../Greenfield-vs-MIT-Comparison.md` + for the head-to-head analysis and final recommendation. + +2. **If you choose the MIT approach** (recommended): + - Phase 0: Spike (1 week) — validate libwayland ↔ asyncio integration + - Phase 1: Browser compositor MVP (25 weeks, clean-room TS) + - Phase 2: Python proxy MVP (3.5 weeks, aiohttp + pywayland + GStreamer) + - Phase 3: Odoo integration (3.5 weeks, OWL shell + backend addon + JWT) + - Phase 4: Sandboxing v1 (1.5 weeks, bubblewrap + seccomp) + - Phase 5: Polish & testing (3 weeks) + +3. **Critical first step**: Set up the clean-room process. + - Designate Person A (spec author) and Person B (implementer) + - Person A reads Greenfield AGPL source + writes `SPEC.md` + - Person B implements from `SPEC.md` + MIT references only + - Maintain `CLEAN_ROOM_LOG.md` as a legal-defensibility artifact + +4. **CI/CD license hygiene gates**: Add the 4 license checks to your CI + pipeline from day one. See `11-pitfalls.md` for the commands. + +5. **Engage legal counsel** before starting the clean-room process. This + bundle is engineering analysis, not legal advice. diff --git a/README.md b/README.md new file mode 100644 index 0000000..84bd86f --- /dev/null +++ b/README.md @@ -0,0 +1,282 @@ +# MIT-Licensed Wayland-to-HTML5 Compositor — Architecture & Build Plan + +> **Purpose.** This bundle documents the architecture, components, and +> implementation plan for building a **fully proprietary** Wayland-to-HTML5 +> compositor that integrates with an Odoo 20 fork. It is the alternative to +> the AGPL-licensed Greenfield codebase (`../greenfield-analysis/`). +> +> **Goal.** Run real Linux applications (GTK3/4, Qt, xterm, X11 via XWayland) +> **natively inside the Odoo web client**, as if they were Odoo-native apps — +> while a real Linux machine under the hood actually executes them. +> +> **License strategy.** Use every MIT/Apache-2.0/BSD-licensed Greenfield +> library as a building block. Replace the AGPL-licensed packages +> (`@gfld/compositor`, `@gfld/compositor-proxy`, `@gfld/compositor-shell`, +> `@gfld/compositor-proxy-cli`) with **clean-room reimplementations** in +> TypeScript (browser) and Python (Linux host). Ship the result as a +> proprietary addon for the Odoo fork. + +## How to read this bundle + +The bundle is split into 13 Markdown files mirroring the Greenfield +analysis structure (so you can compare them side-by-side). Files are +ordered from the highest-level view down to implementation details, +then pitfalls, then appendix. + +| # | File | What you'll find inside | +|---|------|-------------------------| +| 0 | `00-overview.md` | The MIT approach at a glance, component inventory, license summary, architecture diagram | +| 1 | `01-architecture.md` | Layered map of the proposed architecture, side-by-side with Greenfield's layers | +| 2 | `02-wayland-protocol.md` | How to handle the Wayland protocol layer with MIT libs only (pywayland + Greenfield's MIT @gfld libs) | +| 3 | `03-html5-rendering.md` | Custom WebGL2 + Web Workers + OffscreenCanvas browser compositor (replacing @gfld/compositor) | +| 4 | `04-transport.md` | Python aiohttp WebSocket + KCP, replacing Node's ws + KCP TS | +| 5 | `05-input-handling.md` | Python xkbcommon for keymap compilation, browser-side xkbcommon WASM for state | +| 6 | `06-data-clipboard.md` | Clipboard + DnD bridge, with primary selection (which Greenfield lacks) | +| 7 | `07-native-app-bridge.md` | pywayland + bubblewrap + seccomp + systemd-run, with proper sandboxing (Greenfield has none) | +| 8 | `08-app-lifecycle.md` | xdg-shell state machine — what to implement, with a clean-room spec | +| 9 | `09-hardware-accel.md` | GStreamer Python + WebCodecs (no GPL x264, no GPL FFmpeg WASM) | +| 10 | `10-odoo-integration.md` | Odoo 20 OWL shell + backend addon, JWT auth, multi-tenancy | +| 11 | `11-pitfalls.md` | Pitfalls of the MIT approach + risk analysis + the 3 GPL contaminants found | +| 12 | `12-appendix.md` | Glossary, file index, references | +| — | `../Greenfield-vs-MIT-Comparison.md` | Top-level comparison document with person-week estimates and strong recommendation | + +## TL;DR — can we really do this? + +**YES — with three mandatory substitutions.** + +The 9 MIT/Apache/BSD Greenfield libraries are reusable as-is, with +**zero AGPL contamination** (verified by grep'ing every `import` statement +in `libs/`). But the audit found three GPL contaminants that must be +replaced: + +1. **`@gfld/compositor-ffmpeg-h264` is GPL-3.0, not Apache-2.0** as its + `package.json` claims. The TypeScript wrapper is Apache-2.0, but the + compiled FFmpeg WASM binary is GPL-3.0 (built with `--enable-gpl + --enable-version3 --enable-nonfree`). **Replace with WebCodecs + `VideoDecoder` API** (browser-native, MIT/W3C standard). + +2. **`x264enc` GStreamer plugin is GPL-2.0+**. Using it in a proprietary + product forces GPL on the entire combined work. **Replace with + `vaapih264enc` (Intel/AMD), `nvh264enc` (NVIDIA), or `openh264enc` + (BSD libopenh264, CPU fallback)**. + +3. **`gstreamer1.0-libav` on Debian is GPL** — the source is LGPL-2.1+, + but Debian's `libavcodec.so.61` is built with `--enable-gpl` and + directly links `libx264.so.164` + `libx265.so.215`. **Don't install + this package** — use only gst-plugins-base + gst-plugins-good + + gst-vaapi + BSD subset of gst-plugins-bad. + +With these substitutions, **the proprietary strategy is unambiguously +viable**. See `11-pitfalls.md` for the full audit table and +`../Greenfield-vs-MIT-Comparison.md` for the final recommendation. + +## Architecture in one diagram + +``` +┌──────────────────────────────────────────────────────────────────────────────────┐ +│ BROWSER (Odoo 20 web client) │ +│ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Odoo 20 OWL Shell addon (PROPRIETARY — your code) │ │ +│ │ - Application launcher, window manager, keyboard picker │ │ +│ │ - Per-output element │ │ +│ │ - Implements UserShellApi.events.* callbacks (clean-room spec) │ │ +│ │ - Calls userShell.actions.* (initScene, activateSurface, ...) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ userShell.events + actions (clean-room API) │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Custom Browser Compositor Library (PROPRIETARY — your code, clean-room) │ │ +│ │ - Surface.ts, View.ts, Output.ts, Seat.ts, Keyboard.ts, Pointer.ts, ... │ │ +│ │ - render/ (WebGL2 + OffscreenCanvas + Web Workers) │ │ +│ │ - transport/ (WebSocket + KCP + ReconnectingWebSocket) │ │ +│ │ - roles/ (xdg-shell state machine, clean-room from spec) │ │ +│ │ - desktop/, input/, data/, math/ │ │ +│ │ ~13,400 LOC TypeScript, ~25 dev-weeks for MVP │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ Reuses MIT libs as npm deps: │ +│ │ - @gfld/common (MIT, wire format) │ +│ │ - @gfld/compositor-protocol (MIT, server stubs)│ +│ │ - @gfld/client-protocol (MIT, browser apps) │ +│ │ - @gfld/compositor-wasm (MIT, pixman+xkbcommon)│ +│ │ - @gfld/xtsb (MIT, X11 — only if XWayland) │ +│ │ - WebCodecs API (W3C standard, replaces GPL │ +│ │ FFmpeg WASM) │ +└──────────────────────────────────────────────────────────────────────────────────┘ + │ + │ WebSocket + KCP-over-WS + │ 9 endpoint paths per session + │ JWT (HS256) auth + ▼ +┌──────────────────────────────────────────────────────────────────────────────────┐ +│ PYTHON HOST (Linux server) │ +│ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Odoo 20 backend addon (PROPRIETARY — your code) │ │ +│ │ - /web/compositor/token → JWT issuance (HS256, 15-min expiry) │ │ +│ │ - /web/compositor/apps → per-user allowed apps list │ │ +│ │ - /web/compositor/launch → triggers proxy to spawn app │ │ +│ │ - /web/compositor/audit → audit log intake │ │ +│ │ - Models: compositor.app, compositor.session, compositor.audit.log │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ subprocess / IPC │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Python Compositor Proxy (PROPRIETARY — your code, ~2880 LOC) │ │ +│ │ - aiohttp WebSocket server (9 endpoints) on :8081 │ │ +│ │ - asyncio event loop integrating libwayland FD (5-line cffi shim) │ │ +│ │ - pywayland (Apache-2.0) for Wayland protocol dispatch │ │ +│ │ - PyGObject+GStreamer (LGPL) for H.264/PNG encoding │ │ +│ │ - Uses vaapih264enc / nvh264enc / openh264enc (NOT x264enc — GPL) │ │ +│ │ - python-xkbcommon (MIT) for keymap │ │ +│ │ - kcp PyPI package (MIT, Cython wrapper over ikcp.c) │ │ +│ │ - bubblewrap + seccomp + systemd-run for app sandboxing │ │ +│ │ - multiprocessing.Process per Odoo session (fork) │ │ +│ │ - JWT validation (pyjwt, MIT) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +│ │ │ +│ │ Reuses Greenfield MIT C files as vendored src:│ +│ │ - westfield-buffer.c, westfield-dmabuf.c, │ +│ │ westfield-egl.c, westfield-xwayland.c │ +│ │ - wlr_drm.c, wlr_linux_dmabuf_v1.c (all MIT) │ +│ │ - Greenfield's changes.diff (patched │ +│ │ libwayland-server, MIT) │ +│ │ │ +│ │ WAYLAND_DISPLAY=wayland-N │ +│ ▼ │ +│ ┌────────────────────────────────────────────────────────────────────────────┐ │ +│ │ Native Linux apps (GTK3/4, Qt, xterm, X11 via XWayland) │ │ +│ │ - Spawned via bwrap + seccomp + systemd-run --scope │ │ +│ │ - cgroup limits: MemoryMax=1G, CPUQuota=200%, TasksMax=256 │ │ +│ │ - Filesystem: /usr ro-bind, minimal /etc, $HOME overlay (discarded) │ │ +│ │ - GPU: /dev/dri/renderD128 only (no card0) │ │ +│ │ - Network: --unshare-net by default (most apps don't need it) │ │ +│ │ - UID remap to 100000-165535 (per-app, /etc/subuid) │ │ +│ └────────────────────────────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────────────────┘ +``` + +## Effort summary (single engineer, sequential) + +| Phase | Weeks | Deliverable | +|---|---|---| +| **Phase 0: Spike** | 1 | Validate libwayland ↔ asyncio integration (5-line cffi shim), validate kcp Python package works | +| **Phase 1: Browser compositor MVP** | 25 | Clean-room TypeScript compositor library, ~13,400 LOC, runs a single Linux app in `` | +| **Phase 2: Python proxy MVP** | 3.5 (~17 dev-days) | aiohttp + pywayland + GStreamer, ~2880 LOC Python | +| **Phase 3: Odoo integration** | 3.5 (~18 dev-days) | OWL shell + backend addon + JWT auth + Nginx + Docker | +| **Phase 4: Sandboxing** | 1.5 (~7-9 dev-days) | bubblewrap + seccomp + systemd-run, v1 shippable | +| **Phase 5: Polish & testing** | 3 | Tests, hardening, docs, XWayland (optional +8 weeks) | +| **TOTAL (MVP)** | **~37 weeks** | Single-tenant, single-app, fully proprietary, sandboxed | +| **TOTAL (with XWayland + multi-tenant)** | **~46 weeks** | Production-ready | + +With 2 senior engineers in parallel: **~5-6 months to MVP**, **~7-9 months to production**. + +See `../Greenfield-vs-MIT-Comparison.md` for the detailed comparison and +recommendation. + +## Key findings to remember + +1. **License is clean** — with 3 substitutions (no GPL x264, no GPL FFmpeg WASM, + no Debian gstreamer1.0-libav), the entire stack is MIT/Apache-2.0/BSD/LGPL. + Fully proprietary product is shippable. (See `11-pitfalls.md`.) + +2. **Clean-room is mandatory** — you CANNOT read Greenfield's AGPL + `@gfld/compositor` source and reimplement it as "inspired-by" code. + Under the *Computer Associates v. Altai* Abstraction-Filtration-Comparison + test, even non-literal similarity (class names, method names, code + structure) is sufficient to find derivative-work infringement. AGPL §13 + network-use clause would then force you to open-source your entire Odoo + fork. **One person writes the spec (may read AGPL), another implements + (must NOT read AGPL).** See `11-pitfalls.md` §Clean-room. + +3. **pywayland is Apache-2.0, not MIT** — but Apache-2.0 is permissive + (allows proprietary use, modification, distribution). The user's + "MIT-only" preference is satisfied in spirit; the only obligation is + to keep the LICENSE + NOTICE files. (See `02-wayland-protocol.md`.) + +4. **pywlroots is NOT recommended** — it operates ABOVE the wire-dispatch + layer (wlroots handles each request locally), incompatible with + Greenfield's wire-forwarding architecture. Instead: pywayland + 5-line + cffi shim + vendored Greenfield C files (MIT) for dmabuf/egl/xwayland. + (See `07-native-app-bridge.md`.) + +5. **Odoo's built-in WebSocket server is unsuitable** — `addons/bus/websocket.py` + has a 1 MB frame cap, shared PostgreSQL pool, single gevent process. + Run a separate aiohttp Python process on :8081 and route via Nginx. + (See `10-odoo-integration.md`.) + +6. **Sandboxing is critical and achievable in 7-9 days** — Greenfield has + zero sandboxing (pitfall M). With bubblewrap + seccomp + systemd-run + + cgroups v2, v1 shippable in 7-9 engineer-days. (See + `07-native-app-bridge.md`.) + +7. **The browser compositor is the biggest effort** — ~25 dev-weeks for + MVP, ~30-36 weeks total with XWayland. This is the bottleneck. See + `03-html5-rendering.md` for the file structure and component-by-component + estimate. + +## How this analysis was produced + +This bundle synthesizes findings from 10 parallel research agents (one +failed with a context deadline, but its scope was covered by other agents): + +- **Agent K** — pywayland deep dive (API surface, asyncio integration, + license verification) +- **Agent L** — wlroots + pywlroots + Smithay comparison, recommendation + to NOT use pywlroots +- **Agent M** — Greenfield MIT libs audit (per-library license verification, + found GPL contaminant in compositor-ffmpeg-h264) +- **Agent N** — GStreamer Python bindings feasibility, found x264enc GPL + issue +- **Agent P** — Custom browser compositor library design, clean-room + legal analysis, file structure, effort estimate +- **Agent Q** — Python WebSocket + KCP transport, recommended aiohttp + + kcp PyPI package +- **Agent R** — Odoo 20 integration patterns, JWT auth, multi-tenancy +- **Agent S** — App sandboxing with bubblewrap + seccomp + systemd-run +- **Agent T** — Comprehensive license audit, found 3 GPL contaminants, + verified all other components are clean + +Each agent read the worklog (`/home/z/my-project/worklog.md`), researched +its assigned area, and appended findings. This bundle is the synthesis. + +## License summary table + +| Component | License | Compatible with proprietary? | +|---|---|---| +| **Browser side** | | | +| @gfld/common | MIT | ✅ | +| @gfld/client-protocol | MIT | ✅ | +| @gfld/compositor-protocol | MIT | ✅ (verified NO @gfld/compositor import) | +| @gfld/client-generator | MIT | ✅ | +| @gfld/compositor-generator | MIT | ✅ | +| @gfld/compositor-wasm (pixman+xkbcommon) | MIT | ✅ | +| @gfld/xtsb (X11 bindings) | MIT | ✅ (only if XWayland needed) | +| ~~@gfld/compositor-ffmpeg-h264~~ | ~~GPL-3.0~~ | ❌ REPLACE with WebCodecs | +| Custom browser compositor (your code) | PROPRIETARY | ✅ (clean-room) | +| **Python backend** | | | +| pywayland | Apache-2.0 | ✅ | +| python-xkbcommon (`xkbcommon` PyPI) | MIT | ✅ | +| PyGObject (gi.repository.Gst) | LGPL-2.1+ | ✅ (dynamic linking) | +| GLib | LGPL-2.1+ | ✅ (dynamic linking) | +| GStreamer core | LGPL-2.0+ | ✅ (dynamic linking) | +| gst-plugins-base, gst-plugins-good | LGPL | ✅ | +| gst-vaapi (vaapih264enc) | LGPL + MIT (libva) | ✅ | +| gst-plugins-bad (nvh264enc, openh264enc) | BSD subset | ✅ | +| ~~gst-plugins-ugly (x264enc)~~ | ~~GPL-2.0+~~ | ❌ DROP | +| ~~gst-libav (libavcodec)~~ | ~~LGPL source, GPL binary on Debian~~ | ❌ DROP | +| libwayland | MIT | ✅ | +| libdrm | MIT | ✅ | +| mesa (libGL, libEGL, libgbm) | MIT | ✅ | +| aiohttp | Apache-2.0 | ✅ | +| kcp (PyPI, RealistikDash) | MIT (Cython wrapper over ikcp.c) | ✅ | +| PyJWT | MIT | ✅ | +| bubblewrap (bwrap) | LGPL-2.0+ | ✅ (dynamic linking) | +| libseccomp | LGPL-2.1 | ✅ | +| **Odoo integration** | | | +| Odoo 20 Community | LGPL-3.0 | ✅ (separate process, no contamination) | +| Nginx | BSD-2-Clause | ✅ | +| Docker | Apache-2.0 | ✅ |