Initial commit: MIT-licensed compositor architecture & build plan (14 files, ~80 pages, 10 parallel research agents)
This commit is contained in:
1b-agent committed
commit
582e61e942
15 files changed
+4970
No files matched your search
+24
@@ -0,0 +1,24 @@
|
||||
# Python
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.pyo
|
||||
.venv/
|
||||
venv/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Editor
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Build
|
||||
node_modules/
|
||||
dist/
|
||||
build/
|
||||
*.egg-info/
|
||||
+312
@@ -0,0 +1,312 @@
|
||||
# 00 — The MIT Approach at a Glance
|
||||
|
||||
## The one-paragraph summary
|
||||
|
||||
Build a fully proprietary Wayland-to-HTML5 compositor for the Odoo 20 fork
|
||||
by **reusing Greenfield's 7 MIT-licensed libraries** as browser-side
|
||||
building blocks (the protocol runtime, the generators, the WASM modules
|
||||
for pixman+xkbcommon, the X11 bindings if XWayland is needed) and
|
||||
**replacing the 4 AGPL packages** (`@gfld/compositor`,
|
||||
`@gfld/compositor-proxy`, `@gfld/compositor-shell`,
|
||||
`@gfld/compositor-proxy-cli`) with **clean-room reimplementations** in
|
||||
TypeScript (browser) and Python (Linux host). The result integrates with
|
||||
Odoo 20 as a custom OWL shell + backend addon, with JWT-based auth and
|
||||
bubblewrap/seccomp app sandboxing — features Greenfield lacks entirely.
|
||||
|
||||
## What's reused (MIT/Apache/BSD — no AGPL contamination)
|
||||
|
||||
### Browser side — reused from Greenfield
|
||||
|
||||
| Library | License | Purpose |
|
||||
|---|---|---|
|
||||
| `@gfld/common` | MIT | Wayland wire format codec (Connection, WlObject, Fixed, marshallers) |
|
||||
| `@gfld/client-protocol` | MIT | Browser-side Wayland client bindings (for browser-resident JS/WASM apps that want to be Wayland clients of OUR compositor) |
|
||||
| `@gfld/compositor-protocol` | MIT | Browser-side Wayland server bindings — the **strategic keystone lib**. Verified: imports ZERO code from `@gfld/compositor` (AGPL). |
|
||||
| `@gfld/client-generator` | MIT | Build-time CLI: XML → TS browser client bindings |
|
||||
| `@gfld/compositor-generator` | MIT | Build-time CLI: XML → TS browser server bindings |
|
||||
| `@gfld/compositor-wasm` | MIT | pixman 0.42.2 + xkbcommon 1.5.0 compiled to WASM |
|
||||
| `@gfld/xtsb` | MIT | X11 TypeScript bindings (only if XWayland support is needed) |
|
||||
|
||||
### Browser side — native APIs (W3C standards, MIT-equivalent)
|
||||
|
||||
| API | Standard | Purpose |
|
||||
|---|---|---|
|
||||
| WebGL2 | W3C | GPU-accelerated 2D/3D rendering (replaces Greenfield's WebGL1) |
|
||||
| WebCodecs `VideoDecoder` | W3C | Hardware H.264/VP9/AV1 decode (replaces Greenfield's GPL FFmpeg WASM) |
|
||||
| OffscreenCanvas | W3C | Move WebGL to a Web Worker (Greenfield doesn't use this) |
|
||||
| Web Workers | W3C | Parallel decode + render (Greenfield uses only 2 workers for the fallback path) |
|
||||
| WebSocket | W3C | Binary transport (same as Greenfield) |
|
||||
| SharedArrayBuffer + Atomics | W3C | Zero-copy between workers (needs COOP/COEP headers) |
|
||||
| ImageBitmap | W3C | GPU-backed bitmap for the `web-bitmapbuf` fast path |
|
||||
| WebTransport | W3C (future) | UDP-over-QUIC transport (when browser support is widespread) |
|
||||
|
||||
### Linux side — Python ecosystem
|
||||
|
||||
| Component | License | Purpose |
|
||||
|---|---|---|
|
||||
| pywayland | Apache-2.0 | Python bindings to libwayland-server (replaces Greenfield's `wayland-server-addon.node` + `wayland-server.ts`) |
|
||||
| python-xkbcommon (`xkbcommon` on PyPI) | MIT | Python bindings to libxkbcommon |
|
||||
| PyGObject (`gi.repository.Gst`) | LGPL-2.1+ | Python bindings to GStreamer (replaces Greenfield's `proxy-encoding-addon.node` + `libproxy-encoding.so`) |
|
||||
| GLib | LGPL-2.1+ | GStreamer runtime dep |
|
||||
| GStreamer core | LGPL-2.0+ | Media pipeline framework |
|
||||
| gst-plugins-base, gst-plugins-good | LGPL | `pngenc`, `appsrc`, `appsink`, `videoconvert` |
|
||||
| gst-vaapi | LGPL + MIT (libva) | `vaapih264enc` for Intel/AMD GPU encoding |
|
||||
| gst-plugins-bad (BSD subset) | BSD | `nvh264enc` for NVIDIA NVENC, `openh264enc` for software fallback |
|
||||
| libwayland | MIT | The Wayland protocol C library |
|
||||
| libdrm | MIT | Direct Rendering Manager |
|
||||
| mesa (libGL, libEGL, libgbm) | MIT | OpenGL/EGL/GBM for GPU buffer management |
|
||||
| aiohttp | Apache-2.0 | Async HTTP + WebSocket server |
|
||||
| kcp (PyPI, RealistikDash) | MIT | Cython wrapper over upstream `ikcp.c` |
|
||||
| PyJWT | MIT | JWT issuance + validation |
|
||||
| bubblewrap (bwrap) | LGPL-2.0+ | Unprivileged sandboxing |
|
||||
| libseccomp | LGPL-2.1 | Seccomp BPF filter compilation |
|
||||
|
||||
### Linux side — vendored Greenfield C files (MIT)
|
||||
|
||||
These are the C files in `packages/compositor-proxy/native/wayland/src/`
|
||||
that are MIT-licensed (Greenfield's own work, NOT wlroots-derived unless
|
||||
noted) and can be **reused verbatim** in our Python proxy:
|
||||
|
||||
| File | Lines | Purpose |
|
||||
|---|---|---|
|
||||
| `westfield-buffer.c` | ~150 | Buffer helpers |
|
||||
| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM |
|
||||
| `westfield-egl.c` | ~900 | EGL zero-copy setup (creates EGLDisplay, EGLContext, wraps dmabuf as EGLImage) |
|
||||
| `westfield-xwayland.c` | ~465 | XWayland spawn + bridge |
|
||||
| `wlr_drm.c` | ~244 | `wl_drm` global impl (vendored from wlroots, MIT) |
|
||||
| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots, MIT) |
|
||||
| `drm_format_set.c` | ~239 | DRM format helpers (vendored from wlroots, MIT) |
|
||||
| `addon.c` | ~73 | wlroots addon system helper |
|
||||
| `changes.diff` | ~593 | Patches to libwayland-server adding wire-message interception hooks |
|
||||
|
||||
**Total**: ~3,974 lines of MIT-licensed C that can be reused as-is.
|
||||
|
||||
## What's replaced (custom code, written clean-room)
|
||||
|
||||
### Browser side — custom TypeScript compositor library
|
||||
|
||||
**Replaces**: `@gfld/compositor` (AGPL, ~21,101 LOC)
|
||||
|
||||
| Component | Lines | Weeks | Notes |
|
||||
|---|---|---|---|
|
||||
| `surface/` (Surface, SurfaceRole, Subsurface, Subcompositor, Region, FrameCallback) | ~1,200 | 3 | Double-buffered state, commit semantics |
|
||||
| `render/` (Renderer, Scene, View, Output, RenderState, Texture, shaders) | ~1,800 | 3 | WebGL2 + OffscreenCanvas + Web Workers |
|
||||
| `buffer/` (Shm, BufferContents, ImageBitmapBuffer, EncodedFrameBuffer) | ~600 | 1.5 | SHM with BMP-header hack |
|
||||
| `transport/` (Channel, KCP, ReconnectingWebSocket, BufferStream, decoders) | ~3,500 | 4 | KCP ported fresh from MIT C ref |
|
||||
| `input/` (Seat, Keyboard, Pointer, Touch, Xkb, InputQueue, InputOutput) | ~2,500 | 3 | Uses @gfld/compositor-wasm |
|
||||
| `roles/` (XdgWmBase, XdgSurface, XdgToplevel, XdgPopup, XdgPositioner, Shell) | ~1,800 | 3 | xdg-shell state machine, biggest risk |
|
||||
| `data/` (DataDeviceManager, DataSource, DataOffer, WaylandDataSource) | ~600 | 1.5 | Clipboard + DnD + primary selection (Greenfield lacks primary) |
|
||||
| `desktop/` (Desktop, FloatingDesktopSurface, FullscreenDesktopSurface) | ~800 | 1.5 | Layout policy |
|
||||
| `math/` (Mat4, Vec4, Point, Size, Rect, Transformations) | ~500 | 0.5 | |
|
||||
| `userapi/` (UserShellApi) | ~120 | 0.5 | Odoo OWL shell integration seam |
|
||||
| `browser/` (input.ts, pointer.ts, dnd.ts, selection.ts) | ~750 | 2 | DOM glue |
|
||||
| `decoders/` (WebCodecsFrameDecoder, OpenH264FrameDecoder) | ~500 | 1 | NO FFmpeg Wasm (GPL) |
|
||||
| `core/` (Session, Globals) | ~200 | 0.5 | |
|
||||
| **MVP TOTAL** | **~13,400** | **~25 weeks** | |
|
||||
| `xwayland/` (XWindowManager, XWindow, XWindowFrame) | ~5,000 | 8 | Optional, only for X11 apps |
|
||||
| Tests + hardening | — | 3 | |
|
||||
| **GRAND TOTAL** | **~18,900** | **~30-36 weeks** | |
|
||||
|
||||
### Linux side — custom Python proxy
|
||||
|
||||
**Replaces**: `@gfld/compositor-proxy` + `@gfld/compositor-proxy-cli` (AGPL, ~5,000 LOC TS + ~3,974 LOC C)
|
||||
|
||||
| Component | Lines | Dev-days | Notes |
|
||||
|---|---|---|---|
|
||||
| aiohttp WS server (9 endpoints) | 600 | 3 | /signal, /channel, /mkfifo, /mkstemp-mmap, /read-fd, /read-fd-as-stream, /write-fd-as-stream, /close-fd, /request-keyframe |
|
||||
| `Channel` / `ARQChannel` / `SimpleChannel` port | 250 | 1.5 | |
|
||||
| `kcp` PyPI package integration | 100 | 0.5 | |
|
||||
| `NativeAppContext` (signaling, kill timers) | 350 | 2 | 10-min SIGHUP grace, SIGTERM escalation |
|
||||
| `ProxyInputOutput` (FD workaround over WS) | 300 | 2 | memfd_create, mkfifo via os.pipe |
|
||||
| libwayland event-loop integration | 80 | 1 | 5-line cffi shim for `wl_event_loop_get_fd` |
|
||||
| Multiprocessing scaffolding + FD passing | 200 | 1.5 | `multiprocessing.reduction.send_handle` |
|
||||
| GStreamer encoder thread + marshalling | 250 | 2 | `asyncio.run_coroutine_threadsafe` |
|
||||
| Reconnection & lifecycle | 150 | 1 | |
|
||||
| Tests (pytest + pytest-asyncio) | 600 | 3 | |
|
||||
| **TOTAL** | **~2,880** | **~17 days** | |
|
||||
|
||||
### Odoo 20 integration — custom addon
|
||||
|
||||
| Component | Dev-days | Notes |
|
||||
|---|---|---|
|
||||
| Odoo addon skeleton (models + controllers + views + security) | 2 | compositor.app, compositor.session, compositor.audit.log |
|
||||
| JWT issuance + audit log model | 1.5 | HS256, 15-min expiry |
|
||||
| OWL 3 fullscreen client action + canvas + WS wiring | 2.5 | `ir.actions.client` with `target=fullscreen` |
|
||||
| Browser compositor library integration | 2 | `userShell.actions.initScene(canvasCreator)` |
|
||||
| Compositor proxy JWT middleware + WS handshake | 1.5 | |
|
||||
| Audit POSTback + Nginx + Docker Compose + GPU passthrough | 2 | |
|
||||
| Multi-tenant: per-DB proxy spawning + db claim scoping | 1.5 | |
|
||||
| Tests (HttpCase + TransactionCase) | 2 | |
|
||||
| Documentation + buffer | 3 | |
|
||||
| **TOTAL** | **~18 days** | |
|
||||
|
||||
### Sandboxing — NEW (Greenfield has none)
|
||||
|
||||
| Component | Dev-days | Notes |
|
||||
|---|---|---|
|
||||
| bubblewrap integration | 2-3 | |
|
||||
| seccomp profile + compile | 1-2 | Docker default profile + CLONE_NEWUSER allow |
|
||||
| systemd-run cgroup wrapper | 1 | MemoryMax=1G, CPUQuota=200%, TasksMax=256 |
|
||||
| Wayland socket ownership fix | 0.5 | chmod 0660 + add sandbox group |
|
||||
| GPU passthrough testing | 1 | /dev/dri/renderD128 only |
|
||||
| Schema + docs | 1 | |
|
||||
| **v1 TOTAL** | **~7-9 days** | shippable, XWayland unsandboxed |
|
||||
| XWayland sandbox (v2) | +4-9 | |
|
||||
| veth + nftables networking (v2) | +2 | |
|
||||
| Per-tenant hierarchical cgroups (v2) | +2 | |
|
||||
| XDG Desktop Portal (FileChooser/Secret) (v2) | +3 | |
|
||||
|
||||
## Architecture at a glance
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER (Odoo 20 web client) │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Odoo 20 OWL Shell addon (PROPRIETARY) │ │
|
||||
│ │ - ir.actions.client target=fullscreen │ │
|
||||
│ │ - <canvas t-ref="canvas"> owned by OWL component │ │
|
||||
│ │ - Implements userShell.events.* callbacks │ │
|
||||
│ │ - Calls userShell.actions.initScene, activateSurface, ... │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ userShell.events + actions (clean-room spec) │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Custom Browser Compositor Library (PROPRIETARY, clean-room TS) │ │
|
||||
│ │ - Session, Compositor, Globals, Surface, View, Output, Seat, ... │ │
|
||||
│ │ - render/ (WebGL2 + OffscreenCanvas + Web Workers) │ │
|
||||
│ │ - transport/ (WebSocket + KCP + ReconnectingWebSocket) │ │
|
||||
│ │ - roles/ (xdg-shell state machine, clean-room from spec) │ │
|
||||
│ │ - desktop/, input/, data/, math/ │ │
|
||||
│ │ - userapi/ (UserShellApi — the OWL shell integration seam) │ │
|
||||
│ │ ~13,400 LOC, ~25 dev-weeks for MVP │ │
|
||||
│ │ │ │
|
||||
│ │ Reuses MIT libs as npm deps: │ │
|
||||
│ │ - @gfld/common, @gfld/compositor-protocol, @gfld/client-protocol (MIT) │ │
|
||||
│ │ - @gfld/compositor-wasm (pixman + xkbcommon WASM, MIT) │ │
|
||||
│ │ - @gfld/xtsb (MIT, X11 — only if XWayland needed) │ │
|
||||
│ │ - WebCodecs API (W3C standard, replaces GPL FFmpeg WASM) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ WebSocket (TCP) + KCP-over-WS
|
||||
│ 9 endpoint paths per session
|
||||
│ JWT (HS256) auth, 15-min expiry
|
||||
│ Same-origin via Nginx reverse proxy
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ PYTHON HOST (Linux server) │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Nginx (TLS termination, BSD-2-Clause) │ │
|
||||
│ │ / → Odoo :8069 (HTTP workers) │ │
|
||||
│ │ /websocket → Odoo :8072 (gevent, Discuss/chat) │ │
|
||||
│ │ /compositor/ → Compositor proxy :8081 (aiohttp WS, proxy_buffering off) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Odoo 20 (LGPL-3.0, separate process — no contamination) │ │
|
||||
│ │ - HTTP workers (preforked, N processes) │ │
|
||||
│ │ - Controllers: /web/compositor/{token, apps, launch, terminate, audit} │ │
|
||||
│ │ - Models: compositor.app, compositor.session, compositor.audit.log │ │
|
||||
│ │ - JWT issuer (HS256 secret in ir.config_parameter) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ subprocess + IPC │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Python Compositor Proxy (PROPRIETARY, ~2880 LOC) │ │
|
||||
│ │ - aiohttp WebSocket server on :8081, 9 endpoints │ │
|
||||
│ │ - asyncio + 5-line cffi shim for libwayland event loop │ │
|
||||
│ │ - multiprocessing.Process per Odoo session (fork) │ │
|
||||
│ │ - JWT validation (pyjwt) │ │
|
||||
│ │ - pywayland (Apache-2.0) for Wayland protocol dispatch │ │
|
||||
│ │ - python-xkbcommon (MIT) for keymap compilation │ │
|
||||
│ │ - PyGObject+GStreamer (LGPL) for H.264/PNG encoding │ │
|
||||
│ │ - Uses vaapih264enc / nvh264enc / openh264enc (NOT x264enc — GPL) │ │
|
||||
│ │ - kcp (PyPI, MIT, Cython wrapper over ikcp.c) for ARQ channels │ │
|
||||
│ │ - Vendored Greenfield C files (MIT) for dmabuf/egl/xwayland helpers │ │
|
||||
│ │ - Greenfield's changes.diff applied to libwayland-server (MIT patches) │ │
|
||||
│ │ - bubblewrap + seccomp + systemd-run --scope for app sandboxing │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ WAYLAND_DISPLAY=wayland-N │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Native Linux apps (gtk4-demo, xterm, kwrite, gnome-calculator, ...) │ │
|
||||
│ │ - Spawned via: systemd-run --user --scope -p MemoryMax=1G -p CPUQuota=200% │ │
|
||||
│ │ → bwrap --unshare-all --clearenv --seccomp <memfd> --ro-bind /usr /usr │ │
|
||||
│ │ --dev-bind /dev/dri/renderD128 /dev/dri/renderD128 │ │
|
||||
│ │ --ro-bind $HOST_XDG/wayland-N /run/user/<UID>/wayland-N │ │
|
||||
│ │ --uid <100000-165535 per-app> --gid <same> │ │
|
||||
│ │ --new-session --die-with-parent │ │
|
||||
│ │ /usr/bin/<app> │ │
|
||||
│ │ - Filesystem: /usr ro-bind, minimal /etc whitelist, $HOME overlay │ │
|
||||
│ │ - Network: --unshare-net by default (most apps don't need it) │ │
|
||||
│ │ - GPU: /dev/dri/renderD128 only (kernel render-node access control) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## What we GAIN over Greenfield
|
||||
|
||||
1. **Full proprietary control** — no AGPL contamination, ship as a closed-source
|
||||
Odoo addon.
|
||||
2. **Modern browser stack** — WebGL2 (vs Greenfield's WebGL1), OffscreenCanvas +
|
||||
Web Workers (vs Greenfield's main-thread WebGL), WebCodecs with hardware
|
||||
decode enabled (Greenfield disables it with a FIXME).
|
||||
3. **Sandboxing** — bubblewrap + seccomp + systemd-run + cgroups. Greenfield
|
||||
has zero sandboxing.
|
||||
4. **Multi-tenancy** — per-Odoo-database proxy isolation, JWT-scoped sessions.
|
||||
Greenfield is single-user single-session.
|
||||
5. **Odoo integration** — auth via Odoo session, audit log, app launcher from
|
||||
Odoo's app catalog. Greenfield has its own Preact shell.
|
||||
6. **Adaptive bitrate** — consume the `EncoderFeedback` data Greenfield produces
|
||||
but doesn't use.
|
||||
7. **Primary selection** — middle-click paste, which Greenfield lacks.
|
||||
8. **Real touch input** — Greenfield has a 46-line stub that never fires.
|
||||
9. **Modern codecs** — VP9/AV1 support via WebCodecs (Greenfield is H.264 only).
|
||||
10. **Clean upgrade path** — no native Node addons (no `.node` files), no
|
||||
Node.js ABI lock-in, no CMake/Ninja build chain.
|
||||
|
||||
## What we LOSE vs Greenfield
|
||||
|
||||
1. **~25 dev-weeks of engineering** — the browser compositor is the
|
||||
biggest cost. Clean-room rewrite is mandatory for legal safety.
|
||||
2. **Greenfield's battle-tested codebase** — we'd be writing fresh Wayland
|
||||
state machines, which is subtle (xdg-shell configure/ack round-trips,
|
||||
subsurface sync/desync cascades, popup positioner math).
|
||||
3. **Time-to-market** — Greenfield works today; our MIT approach takes
|
||||
~6-8 months to MVP.
|
||||
4. **Greenfield's WASM app SDK** — the `sdk/` directory is for compiling
|
||||
native Wayland apps to WASM and running them in the browser. If you
|
||||
need this feature, it's a separate ~4-8 week project to port.
|
||||
|
||||
## Total cost summary (single engineer, sequential)
|
||||
|
||||
| Phase | Weeks | Cumulative |
|
||||
|---|---|---|
|
||||
| Phase 0: Spike (validate libwayland ↔ asyncio) | 1 | 1 |
|
||||
| Phase 1: Browser compositor MVP (clean-room TS) | 25 | 26 |
|
||||
| Phase 2: Python proxy MVP (aiohttp + pywayland + GStreamer) | 3.5 | 29.5 |
|
||||
| Phase 3: Odoo integration (OWL shell + backend addon + JWT) | 3.5 | 33 |
|
||||
| Phase 4: Sandboxing v1 (bubblewrap + seccomp) | 1.5 | 34.5 |
|
||||
| Phase 5: Polish & testing | 3 | 37.5 |
|
||||
| **MVP TOTAL** | **~37.5 weeks** | (~9 months) |
|
||||
| Phase 6 (optional): XWayland + multi-tenant + portal | +9 | ~46.5 weeks |
|
||||
|
||||
With 2 senior engineers in parallel (one on browser compositor, one on
|
||||
Python proxy + Odoo integration):
|
||||
|
||||
| Phase | Weeks | Cumulative |
|
||||
|---|---|---|
|
||||
| Phase 0: Spike | 1 | 1 |
|
||||
| Phase 1+2 (parallel) | 25 (browser is critical path) | 26 |
|
||||
| Phase 3+4 (parallel) | 3.5 | 29.5 |
|
||||
| Phase 5: Polish & testing | 3 | 32.5 |
|
||||
| **MVP TOTAL** | **~32.5 weeks** | (~8 months) |
|
||||
| Phase 6 (optional): XWayland + multi-tenant + portal | +6 | ~38.5 weeks |
|
||||
|
||||
See `../Greenfield-vs-MIT-Comparison.md` for the head-to-head comparison.
|
||||
@@ -0,0 +1,194 @@
|
||||
# 01 — Layered Architecture Map
|
||||
|
||||
This chapter walks the proposed MIT-licensed architecture layer by layer,
|
||||
mirroring the Greenfield analysis structure (see `../greenfield-analysis/01-architecture.md`)
|
||||
so you can compare them side-by-side.
|
||||
|
||||
## The eight layers (MIT approach)
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────┐
|
||||
│ Layer 8: USER SHELL (Odoo 20 OWL addon, PROPRIETARY) │
|
||||
│ Hooks: UserShellApi (events + actions) — clean-room spec │
|
||||
│ See 10-odoo-integration.md │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 7: DESKTOP / LAYOUT POLICY (PROPRIETARY TS, clean-room) │
|
||||
│ Floating vs AlwaysFullScreen, view stacking, focus activation refcount │
|
||||
│ Custom: desktop/Desktop.ts (~800 LOC) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 6: SURFACE LIFECYCLE / ROLES (PROPRIETARY TS, clean-room) │
|
||||
│ XdgWmBase, XdgSurface, XdgToplevel, XdgPopup, XdgPositioner, Shell, │
|
||||
│ ShellSurface, Subcompositor, Subsurface, SurfaceRole │
|
||||
│ Custom: roles/*.ts (~1,800 LOC) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 5: RENDERING PIPELINE (PROPRIETARY TS, clean-room, modernized) │
|
||||
│ Surface (state stack), View, Output (WebGL2 canvas), Renderer, Scene, │
|
||||
│ Shaders (YUV→RGBA, YUVA→RGBA, RGB+A→RGBA), Loop (RAF), Callback │
|
||||
│ NEW: OffscreenCanvas + Web Workers + WebGL2 (Greenfield uses WebGL1) │
|
||||
│ Custom: render/*.ts (~1,800 LOC), surface/*.ts (~1,200 LOC) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 4: TRANSPORT (browser side, PROPRIETARY TS, clean-room) │
|
||||
│ Channel (ARQ + Simple), ReconnectingWebSocket, RemoteAppLauncher, │
|
||||
│ StreamingBuffer, BufferStream, EncodedFrame, EncoderFeedback, decoders │
|
||||
│ NEW: WebCodecs VideoDecoder replaces FFmpeg WASM (was GPL) │
|
||||
│ Custom: transport/*.ts (~3,500 LOC), decoders/*.ts (~500 LOC) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 4: TRANSPORT (Python side, PROPRIETARY, clean-room) │
|
||||
│ aiohttp WebSocket server (9 endpoints), asyncio + cffi shim, │
|
||||
│ kcp PyPI package, multiprocessing.Process per session, JWT auth │
|
||||
│ Custom: proxy/*.py (~2,880 LOC) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 3: WAYLAND RUNTIME (browser side) │
|
||||
│ REUSED: @gfld/common (Connection, WlObject, Fixed, marshallers) — MIT │
|
||||
│ REUSED: @gfld/compositor-protocol (Display, Client, Resource, Global) │
|
||||
│ — MIT, verified NO @gfld/compositor import │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 3: WAYLAND RUNTIME (Python side) │
|
||||
│ REUSED: pywayland (Apache-2.0) — Display, Client, Resource, Global │
|
||||
│ + 5-line cffi shim for wl_event_loop_get_fd (asyncio integration) │
|
||||
│ + Greenfield's changes.diff (MIT patches) applied to libwayland-server │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 2: PROTOCOL GENERATORS (REUSED) │
|
||||
│ REUSED: @gfld/client-generator, @gfld/compositor-generator (MIT) │
|
||||
│ REUSED: pywayland-protocols (auto-generated from wayland-protocols repo) │
|
||||
├──────────────────────────────────────────────────────────────────────────┤
|
||||
│ Layer 1: WAYLAND PROTOCOL XML (REUSED) │
|
||||
│ REUSED: wayland.xml, xdg-shell.xml, wayland-drm.xml (upstream MIT) │
|
||||
│ REUSED: web-bitmapbuf-unstable-v1.xml (Greenfield's custom extension) │
|
||||
└──────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Side-by-side comparison with Greenfield
|
||||
|
||||
| Layer | Greenfield | MIT approach |
|
||||
|---|---|---|
|
||||
| Layer 8: User Shell | `@gfld/compositor-shell` (AGPL, Preact) | Custom Odoo 20 OWL addon (proprietary) |
|
||||
| Layer 7: Desktop policy | `desktop/Desktop.ts` (AGPL) | Clean-room rewrite (~800 LOC TS) |
|
||||
| Layer 6: Surface roles | `Xdg*.ts`, `Shell*.ts`, `Sub*.ts` (AGPL) | Clean-room rewrite (~1,800 LOC TS) |
|
||||
| Layer 5: Rendering | `render/*.ts`, `Surface.ts`, etc. (AGPL, WebGL1, main-thread) | Clean-room rewrite (~3,000 LOC TS, WebGL2 + OffscreenCanvas + Workers) |
|
||||
| Layer 4: Transport (browser) | `remote/*.ts` (AGPL) + FFmpeg WASM (GPL) | Clean-room rewrite (~3,500 LOC TS) + WebCodecs (W3C standard) |
|
||||
| Layer 4: Transport (host) | `compositor-proxy/src/*.ts` (AGPL) + 3 N-API addons | Clean-room Python rewrite (~2,880 LOC Python, NO native addons) |
|
||||
| Layer 3: Wayland runtime (browser) | `@gfld/common` + `@gfld/compositor-protocol` (MIT) | **REUSED AS-IS** |
|
||||
| Layer 3: Wayland runtime (host) | `wayland-server-addon.node` (AGPL, 1231 lines C, patched libwayland) | **REPLACED** with pywayland (Apache-2.0) + 5-line cffi shim + Greenfield's `changes.diff` (MIT patches) |
|
||||
| Layer 2: Generators | `@gfld/*-generator` (MIT) | **REUSED AS-IS** + pywayland-protocols (auto-generated) |
|
||||
| Layer 1: Protocol XML | `protocol/*.xml` (upstream MIT + Greenfield's web-bitmapbuf) | **REUSED AS-IS** |
|
||||
|
||||
## How data flows across the layers
|
||||
|
||||
```
|
||||
NATIVE APP (e.g. gtk4-demo, X11 via XWayland)
|
||||
│
|
||||
│ libwayland-client → UNIX socket + SCM_RIGHTS FDs
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ PYTHON PROXY (aiohttp + pywayland + GStreamer + bubblewrap) │
|
||||
│ │
|
||||
│ Layer 3 (Python): pywayland + 5-line cffi shim │
|
||||
│ ↳ VANILLA libwayland-server (NO patches needed for basic use) │
|
||||
│ ↳ Apply Greenfield's changes.diff (MIT) IF we want wire-message │
|
||||
│ interception hooks (recommended for the fast-sync optimization) │
|
||||
│ ↳ Per-resource dispatchers (auto-generated by pywayland scanner) │
|
||||
│ ↳ FDs wrapped as ProxyFD JSON │
|
||||
│ │
|
||||
│ Layer 4 (Python): aiohttp WebSocket + kcp PyPI package │
|
||||
│ ↳ 9 endpoints: /signal, /channel, /mkfifo, /mkstemp-mmap, │
|
||||
│ /read-fd, /read-fd-as-stream, /write-fd-as-stream, │
|
||||
│ /close-fd, /request-keyframe │
|
||||
│ ↳ asyncio integrates libwayland FD via loop.add_reader() │
|
||||
│ ↳ multiprocessing.Process per Odoo session (fork) │
|
||||
│ ↳ JWT validation (pyjwt) │
|
||||
│ │
|
||||
│ Layer 5 (encoding): PyGObject + GStreamer │
|
||||
│ ↳ gst_parse_launch: same pipeline strings as Greenfield │
|
||||
│ ↳ x264enc REPLACED with vaapih264enc / nvh264enc / openh264enc │
|
||||
│ (x264enc is GPL — see 11-pitfalls.md) │
|
||||
│ ↳ Zero-copy dmabuf via EGLImage → GstGLMemoryEGL (same as Greenfield)│
|
||||
│ ↳ GLib MainLoop in separate Python thread │
|
||||
│ ↳ asyncio.run_coroutine_threadsafe for cross-thread marshalling │
|
||||
│ │
|
||||
│ Layer 7 (sandboxing): bubblewrap + seccomp + systemd-run --scope │
|
||||
│ ↳ Per-app: MemoryMax=1G, CPUQuota=200%, TasksMax=256 │
|
||||
│ ↳ --unshare-all, --clearenv, --ro-bind /usr /usr │
|
||||
│ ↳ --dev-bind /dev/dri/renderD128 /dev/dri/renderD128 (single GPU) │
|
||||
│ ↳ --ro-bind $HOST_XDG/wayland-N /run/user/<UID>/wayland-N │
|
||||
│ ↳ --uid <100000-165535 per-app> --gid <same> │
|
||||
│ ↳ --unshare-net by default │
|
||||
│ │
|
||||
│ Layer 1 (XML) → Layer 2 (pywayland scanner) feeds Layer 3 dispatchers │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ WebSocket (TCP) + KCP-over-WS, JWT auth
|
||||
│ Same-origin via Nginx reverse proxy
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER COMPOSITOR (Custom TypeScript + WebGL2 + WebCodecs + WASM) │
|
||||
│ │
|
||||
│ Layer 4 (browser): Custom Channel + KCP + ReconnectingWebSocket │
|
||||
│ ↳ ARQChannel + SimpleChannel (clean-room, ~250 LOC) │
|
||||
│ ↳ KCP ported fresh from MIT C reference (skywind3000/kcp) │
|
||||
│ ↳ ReconnectingWebSocket (vendored from pladaria, MIT) │
|
||||
│ ↳ WebCodecs VideoDecoder (primary, hardware-accelerated) │
|
||||
│ ↳ OpenH264 BSD Wasm (fallback, NOT FFmpeg — FFmpeg is GPL) │
|
||||
│ │
|
||||
│ Layer 3 (browser): REUSED @gfld/common + @gfld/compositor-protocol │
|
||||
│ ↳ Standard Wayland wire format, byte-for-byte │
|
||||
│ ↳ WlObject registry, opcode dispatch via wlObject[opcode](msg) │
|
||||
│ ↳ Per-client Connections (slow client never blocks another) │
|
||||
│ │
|
||||
│ Layer 5: Custom Surface.ts + View.ts + Output.ts + render/ │
|
||||
│ ↳ Double-buffered state (state vs pendingState) │
|
||||
│ ↳ Damage tracking via @gfld/compositor-wasm pixman regions │
|
||||
│ ↳ WebGL2 with FBOs for YUV→RGBA, RGB+Alpha→RGBA conversion │
|
||||
│ ↳ OffscreenCanvas in Web Worker (NEW vs Greenfield) │
|
||||
│ ↳ Frame callbacks fired on requestAnimationFrame vsync │
|
||||
│ │
|
||||
│ Layer 6: Custom XdgWmBase + XdgSurface + XdgToplevel + XdgPopup + ... │
|
||||
│ ↳ Configure/ack-configure round-trips │
|
||||
│ ↳ Popup positioner math (anchor + gravity + constraints) │
|
||||
│ ↳ Subsurface sync/desync mode cascading │
|
||||
│ ↳ Primary selection (NEW — Greenfield lacks this) │
|
||||
│ │
|
||||
│ Layer 7: Custom desktop/Desktop.ts │
|
||||
│ ↳ Floating vs AlwaysFullScreen mode │
|
||||
│ ↳ View stacking, focus refcount │
|
||||
│ ↳ Adaptive bitrate (NEW — consumes EncoderFeedback data) │
|
||||
│ │
|
||||
│ Layer 8: UserShellApi (clean-room spec) + Odoo 20 OWL shell │
|
||||
│ ↳ Events: clientCreated, surfaceCreated, surfaceTitleUpdated, │
|
||||
│ surfaceActivationUpdated, ... │
|
||||
│ ↳ Actions: initScene, refreshScene, destroyScene, │
|
||||
│ closeClient, activateSurface, setUserConfiguration │
|
||||
│ ↳ OWL shell wires events to reactive state │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ userShell.events.* fired by compositor
|
||||
│ userShell.actions.* called by OWL shell
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ USER SHELL (Odoo 20 OWL addon — PROPRIETARY) │
|
||||
│ │
|
||||
│ - ir.actions.client with target=fullscreen │
|
||||
│ - Application launcher (lists apps from Odoo /web/compositor/apps) │
|
||||
│ - Window management UI (taskbar, decorations, minimize/maximize) │
|
||||
│ - Keyboard layout picker (RMLVO) │
|
||||
│ - Per-output <canvas> element (OWL component owns the lifecycle) │
|
||||
│ - Triggers app launches via /web/compositor/launch │
|
||||
│ - Receives JWT from /web/compositor/token │
|
||||
│ │
|
||||
│ Not a Greenfield component — entirely your code. │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Reading guide for the rest of the bundle
|
||||
|
||||
- `02-wayland-protocol.md` — Layer 1, 2, 3 (XML → generators → runtime, both sides)
|
||||
- `03-html5-rendering.md` — Layer 5 (custom WebGL2 + Workers + OffscreenCanvas)
|
||||
- `04-transport.md` — Layer 4 (Python aiohttp + kcp, browser side clean-room)
|
||||
- `05-input-handling.md` — Layer 5 input subsystem (Python xkbcommon, browser side)
|
||||
- `06-data-clipboard.md` — Layer 5 data device (with primary selection, Greenfield lacks)
|
||||
- `07-native-app-bridge.md` — Layer 4 Python + Layer 3 Python + sandboxing
|
||||
- `08-app-lifecycle.md` — Layer 6 + Layer 8 (UserShellApi clean-room spec)
|
||||
- `09-hardware-accel.md` — Layer 5 encoding + Layer 4 frame channel + WebCodecs
|
||||
- `10-odoo-integration.md` — Layer 8 + Odoo 20 backend addon + JWT + Nginx + Docker
|
||||
- `11-pitfalls.md` — 3 GPL contaminants found + clean-room legal requirements
|
||||
- `12-appendix.md` — glossary, file index, references
|
||||
@@ -0,0 +1,291 @@
|
||||
# 02 — Wayland Protocol Layer (MIT-licensed)
|
||||
|
||||
This chapter covers how the Wayland protocol layer works in the MIT
|
||||
approach: protocol XML → generators → runtime, for both the browser side
|
||||
and the Python host side.
|
||||
|
||||
## Layer 1 — Protocol XML (REUSED AS-IS)
|
||||
|
||||
Same 4 XML files as Greenfield, all upstream MIT or Greenfield's own MIT
|
||||
extension:
|
||||
|
||||
- `protocol/wayland.xml` — upstream Wayland core (MIT)
|
||||
- `protocol/xdg-shell.xml` — stable xdg-shell (MIT)
|
||||
- `protocol/wayland-drm.xml` — legacy wl_drm (MIT, host-side only)
|
||||
- `protocol/web-bitmapbuf-unstable-v1.xml` — Greenfield's custom extension
|
||||
for browser JS apps (MIT)
|
||||
|
||||
**No changes needed.** These files are the source of truth.
|
||||
|
||||
## Layer 2 — Protocol Generators (REUSED AS-IS + pywayland scanner)
|
||||
|
||||
### Browser side — REUSED from Greenfield (MIT)
|
||||
|
||||
- `@gfld/client-generator` (MIT) — generates TS `*Proxy` classes for browser
|
||||
apps that want to be Wayland clients of OUR compositor.
|
||||
- `@gfld/compositor-generator` (MIT) — generates TS `*Resource` + `*Requests`
|
||||
classes for our custom browser compositor.
|
||||
- `@gfld/compositor-proxy-generator` — **NOT NEEDED** (this generates
|
||||
interceptor files for the Node N-API addon; we don't have one).
|
||||
|
||||
### Python side — pywayland's built-in scanner
|
||||
|
||||
pywayland ships its own scanner that parses the same `wayland.xml` +
|
||||
`wayland-protocols/*.xml` and emits Python bindings
|
||||
(`pywayland/protocol/*.py`). It uses the upstream `wayland-scanner`
|
||||
C tool under the hood.
|
||||
|
||||
**No new generator code to write.** The scanner produces:
|
||||
- `pywayland/protocol/wayland.py` — wl_display, wl_registry, wl_compositor,
|
||||
wl_surface, wl_buffer, wl_seat, wl_keyboard, wl_pointer, wl_touch,
|
||||
wl_output, wl_shm, wl_data_device_manager, wl_subcompositor
|
||||
- `pywayland/protocol/xdg_shell.py` — xdg_wm_base, xdg_surface,
|
||||
xdg_toplevel, xdg_popup, xdg_positioner
|
||||
- 60+ additional files for stable and unstable Wayland protocols
|
||||
|
||||
For Greenfield's custom `web-bitmapbuf-unstable-v1.xml`: **not needed on
|
||||
the Python side** (native apps don't use it; it's browser-only). If you
|
||||
need to expose it to native apps later, pywayland supports custom
|
||||
protocol XMLs via `pywayland.scanner`.
|
||||
|
||||
## Layer 3 — Wayland Runtime
|
||||
|
||||
### Browser side — REUSED from Greenfield (MIT, verified clean)
|
||||
|
||||
`@gfld/common` (`libs/common/src/Connection.ts`) — the shared wire format
|
||||
codec. Standard upstream Wayland wire format, byte-for-byte. 8-byte
|
||||
header (`u32 object_id | u16 opcode | u16 size`), per-arg bytes.
|
||||
|
||||
`@gfld/compositor-protocol` (`libs/compositor-protocol/src/`) — the
|
||||
server-side runtime. Provides `Display`, `Client`, `Resource`, `Global`,
|
||||
`Registry`, plus generated `*Resource` classes for wayland, xdg_shell,
|
||||
web_bitmapbuf.
|
||||
|
||||
**CRITICAL VERIFICATION** (from Agent M's audit): `@gfld/compositor-protocol`
|
||||
imports ZERO code from `@gfld/compositor` (AGPL). It only depends on
|
||||
`@gfld/common` (MIT). So it's safe to reuse in a proprietary codebase.
|
||||
|
||||
### Python side — pywayland (Apache-2.0)
|
||||
|
||||
pywayland is the Python bindings to libwayland-server. License:
|
||||
**Apache-2.0** (not MIT as the task brief assumed, but Apache-2.0 is
|
||||
permissive — allows proprietary use, modification, distribution).
|
||||
|
||||
#### API surface
|
||||
|
||||
**`Display`** — wraps `struct wl_display*`:
|
||||
- `Display()` — create
|
||||
- `.add_socket(name=None)` → returns `"wayland-N"` (becomes WAYLAND_DISPLAY)
|
||||
- `.get_event_loop()` → `EventLoop`
|
||||
- `.run()` — block (we don't call this — we use asyncio integration instead)
|
||||
- `.terminate()` — break the run loop
|
||||
- `.init_shm()` — register wl_shm global
|
||||
- `.add_shm_format(fmt)` — add SHM format (YUV420, NV12, etc.)
|
||||
- `.flush_clients()` — send pending events
|
||||
- `.get_serial()` → next serial number
|
||||
- `.next_serial()` — allocate a serial
|
||||
- `.destroy()`
|
||||
- `.add_destroy_listener(listener)`
|
||||
|
||||
**`EventLoop`** — wraps `struct wl_event_loop*`:
|
||||
- `.add_fd(fd, cb, mask, data)` — register an FD for polling
|
||||
- `.add_signal(signum, cb, data)`
|
||||
- `.add_timer(cb, data)` → `Timer`
|
||||
- `.add_idle(cb, data)`
|
||||
- `.dispatch(timeout_ms)` — process pending events (we call this from asyncio)
|
||||
- `.dispatch_idle()`
|
||||
- `.add_destroy_listener(listener)`
|
||||
- **MISSING: `.get_fd()`** — needed for asyncio integration, see §asyncio below
|
||||
|
||||
**`Client`** — wraps `struct wl_client*`:
|
||||
- `Client(display, fd)` — create from inbound socketpair
|
||||
- `Client(ptr=existing_ptr)` — wrap existing
|
||||
- `.destroy()`
|
||||
- `.flush()`
|
||||
- `.get_credentials()` → `(pid, uid, gid)` — used to identify which app spawned this client
|
||||
- `.add_destroy_listener(listener)`
|
||||
- `.get_object(id)` → `Resource`
|
||||
- `Client.from_resource(res)` — static method
|
||||
- **MISSING: `wl_display_add_client_created_listener`** — for new-client notifications. Available in upstream libwayland, exposed via 5-line cffi shim.
|
||||
|
||||
**`Resource`** (base class for generated `Wl*` types):
|
||||
- `Resource(client, version, id=0)`
|
||||
- `.destroy()`
|
||||
- `.get_id()` → int
|
||||
- `.add_destroy_listener(listener)`
|
||||
- `._post_event(opcode, *args)` — emit event to client
|
||||
- `._post_error(code, msg)` — send protocol error
|
||||
- `.dispatcher[opcode_or_name] = func` — per-resource dispatch table (KEY mechanism)
|
||||
- `.dispatcher.destructor = func`
|
||||
|
||||
**`Global`** (base class for generated `Wl*.global_class`):
|
||||
- `WlFoo.global_class(display)` — instantiate
|
||||
- `.bind_func = handler` — callback when a client binds
|
||||
- `.destroy()`
|
||||
|
||||
**`Listener`** + **`Signal`** — standard observer pattern:
|
||||
- `Listener(callable)`
|
||||
- `Signal.add(listener)`
|
||||
- `Signal.emit(data)`
|
||||
- `Listener.remove()`
|
||||
|
||||
#### asyncio integration (verified working)
|
||||
|
||||
pywayland bundles vanilla upstream libwayland 1.26.0 (verified by `objdump`
|
||||
of the bundled `.so` — only standard `wl_*` exports, NO Greenfield patch
|
||||
additions). To integrate with asyncio:
|
||||
|
||||
```python
|
||||
import asyncio, cffi, os, pywayland
|
||||
from pywayland.server import Display
|
||||
|
||||
# 5-line cffi shim for wl_event_loop_get_fd (pywayland doesn't expose it)
|
||||
extras_ffi = cffi.FFI()
|
||||
extras_ffi.cdef("""
|
||||
int wl_event_loop_get_fd(void *loop);
|
||||
void *wl_display_get_event_loop(void *display);
|
||||
int wl_event_loop_dispatch(void *loop, int timeout);
|
||||
void wl_display_flush_clients(void *display);
|
||||
""")
|
||||
_pywl_libs = os.path.join(
|
||||
os.path.dirname(os.path.dirname(pywayland.__file__)),
|
||||
'pywayland.libs'
|
||||
)
|
||||
_lib = extras_ffi.dlopen([
|
||||
os.path.join(_pywl_libs, f) for f in os.listdir(_pywl_libs)
|
||||
if 'libwayland-server' in f
|
||||
][0])
|
||||
|
||||
display = Display()
|
||||
display.add_socket() # returns "wayland-N"
|
||||
display.init_shm()
|
||||
|
||||
_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr))
|
||||
_display_voidp = extras_ffi.cast('void *', _ptr_int)
|
||||
_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp)
|
||||
_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp)
|
||||
|
||||
async def main():
|
||||
loop = asyncio.get_running_loop()
|
||||
def on_ready():
|
||||
_lib.wl_event_loop_dispatch(_loop_voidp, 0)
|
||||
display.flush_clients()
|
||||
loop.add_reader(_wl_fd, on_ready)
|
||||
await asyncio.Future() # run forever
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
This pattern is used in production by **qtile's Wayland backend**
|
||||
(`libqtile/backend/wayland/core.py`). The qtile code is GPL-2.0 so
|
||||
**don't copy** — but the architectural pattern is well-documented and
|
||||
reusable.
|
||||
|
||||
#### Does pywayland need a patched libwayland-server?
|
||||
|
||||
**NO** — this is one of pywayland's biggest advantages over Greenfield.
|
||||
|
||||
- pywayland bundles and links against **vanilla upstream libwayland 1.26.0**
|
||||
(no patches needed for basic use).
|
||||
- Greenfield patches upstream libwayland-server with 9+ additions (in
|
||||
`native/wayland/src/wayland-server/changes.diff`, MIT-licensed) to add
|
||||
wire-message interception hooks: `wl_client_set_wire_message_cb`,
|
||||
`wl_client_set_registry_created_cb`, `wl_client_set_sync_done_cb`, etc.
|
||||
|
||||
**Should we apply Greenfield's `changes.diff` to our libwayland-server?**
|
||||
|
||||
- **Optional, recommended** for the fast-sync optimization (skip browser
|
||||
round-trip when no prior browser-bound request occurred).
|
||||
- The patches are MIT-licensed, so applying them to a vendored
|
||||
libwayland-server is fine.
|
||||
- Without the patches: per-resource dispatchers work fine but with slightly
|
||||
more overhead (every request goes through the full Python dispatch path).
|
||||
- With the patches: wire-message interception hook lets us route requests
|
||||
to `{native: true}` (handle in proxy) or `{browser: true}` (forward to
|
||||
browser compositor) before they hit the per-resource dispatcher.
|
||||
|
||||
#### Comparison to Greenfield's `wayland-server.ts` + `wayland-server-addon.node`
|
||||
|
||||
| Greenfield function | pywayland equivalent | Status |
|
||||
|---|---|---|
|
||||
| `createDisplay`, `destroyDisplay`, `addSocketAuto` | `Display()`, `Display.destroy()`, `Display.add_socket()` | ✅ |
|
||||
| `flush`, `getCredentials`, `initShm` | `Display.flush_clients()`, `Client.get_credentials()`, `Display.init_shm()` | ✅ |
|
||||
| `getFd`, `dispatchRequests` (libuv integration) | NOT exposed — 5-line cffi shim | ✅ (shimmable) |
|
||||
| `setWireMessageCallback`, `setWireMessageEndCallback` | NO EQUIVALENT in upstream — Greenfield patch | Use per-resource dispatchers + apply changes.diff if you want the hook |
|
||||
| `setRegistryCreatedCallback`, `setSyncDoneCallback`, `setBufferCreatedCallback` | NO EQUIVALENT in upstream — Greenfield patch | Reimplement by intercepting corresponding requests via per-resource dispatchers |
|
||||
| `setClientDestroyedCallback` | `Client.add_destroy_listener(Listener(cb))` | ✅ (different style) |
|
||||
| `destroyClient`, `sendEvents`, `createWlResource` | `Client.destroy()`, `_post_event()`, `WlFoo.resource_class(client, version, id)` | ✅ |
|
||||
| `initDrm`, `setupXWayland` | NO EQUIVALENT — use vendored Greenfield C files (MIT) | ✅ (vendored) |
|
||||
| `createMemoryMappedFile`, `makePipe` | `pywayland.utils.AnonymousFile(size)`, `os.pipe()` | ✅ |
|
||||
|
||||
## Wire format (REUSED, same as Greenfield)
|
||||
|
||||
Standard upstream Wayland, byte-for-byte. See
|
||||
`../greenfield-analysis/02-wayland-protocol.md` for the full byte layout.
|
||||
|
||||
## Object lifecycle (REUSED semantics, different API)
|
||||
|
||||
pywayland uses standard libwayland object lifecycle:
|
||||
- Client-allocated IDs (1-indexed, recycled via `wl_display.delete_id`)
|
||||
- Server-allocated IDs (from `0xff000000`)
|
||||
- `Resource.add_destroy_listener(Listener(cb))` for cleanup
|
||||
- `Client.add_destroy_listener(Listener(cb))` for client disconnect
|
||||
|
||||
## File descriptor handling (REUSED approach)
|
||||
|
||||
Same ProxyFD JSON descriptor as Greenfield:
|
||||
```
|
||||
ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }
|
||||
```
|
||||
|
||||
In Python:
|
||||
- `os.pipe()` for pipes
|
||||
- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
- Subprocess + WebSocket endpoints for the FD-over-WebSocket workaround (same 9 endpoints)
|
||||
|
||||
## Why this works
|
||||
|
||||
The protocol layer is fundamentally just:
|
||||
1. XML describing the protocol surface (upstream MIT)
|
||||
2. Code generators that produce typed bindings (MIT CLIs from Greenfield + pywayland's scanner)
|
||||
3. A runtime that marshalls/unmarshals wire bytes (MIT @gfld/common + Apache-2.0 pywayland)
|
||||
|
||||
None of these layers require AGPL code. The AGPL parts of Greenfield
|
||||
(`@gfld/compositor`, `@gfld/compositor-proxy`) are the **consumers** of
|
||||
these protocol layers — they implement the actual Wayland semantics
|
||||
(Surface state, Seat, Keyboard, etc.). We replace those consumers with
|
||||
custom clean-room implementations.
|
||||
|
||||
## File reference
|
||||
|
||||
### Reused from Greenfield (MIT)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `protocol/wayland.xml` | Upstream Wayland core protocol |
|
||||
| `protocol/xdg-shell.xml` | Stable xdg-shell protocol |
|
||||
| `protocol/wayland-drm.xml` | Legacy wl_drm for DMA-BUF |
|
||||
| `protocol/web-bitmapbuf-unstable-v1.xml` | Greenfield's custom extension for browser JS apps |
|
||||
| `libs/common/src/Connection.ts` | Wire format codec (REUSED) |
|
||||
| `libs/common/src/index.ts` | Public exports |
|
||||
| `libs/compositor-protocol/src/westfield-runtime-server.ts` | Server-side runtime (REUSED) |
|
||||
| `libs/compositor-protocol/src/protocol/` | Generated server bindings (REUSED) |
|
||||
| `libs/client-protocol/src/westfield-runtime-client.ts` | Client-side runtime (REUSED) |
|
||||
| `libs/client-protocol/src/protocol/` | Generated client bindings (REUSED) |
|
||||
| `libs/client-generator/{ProtocolParser,ProtocolArguments,index}.mjs` | Browser client bindings generator (REUSED) |
|
||||
| `libs/compositor-generator/{ProtocolParser,ProtocolArguments,index}.mjs` | Browser server bindings generator (REUSED) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/wayland-server/changes.diff` | Patches to libwayland-server (MIT, optional but recommended) |
|
||||
|
||||
### Python equivalents (Apache-2.0 / MIT)
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| pywayland | Python bindings to libwayland-server |
|
||||
| `pywayland.server.Display` | wl_display wrapper |
|
||||
| `pywayland.server.EventLoop` | wl_event_loop wrapper |
|
||||
| `pywayland.server.Client` | wl_client wrapper |
|
||||
| `pywayland.server.Resource` | wl_resource base class |
|
||||
| `pywayland.server.Global` | wl_global base class |
|
||||
| `pywayland.server.Listener`, `Signal` | Observer pattern |
|
||||
| `pywayland.protocol.*` | Auto-generated protocol bindings (60+ interfaces) |
|
||||
| `pywayland.utils.AnonymousFile` | memfd_create wrapper (replaces Greenfield's createMemoryMappedFile) |
|
||||
@@ -0,0 +1,290 @@
|
||||
# 03 — HTML5 Rendering Pipeline (Custom WebGL2 + Workers + OffscreenCanvas)
|
||||
|
||||
This chapter describes the **custom browser-side rendering pipeline** that
|
||||
replaces Greenfield's AGPL `@gfld/compositor`. It's the largest single
|
||||
component of the MIT approach (~3,000 LOC TypeScript, ~5 dev-weeks).
|
||||
|
||||
## Critical: clean-room requirement
|
||||
|
||||
**You CANNOT read Greenfield's `@gfld/compositor` source and reimplement it.**
|
||||
Even "inspired-by" code is risky under the *Computer Associates v. Altai*
|
||||
Abstraction-Filtration-Comparison test. AGPL §13 network-use clause would
|
||||
force you to open-source your entire Odoo fork if a court finds your
|
||||
compositor is an AGPL derivative.
|
||||
|
||||
**Right approach**: True clean-room.
|
||||
- Person A (spec author, may read AGPL source) writes a behavioral spec
|
||||
(inputs/outputs/edge cases, NO code).
|
||||
- Person B (implementer, never opens `packages/compositor/src/`) implements
|
||||
from spec + MIT references only:
|
||||
- Wayland protocol spec (https://wayland.freedesktop.org/docs/html/)
|
||||
- wlroots C source (MIT, https://gitlab.freedesktop.org/wlroots/wlroots)
|
||||
- Weston C source (MIT, https://gitlab.freedesktop.org/wayland/weston)
|
||||
- Smithay Rust source (MIT, https://github.com/Smithay/smithay)
|
||||
- WebGL2/WebCodecs/KCP MDN docs
|
||||
|
||||
Maintain `SPEC.md` and `CLEAN_ROOM_LOG.md` as legal-defensibility artifacts.
|
||||
|
||||
## Modernization wins over Greenfield
|
||||
|
||||
Greenfield's browser compositor uses:
|
||||
- WebGL1 (no OffscreenCanvas, no WebGL2)
|
||||
- Main-thread WebGL (jank on heavy frames)
|
||||
- WebCodecs with hardware decode explicitly disabled (`// FIXME hardware decoding frame offset is wrong`)
|
||||
- FFmpeg Wasm fallback (GPL — we can't use it)
|
||||
|
||||
Our MIT approach modernizes to:
|
||||
- **WebGL2** — uniform buffer objects, transform feedback, simplified YUV→RGB shaders
|
||||
- **OffscreenCanvas + Web Workers** — render in a worker, never block main thread
|
||||
- **WebCodecs with hardware decode enabled** — fix Greenfield's "frame offset is wrong" bug
|
||||
- **OpenH264 BSD Wasm fallback** — replaces Greenfield's GPL FFmpeg Wasm
|
||||
- **Adaptive bitrate** — consume `EncoderFeedback` data Greenfield produces but doesn't use
|
||||
- **Real touch input** — Greenfield has a 46-line stub that never fires
|
||||
|
||||
## Proposed file structure
|
||||
|
||||
```
|
||||
packages/compositor/ # PROPRIETARY, clean-room TS
|
||||
├── SPEC.md # clean-room behavioral spec
|
||||
├── CLEAN_ROOM_LOG.md # who read AGPL, who didn't, when
|
||||
├── src/
|
||||
│ ├── index.ts # public API: createCompositorSession()
|
||||
│ ├── core/
|
||||
│ │ ├── Session.ts # top-level session
|
||||
│ │ ├── Compositor.ts # wl_compositor global
|
||||
│ │ └── Globals.ts # Wayland globals registry
|
||||
│ ├── surface/ # ~1,200 LOC, 3 weeks
|
||||
│ │ ├── Surface.ts # wl_surface, double-buffered state
|
||||
│ │ ├── SurfaceState.ts # state struct
|
||||
│ │ ├── SurfaceRole.ts # role abstraction
|
||||
│ │ ├── SurfaceTree.ts # parent/child tree
|
||||
│ │ ├── Subsurface.ts # wl_subsurface
|
||||
│ │ ├── Subcompositor.ts # wl_subcompositor global
|
||||
│ │ ├── Region.ts # wl_region (pixman-WASM)
|
||||
│ │ └── FrameCallback.ts # wl_callback
|
||||
│ ├── render/ # ~1,800 LOC, 3 weeks
|
||||
│ │ ├── Renderer.ts # 4-phase render pipeline
|
||||
│ │ ├── Scene.ts # one per output (canvas)
|
||||
│ │ ├── View.ts # positioned surface for compositing
|
||||
│ │ ├── Output.ts # wl_output, OffscreenCanvas binding
|
||||
│ │ ├── RenderState.ts # per-view-per-scene texture + transform
|
||||
│ │ ├── Texture.ts # WebGL2 texture wrapper
|
||||
│ │ ├── ShaderProgram.ts # WebGL2 program wrapper
|
||||
│ │ ├── Loop.ts # requestAnimationFrame integration
|
||||
│ │ └── shaders/
|
||||
│ │ ├── YUVA2RGBA.glsl # BT.601 YUVA → RGBA
|
||||
│ │ ├── YUV2RGB.glsl # YUV → RGB (no alpha)
|
||||
│ │ ├── RGBandA2RGBA.glsl # RGB+Alpha → RGBA
|
||||
│ │ └── SceneShader.glsl # final pass: textured quads
|
||||
│ ├── buffer/ # ~600 LOC, 1.5 weeks
|
||||
│ │ ├── BufferContents.ts # buffer abstraction (mimeType)
|
||||
│ │ ├── BufferImplementation.ts # interface
|
||||
│ │ ├── Shm.ts # wl_shm, BMP-header hack
|
||||
│ │ ├── ImageBitmapBuffer.ts # web-bitmapbuf fast path
|
||||
│ │ └── EncodedFrameBuffer.ts # H.264/PNG from proxy
|
||||
│ ├── transport/ # ~3,500 LOC, 4 weeks
|
||||
│ │ ├── Channel.ts # ARQChannel + SimpleChannel
|
||||
│ │ ├── kcp.ts # ported fresh from MIT C ref
|
||||
│ │ ├── ReconnectingWebSocket.ts # vendored from pladaria (MIT)
|
||||
│ │ ├── BufferStream.ts # serial → frame promise resolution
|
||||
│ │ ├── StreamingBuffer.ts # per-surface encoded-frame buffer
|
||||
│ │ ├── EncodedFrame.ts # wire format parser
|
||||
│ │ ├── EncoderFeedback.ts # 4-byte decode feedback, 1 Hz
|
||||
│ │ ├── RemoteInputOutput.ts # FD-over-WebSocket for native apps
|
||||
│ │ ├── RemoteAppLauncher.ts # channel setup per app
|
||||
│ │ ├── RemoteOutOfBandChannel.ts # OOB opcode dispatch
|
||||
│ │ └── decoders/
|
||||
│ │ ├── FrameDecoder.ts # interface
|
||||
│ │ ├── WebCodecsFrameDecoder.ts # primary: VideoDecoder (HW-accelerated)
|
||||
│ │ └── OpenH264FrameDecoder.ts # fallback: OpenH264 BSD Wasm
|
||||
│ ├── input/ # ~2,500 LOC, 3 weeks
|
||||
│ │ ├── Seat.ts # wl_seat global
|
||||
│ │ ├── Keyboard.ts # wl_keyboard, keymap, modifiers
|
||||
│ │ ├── Pointer.ts # wl_pointer, motion, buttons, axis
|
||||
│ │ ├── Touch.ts # wl_touch (REAL implementation, not stub)
|
||||
│ │ ├── Xkb.ts # xkbcommon WASM wrapper
|
||||
│ │ ├── InputQueue.ts # event coalescing + RAF dispatch
|
||||
│ │ ├── InputOutput.ts # FD abstraction
|
||||
│ │ └── events.ts # KeyEvent, ButtonEvent, AxisEvent
|
||||
│ ├── roles/ # ~1,800 LOC, 3 weeks (HIGH RISK)
|
||||
│ │ ├── XdgWmBase.ts # xdg_wm_base global, ping
|
||||
│ │ ├── XdgSurface.ts # xdg_surface, configure/ack
|
||||
│ │ ├── XdgToplevel.ts # xdg_toplevel, window state
|
||||
│ │ ├── XdgPopup.ts # xdg_popup, transient windows, grabs
|
||||
│ │ ├── XdgPositioner.ts # popup positioning math
|
||||
│ │ ├── Shell.ts # legacy wl_shell (optional)
|
||||
│ │ └── ShellSurface.ts # legacy wl_shell_surface
|
||||
│ ├── data/ # ~600 LOC, 1.5 weeks
|
||||
│ │ ├── DataDeviceManager.ts # wl_data_device_manager
|
||||
│ │ ├── DataSource.ts # wl_data_source
|
||||
│ │ ├── DataOffer.ts # wl_data_offer
|
||||
│ │ ├── WaylandDataSource.ts # native source wrapper
|
||||
│ │ ├── PrimarySelectionDeviceManager.ts # NEW: zwp_primary_selection_v1
|
||||
│ │ └── PrimarySelectionSource.ts # NEW: middle-click paste
|
||||
│ ├── desktop/ # ~800 LOC, 1.5 weeks
|
||||
│ │ ├── Desktop.ts # Floating vs AlwaysFullScreen
|
||||
│ │ ├── DesktopSurface.ts # abstraction
|
||||
│ │ ├── FloatingDesktopSurface.ts # default mode
|
||||
│ │ └── FullscreenDesktopSurface.ts # kiosk mode
|
||||
│ ├── math/ # ~500 LOC, 0.5 weeks
|
||||
│ │ ├── Mat4.ts # 4×4 matrix
|
||||
│ │ ├── Vec4.ts # 4-vector
|
||||
│ │ ├── Point.ts # 2D point
|
||||
│ │ ├── Size.ts # 2D size
|
||||
│ │ ├── Rect.ts # 2D rectangle
|
||||
│ │ └── Transformations.ts # affine transforms
|
||||
│ ├── userapi/ # ~120 LOC, 0.5 weeks
|
||||
│ │ └── UserShellApi.ts # OWL shell integration seam
|
||||
│ ├── browser/ # ~750 LOC, 2 weeks
|
||||
│ │ ├── input.ts # DOM keyboard events
|
||||
│ │ ├── pointer.ts # DOM pointer events
|
||||
│ │ ├── dnd.ts # DnD bridge
|
||||
│ │ ├── selection.ts # clipboard bridge (native → DOM)
|
||||
│ │ ├── text-selection.ts # Firefox fallback (text only)
|
||||
│ │ ├── attributes.ts # canvas attributes
|
||||
│ │ └── capabilities.ts # browser feature detection
|
||||
│ └── (optional, Phase 6)
|
||||
│ ├── webapp/ # ~500 LOC, 1 week (WASM web apps)
|
||||
│ │ ├── WebAppLauncher.ts
|
||||
│ │ ├── WebConnectionHandler.ts
|
||||
│ │ ├── WebInputOutput.ts
|
||||
│ │ └── WebBuffersFactory.ts
|
||||
│ └── xwayland/ # ~5,000 LOC, 8 weeks (X11 apps)
|
||||
│ ├── XWindowManager.ts
|
||||
│ ├── XWindow.ts
|
||||
│ ├── XWindowFrame.ts
|
||||
│ └── ... (uses @gfld/xtsb MIT)
|
||||
└── test/
|
||||
├── unit/ # ~1,000 LOC, Jest unit tests
|
||||
├── protocol/ # ~2,000 LOC, Weston test clients ported to TS
|
||||
└── e2e/ # ~500 LOC, Playwright E2E with real Linux apps
|
||||
```
|
||||
|
||||
## Modern rendering pipeline (WebGL2 + OffscreenCanvas + Workers)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────┐
|
||||
│ MAIN THREAD (Odoo OWL shell + DOM events) │
|
||||
│ │
|
||||
│ <canvas> element owned by OWL component │
|
||||
│ │ │
|
||||
│ │ canvas.transferControlToOffscreen() → OffscreenCanvas │
|
||||
│ │ │
|
||||
│ │ postMessage({canvas: offscreenCanvas}, [offscreenCanvas]) │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Render Worker (WebGL2 + OffscreenCanvas) │ │
|
||||
│ │ │ │
|
||||
│ │ Receives: │ │
|
||||
│ │ - OffscreenCanvas (transferred) │ │
|
||||
│ │ - EncodedVideoChunk / ImageBitmap (transferred from decode) │ │
|
||||
│ │ - Surface state updates (postMessage) │ │
|
||||
│ │ - Frame callback requests (postMessage) │ │
|
||||
│ │ │ │
|
||||
│ │ Runs the 4-phase render pipeline: │ │
|
||||
│ │ PHASE 1 — pixel upload (per view per scene) │ │
|
||||
│ │ PHASE 2 — requestAnimationFrame │ │
|
||||
│ │ PHASE 3 — draw (per scene: viewport + clear + drawArrays) │ │
|
||||
│ │ PHASE 4 — fire frame callbacks │ │
|
||||
│ │ │ │
|
||||
│ │ Sends: │ │
|
||||
│ │ - Frame callback timestamps (postMessage to main thread) │ │
|
||||
│ │ │ │
|
||||
│ └────────────────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Decode Worker (WebCodecs VideoDecoder, hardware-accelerated) │ │
|
||||
│ │ │ │
|
||||
│ │ Receives: │ │
|
||||
│ │ - EncodedVideoChunk (transferred from transport) │ │
|
||||
│ │ │ │
|
||||
│ │ Decodes: │ │
|
||||
│ │ - VideoDecoder.decode(chunk) │ │
|
||||
│ │ - VideoFrame output (zero-copy GPU texture source) │ │
|
||||
│ │ │ │
|
||||
│ │ Sends: │ │
|
||||
│ │ - VideoFrame (transferred to render worker) │ │
|
||||
│ │ - Decode duration telemetry (for adaptive bitrate) │ │
|
||||
│ │ │ │
|
||||
│ └────────────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Why WebGL2 over WebGL1
|
||||
|
||||
- **Uniform Buffer Objects** — share state between shaders efficiently
|
||||
- **Transform feedback** — could be used for vertex skinning (not needed for compositor)
|
||||
- **Simplified YUV→RGB shaders** — `texelFetch` for direct texel access
|
||||
- **3D textures** — could simplify YUV planar upload (Y, U, V as 3 slices)
|
||||
- **Better browser support** — 97%+ global browser support per caniuse
|
||||
|
||||
## Why OffscreenCanvas + Web Workers
|
||||
|
||||
- **Never block main thread** — heavy decode + render happens off-thread
|
||||
- **Smooth UI** — Odoo shell stays responsive during video playback
|
||||
- **Multi-output scaling** — one worker per output for parallel rendering
|
||||
- **Recovery** — if render worker crashes, restart without losing Odoo session
|
||||
|
||||
## WebCodecs vs FFmpeg Wasm
|
||||
|
||||
| Aspect | WebCodecs (primary) | OpenH264 Wasm (fallback) |
|
||||
|---|---|---|
|
||||
| License | W3C standard (MIT-equivalent) | BSD-2-Clause |
|
||||
| Hardware decode | Yes (Intel/AMD/NVIDIA) | No (CPU only) |
|
||||
| Latency | ~5-10ms per frame | ~30-50ms per frame |
|
||||
| CPU usage | Low (offloaded to GPU) | High (full decode on CPU) |
|
||||
| Browser support | Chrome 94+, Edge 94+, Safari 16+, Firefox 130+ | All browsers with Wasm |
|
||||
| Codecs | H.264, VP9, AV1, H.265 (where supported) | H.264 only |
|
||||
|
||||
**Recommended approach**: Probe `VideoDecoder.isConfigSupported()` at session
|
||||
startup. If supported, use WebCodecs. Otherwise, fall back to OpenH264 Wasm
|
||||
(loaded lazily, only when needed).
|
||||
|
||||
## Wire format for encoded frames (REUSED from Greenfield)
|
||||
|
||||
Same wire format as Greenfield (see `../greenfield-analysis/09-hardware-accel.md`
|
||||
for byte layout). The browser side parses `{contentSerial, mimeType, size,
|
||||
encodedSize, opaque, alpha?}` and dispatches to the right decoder.
|
||||
|
||||
## Engineering observations
|
||||
|
||||
1. **Vestigial damage tracking in Greenfield** — `damageRects` are accumulated
|
||||
but not consumed by the renderer. Our MIT approach should actually use
|
||||
them for scissoring (per-rect partial repaints).
|
||||
2. **Multi-output texture cost** — each view has one texture per scene. Our
|
||||
approach should share textures across scenes where possible.
|
||||
3. **SHM xrgb8888 BMP-header hack** — cleverly offloads pixel conversion
|
||||
to the browser's BMP decoder. Worth re-implementing.
|
||||
4. **Encoder feedback telemetry** — Greenfield produces it but doesn't use
|
||||
it. Our approach should consume it for adaptive bitrate.
|
||||
5. **Cursor bypass** — cursor is rendered as CSS `cursor: url(dataURL)`,
|
||||
not via WebGL. Same approach in our MIT version.
|
||||
6. **Frame-callback coalescing** — multiple `wl_surface.frame` requests
|
||||
within one render pass fire on the same RAF tick. Same in our version.
|
||||
7. **Subsurface sync cascade** — three-layer mechanism (cachedState,
|
||||
commitCache, effectiveSync). Hard to get right; clean-room from spec.
|
||||
|
||||
## Effort estimate
|
||||
|
||||
| Component | LOC | Weeks (1 dev) | Risk |
|
||||
|---|---|---|---|
|
||||
| Surface state stack | 1,200 | 3 | Med |
|
||||
| Rendering pipeline (WebGL2 + OffscreenCanvas) | 1,800 | 3 | Med |
|
||||
| Buffers (SHM + BMP hack) | 600 | 1.5 | Low |
|
||||
| Transport (incl. fresh KCP port from MIT C ref) | 3,500 | 4 | **High** |
|
||||
| Input (seat/kbd/ptr/touch/xkb) | 2,500 | 3 | Med |
|
||||
| Roles (xdg-shell state machine) | 1,800 | 3 | **High** |
|
||||
| Data device / clipboard / DnD + primary selection | 600 | 1.5 | Low |
|
||||
| Desktop policy | 800 | 1.5 | Low |
|
||||
| Math | 500 | 0.5 | Low |
|
||||
| User shell API | 120 | 0.5 | Low |
|
||||
| Browser glue | 750 | 2 | Med |
|
||||
| H.264 decoder (WebCodecs + OpenH264 BSD) | 500 | 1 | Low |
|
||||
| Core (Session/Globals) | 200 | 0.5 | Low |
|
||||
| **MVP TOTAL** | **~13,400** | **~25 weeks** | |
|
||||
| XWayland (optional Phase 6) | 5,000 | 8 | High |
|
||||
| WASM web app (optional) | 500 | 1 | Low |
|
||||
| Test suite + hardening | — | 3 | — |
|
||||
| **GRAND TOTAL** | **~18,900** | **~30-36 weeks** | |
|
||||
|
||||
With 2 senior devs in parallel: **~15-18 weeks for MVP**.
|
||||
+332
@@ -0,0 +1,332 @@
|
||||
# 04 — Transport Layer (Python aiohttp + KCP, browser side clean-room)
|
||||
|
||||
This chapter covers the transport layer for the MIT approach: how Wayland
|
||||
wire traffic flows between the Python proxy and the browser compositor.
|
||||
|
||||
## Recommended Python transport stack
|
||||
|
||||
| Layer | Choice | Why |
|
||||
|---|---|---|
|
||||
| **WebSocket framework** | **aiohttp.web** (v3.10+) | Best binary perf (300-600 Mbps on CPython, 2× on PyPy), native backpressure via `await ws.drain()`, single dependency, asyncio-native — eliminates the ASGI indirection tax of Starlette/uvicorn and the framework overhead of FastAPI |
|
||||
| **KCP library** | **`kcp` PyPI package** (RealistikDash/kcp.py, v0.1.6, MIT, Cython wrapper over upstream `ikcp.c`) | API (`enqueue`/`receive`/`get_all_received`/`update`/`outbound_handler`) maps cleanly to Greenfield's `Kcp` class. ~1500 LoC port of Greenfield's TS KCP would buy nothing — the C-backed package is faster and battle-tested |
|
||||
| **libwayland binding** | **pywayland 0.4.19** (Apache-2.0, CFFI) + **cffi escape hatch for `wl_event_loop_get_fd`** | pywayland's `EventLoop` wrapper omits `wl_event_loop_get_fd()`. Fix: call `pywayland.lib.wl_event_loop_get_fd(loop._ptr)` directly via cffi, then `asyncio.get_running_loop().add_reader(wl_fd, dispatch_wl)`. Replaces Greenfield's entire `proxy-poll-addon.node` C addon with ~10 lines of Python |
|
||||
| **Multi-process model** | **`multiprocessing.Process` (fork start method) per Odoo session** | Matches Greenfield's per-`compositorSessionId` forking. FD passing of WS upgrade socket via `multiprocessing.reduction.send_handle`/`recv_handle` (uses SCM_RIGHTS on Linux). Preserves the SIGHUP/SIGKILL escalation semantics |
|
||||
| **Encoder threading** | **`threading.Thread` + GMainLoop + `loop.call_soon_threadsafe`** | Mirrors Greenfield's `napi_threadsafe_function` pattern. GIL released during C encoding calls (GStreamer). Reuse Greenfield's MIT C files for dmabuf/egl via cffi — preserves all EGL/GBM/GStreamer pipelines |
|
||||
| **Binary message format** | **Keep Greenfield's verbatim** | Browser-side TS unchanged; no need for a version field until v2. KCP gives message boundaries |
|
||||
| **ReconnectingWebSocket** | **Reuse Greenfield's vendored copy** (Pedro Ladaria, MIT, 514 lines TS) | Browser-side; copy with attribution into Odoo OWL shell assets |
|
||||
|
||||
## Channel types (same as Greenfield)
|
||||
|
||||
| Type | Class | KCP? | Purpose |
|
||||
|------|-------|------|---------|
|
||||
| PROTOCOL | ARQChannel | Yes | Wayland wire protocol, per `wl_client` |
|
||||
| FRAME | ARQChannel | Yes | H.264/PNG encoded surface content, per `wl_surface` |
|
||||
| XWM | ARQChannel | Yes | XWayland window manager protocol |
|
||||
| FEEDBACK | SimpleChannel | No | 4-byte telemetry (refreshInterval + decode avg), 1 Hz |
|
||||
|
||||
## WebSocket endpoints (9 paths, same as Greenfield)
|
||||
|
||||
```
|
||||
/signal # JSON control plane (per-app)
|
||||
/channel # binary data channels (multiplexed)
|
||||
/mkfifo # create a POSIX pipe, return 2 ProxyFDs
|
||||
/mkstemp-mmap # create a memfd-backed shm, return 1 ProxyFD
|
||||
/read-fd # read bytes from a ProxyFD (one-shot)
|
||||
/read-fd-as-stream # read bytes from a ProxyFD (streaming)
|
||||
/write-fd-as-stream # write bytes to a ProxyFD (streaming)
|
||||
/close-fd # close a ProxyFD
|
||||
/request-keyframe # ask the encoder for a keyframe
|
||||
```
|
||||
|
||||
All endpoints require JWT auth (HS256, validated by `pyjwt`).
|
||||
|
||||
## asyncio + libwayland integration (the riskiest piece)
|
||||
|
||||
**Verified working** in the sandbox:
|
||||
|
||||
```python
|
||||
import asyncio, cffi
|
||||
from pywayland.server import Display
|
||||
|
||||
extras_ffi = cffi.FFI()
|
||||
extras_ffi.cdef("""
|
||||
int wl_event_loop_get_fd(void *loop);
|
||||
void *wl_display_get_event_loop(void *display);
|
||||
int wl_event_loop_dispatch(void *loop, int timeout);
|
||||
void wl_display_flush_clients(void *display);
|
||||
""")
|
||||
_lib = extras_ffi.dlopen("libwayland-server.so.0") # or pywayland's bundled
|
||||
|
||||
display = Display()
|
||||
display.add_socket()
|
||||
display.init_shm()
|
||||
|
||||
_ptr_int = int(pywayland.ffi.cast('uintptr_t', display._ptr))
|
||||
_display_voidp = extras_ffi.cast('void *', _ptr_int)
|
||||
_loop_voidp = _lib.wl_display_get_event_loop(_display_voidp)
|
||||
_wl_fd = _lib.wl_event_loop_get_fd(_loop_voidp)
|
||||
|
||||
async def main():
|
||||
loop = asyncio.get_running_loop()
|
||||
def on_ready():
|
||||
_lib.wl_event_loop_dispatch(_loop_voidp, 0)
|
||||
display.flush_clients()
|
||||
loop.add_reader(_wl_fd, on_ready)
|
||||
await asyncio.Future() # run forever
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
This pattern is used in production by **qtile's Wayland backend**
|
||||
(`libqtile/backend/wayland/core.py`). qtile is GPL-2.0 so don't copy code,
|
||||
but the architectural pattern is documented and reusable.
|
||||
|
||||
## Process model — three layers (same as Greenfield)
|
||||
|
||||
### Layer 1: Main process (HTTP + WS hub)
|
||||
|
||||
Single Python process listens on `:8081`. Per Odoo session, it forks
|
||||
one `SessionProcess` child — one child per `compositor_session_id`, **not**
|
||||
per app. Multiple apps in the same browser tab share one child.
|
||||
|
||||
WebSocket upgrades are handed off to the matching child via
|
||||
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
|
||||
|
||||
### Layer 2: Session process
|
||||
|
||||
The forked child. Receives IPC messages:
|
||||
|
||||
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
|
||||
- `launch_app` — spawns the actual app binary via `subprocess.Popen` + bubblewrap
|
||||
- `ws_upgrade` — hands WS to the dispatcher
|
||||
|
||||
On `session.close()` the process exits.
|
||||
|
||||
### Layer 3: Per-client session
|
||||
|
||||
For each new `wl_client` (one per Wayland connection), creates a
|
||||
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
|
||||
JSON. Routes requests via per-resource dispatchers.
|
||||
|
||||
## FD virtualization (same approach as Greenfield)
|
||||
|
||||
`ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }`
|
||||
|
||||
In Python:
|
||||
- `os.pipe()` for pipes
|
||||
- `os.memfd_create(name, flags=0)` for shared memory (Python 3.8+ builtin,
|
||||
replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
- `asyncio.StreamReader` / `StreamWriter` for streaming reads/writes over
|
||||
separate WebSocket endpoints
|
||||
|
||||
## Reconnection & lifecycle (same as Greenfield)
|
||||
|
||||
### Browser side
|
||||
|
||||
Vendored `ReconnectingWebSocket` (MIT):
|
||||
- `minDelay=1-5s`, `factor=1.3`, `maxDelay=10s`
|
||||
- `minUptime=5s`, `connTimeout=4s`, `retries=∞`
|
||||
- Close codes 4000-4999 tear down KCP; lower codes auto-reconnect
|
||||
|
||||
### Python side
|
||||
|
||||
No auto-reconnect (it's the server). Channel object lives on, waiting for
|
||||
new WS upgrade with matching `id` + `key`. Outbound messages queue and
|
||||
flush on reconnect.
|
||||
|
||||
### App kill timers
|
||||
|
||||
When browser disconnects from `/signal` WebSocket:
|
||||
- 10-minute SIGHUP grace period
|
||||
- After 10 min: SIGTERM, then SIGKILL after another 10s
|
||||
- Close code 1001 → immediate SIGHUP (intentional tab close)
|
||||
- Close code 4001 → immediate SIGTERM (force quit)
|
||||
|
||||
## JWT auth (NEW vs Greenfield)
|
||||
|
||||
Every WebSocket connection must include a valid JWT (HS256, 15-min expiry):
|
||||
|
||||
```
|
||||
Authorization: Bearer <jwt>
|
||||
```
|
||||
|
||||
Or as a query param: `?token=<jwt>`
|
||||
|
||||
JWT payload:
|
||||
```json
|
||||
{
|
||||
"iss": "odoo.example.com",
|
||||
"aud": "compositor-proxy",
|
||||
"sub": 42, // Odoo user ID
|
||||
"db": "production", // Odoo database name
|
||||
"company_ids": [1, 3],
|
||||
"scope": "compositor",
|
||||
"compositor_session_id": "uuid-here",
|
||||
"iat": 1736000000,
|
||||
"exp": 1736000900
|
||||
}
|
||||
```
|
||||
|
||||
Validated by the proxy using `compositor.jwt_secret` from Odoo's
|
||||
`ir.config_parameter` (shared via env var at proxy startup).
|
||||
|
||||
## ASCII diagram
|
||||
|
||||
```
|
||||
PYTHON HOST (Linux) BROWSER (Odoo OWL shell)
|
||||
========================== ========================
|
||||
|
||||
┌─────────────────────────┐ ┌──────────────────────────────┐
|
||||
│ Native app (gtk4-demo) │ │ Odoo web client │
|
||||
│ (sandboxed in bwrap) │ │ (Odoo 20 OWL shell addon) │
|
||||
└───────────┬─────────────┘ └──────────────┬───────────────┘
|
||||
│ WAYLAND_DISPLAY=wayland-0 │
|
||||
│ Unix socket + SCM_RIGHTS FD passing │ DOM events → input
|
||||
▼ │ WebSocket API
|
||||
┌──────────────────────────────────────────┐ │
|
||||
│ SessionProcess (fork'd Python child) │ │
|
||||
│ ┌────────────────────────────────────┐ │ │
|
||||
│ │ pywayland (Apache-2.0) │ │ │
|
||||
│ │ + 5-line cffi shim │ │ │
|
||||
│ │ - wl_display + wl_client │ │ │
|
||||
│ │ - asyncio.add_reader for wl_fd │ │ │
|
||||
│ └─────────────┬──────────────────────┘ │ │
|
||||
│ │ wire msg + FDs │ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ Per-resource dispatchers │ │ │
|
||||
│ │ (auto-generated by pywayland) │ │ │
|
||||
│ │ - route to {native, browser} │ │ │
|
||||
│ │ - serialize FDs as ProxyFD JSON │ │ │
|
||||
│ │ - fast-sync optimization │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ │ outbound buffer │ │
|
||||
│ ▼ │ │
|
||||
│ ┌─────────────────────────────────────┐ │ │
|
||||
│ │ ARQChannel (KCP-wrapped) │ │ │
|
||||
│ │ - kcp PyPI package (MIT) │ │ │
|
||||
│ │ - MTU=64000, snd_wnd=128, rcv=1024 │ │ │
|
||||
│ │ - kcp.update() every 100ms │ │ │
|
||||
│ │ - PROTOCOL (per wl_client) │ │ │
|
||||
│ │ - FRAME (per wl_surface) │ │ │
|
||||
│ │ - XWM (per XWayland) │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────┴───────────────────────┐ │ │
|
||||
│ │ SimpleChannel (FEEDBACK, no KCP) │ │ │
|
||||
│ │ 4 bytes: refreshIntervalMs + │ │ │
|
||||
│ │ avgDecodeDurationMs │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ NativeAppContext │ │ │
|
||||
│ │ - signaling WebSocket (JSON) │ │ │
|
||||
│ │ - kill timers: SIGHUP@10min, │ │ │
|
||||
│ │ SIGKILL@+10s │ │ │
|
||||
│ │ - bubblewrap + seccomp sandbox │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ SessionController │ │ │
|
||||
│ │ - aiohttp.web.Application │ │ │
|
||||
│ │ - 9 endpoints │ │ │
|
||||
│ │ - JWT validation (pyjwt) │ │ │
|
||||
│ │ - perMessageDeflate: false │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
│ ┌─────────────▼───────────────────────┐ │ │
|
||||
│ │ Encoder thread (GStreamer + GLib │ │ │
|
||||
│ │ MainLoop in separate thread) │ │ │
|
||||
│ │ - vaapih264enc / nvh264enc / │ │ │
|
||||
│ │ openh264enc (NOT x264enc — GPL) │ │ │
|
||||
│ │ - asyncio.run_coroutine_threadsafe │ │ │
|
||||
│ │ for cross-thread marshalling │ │ │
|
||||
│ └─────────────┬───────────────────────┘ │ │
|
||||
└────────────────┼─────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ main.py (HTTP+WS hub) │ │
|
||||
│ │ - socket.pause() + IPC │ │
|
||||
│ │ send to session proc │ │
|
||||
│ │ - multiprocessing.fork │ │
|
||||
│ │ - JWT issuance delegated │ │
|
||||
│ │ to Odoo backend │ │
|
||||
│ └────────────┬─────────────┘ │
|
||||
│ │ │
|
||||
│ ◄──── 1 WebSocket per channel ────► │
|
||||
│ ◄── 1 signaling WS per app ──► │
|
||||
│ ◄── ad-hoc WebFD streams ─► │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ ReconnectingWebSocket (vendored MIT) │ │
|
||||
│ │ min 1-5s, factor 1.3, max 10s │ │
|
||||
│ │ minUptime 5s, connTimeout 4s │ │
|
||||
│ │ + Authorization: Bearer <jwt> │ │
|
||||
│ └────────────┬───────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ ARQChannel / SimpleChannel (clean-room TS) │ │
|
||||
│ │ snd_wnd=1024, rcv_wnd=128 │ │
|
||||
│ │ kcp.input → peekSize/recv → onMessage │ │
|
||||
│ └────────────┬───────────────────────────────┘ │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Custom Browser Compositor (clean-room TS) │ │
|
||||
│ │ - WebCodecs VideoDecoder (HW-accelerated)│ │
|
||||
│ │ - WebGL2 + OffscreenCanvas in Worker │ │
|
||||
│ │ - Surface/View/Output/Seat/Keyboard/... │ │
|
||||
│ └────────────┬───────────────────────────────┘ │
|
||||
│ │ input events flow back via the │
|
||||
│ │ PROTOCOL channel as ordinary │
|
||||
│ │ Wayland requests (same framing) │
|
||||
└──────────────────┘ │
|
||||
Browser DOM ◄─┘
|
||||
```
|
||||
|
||||
## Effort estimate (Python transport)
|
||||
|
||||
| Component | LoC | Dev-days | Risk |
|
||||
|---|---|---|---|
|
||||
| aiohttp WS server (9 endpoints + HTTP launcher) | 600 | 3 | Low |
|
||||
| `Channel`/`ARQChannel`/`SimpleChannel` port | 250 | 1.5 | Low |
|
||||
| `kcp.py` integration adapter | 100 | 0.5 | Low |
|
||||
| `NativeAppContext` port (signaling, escalation) | 350 | 2 | Low |
|
||||
| `ProxyInputOutput` port (FD workaround) | 300 | 2 | Medium |
|
||||
| libwayland event-loop integration | 80 | 1 | **Medium-High** (under-documented) |
|
||||
| Multiprocessing scaffolding + FD passing | 200 | 1.5 | Medium |
|
||||
| GStreamer encoder thread + marshalling | 250 | 2 | Medium |
|
||||
| Reconnection & lifecycle | 150 | 1 | Low |
|
||||
| Tests (pytest + pytest-asyncio) | 600 | 3 | Low |
|
||||
| **Total** | **2880** | **~17** | |
|
||||
|
||||
## Top risks & mitigations
|
||||
|
||||
1. **Highest risk**: libwayland ↔ asyncio event-loop integration (under-documented).
|
||||
**Mitigation**: Spend 1 day on a standalone prototype (the 10-line script
|
||||
above) before committing.
|
||||
2. **Medium risk**: FD passing via `multiprocessing.reduction`. Fallback:
|
||||
single-process with asyncio concurrency.
|
||||
3. **Medium risk**: GStreamer threading. Mitigation: reuse Greenfield's
|
||||
MIT C files via cffi — only the marshalling needs Python.
|
||||
|
||||
## Dependency manifest
|
||||
|
||||
```
|
||||
# requirements.txt
|
||||
aiohttp>=3.10,<4 # WebSocket server + HTTP
|
||||
kcp>=0.1.6 # Cython KCP bindings (RealistikDash)
|
||||
pywayland>=0.4.19 # libwayland CFFI wrapper (Apache-2.0)
|
||||
cffi>=1.16 # for wl_event_loop_get_fd escape hatch
|
||||
pygobject>=3.48 # GStreamer Python bindings
|
||||
pyjwt>=2.8 # JWT validation
|
||||
```
|
||||
|
||||
System packages (Debian):
|
||||
```
|
||||
libwayland-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
|
||||
libgstreamer-plugins-bad1.0-dev libgbm-dev libdrm-dev libegl-dev
|
||||
gstreamer1.0-vaapi # Intel/AMD hardware encoding
|
||||
libnvidia-encode1 # NVIDIA hardware encoding (if applicable)
|
||||
bubblewrap libseccomp-dev
|
||||
python3-dev build-essential pkg-config
|
||||
```
|
||||
|
||||
**DO NOT INSTALL**: `gstreamer1.0-plugins-ugly` (provides x264enc, GPL) or
|
||||
`gstreamer1.0-libav` (LGPL source but GPL binary on Debian).
|
||||
@@ -0,0 +1,347 @@
|
||||
# 05 — Input Handling (Python xkbcommon + Browser WASM)
|
||||
|
||||
This chapter covers input handling in the MIT approach: keyboard (with
|
||||
xkbcommon), pointer, touch, focus management, and how input events travel
|
||||
between browser and native apps.
|
||||
|
||||
## Architecture (same as Greenfield, but with improvements)
|
||||
|
||||
The browser is the source of truth for the keymap (same as Greenfield).
|
||||
The keymap is compiled in the browser using @gfld/compositor-wasm's xkbcommon
|
||||
WASM module, then shipped to the Python proxy as a real native shm fd via
|
||||
the `/mkstemp-mmap` WebSocket endpoint. The proxy then sends it to native
|
||||
apps via libwayland-server's SCM_RIGHTS FD passing.
|
||||
|
||||
**Improvements over Greenfield**:
|
||||
1. **Real touch input** — Greenfield has a 46-line stub that never fires
|
||||
2. **Primary selection** — middle-click paste, which Greenfield lacks
|
||||
3. **Adaptive keyboard layout** — easier to switch layouts at runtime
|
||||
|
||||
## Browser side — Custom TypeScript (clean-room)
|
||||
|
||||
### Seat abstraction
|
||||
|
||||
Same architecture as Greenfield (clean-room from spec + Wayland docs):
|
||||
- `Seat.ts` — `wl_seat` global, multi-seat support, focus management
|
||||
- `Keyboard.ts` — `wl_keyboard` impl, keymap, modifiers, key events
|
||||
- `Pointer.ts` — `wl_pointer` impl, motion, buttons, axis
|
||||
- `Touch.ts` — `wl_touch` (REAL implementation, not stub)
|
||||
- `Xkb.ts` — xkbcommon WASM wrapper (REUSED @gfld/compositor-wasm)
|
||||
- `InputQueue.ts` — event coalescing + RAF dispatch
|
||||
- `InputOutput.ts` — FD abstraction
|
||||
- `events.ts` — KeyEvent, ButtonEvent, AxisEvent wrappers
|
||||
|
||||
### xkbcommon WASM (REUSED)
|
||||
|
||||
`@gfld/compositor-wasm` (MIT) bundles:
|
||||
- libxkbcommon 1.5.0 compiled to WASM via emsdk 3.1.46
|
||||
- Embedded xkeyboard-config database
|
||||
- Exports `init()` + `lib.xkbcommon` singleton with keymap/state functions
|
||||
|
||||
This is REUSED AS-IS. No custom code needed.
|
||||
|
||||
### DOM → Wayland key event translation (clean-room)
|
||||
|
||||
Same approach as Greenfield (verified correct Wayland behavior):
|
||||
1. `event.code` (e.g. `"KeyA"`) looked up in EvDevKeyCode reverse-name-map → evdev keycode (e.g. 30)
|
||||
2. `x11KeyCode = evdevKeyCode + 8` (Linux input protocol offset)
|
||||
3. `capsLock`/`numLock` read from `event.getModifierState(...)`
|
||||
4. DOM `.key` glyph IGNORED — only physical `.code` used (mirrors evdev)
|
||||
|
||||
### Modifier tracking (REUSED WASM)
|
||||
|
||||
Via @gfld/compositor-wasm's xkbcommon state machine:
|
||||
1. `Seat.updateModifierState` calls `xkb.keyDown(x11)` or `xkb.keyUp(x11)`
|
||||
2. `_xkb_state_update_key` invoked internally
|
||||
3. `notifyModifiers` reads `xkb.modsDepressed` / `modsLatched` / `modsLocked` / `group`
|
||||
4. `Keyboard.sendModifiers(modsDepressed, modsLatched, modsLocked, group)` sends to native client
|
||||
|
||||
### Touch input (NEW vs Greenfield)
|
||||
|
||||
Greenfield has a 46-line stub for `Touch.ts`. Our MIT approach implements
|
||||
real touch:
|
||||
|
||||
```typescript
|
||||
// Pseudocode for Touch.ts (clean-room)
|
||||
export class Touch {
|
||||
private touchPoints: Map<number, TouchPoint> = new Map();
|
||||
|
||||
constructor(public seat: Seat) {
|
||||
// Listen to DOM pointer events with pointerType === 'touch'
|
||||
document.addEventListener('pointerdown', this.onPointerDown.bind(this));
|
||||
document.addEventListener('pointermove', this.onPointerMove.bind(this));
|
||||
document.addEventListener('pointerup', this.onPointerUp.bind(this));
|
||||
document.addEventListener('pointercancel', this.onPointerCancel.bind(this));
|
||||
}
|
||||
|
||||
private onPointerDown(ev: PointerEvent) {
|
||||
if (ev.pointerType !== 'touch') return;
|
||||
// Map to wl_touch.down event
|
||||
this.touchPoints.set(ev.pointerId, { x: ev.offsetX, y: ev.offsetY });
|
||||
this.sendDown(ev.pointerId, ev.offsetX, ev.offsetY);
|
||||
}
|
||||
|
||||
// ... etc
|
||||
}
|
||||
```
|
||||
|
||||
### Keymap FD workaround (REUSED approach)
|
||||
|
||||
Same as Greenfield (see `../greenfield-analysis/05-input-handling.md`):
|
||||
|
||||
1. Browser compiles keymap via @gfld/compositor-wasm's xkbcommon
|
||||
2. `inputOutput.mkstempMmap(new Blob([utf8(keymapString)]))` opens a
|
||||
`/mkstemp-mmap` WebSocket, sends the blob
|
||||
3. Python proxy creates a real native shm fd via `os.memfd_create()` (replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
4. Returns `ProxyFD { handle: <native_fd>, type: 'shm', host: baseURL }`
|
||||
5. Browser sends `wl_keyboard.keymap(format, ProxyFD, size)`
|
||||
6. Proxy deserializes, looks up native fd via `proxyFDtoNativeFD`, ships via
|
||||
libwayland-server's SCM_RIGHTS
|
||||
|
||||
## Python side — pywayland + python-xkbcommon
|
||||
|
||||
### python-xkbcommon (MIT)
|
||||
|
||||
The `xkbcommon` PyPI package (by Stephen Early, MIT-licensed) provides
|
||||
Python bindings to libxkbcommon. Useful if the proxy needs to compile
|
||||
keymaps (e.g., for XWayland apps that don't speak Wayland natively).
|
||||
|
||||
For most use cases, the proxy doesn't need xkbcommon — it just forwards
|
||||
the keymap from browser to native apps. The browser does the compilation.
|
||||
|
||||
### Pointer (clean-room)
|
||||
|
||||
Same approach as Greenfield:
|
||||
- DOM `pointermove` / `pointerdown` / `pointerup` / `wheel` events
|
||||
- Mapped to `ButtonEvent` / `AxisEvent` wrappers
|
||||
- `InputQueue` coalesces and dispatches on next RAF
|
||||
- `DefaultPointerGrab.focus()` runs on every motion: `Renderer.pickView`
|
||||
walks the view stack front-to-back, returns first view whose
|
||||
`Surface.isWithinInputRegion(point)` (pixman-WASM region) is true
|
||||
- On change, `Pointer.setFocus` emits `wl_pointer.leave` + `frame` to old
|
||||
surface, then `wl_pointer.enter` + `frame` (+ modifiers if keyboard is
|
||||
grabbed) to new surface
|
||||
|
||||
### Pointer capture / window leave (REUSED approach)
|
||||
|
||||
- `canvas.setPointerCapture` on `pointerdown`
|
||||
- `pointerleave` calls `notifyKeyboardFocusOut` (saves keyboard focus,
|
||||
releases depressed keys, sets `needFocusInit=true` for re-sync),
|
||||
resets `buttonCount=0`
|
||||
- DnD suppresses this handler mid-drag
|
||||
|
||||
### Cursor images (REUSED approach)
|
||||
|
||||
CSS `cursor: url(data:...) x y, auto`:
|
||||
1. Native client calls `wl_pointer.set_cursor`
|
||||
2. Cursor surface commits a buffer
|
||||
3. `CursorRole` draws the bitmap to a hidden canvas
|
||||
4. `toDataURL()` produces a data URL
|
||||
5. Set on `document.body.style.cursor`
|
||||
|
||||
OS composites the cursor. No rotation/scale transforms.
|
||||
|
||||
**NEW**: Use SVG cursor sprites (e.g., from the Phosphor icon set, MIT)
|
||||
instead of bundled PNGs (Greenfield's bundled PNGs are AGPL-contaminated
|
||||
since they're inside `@gfld/compositor`).
|
||||
|
||||
### Scroll wheel / touchpad (REUSED approach)
|
||||
|
||||
`WheelEvent.deltaMode`:
|
||||
- `DOM_DELTA_LINE × 12` — discrete wheel clicks
|
||||
- `DOM_DELTA_PAGE × surface size` — page-sized scrolls
|
||||
- `DOM_DELTA_PIXEL ÷ 3` — fine-grained touchpad
|
||||
|
||||
Each non-zero axis → `wl_pointer.axis(time, axis, Fixed)`. Wayland version
|
||||
≥5 also gets `axis_source(wheel)` + `frame()` (groups events).
|
||||
|
||||
### Linux button codes (REUSED approach)
|
||||
|
||||
DOM `event.button` (0=left, 1=middle, 2=right, 3=Back, 4=forward) mapped
|
||||
to Linux input protocol button codes via the `linuxInput` table.
|
||||
|
||||
## Focus management (REUSED approach)
|
||||
|
||||
### DOM focus
|
||||
|
||||
The `<canvas>` owns keyboard focus permanently (`tabIndex=1`).
|
||||
`Keyboard.setFocus(surface)` emits `wl_keyboard.leave` / `enter` between
|
||||
native surfaces. Activation is driven by `XdgToplevel` / `ShellSurface`
|
||||
activation handlers.
|
||||
|
||||
### DOM → Wayland focus
|
||||
|
||||
- Browser DOM `focus` event → `seat.notifyKeyboardFocusIn()` (restores
|
||||
saved focus, re-syncs modifiers)
|
||||
- Browser DOM `blur` event → `seat.notifyKeyboardFocusOut()` (saves
|
||||
focus, releases depressed keys, sets `needFocusInit=true`)
|
||||
|
||||
### Surface-level focus
|
||||
|
||||
- Click on a surface → `XdgToplevel` activation request →
|
||||
`desktopSurface.activate()` → `gainFocus()` (refcounted via
|
||||
`focusCount`, fires `surfaceActivationUpdated` event to the user shell
|
||||
on 0↔1 transitions)
|
||||
- Popup grabs take keyboard + pointer focus via `seat.popupGrabStart`,
|
||||
validating the serial against the keyboard/pointer `grabSerial`
|
||||
|
||||
## Input queue (REUSED approach)
|
||||
|
||||
`InputQueue.ts` coalesces events and dispatches them on the next RAF
|
||||
(`InputQueue.drainQueues`). Batches bursts of input events (e.g. fast
|
||||
mouse motion) into a single dispatch, avoiding redundant work.
|
||||
|
||||
FIFO per event type (motion, axis, key, button). Each dispatch calls the
|
||||
appropriate `seat.notify*` method.
|
||||
|
||||
## ASCII event-flow diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER (DOM) │
|
||||
│ │
|
||||
│ <canvas tabIndex=1> ─┐ │
|
||||
│ pointermove/dn/up ───┤── createButtonEventFromMouseEvent() ── ButtonEvent │
|
||||
│ wheel ────┤── createAxisEventFromWheelEvent() ── AxisEvent │
|
||||
│ keydown / keyup ────┤── createKeyEventFromKeyboardEvent() ── KeyEvent │
|
||||
│ pointerdown (touch) ─┤── createTouchEventFromPointerEvent() ── TouchEvent (NEW) │
|
||||
│ │ • evdevKeyCode = EvDevKeyCode[event.code] │
|
||||
│ │ • x11KeyCode = evdevKeyCode + 8 │
|
||||
│ │ • capsLock/numLock = event.getModifierState(...) │
|
||||
│ pointerleave / blur ──┤── seat.notifyKeyboardFocusOut() (saves focus, resets mods) │
|
||||
│ focus ────┴── seat.notifyKeyboardFocusIn() (restores saved focus) │
|
||||
└──────────────────────────────┬──────────────────────────────────────────────────────┘
|
||||
│ session.inputQueue.queue{Motion,Axis,Key,Button,Touch}
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER COMPOSITOR (InputQueue.drainQueues, scheduled on next animation frame) │
|
||||
│ │
|
||||
│ seat.notifyMotion / notifyAxis / notifyButton / notifyKey / notifyFrame / notifyTouch│
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyKey ─► Keyboard.grab.key(event) │
|
||||
│ │ └─ Keyboard.sendKey(event) │
|
||||
│ │ └─ keyboardResource.key(serial,time,evdev,st) │
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyButton ─► runButtonBindings + grab.button(event) │
|
||||
│ │ └─ Pointer.sendButton │
|
||||
│ │ └─ pointerResource.button(serial,time, │
|
||||
│ │ linuxInput[btn], state) │
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyMotion ─► grab.motion ─► Pointer.moveTo / sendMotion │
|
||||
│ │ └─ pointerResource.motion(time, sx_f, sy_f) │
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyAxis ─► grab.axis ─► Pointer.sendAxis │
|
||||
│ │ deltaMode→scale: line=×12, page=×surface-size, pixel=÷3 │
|
||||
│ │ └─ pointerResource.axis(time, axis, Fixed) │
|
||||
│ │ + axisSource(wheel) + frame() if v>=5 │
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyTouch (NEW) ─► Touch.sendDown/Motion/Up │
|
||||
│ │ └─ touchResource.down/motion/up/cancel │
|
||||
│ │ │
|
||||
│ ├─► Seat.notifyFrame ─► pointerResource.frame() (groups events) │
|
||||
│ │ │
|
||||
│ └─► Seat.notifyModifiers ─ xkb.keyDown/Up(x11) ─► xkb_state_serialize_mods │
|
||||
│ └─ Keyboard.sendModifiers(modsDepressed,Latched, │
|
||||
│ Locked,group) │
|
||||
│ │
|
||||
│ Keymap path (once per WlKeyboardResource): │
|
||||
│ Seat.notifyUpdateKeymap(nrmlvo) → createFromNames(rules,model,layout,variant,opt) │
|
||||
│ → xkb_context_new + xkb_keymap_new_from_names (@gfld/compositor-wasm, MIT) │
|
||||
│ → xkb_keymap_get_as_string ──► keymapString │
|
||||
│ Keyboard.sendKeymap(resource): │
|
||||
│ inputOutput.mkstempMmap(new Blob([utf8(keymapString)])) ──► ProxyFD │
|
||||
│ (Python proxy creates memfd via os.memfd_create()) │
|
||||
│ keyboardResource.keymap(xkbV1, ProxyFD, size) │
|
||||
└──────────────────────────────┬──────────────────────────────────────────────────────┘
|
||||
│ client.connection.onFlush(wireMessages)
|
||||
│ ─ one big Uint32Array, FDs serialized as ProxyFD JSON
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ WIRE (ARQChannel = KCP-over-WebSocket, custom TS, clean-room) │
|
||||
│ Browser ⇄ ReconnectingWebSocket ⇄ Internet ⇄ aiohttp on proxy :8081 │
|
||||
│ Message layout (Uint32Array): │
|
||||
│ [0] outOfBandOpcode (0 = in-band) │
|
||||
│ [1] eventSerial │
|
||||
│ [2] = nFds, then for each fd: [len][JSON bytes, 4-byte aligned] │
|
||||
│ then concatenated Wayland wire messages │
|
||||
│ + Authorization: Bearer <jwt> │
|
||||
└──────────────────────────────┬──────────────────────────────────────────────────────┘
|
||||
│ NativeWaylandClientSession.onMessage (Python)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ PYTHON PROXY (aiohttp + pywayland) │
|
||||
│ │
|
||||
│ SessionController endpoints: /mkfifo /mkstemp-mmap /read-fd /close-fd │
|
||||
│ /read-fd-as-stream /write-fd-as-stream │
|
||||
│ /signal /channel /request-keyframe │
|
||||
│ │
|
||||
│ mkstempMmap(ws): │
|
||||
│ receives Blob → os.memfd_create() → write bytes → return fd │
|
||||
│ returns ProxyFD { handle: <native_fd>, type: 'shm', host: baseURL } │
|
||||
│ │
|
||||
│ onWireMessageEvents: │
|
||||
│ for each ProxyFD in message: │
|
||||
│ nativeFd = webFS.proxyFDtoNativeFD(proxyFD) # identity when host matches │
|
||||
│ └─ if foreign host → spawn bridging pipe + WebSocket │
|
||||
│ pywayland per-resource dispatcher fires │
|
||||
│ # For seat events, dispatcher is a no-op (events just pass through) │
|
||||
│ # libwayland-server sends via wl_resource_post_event_array │
|
||||
│ flush_clients() │
|
||||
└──────────────────────────────┬──────────────────────────────────────────────────────┘
|
||||
│ libwayland-server sends wire msg + SCM_RIGHTS FDs
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ NATIVE (Wayland client, sandboxed in bubblewrap) │
|
||||
│ wl_display.fd readable → libwayland-client demarshals │
|
||||
│ wl_keyboard.keymap(fd) → mmap(fd) → xkb_keymap_new_from_string (native xkbcommon) │
|
||||
│ wl_keyboard.key / enter / leave / modifiers → client app │
|
||||
│ wl_pointer.enter / motion / button / axis / frame → client app │
|
||||
│ wl_touch.down / motion / up / cancel → client app (NEW: actually works) │
|
||||
└──────────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Key answers (quick reference)
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| DOM→Wayland key event translation | `event.code` → EvDevKeyCode → evdev keycode → +8 = x11KeyCode. DOM `.key` glyph ignored. |
|
||||
| Modifier tracking | @gfld/compositor-wasm xkbcommon state machine (REUSED MIT) |
|
||||
| Focus: DOM ↔ native surfaces | `<canvas>` owns keyboard focus. `Keyboard.setFocus(surface)` emits leave/enter between native surfaces. |
|
||||
| Pointer capture / window leave | `canvas.setPointerCapture` on pointerdown. `pointerleave` saves focus + releases keys. |
|
||||
| Touch event mapping | **NEW: REAL implementation** via `pointerdown`/`pointermove`/`pointerup` with `pointerType === 'touch'`. Greenfield has only a stub. |
|
||||
| Pointer enter/leave between surfaces | `DefaultPointerGrab.focus()` runs on every motion. `Renderer.pickView` walks view stack. |
|
||||
| Pointer acceleration / cursor images | No acceleration. CSS `cursor: url(data:...) x y, auto`. NEW: SVG sprites (MIT) instead of bundled PNGs (AGPL). |
|
||||
| Scroll wheel / touchpad → `wl_pointer.axis` | `WheelEvent.deltaMode`: LINE×12, PAGE×size, PIXEL÷3. v≥5 gets `axis_source(wheel)` + `frame()`. |
|
||||
| WASM xkbcommon? | Yes — @gfld/compositor-wasm (MIT, REUSED). libxkbcommon 1.5.0 + embedded xkeyboard-config. |
|
||||
| Keymap FD workaround for WebSocket | Browser → `/mkstemp-mmap` WebSocket → Python proxy `os.memfd_create()` → ProxyFD JSON → libwayland SCM_RIGHTS to native app. |
|
||||
| Primary selection (middle-click paste)? | **NEW: implemented** via `zwp_primary_selection_device_manager_v1` (Greenfield lacks). |
|
||||
|
||||
## File reference
|
||||
|
||||
| File (custom, clean-room) | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/input/Seat.ts` | `wl_seat` global |
|
||||
| `packages/compositor/src/input/Keyboard.ts` | `wl_keyboard` impl |
|
||||
| `packages/compositor/src/input/Pointer.ts` | `wl_pointer` impl |
|
||||
| `packages/compositor/src/input/Touch.ts` | `wl_touch` impl (REAL, not stub) |
|
||||
| `packages/compositor/src/input/Xkb.ts` | @gfld/compositor-wasm wrapper (REUSED MIT) |
|
||||
| `packages/compositor/src/input/InputQueue.ts` | Event coalescing |
|
||||
| `packages/compositor/src/input/InputOutput.ts` | FD abstraction |
|
||||
| `packages/compositor/src/input/events.ts` | Event wrappers |
|
||||
| `packages/compositor/src/browser/input.ts` | DOM keyboard events |
|
||||
| `packages/compositor/src/browser/pointer.ts` | DOM pointer events |
|
||||
| `packages/compositor/src/browser/touch.ts` | DOM touch events (NEW) |
|
||||
|
||||
### Reused (MIT)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `libs/compositor-wasm/src/index.ts` | xkbcommon + pixman WASM (REUSED) |
|
||||
|
||||
### Python side
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| `pywayland.server.Client` | wl_client wrapper |
|
||||
| `pywayland.server.Resource` | wl_resource base (per-resource dispatchers) |
|
||||
| `xkbcommon` PyPI package | Optional, only needed for XWayland keymap compilation |
|
||||
@@ -0,0 +1,276 @@
|
||||
# 06 — Data Device, Selection, DnD (`wl_data_device`)
|
||||
|
||||
This chapter covers the data device / clipboard / drag-and-drop layer in
|
||||
the MIT approach. Same architecture as Greenfield (clean-room), but with
|
||||
**primary selection** added (which Greenfield lacks).
|
||||
|
||||
## Faithful Wayland model + primary selection (NEW)
|
||||
|
||||
`DataDeviceManager` is a v3 global singleton. `wl_data_device` is per
|
||||
seat-per-client and the implementation lives directly on `Seat`.
|
||||
`WaylandDataSource` / `DataOffer` are the standard source/offer pair.
|
||||
|
||||
**NEW**: Implement `zwp_primary_selection_device_manager_v1` for middle-click
|
||||
paste, which Greenfield lacks entirely (grep returns zero hits).
|
||||
|
||||
## FD-over-WebSocket workaround (REUSED approach)
|
||||
|
||||
Same `ProxyFD` JSON descriptor as Greenfield:
|
||||
```
|
||||
ProxyFD = { handle: number, type: 'pipe-read'|'pipe-write'|'shm'|'unknown', host: string }
|
||||
```
|
||||
|
||||
Three concrete `InputOutput` / `InputOutputFD` implementations:
|
||||
|
||||
### `webInputOutput` — for in-browser web apps
|
||||
|
||||
Pipes are `MessageChannel` ports, transferred as `ArrayBuffer`s.
|
||||
|
||||
### `RemoteInputOutput` — for native apps
|
||||
|
||||
All fd operations go over a secondary WebSocket to endpoints: `/mkfifo`,
|
||||
`/mkstemp-mmap`, `/read-fd-as-stream`, `/write-fd-as-stream`, `/read-fd`,
|
||||
`/close-fd`.
|
||||
|
||||
### `ProxyInputOutput` (Python side) — owns the real OS pipes and memfds
|
||||
|
||||
In Python:
|
||||
- `os.pipe()` for pipes
|
||||
- `os.memfd_create()` for shared memory (replaces Greenfield's `createMemoryMappedFile` native addon)
|
||||
|
||||
`proxyFDtoNativeFD(proxyFD)` returns the raw int fd because every ProxyFD
|
||||
has `host === proxy.baseURL`.
|
||||
|
||||
**Crucial consequence**: For native→native clipboard transfers (user
|
||||
copies in one Linux app, pastes in another), the browser compositor never
|
||||
sees a single byte of the data — it only relays the fd reference. The
|
||||
actual content stays inside the proxy host's OS kernel pipe.
|
||||
|
||||
## MIME-type negotiation (IMPROVED over Greenfield)
|
||||
|
||||
Same hard-coded allow-list initially: `['text/plain', 'text/html',
|
||||
'image/png']`. **IMPROVEMENT**: use exact prefix matching, not substring
|
||||
matching (Greenfield's substring match can collide —
|
||||
`text/plain;charset=iso-8859-1` silently becomes `text/plain`).
|
||||
|
||||
### Native → browser
|
||||
|
||||
`selection.ts:handleWaylandDataSourceUpdate`: each source mime type is
|
||||
matched against the allow-list (exact prefix match), survivors pulled as
|
||||
separate blobs via `mkfifo + send + readBlob`, then
|
||||
`navigator.clipboard.write([new ClipboardItem(...)])`.
|
||||
|
||||
### Browser → native
|
||||
|
||||
`BrowserDataSource.ts`: `ClipboardItem.types` become the source mime types,
|
||||
plus a synthetic `text/plain;charset=utf-8` alias appended (Linux apps
|
||||
expect the charset form). On `send`, the alias is normalised back to
|
||||
`text/plain` for the lookup.
|
||||
|
||||
## Clipboard mirroring (REUSED approach)
|
||||
|
||||
The browser has no `clipboardchange` event (Chromium issue 933608 cited in
|
||||
Greenfield's code). MIT approach polls `navigator.clipboard.read()`:
|
||||
|
||||
- on init
|
||||
- on every `window.focus` event
|
||||
|
||||
De-duplicates offers by mime-type set + total blob size, then calls
|
||||
`seat.setSelectionInternal(createBrowserDataSource(offers), serial)`.
|
||||
|
||||
The feedback loop is broken (same as Greenfield) because
|
||||
`handleWaylandDataSourceUpdate` returns early when `dataSource instanceof
|
||||
BrowserDataSource`. Watch for the `clipboardchange` event in modern
|
||||
browsers (currently behind flags in Chrome) and switch to event-driven
|
||||
when available.
|
||||
|
||||
### Firefox fallback (REUSED approach)
|
||||
|
||||
`text-selection.ts` uses only `readText`/`writeText` and a
|
||||
`BrowserTextDataSource` that serves `text/plain;charset=utf-8` — no images
|
||||
or HTML.
|
||||
|
||||
## Drag-and-drop (REUSED approach + NEW)
|
||||
|
||||
### Wayland-side (Seat.startDrag)
|
||||
|
||||
1. Validates the implicit pointer grab.
|
||||
2. Attaches `DragIconRole` to the optional icon surface.
|
||||
3. Installs a `PointerDrag` grab on both pointer and keyboard.
|
||||
|
||||
`PointerDrag` tracks the focused data device, sends
|
||||
`wl_data_device.enter`/`leave`/`motion`/`drop`, creates fresh
|
||||
`wl_data_offer`s via `sendOffer`, and runs the v3 action negotiation.
|
||||
|
||||
### Browser-side (`BrowserDnD` class)
|
||||
|
||||
Creates a hidden full-canvas `dragHelper` div with `draggable=true` when
|
||||
the Wayland drag-icon surface commits, so the browser shows the Wayland
|
||||
icon via `event.dataTransfer.setDragImage(cursorCanvas, ...)`. While the
|
||||
user drags, `dragover` events on the dragHelper are converted back to
|
||||
`ButtonEvent`s and pushed through `session.inputQueue.queueMotion`,
|
||||
keeping `PointerDrag` fed.
|
||||
|
||||
**NEW: Native→DOM DnD** — implement the dataTransfer side of `BrowserDnD`
|
||||
so a Wayland drag can deliver bytes to a real DOM drop target. Greenfield
|
||||
has a FIXME admitting this isn't implemented.
|
||||
|
||||
## XWayland bridge (REUSED approach, IF XWayland enabled)
|
||||
|
||||
`XWindowManager` listens on the X11 `CLIPBOARD` atom via `XFixes` and
|
||||
builds `XDataSource` (clipboard) or `XDnDDataSource` (XDnD). When the
|
||||
receiver pulls data:
|
||||
|
||||
1. `XDataSource.send` calls `XConvertSelection` to have the X server push
|
||||
the bytes into a property on `selectionWindow`.
|
||||
2. `writeProperty` copies them into the receiver's `InputOutputFD`.
|
||||
3. INCR transfers are handled (`getIncrChunk`, `sendIncrChunk`).
|
||||
|
||||
Uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings.
|
||||
|
||||
## Primary selection (NEW — Greenfield lacks this)
|
||||
|
||||
Implement `zwp_primary_selection_device_manager_v1` (or the older
|
||||
`gtk_primary_selection`) on the compositor side. Forward `PRIMARY`
|
||||
notifications in `XWindowManager`. This unlocks middle-click paste from
|
||||
native apps.
|
||||
|
||||
## ASCII diagram — clipboard flow in both directions
|
||||
|
||||
```
|
||||
CLIPBOARD / SELECTION FLOW
|
||||
|
||||
=================================================================
|
||||
A) NATIVE APP COPIES ──► BROWSER DOM CLIPBOARD (Ctrl+C in Linux app)
|
||||
=================================================================
|
||||
|
||||
┌──────────────┐ wl_data_device.set_selection(src,serial) ┌────────────────────────┐
|
||||
│ Native app │ ─────────────────────────────────────────────► │ compositor │
|
||||
│ (sandboxed │ │ (browser, in DOM) │
|
||||
│ in bwrap) │ ◄────── wl_data_offer.offer("text/plain") ─── │ │
|
||||
└──────────────┘ (only if this app has keyboard focus)└───────────┬────────────┘
|
||||
│ selectionListeners fire
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ handleWaylandDataSource │
|
||||
│ Update() (selection.ts)│
|
||||
└──────────┬──────────────┘
|
||||
mkfifo() ──────────────────────────► │ Python /mkfifo
|
||||
ProxyFD read+write (os.pipe) ◄───── returns 2 ProxyFDs
|
||||
▼
|
||||
dataSource.send("text/plain", writeFD)
|
||||
│
|
||||
▼
|
||||
wl_data_source.send("text/plain", writeFDProxyFD)
|
||||
─────────────────────────────────────────► │ Native source
|
||||
│ receives real write fd
|
||||
│ (proxyFDtoNativeFD
|
||||
│ is a passthrough)
|
||||
▼
|
||||
Native app writes bytes
|
||||
──────────────► OS pipe ───┐
|
||||
│
|
||||
readFD.readBlob() ◄──────────── read from /read-fd-as-stream ◄─┘
|
||||
▼ (proxy streams over WS)
|
||||
Blob {type:"text/plain"}
|
||||
▼
|
||||
navigator.clipboard.write([ new ClipboardItem({"text/plain": blob}) ])
|
||||
▼
|
||||
┌────────────────────────────┐
|
||||
│ DOM clipboard populated │
|
||||
│ (Ctrl+V anywhere now works) │
|
||||
└────────────────────────────┘
|
||||
|
||||
|
||||
=================================================================
|
||||
B) USER COPIES IN BROWSER ──► NATIVE APP GETS SELECTION (Ctrl+C in DOM)
|
||||
=================================================================
|
||||
|
||||
┌─────────────────────────────┐
|
||||
│ User copies in DOM │
|
||||
│ (browser tab, textarea...) │
|
||||
└──────────────┬──────────────┘
|
||||
│ on window 'focus' event (or initial load)
|
||||
▼
|
||||
navigator.clipboard.read() ──► ClipboardItems[]
|
||||
▼
|
||||
handleBrowserDataSourceUpdate() — de-dup vs previous offers
|
||||
▼
|
||||
createBrowserDataSource(offers)
|
||||
.mimeTypes = offer.types + ['text/plain;charset=utf-8'] alias
|
||||
▼
|
||||
seat.setSelectionInternal(browserDataSource, serial)
|
||||
│
|
||||
│ 1) selectionListeners fire — handleWaylandDataSourceUpdate
|
||||
│ returns early (instanceof BrowserDataSource), breaking loop
|
||||
│
|
||||
│ 2) sendSelection(focusedClient)
|
||||
▼
|
||||
┌────────────────────────┐ wl_data_offer.offer("text/plain"), ┌──────────────┐
|
||||
│ compositor │ ─ wl_data_device.selection(offer) ──► │ Native app │
|
||||
│ (browser, in DOM) │ │ (focused) │
|
||||
└───────────┬────────────┘ └──────┬───────┘
|
||||
│ │
|
||||
│ When the native app calls wl_data_offer.receive("text/plain", fd):
|
||||
│ │
|
||||
│ ◄──────── wl_data_offer.receive(mimeType, writeFDProxyFD) ◄── native pipe
|
||||
▼
|
||||
BrowserDataSource.send(mimeType, ioFD)
|
||||
│
|
||||
▼
|
||||
matchingOffer.getType(mimeType) ─► Promise<Blob> (from ClipboardItem)
|
||||
│
|
||||
▼
|
||||
ioFD.write(blob) ─► POST /write-fd-as-stream?fd=<handle> ─► Python proxy
|
||||
│
|
||||
▼
|
||||
asyncio.StreamWriter ──► native write fd
|
||||
│
|
||||
▼
|
||||
Bytes flow through OS pipe into native app
|
||||
│
|
||||
▼
|
||||
ioFD.close() ─► /close-fd ─► EOF ─► native app's read returns 0
|
||||
|
||||
|
||||
=================================================================
|
||||
C) PRIMARY SELECTION (NEW — middle-click paste) ──► same flow but via
|
||||
zwp_primary_selection_device_manager_v1 instead of wl_data_device
|
||||
=================================================================
|
||||
|
||||
Native app selects text ──► zwp_primary_selection_source.set_selection
|
||||
│
|
||||
▼
|
||||
Browser receives primary selection offer ──► stores in seat.primarySelectionDataSource
|
||||
│
|
||||
▼
|
||||
User middle-clicks on a surface ──► seat.notifyPrimarySelectionRequest
|
||||
│
|
||||
▼
|
||||
primarySelectionSource.send(mimeType, writeFD) ──► same FD-over-WS flow as A/B
|
||||
```
|
||||
|
||||
## File reference
|
||||
|
||||
### Custom (clean-room TS)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` global |
|
||||
| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` impl |
|
||||
| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` impl |
|
||||
| `packages/compositor/src/data/WaylandDataSource.ts` | Native source wrapper |
|
||||
| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** |
|
||||
| `packages/compositor/src/data/PrimarySelectionSource.ts` | **NEW: middle-click paste source** |
|
||||
| `packages/compositor/src/browser/selection.ts` | Browser clipboard bridge |
|
||||
| `packages/compositor/src/browser/text-selection.ts` | Firefox fallback |
|
||||
| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD implemented) |
|
||||
| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
||||
|
||||
### Python side
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| `os.pipe()` | Create pipes (replaces Greenfield's `makePipe` native addon) |
|
||||
| `os.memfd_create()` | Create shared memory (replaces Greenfield's `createMemoryMappedFile`) |
|
||||
| `asyncio.StreamReader` / `StreamWriter` | Streaming reads/writes over WebSocket endpoints |
|
||||
@@ -0,0 +1,423 @@
|
||||
# 07 — Native App Bridge (pywayland + bubblewrap + seccomp + systemd-run)
|
||||
|
||||
This chapter covers the native app bridge in the MIT approach: how real
|
||||
Linux applications get launched, sandboxed, connected to the Python proxy,
|
||||
and routed to the browser. **This is the most important chapter for the
|
||||
user** — they want to run real Linux apps in their Odoo 20 web app.
|
||||
|
||||
## Process model — three layers (same as Greenfield, but in Python)
|
||||
|
||||
### Layer 1: Main process (HTTP + WS hub)
|
||||
|
||||
Single Python process (aiohttp) listens on `:8081`. Per Odoo session, it
|
||||
forks one `SessionProcess` child — one child per `compositor_session_id`,
|
||||
**not** per app. Multiple apps in the same browser tab share one child.
|
||||
|
||||
WebSocket upgrades are handed off to the matching child via
|
||||
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
|
||||
|
||||
### Layer 2: Session process
|
||||
|
||||
The forked child. Receives IPC messages:
|
||||
|
||||
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
|
||||
- `launch_app` — spawns the actual app binary via `subprocess.Popen` +
|
||||
`bubblewrap` + `seccomp` + `systemd-run --scope`
|
||||
- `ws_upgrade` — hands WS to the dispatcher
|
||||
|
||||
On `session.close()` the process exits.
|
||||
|
||||
### Layer 3: Per-client session
|
||||
|
||||
For each new `wl_client` (one per Wayland connection), creates a
|
||||
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
|
||||
JSON. Routes requests via per-resource dispatchers.
|
||||
|
||||
## App launch flow (definitive answer)
|
||||
|
||||
1. Browser sends `GET http://proxy:8081/<path>` with
|
||||
`Authorization: Bearer <jwt>` header.
|
||||
2. Main process validates JWT (HS256, via `pyjwt`).
|
||||
3. Main process forks a `SessionProcess` if none exists for that
|
||||
`compositor_session_id`, sends `start` IPC.
|
||||
4. Main sends `launch_app` IPC with `{name, executable, args, env, serial}`.
|
||||
5. `SessionProcess` calls `launch_application()`:
|
||||
- `appEnv = {...os.environ, ...env, WAYLAND_DISPLAY: session.wayland_display}`
|
||||
- `subprocess.Popen(['systemd-run', '--user', '--scope', '-p',
|
||||
'MemoryMax=1G', '-p', 'CPUQuota=200%', '-p', 'TasksMax=256', '--',
|
||||
'bwrap', '--unshare-all', '--clearenv', '--seccomp', '<memfd>',
|
||||
'--ro-bind', '/usr', '/usr', '--dev-bind', '/dev/dri/renderD128',
|
||||
'/dev/dri/renderD128', '--ro-bind',
|
||||
f'{host_xdg}/wayland-N', f'/run/user/{uid}/wayland-N',
|
||||
'--uid', str(uid), '--gid', str(gid),
|
||||
'--new-session', '--die-with-parent', executable, *args],
|
||||
env=appEnv)`
|
||||
6. On `spawn` event, captures `pid`, creates `NativeAppContext`, registers
|
||||
`exit` listener.
|
||||
7. Replies to main with `{pid, key}` → main returns HTTP 201 with
|
||||
`{baseURL, signalURL, key, pid, name}`.
|
||||
8. Browser opens `signalURL` as WebSocket → control channel established.
|
||||
9. App connects to `$XDG_RUNTIME_DIR/wayland-N` → libwayland fires
|
||||
`wl_client` creation → pywayland's per-resource dispatchers route
|
||||
requests to `{native, browser}`.
|
||||
10. For each new `wl_client`, creates a `PROTOCOL` ARQ channel and sends
|
||||
`CONNECT_CHANNEL` over the signaling WS; browser opens a new WS at
|
||||
`/channel?id=…`.
|
||||
11. Wayland wire messages now flow: app → libwayland → pywayland
|
||||
dispatcher → routed to `{native, browser}`.
|
||||
|
||||
## Sandboxing (NEW — Greenfield has none)
|
||||
|
||||
This is the **critical gap** in Greenfield (pitfall M). The MIT approach
|
||||
ships with proper sandboxing from day one.
|
||||
|
||||
### Recommended stack
|
||||
|
||||
- **bubblewrap (bwrap)** — LGPL-2.0+, unprivileged, setuid-free, Flatpak-grade
|
||||
- **seccomp BPF** — libseccomp LGPL-2.1, Docker's default profile + allow `CLONE_NEWUSER`
|
||||
- **systemd-run --scope** — native cgroup v2 integration (MemoryMax, CPUQuota, TasksMax)
|
||||
- **Linux user namespaces** — UID remap to 100000-165535 via `/etc/subuid`/`/etc/subgid`
|
||||
- **cgroups v2** — memory, CPU, PID limits per app
|
||||
|
||||
### Reference bwrap command line (GTK4 app, GPU, no network)
|
||||
|
||||
```bash
|
||||
systemd-run --user --scope --unit=app-odoo-calculator-$(uuid) \
|
||||
-p MemoryMax=256M -p MemorySwapMax=0 -p CPUQuota=50% -p TasksMax=32 -p Delegate=yes -- \
|
||||
bwrap \
|
||||
--clearenv --unshare-all --hostname sandbox --new-session --die-with-parent \
|
||||
--symlink usr/lib /lib --symlink usr/lib64 /lib64 \
|
||||
--symlink usr/bin /bin --symlink usr/bin /sbin \
|
||||
--ro-bind /usr /usr --ro-bind /opt /opt \
|
||||
--ro-bind /etc/passwd /etc/passwd --ro-bind /etc/group /etc/group \
|
||||
--ro-bind /etc/hosts /etc/hosts --ro-bind /etc/localtime /etc/localtime \
|
||||
--ro-bind /etc/locale.conf /etc/locale.conf --ro-bind /etc/machine-id /etc/machine-id \
|
||||
--ro-bind /etc/fonts /etc/fonts --ro-bind /etc/ssl /etc/ssl \
|
||||
--ro-bind /etc/ca-certificates /etc/ca-certificates \
|
||||
--proc /proc --dev /dev \
|
||||
--dev-bind-try /dev/dri/renderD128 /dev/dri/renderD128 \
|
||||
--tmpfs /tmp --tmpfs /var/tmp --tmpfs /run \
|
||||
--dir /run/user/100000 \
|
||||
--ro-bind "$HOST_XDG/wayland-0" /run/user/100000/wayland-0 \
|
||||
--overlay /usr/share/compositor-proxy/skel/gnome-calculator:/var/lib/compositor-proxy/tenants/$TENANT/appdata/gnome-calculator-upper /home/appuser \
|
||||
--setenv HOME /home/appuser \
|
||||
--setenv XDG_RUNTIME_DIR /run/user/100000 \
|
||||
--setenv WAYLAND_DISPLAY wayland-0 \
|
||||
--setenv XDG_DATA_DIRS /usr/share --setenv PATH /usr/bin --setenv LANG en_US.UTF-8 \
|
||||
--uid 100000 --gid 100000 \
|
||||
--seccomp /proc/self/fd/3 \
|
||||
/usr/bin/gnome-calculator
|
||||
```
|
||||
|
||||
### Key design decisions
|
||||
|
||||
| Concern | Decision | Rationale |
|
||||
|---|---|---|
|
||||
| Sandbox runner | **bubblewrap** | MIT license, unprivileged, setuid-free, Flatpak-grade |
|
||||
| cgroup limits | **`systemd-run --user --scope` wrapping bwrap** | Native cgroup v2 integration; no manual cgroupfs writes |
|
||||
| Wayland socket | **Single-file `--ro-bind` of `wayland-N`** into fresh `/run/user/<UID>` tmpfs | Critical: prevents access to PulseAudio, pipewire, D-Bus, keyring, and sibling tenants' sockets |
|
||||
| GPU | **`--dev-bind /dev/dri/renderD128`** (single render node, NOT `card0`) | Kernel render-node access control already restricts to non-display ioctls |
|
||||
| Network | **`--unshare-net` by default** (loopback-down only); per-app opt-in to veth+nftables filter | Most GTK apps need no network. Firefox/Odoo clients opt in. |
|
||||
| Seccomp | **Docker's default profile** (allowlist of ~250 syscalls, blocking ~44) + allow `CLONE_NEWUSER` for nested sandboxes | Compile to BPF at proxy startup; pass via `bwrap --seccomp <memfd>` |
|
||||
| Filesystem | **`/usr` ro-bind, minimal `/etc` whitelist, `/tmp` tmpfs, `$HOME` overlay (changes discarded)** | No host write access; per-app `$HOME` reset on each launch kills malware persistence |
|
||||
| User ns | **UID remap to 100000–165535 via `/etc/subuid`/`/etc/subgid`** | Even on kernel escape the app is unprivileged on host; per-app UID enables nftables filtering and audit |
|
||||
| XWayland | **v1: unsandboxed (documented limitation); v2: per-Xwayland bwrap sandbox** | Hardest piece (3–4 days). v1 ships like Flatpak's old `socket=x11`. |
|
||||
| `$XDG_RUNTIME_DIR` | **`--clearenv` then explicit `--setenv` whitelist** | No host env leak (DB password, Odoo session cookies, AWS keys, etc.) |
|
||||
|
||||
## Native addon architecture (NO native Node addons — pure Python)
|
||||
|
||||
**No N-API addons, no `.node` files, no CMake/Ninja build chain.** The
|
||||
entire proxy is pure Python.
|
||||
|
||||
### Replaces Greenfield's 3 native C addons
|
||||
|
||||
| Greenfield native | Python replacement | Notes |
|
||||
|---|---|---|
|
||||
| `wayland-server-addon.node` (1231 lines C, patched libwayland) | **pywayland** (Apache-2.0) + 5-line cffi shim | Vanilla upstream libwayland 1.26.0 bundled in pywayland's wheel. Apply Greenfield's `changes.diff` (MIT patches) optionally for fast-sync optimization. |
|
||||
| `proxy-poll-addon.node` (127 lines C, libuv `uv_poll_t`) | `asyncio.get_running_loop().add_reader(wl_fd, dispatch_fn)` | Built-in asyncio. NO addon needed. |
|
||||
| `proxy-encoding-addon.node` (290 lines C) + `libproxy-encoding.so` (~1400 lines C, GStreamer) | **PyGObject + GStreamer Python bindings** (LGPL) | Same GStreamer, Python wrapper. ~700 LOC Python replaces ~1700 LOC C. |
|
||||
| `westfield-*.c` (EGL, dmabuf, xwayland helpers, ~2000 lines C) | **Vendored Greenfield C files via cffi** (MIT) | Reuse verbatim. ~3000 LOC of MIT C cherry-picked from Greenfield. |
|
||||
|
||||
### Vendored Greenfield C files (MIT, reused as-is)
|
||||
|
||||
These files from `greenfield/packages/compositor-proxy/native/wayland/src/`
|
||||
are MIT-licensed (Greenfield's own work or vendored from wlroots which is
|
||||
also MIT) and can be **reused verbatim** in our Python proxy:
|
||||
|
||||
| File | Lines | Purpose |
|
||||
|---|---|---|
|
||||
| `westfield-buffer.c` | ~150 | Buffer helpers |
|
||||
| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM |
|
||||
| `westfield-egl.c` | ~900 | EGL zero-copy setup |
|
||||
| `westfield-xwayland.c` | ~465 | XWayland spawn + bridge |
|
||||
| `wlr_drm.c` | ~244 | `wl_drm` global impl (vendored from wlroots) |
|
||||
| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) |
|
||||
| `drm_format_set.c` | ~239 | DRM format helpers (vendored from wlroots) |
|
||||
| `addon.c` | ~73 | wlroots addon system helper |
|
||||
| **TOTAL** | **~3,374 lines C** | All MIT, reused verbatim |
|
||||
|
||||
These compile into a single `libwestfield.so` shared library, loaded by
|
||||
the Python proxy via cffi at runtime.
|
||||
|
||||
## XWayland bridge
|
||||
|
||||
### Native side (Python equivalent of westfield-xwayland.c)
|
||||
|
||||
In Python:
|
||||
```python
|
||||
import subprocess, os, socket
|
||||
|
||||
def spawn_xwayland(display_num, wl_client_fd, wm_fd, display_fd):
|
||||
# Create X11 socket
|
||||
x11_socket = f"/tmp/.X11-unix/X{display_num}"
|
||||
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
|
||||
listener.bind(x11_socket)
|
||||
listener.listen(1)
|
||||
|
||||
# Set DISPLAY env
|
||||
os.environ["DISPLAY"] = f":{display_num}"
|
||||
|
||||
# Fork + exec Xwayland
|
||||
subprocess.Popen([
|
||||
"Xwayland", f":{display_num}", "-ac", "-rootless",
|
||||
"-listen", str(listener.fileno()),
|
||||
"-displayfd", str(display_fd),
|
||||
"-wm", str(wm_fd),
|
||||
], pass_fds=[listener.fileno(), display_fd, wm_fd])
|
||||
|
||||
# Xwayland itself becomes a Wayland client of the proxy
|
||||
# (via wl_client_create(wl_display, wl_client_fd))
|
||||
```
|
||||
|
||||
### xtsb (REUSED MIT, browser side)
|
||||
|
||||
`@gfld/xtsb` (MIT) — pure-TS X11 client, no libxcb. Dual-environment
|
||||
(Node `net.Socket` + browser WebSocket). Auto-generated from XCB XML
|
||||
protocol files via `xcbgen/ts_client.py`. The browser-side xtsb
|
||||
implements the actual XWM logic (translates X `CreateNotify` events →
|
||||
`xdg_toplevel` Wayland surfaces).
|
||||
|
||||
**Optional**: skip XWayland in MVP (8 weeks of effort). Add only if you
|
||||
need to run X11-only apps (most modern GTK/Qt apps are Wayland-native).
|
||||
|
||||
## SDK misconception corrected (same as Greenfield)
|
||||
|
||||
The `sdk/` directory is **NOT for native apps**. It's exclusively for
|
||||
**WASM apps** that run inside the browser via Emscripten. The
|
||||
`sysrootlibs/` cross-compiles wayland/gtk/cairo/etc. to wasm32. Native
|
||||
Linux apps on the proxy use the host's own system libraries.
|
||||
|
||||
## ASCII diagram — native app launch & connection
|
||||
|
||||
```
|
||||
Browser (Odoo OWL shell) Python proxy host (Linux) App (sandboxed in bwrap)
|
||||
========================== ==================== ==============================
|
||||
|
||||
[1] User clicks app in shell $ python -m compositor_proxy.cli \
|
||||
│ └─── main.py listens on :8081
|
||||
│ sessionProcesses: Dict[id, Process]
|
||||
│
|
||||
│ HTTP GET /gtk4-demo
|
||||
│ Authorization: Bearer <jwt>
|
||||
├──────────────────────────────► │
|
||||
│ │ validate JWT (HS256, pyjwt)
|
||||
│ │ if no child for <session_id>:
|
||||
│ │ multiprocessing.Process(target=session_main)
|
||||
│ │ IPC: {type:'start', payload:{config, session_id}}
|
||||
│ │ └── create_session() → Display
|
||||
│ │ ├── Display()
|
||||
│ │ ├── display.add_socket() → "wayland-N"
|
||||
│ │ │ (becomes WAYLAND_DISPLAY)
|
||||
│ │ ├── display.init_shm() + YUV420/NV12/...
|
||||
│ │ ├── init_drm("/dev/dri/renderD128")
|
||||
│ │ │ (uses vendored westfield-egl.c via cffi)
|
||||
│ │ │ (registers zwp_linux_dmabuf_v1 + wl_drm globals)
|
||||
│ │ └── asyncio.add_reader(wl_fd, dispatch_wl)
|
||||
│ │ (5-line cffi shim for wl_event_loop_get_fd)
|
||||
│ │
|
||||
│ │ IPC: {type:'launch_app', payload:{name, executable, args, env, serial}}
|
||||
│ │ └── launch_application() (NativeAppContext.py)
|
||||
│ │ ├── appEnv = {...os.environ, ...env,
|
||||
│ │ │ WAYLAND_DISPLAY: "wayland-N"}
|
||||
│ │ ├── subprocess.Popen([
|
||||
│ │ │ 'systemd-run', '--user', '--scope',
|
||||
│ │ │ '-p', 'MemoryMax=1G',
|
||||
│ │ │ '-p', 'CPUQuota=200%',
|
||||
│ │ │ '-p', 'TasksMax=256', '--',
|
||||
│ │ │ 'bwrap',
|
||||
│ │ │ '--unshare-all', '--clearenv',
|
||||
│ │ │ '--seccomp', '<memfd>',
|
||||
│ │ │ '--ro-bind', '/usr', '/usr',
|
||||
│ │ │ '--dev-bind', '/dev/dri/renderD128', '/dev/dri/renderD128',
|
||||
│ │ │ '--ro-bind', '$XDG/wayland-N', '/run/user/100000/wayland-N',
|
||||
│ │ │ '--uid', '100000', '--gid', '100000',
|
||||
│ │ │ '--new-session', '--die-with-parent',
|
||||
│ │ │ '/usr/bin/gtk4-demo'
|
||||
│ │ │ ], env=appEnv)
|
||||
│ │ │ ◄── systemd-run + bwrap, NO chroot, NO container
|
||||
│ │ │ ──────────────────────────────► [2] gtk4-demo (sandboxed)
|
||||
│ │ │ │
|
||||
│ │ │ │ libwayland-client.so
|
||||
│ │ │ │ connects to
|
||||
│ │ │ │ $XDG_RUNTIME_DIR/wayland-N
|
||||
│ │ │ │
|
||||
│ │ │ ▼
|
||||
│ │ │ ┌──────────────────────────┐
|
||||
│ │ │ │ wl_display event loop │
|
||||
│ │ │ │ (in proxy, libwayland- │
|
||||
│ │ │ │ server.so.0 — vanilla, │
|
||||
│ │ │ │ NO patches needed) │
|
||||
│ │ │ │ │
|
||||
│ │ │ ◄── asyncio fires, │ wl_client created → │
|
||||
│ │ │ dispatch_wl() called │ Client.get_credentials │
|
||||
│ │ │ wl_event_loop_dispatch() │ → (pid, uid, gid) │
|
||||
│ │ │ │ find_matching_app_ctx │
|
||||
│ │ │ │ (pid) by PPID walk │
|
||||
│ │ │ │ │
|
||||
│ │ │ │ Per-resource dispatchers│
|
||||
│ │ │ │ (auto-generated by │
|
||||
│ │ │ │ pywayland scanner) │
|
||||
│ │ │ │ route to {native, │
|
||||
│ │ │ │ browser} │
|
||||
│ │ │ └──────────────────────────┘
|
||||
│ │ │
|
||||
│ │ ├── child.on('spawn') → capture pid
|
||||
│ │ │ create_native_app_context(pid, name, sandbox=True)
|
||||
│ │ │ native_app_context.on_disconnect() # start 10-min SIGHUP timer
|
||||
│ │ │
|
||||
│ │ └── child.on('exit') → native_app_context.on_exit({exit_code|signal})
|
||||
│ │
|
||||
│ │ IPC reply: {type:'launch_app_success', payload:{pid, key}}
|
||||
│
|
||||
│ HTTP 201 Created
|
||||
│ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, sandboxed:true }
|
||||
◄────────────────────────────── │
|
||||
│ │
|
||||
│ Open WS: ws://…/signal?key=<16-hex>
|
||||
│ Authorization: Bearer <jwt>
|
||||
├──────────────────────────────► │
|
||||
│ │ aiohttp demux /signal → native_app_context.on_connect(ws)
|
||||
│ │ clears 10-min SIGHUP timer
|
||||
│ │
|
||||
│ │ For each new wl_client (one per wayland connection):
|
||||
│ │ create_protocol_channel() → ARQChannel (kcp over WS)
|
||||
│ │ sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc}
|
||||
│ ◄──── CONNECT_CHANNEL JSON ───┤
|
||||
│ │
|
||||
│ Open WS: ws://…/channel?id=…&key=…
|
||||
│ Authorization: Bearer <jwt>
|
||||
├──────────────────────────────► │ binds WS to the ARQChannel
|
||||
│ │
|
||||
│ Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval):
|
||||
│
|
||||
│ App → wl_surface.commit ──── proxy intercepts via dispatcher ────► browser compositor
|
||||
│ App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄── FrameFeedback paces callback
|
||||
│ App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected)
|
||||
│ App ← wl_keyboard.enter ─── browser → proxy → libwayland-server sends via SCM_RIGHTS
|
||||
│
|
||||
│ Buffer path (per wl_surface):
|
||||
│ - App submits wl_buffer (dmabuf or shm)
|
||||
│ - setBufferCreatedCallback → ProxyBuffer with creationSerial
|
||||
│ - On commit: dispatcher → Encoder.encodeFrame()
|
||||
│ └── gst_parse_launch("appsrc ! glupload ! glcolorconvert ! glshader ! "
|
||||
│ "glcolorconvert ! gldownload ! queue ! vaapih264enc ! appsink")
|
||||
│ (vaapih264enc — LGPL, NOT x264enc which is GPL)
|
||||
│ └── GLib MainLoop in separate thread
|
||||
│ └── encoded H.264 sample → asyncio.run_coroutine_threadsafe → main thread
|
||||
│ └── sent over the surface's FRAME data channel (separate WS)
|
||||
│
|
||||
│ X11 path (xterm etc., IF XWayland enabled):
|
||||
│ - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by spawn_xwayland())
|
||||
│ - First X connect → spawn_xwayland() forks Xwayland: execlp("Xwayland", …, "-rootless", "-wm", wm_fd)
|
||||
│ - Xwayland itself becomes a Wayland client of the proxy (wl_client_create)
|
||||
│ - When Xwayland ready (displayFd readable):
|
||||
│ └── xtsb (REUSED MIT, browser side) speaks X11 → translates X windows → xdg_toplevel surfaces
|
||||
│
|
||||
│ Lifecycle:
|
||||
│ - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP')
|
||||
│ - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM')
|
||||
│ └── if not exited in 10s → SIGKILL
|
||||
│ - App crashes → child 'exit' event → native_app_context.on_exit() → APP_TERMINATED msg → browser UI updates
|
||||
│ - All clients gone for >1s → session.close() → process.exit() in SessionProcess.py → main deletes entry
|
||||
│
|
||||
│ Sandbox (NEW vs Greenfield):
|
||||
│ - bwrap --unshare-all --clearenv --seccomp <memfd>
|
||||
│ - --ro-bind /usr /usr, minimal /etc whitelist
|
||||
│ - --dev-bind /dev/dri/renderD128 (single GPU render node, no card0)
|
||||
│ - --ro-bind $HOST_XDG/wayland-N /run/user/100000/wayland-N (single socket only)
|
||||
│ - --uid 100000-165535 per-app (--gid same)
|
||||
│ - --unshare-net by default
|
||||
│ - cgroup: MemoryMax=1G, CPUQuota=200%, TasksMax=256
|
||||
│ - $HOME overlay (changes discarded on app exit)
|
||||
```
|
||||
|
||||
## Effort estimates
|
||||
|
||||
| Phase | Days | Deliverable |
|
||||
|---|---|---|
|
||||
| **v1 shippable** | **7-9 days** | bubblewrap integration + seccomp + cgroup limits + Wayland socket isolation + GPU render-node passthrough + default-deny net. XWayland unsandboxed (documented). |
|
||||
| v2 full | +4-9 days | + XWayland sandbox + veth+nftables networking + per-tenant hierarchical cgroups + XDG Desktop Portal |
|
||||
|
||||
### v1 breakdown
|
||||
|
||||
| Component | Days |
|
||||
|---|---|
|
||||
| bwrap integration | 2-3 |
|
||||
| seccomp profile + compile | 1-2 |
|
||||
| systemd-run cgroup wrapper | 1 |
|
||||
| Wayland socket ownership fix | 0.5 |
|
||||
| GPU passthrough testing | 1 |
|
||||
| Schema + docs | 1 |
|
||||
| **TOTAL** | **7-9** |
|
||||
|
||||
## File reference
|
||||
|
||||
### Python proxy (custom, ~2880 LOC)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `proxy/main.py` | HTTP + WS hub, forks per session |
|
||||
| `proxy/session.py` | Top-level proxy session |
|
||||
| `proxy/session_controller.py` | aiohttp WS server with 9 endpoint paths |
|
||||
| `proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn |
|
||||
| `proxy/native_wayland_client_session.py` | Per-client message dispatcher |
|
||||
| `proxy/native_wayland_compositor_session.py` | Display creation, initShm, initDrm |
|
||||
| `proxy/channel.py` | ARQChannel + SimpleChannel (clean-room) |
|
||||
| `proxy/kcp_adapter.py` | kcp PyPI package integration |
|
||||
| `proxy/frame_feedback.py` | Receives 4-byte decode feedback, paces wl_frame.done |
|
||||
| `proxy/proxy_buffer.py` | Proxy-side buffer representation |
|
||||
| `proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration |
|
||||
| `proxy/xwayland_session.py` | XWayland spawn + bridge (Python equivalent of westfield-xwayland.c) |
|
||||
| `proxy/wayland_server.py` | pywayland wrapper + cffi shim |
|
||||
| `proxy/io/` | WebSocket server glue |
|
||||
| `proxy/encoding/` | GStreamer encoder orchestration via PyGObject |
|
||||
| `proxy/protocol/` | pywayland-generated dispatchers (auto) |
|
||||
| `proxy/jwt_auth.py` | JWT validation (pyjwt) |
|
||||
| `proxy/sandbox.py` | bubblewrap + seccomp + systemd-run wrappers |
|
||||
|
||||
### Vendored Greenfield C files (MIT, reused as-is)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `native/westfield-buffer.c` | Buffer helpers |
|
||||
| `native/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM |
|
||||
| `native/westfield-egl.c` | EGL zero-copy setup |
|
||||
| `native/westfield-xwayland.c` | XWayland spawn + bridge (used as reference, ported to Python) |
|
||||
| `native/wlr_drm.c` | `wl_drm` global impl (vendored from wlroots) |
|
||||
| `native/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) |
|
||||
| `native/drm_format_set.c` | DRM format helpers (vendored from wlroots) |
|
||||
| `native/addon.c` | wlroots addon system helper |
|
||||
| `native/changes.diff` | Patches to libwayland-server (optional, for fast-sync) |
|
||||
|
||||
### Reused (MIT/Apache)
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| pywayland | Python bindings to libwayland-server |
|
||||
| `xkbcommon` PyPI package | Python bindings to libxkbcommon (optional, for XWayland) |
|
||||
| PyGObject (`gi.repository.Gst`) | GStreamer Python bindings |
|
||||
| `kcp` PyPI package | Cython wrapper over upstream `ikcp.c` |
|
||||
| aiohttp | Async HTTP + WebSocket server |
|
||||
| PyJWT | JWT validation |
|
||||
| bubblewrap | Unprivileged sandboxing |
|
||||
| libseccomp | Seccomp BPF filter compilation |
|
||||
@@ -0,0 +1,287 @@
|
||||
# 08 — App Lifecycle (Xdg Shell, Toplevels, Popups, Subsurfaces)
|
||||
|
||||
This chapter covers the application lifecycle layer in the MIT approach.
|
||||
Same architecture as Greenfield (clean-room from spec), with the
|
||||
`UserShellApi` integration seam documented in detail since that's where
|
||||
the Odoo 20 OWL shell hooks in.
|
||||
|
||||
## Critical: clean-room requirement for roles
|
||||
|
||||
The xdg-shell state machine is the **highest-risk component** of the MIT
|
||||
approach (3 dev-weeks, high risk per Agent P's estimate). Subtle bugs in
|
||||
configure/ack-configure round-trips, popup positioner math, or subsurface
|
||||
sync/desync cascades can cause apps to hang or crash.
|
||||
|
||||
**Approach**:
|
||||
1. Person A writes a behavioral spec from:
|
||||
- Wayland protocol spec (https://wayland.freedesktop.org/docs/html/)
|
||||
- xdg-shell.xml protocol description
|
||||
- wlroots C source (MIT) — `types/xdg_shell/wlr_xdg_*.c`
|
||||
- Weston C source (MIT) — `libweston/desktop/xdg-shell.c`
|
||||
- Smithay Rust source (MIT) — `src/wayland/xdg_shell/`
|
||||
2. Person B implements from spec — never opens `@gfld/compositor` source.
|
||||
|
||||
## Three role families, one shared abstraction (same as Greenfield)
|
||||
|
||||
All surface roles implement `SurfaceRole` (`{ view, optional
|
||||
desktopSurface, onCommit(surface) }`). `XdgToplevel`, `XdgPopup`, and
|
||||
`ShellSurface` additionally implement `DesktopSurfaceRole`, which bridges
|
||||
them to a `DesktopSurface` (`FloatingDesktopSurface` or
|
||||
`AlwaysFullScreenDesktopSurface`, chosen by `session.config.mode`). The
|
||||
`DesktopSurface` is where **every** `userShell.events.*` emission
|
||||
originates — making it the true integration boundary.
|
||||
|
||||
## XdgToplevel lifecycle — the headline diagram (same as Greenfield)
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ wl_surface created (no role, no buffer yet) │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│ client: xdg_wm_base.get_xdg_surface
|
||||
│ (errors if surface already has buffer)
|
||||
▼
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ XdgSurface created │
|
||||
│ • surface.hasKeyboard/Pointer/Touch = true │
|
||||
│ • configured=false, configureList=[] │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│ client: xdg_surface.get_toplevel
|
||||
│ (errors if surface has another role)
|
||||
▼
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ XdgToplevel created │
|
||||
│ • surface.role = xdgToplevel │
|
||||
│ • View created │
|
||||
│ • DesktopSurface created + init() │
|
||||
│ • ackConfigure override installed │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│ client: xdg_toplevel.set_title /
|
||||
│ set_app_id / set_min/max_size /
|
||||
│ set_parent (stored, propagated)
|
||||
▼
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ First wl_surface.commit (buffer === undefined) │
|
||||
│ XdgToplevel.onCommit() → ensureAdded() │
|
||||
│ • desktopSurface.add() │
|
||||
│ → userShell.events.surfaceCreated(...) │
|
||||
│ • scheduleConfigure(first) │
|
||||
│ → microtask → sendConfigure() │
|
||||
│ → resource.configure(w,h,states) │
|
||||
│ • added=true │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│ client receives configure
|
||||
│ client: xdg_surface.ack_configure(serial)
|
||||
▼
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ ackConfigure(serial, configure) │
|
||||
│ • handleAckConfigure trims older serials │
|
||||
│ • configured=true │
|
||||
│ • next.state = configure.state │
|
||||
│ • next.size = configure.size │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│ client: wl_surface.commit (with buffer)
|
||||
▼
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ ★ MAPPED (steady state) ★ │
|
||||
│ onCommit(): │
|
||||
│ • surface.commitPending() (state latch) │
|
||||
│ • xdgSurface.commit() (geometry latch) │
|
||||
│ • current.state ← next.state │
|
||||
│ • current.min/max ← next.min/max │
|
||||
│ • desktopSurface.commit() → render() │
|
||||
└─────────────────────┬─────────────────────────┘
|
||||
│
|
||||
┌────────────────────────────────────────┼────────────────────────────┐
|
||||
│ │ │
|
||||
user/shell requests client requests destruction
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────────────────┐ ┌─────────────────────────────┐ ┌──────────────────────┐
|
||||
│ configureMaximized(b) │ │ setMaximized / unsetMaximized│ │ wl_surface.destroy │
|
||||
│ configureFullscreen(b) │ │ setFullscreen/unsetFullscreen│ │ or xdg_toplevel. │
|
||||
│ configureActivated(b) │ │ setMinimized │ │ destroy │
|
||||
│ configureResizing(b) │ │ setParent / setTitle / │ ├──────────────────────┤
|
||||
│ configureSize(w,h) │ │ setAppId │ │ destroy listener: │
|
||||
└────────────┬─────────────┘ │ move / resize / showWindowMenu│ │ • surface.unmap() │
|
||||
│ └──────────────┬──────────────┘ │ • desktopSurface. │
|
||||
│ each mutates │ │ removed() │
|
||||
│ pending.state / pending.size │ │ → surfaceDestroyed │
|
||||
▼ │ │ • view.destroy() │
|
||||
┌──────────────────────────┐ │ └──────────────────────┘
|
||||
│ scheduleConfigure( │ │
|
||||
│ stateCompareEquals(), │ ┌─────────────────────────────┐
|
||||
│ sendConfigure) │ │ setMinimized → │
|
||||
│ • coalesce via µtask │ │ desktopSurface.minimize() │
|
||||
│ • if pending==current → │ │ • view.mapped=false │
|
||||
│ skip │ │ • seat drops keyboard focus │
|
||||
│ • else: push serial, │ │ if this was the focused │
|
||||
│ emit configure event │ │ surface │
|
||||
│ │ └─────────────────────────────┘
|
||||
└────────────┬─────────────┘
|
||||
│
|
||||
│ cycle repeats: ack_configure → commit → render
|
||||
└────────────────────────────────────► back to ★ MAPPED ★
|
||||
```
|
||||
|
||||
## Configure / ack-configure round-trip (REUSED semantics)
|
||||
|
||||
The most important mechanism in xdg-shell. Async protocol-level sync:
|
||||
|
||||
1. Compositor mutates `pending.{state,size}`.
|
||||
2. Coalesces via `queueCancellableMicrotask` with a `stateCompareEquals`
|
||||
short-circuit (if pending == current, skip).
|
||||
3. Emits `resource.configure(w,h,states)` with a fresh serial.
|
||||
4. Client acks: `xdg_surface.ack_configure(serial)`.
|
||||
5. `handleAckConfigure` trims older serials, sets `configured=true`,
|
||||
copies `configure.{state,size}` into `next`.
|
||||
6. Next commit latches `next → current`.
|
||||
|
||||
The `configured` flag gates buffer attachment: an `unconfiguredBuffer`
|
||||
error is raised if the client tries to attach a buffer before acking.
|
||||
|
||||
## XdgPositioner math (REUSED semantics)
|
||||
|
||||
`anchor` selects a point on the anchor rect (in parent window-geometry
|
||||
space); `gravity` selects which corner/edge of the popup is placed at
|
||||
that point; `offset` is applied last.
|
||||
|
||||
### Constraint adjustment cascade (X-then-Y)
|
||||
|
||||
1. Try **flip** — swap anchor+gravity via `inverseX` / `inverseY` tables,
|
||||
revert if still violating.
|
||||
2. Try **slide** — shift offset by violation delta.
|
||||
3. Try **resize** — shrink `sizeRect`.
|
||||
|
||||
Slides/flips are skipped if popup size on that axis exceeds viewport.
|
||||
|
||||
## Popup grabs (REUSED semantics)
|
||||
|
||||
`XdgPopup.grab` rejects if:
|
||||
- Surface is already mapped, OR
|
||||
- Parent isn't either:
|
||||
- a toplevel (root of new chain), OR
|
||||
- the topmost popup of an existing grab (`notTheTopmostPopup` error)
|
||||
|
||||
`seat.popupGrabStart` validates the serial against the keyboard/pointer
|
||||
`grabSerial`, then starts a `PopupGrab` on both devices. `popupGrabEnd`
|
||||
walks the chain in reverse calling `requestClose()` on each (sends
|
||||
`popup_done` / `xdg_popup.popup_done`).
|
||||
|
||||
## Subsurfaces sync/desync (REUSED semantics)
|
||||
|
||||
Three-layer mechanism:
|
||||
1. **`cachedState`** — full pending-state shadow on the subsurface role.
|
||||
2. **`commitCache`** — flush on parent commit.
|
||||
3. **`effectiveSync`** getter — cascades: if any ancestor is sync, this
|
||||
subsurface is effectively sync.
|
||||
|
||||
### Decision matrix
|
||||
|
||||
| Sync mode | Dirty | Action |
|
||||
|---|---|---|
|
||||
| Sync | yes | Cache to `cachedState`, flush when parent commits |
|
||||
| Sync | no | No-op (already cached) |
|
||||
| Desync | yes | Flush immediately to parent's pending state |
|
||||
| Desync | no | Acts like a normal surface (commit directly to its own pendingState) |
|
||||
|
||||
`setDesync` immediately calls `onCommit` to flush any pending cache.
|
||||
|
||||
## wl_shell (legacy path, OPTIONAL)
|
||||
|
||||
Legacy path. Reduced feature set. State machine via `SurfaceStates` enum
|
||||
(MAXIMIZED/FULLSCREEN/POPUP/TRANSIENT/TOP_LEVEL) and `changeState()`.
|
||||
Per-surface ping (separate from `XdgWmBase`'s per-client ping).
|
||||
|
||||
**Recommendation**: skip `wl_shell` in MVP — modern GTK4/Qt6 apps use
|
||||
xdg-shell exclusively. Only needed for very old apps.
|
||||
|
||||
## The UserShellApi seam — clean-room spec for Odoo 20 OWL shell
|
||||
|
||||
This is the integration point where the Odoo 20 OWL shell hooks in.
|
||||
Designed to match Greenfield's API for compatibility (since it's a
|
||||
reasonable API), but written from spec.
|
||||
|
||||
### Events (compositor → shell) — all optional
|
||||
|
||||
| Event | Payload | Fired from | When |
|
||||
|---|---|---|---|
|
||||
| `clientCreated` | `CompositorClient` | `WebConnectionHandler`, `RemoteAppLauncher` | A new app client connects. |
|
||||
| `clientDestroyed` | `CompositorClient` | same | A client disconnects. |
|
||||
| `clientUnresponsiveUpdated` | `(client, unresponsive: boolean)` | `XdgWmBase`, `ShellSurface` | Ping (2s timeout, every 5s) — `true` = hung, `false` = recovered. |
|
||||
| `surfaceCreated` | `CompositorSurface` | `FloatingDesktopSurface.add()`, `AlwaysFullScreenDesktopSurface` | Toplevel first added to desktop. |
|
||||
| `surfaceDestroyed` | `CompositorSurface` | `.removed()` | Window destroyed. |
|
||||
| `surfaceTitleUpdated` | `(surface, title)` | `.setTitle()` | `xdg_toplevel.set_title`. |
|
||||
| `surfaceAppIdUpdated` | `(surface, appId)` | `.setAppId()` | `xdg_toplevel.set_app_id`. |
|
||||
| `surfaceActivationUpdated` | `(surface, active: boolean)` | `gainFocus()` / `loseFocus()` | Refcounted via `focusCount` — only fires on 0↔1 transitions. |
|
||||
| `notify` | `(variant, message)` | (declared, free-form) | Free-form notification channel. |
|
||||
| `sceneRefreshed` | `sceneId: string` | `Scene` | Render finished. |
|
||||
| **NEW: `surfaceStateChanged`** | `(surface, state)` | `setMaximized`/`setFullscreen`/etc. | Window state transitions (maximized/fullscreen/minimized/resizing). Greenfield lacks this — Odoo shell needs it for window decorations. |
|
||||
|
||||
### Actions (shell → compositor) — provided by default
|
||||
|
||||
| Action | Effect |
|
||||
|---|---|
|
||||
| `initScene(canvasCreator)` | Bootstraps a WebGL2 canvas + input/output. One per output. |
|
||||
| `refreshScene()` | `session.renderer.render()` — manual redraw. |
|
||||
| `destroyScene(sceneId)` | Destroys a specific scene. |
|
||||
| `setUserConfiguration(config)` | Updates `scrollFactor`, `keyboardLayoutName`. |
|
||||
| `closeClient({ id })` | Closes a client connection. |
|
||||
| `activateSurface(compositorSurface)` | **Click-to-focus entry point from the shell's taskbar.** |
|
||||
| **NEW: `querySurfaceState(compositorSurface)`** | Returns current surface state (maximized/fullscreen/minimized/resizing/activated). Greenfield lacks this. |
|
||||
| **NEW: `requestKeyframe(compositorSurface)`** | Triggers an immediate keyframe request for the encoder. Greenfield has plumbing but doesn't call it. |
|
||||
|
||||
### Opaque handle types
|
||||
|
||||
```typescript
|
||||
interface CompositorSurface { id: number; client: CompositorClient }
|
||||
interface CompositorClient { id: string }
|
||||
```
|
||||
|
||||
## Notable gaps addressed (vs Greenfield)
|
||||
|
||||
1. **No maximize/fullscreen/resize/minimize events in Greenfield's
|
||||
UserShellApi** — the shell can't directly observe window-state
|
||||
transitions through this API. **Our MIT approach adds
|
||||
`surfaceStateChanged` event** so the Odoo shell can render window
|
||||
decorations correctly.
|
||||
2. **No geometry/position event in Greenfield** — the shell can't render
|
||||
a frame that hugs surface bounds. **Our MIT approach can extend the API
|
||||
later** if needed (currently out of MVP scope).
|
||||
3. **`notify` declared but unused in Greenfield's compositor core** — dead
|
||||
interface slot. **Our MIT approach implements it** as a free-form
|
||||
notification channel (e.g. for "App not responding" warnings).
|
||||
4. **`surfaceCreated` fires once on first commit; no separate map/unmap
|
||||
signal in Greenfield** — minimization is internal (`view.mapped=false`)
|
||||
and not propagated. **Our MIT approach adds `surfaceMapChanged` event**
|
||||
if needed for taskbar indicators.
|
||||
|
||||
## Risks flagged (same as Greenfield)
|
||||
|
||||
- `XdgWmBase.doPing` should use a proper serial (Greenfield uses 0 with a
|
||||
FIXME). Our clean-room implementation should generate proper serials.
|
||||
- `XdgToplevel.onCommit` should validate buffer size vs configured state
|
||||
(Greenfield has commented-out validation). Our implementation should
|
||||
enforce this.
|
||||
- `XdgSurface.scheduleConfigure`'s `pendingSame` argument inverts meaning
|
||||
depending on whether a microtask is already queued — subtle. Our spec
|
||||
should make this explicit and well-tested.
|
||||
|
||||
## File reference
|
||||
|
||||
### Custom (clean-room TS)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/roles/XdgWmBase.ts` | `xdg_wm_base` global, ping-based unresponsiveness detection |
|
||||
| `packages/compositor/src/roles/XdgSurface.ts` | `xdg_surface` role, configure/ack round-trip |
|
||||
| `packages/compositor/src/roles/XdgToplevel.ts` | `xdg_toplevel` role, window state |
|
||||
| `packages/compositor/src/roles/XdgPopup.ts` | `xdg_popup` role, transient windows, grabs |
|
||||
| `packages/compositor/src/roles/XdgPositioner.ts` | `xdg_positioner`, popup positioning math |
|
||||
| `packages/compositor/src/roles/Shell.ts` | `wl_shell` global (legacy, optional) |
|
||||
| `packages/compositor/src/roles/ShellSurface.ts` | `wl_shell_surface` role (legacy, optional) |
|
||||
| `packages/compositor/src/subsurface/Subcompositor.ts` | `wl_subcompositor` global |
|
||||
| `packages/compositor/src/subsurface/Subsurface.ts` | `wl_subsurface` role, sync/desync mode |
|
||||
| `packages/compositor/src/subsurface/SurfaceRole.ts` | SurfaceRole interface |
|
||||
| `packages/compositor/src/subsurface/SurfaceChild.ts` | Surface tree child abstraction |
|
||||
| `packages/compositor/src/desktop/Desktop.ts` | Floating vs AlwaysFullScreen modes |
|
||||
| `packages/compositor/src/userapi/UserShellApi.ts` | The integration seam — events + actions |
|
||||
@@ -0,0 +1,522 @@
|
||||
# 09 — Hardware Acceleration, Encoding, WebCodecs, GPU Textures
|
||||
|
||||
This chapter covers the encoding/decoding/WASM pipeline in the MIT
|
||||
approach. **Critical changes from Greenfield**:
|
||||
- **No x264enc** (GPL-2.0+) — replaced with `vaapih264enc`/`nvh264enc`/`openh264enc`
|
||||
- **No FFmpeg Wasm** (GPL-3.0 in Greenfield's build) — replaced with WebCodecs primary + OpenH264 BSD fallback
|
||||
- **Hardware decode enabled** (Greenfield disables it with a FIXME)
|
||||
- **Adaptive bitrate** (consumes `EncoderFeedback` data Greenfield produces but doesn't use)
|
||||
|
||||
## What's reused (MIT)
|
||||
|
||||
### Browser side
|
||||
|
||||
| Component | License | Role |
|
||||
|---|---|---|
|
||||
| `@gfld/compositor-wasm` | MIT | pixman 0.42.2 + xkbcommon 1.5.0 WASM (region math + keymap) |
|
||||
|
||||
### Vendored C (MIT, used in Python proxy via cffi)
|
||||
|
||||
| File | Lines | Role |
|
||||
|---|---|---|
|
||||
| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM |
|
||||
| `westfield-egl.c` | ~900 | EGL zero-copy setup (EGLDisplay, EGLContext, EGLImage) |
|
||||
| `wlr_drm.c` | ~244 | `wl_drm` global impl |
|
||||
| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl |
|
||||
| `drm_format_set.c` | ~239 | DRM format helpers |
|
||||
|
||||
## What's replaced (proprietary, clean-room)
|
||||
|
||||
### Browser side
|
||||
|
||||
| Component | Replaces | Why |
|
||||
|---|---|---|
|
||||
| Custom `WebCodecsFrameDecoder.ts` | Greenfield's `webcodec-buffer-decoder.ts` (AGPL) | Clean-room; **enables hardware decode** (Greenfield disables with FIXME) |
|
||||
| Custom `OpenH264FrameDecoder.ts` | Greenfield's `wasm-buffer-decoder.ts` + `H264NALDecoder.worker.ts` (AGPL) + `@gfld/compositor-ffmpeg-h264` (GPL) | Clean-room; uses **OpenH264 BSD Wasm** instead of FFmpeg GPL Wasm |
|
||||
| Custom `YUVA2RGBA` / `RGBXandA2RGBA` shaders | Greenfield's (AGPL) | Clean-room; **WebGL2** (Greenfield uses WebGL1) |
|
||||
| Custom `Renderer` / `Scene` / `SceneShader` | Greenfield's (AGPL) | Clean-room; **OffscreenCanvas + Web Workers** (Greenfield is main-thread) |
|
||||
|
||||
### Python side
|
||||
|
||||
| Component | Replaces | Why |
|
||||
|---|---|---|
|
||||
| Custom `gst_encoder.py` (~700 LOC) | Greenfield's `proxy-encoding-addon.node` (AGPL, 290 lines C) + `libproxy-encoding.so` (AGPL, ~1400 lines C) | Pure Python via PyGObject. Same GStreamer, no N-API addon. |
|
||||
| `vaapih264enc` / `nvh264enc` / `openh264enc` | `x264enc` (GPL) | GPL-free H.264 encoding |
|
||||
|
||||
## Encoder selection (per-frame, deterministic)
|
||||
|
||||
Same logic as Greenfield, but **without x264enc**:
|
||||
|
||||
| name | element | when used | split_alpha | coded-size multiple | License |
|
||||
|---|---|---|---|---|---|
|
||||
| `vaapih264` | `vaapih264enc` | >256×256 + ARGB/XRGB/dmabuf, Intel/AMD GPU | true (parallel alpha H.264 stream) | 128 | LGPL + MIT (libva) |
|
||||
| `nvh264` | `nvh264enc` | same, NVIDIA GPU | true | 128 | BSD plugin + proprietary NVENC blob |
|
||||
| `openh264` | `openh264enc` | same, no GPU (CPU fallback) | true | 128 | BSD-2-Clause (libopenh264) |
|
||||
| `png` | `pngenc` | ≤256×256 + ARGB/XRGB/dmabuf | false (PNG has native alpha) | 16 | LGPL |
|
||||
|
||||
**Rule**: pixel_count ≤ 256×256 → PNG; otherwise the configured H.264
|
||||
encoder (selected by `--encoder=vaapih264|nvh264|openh264`).
|
||||
|
||||
**NOTE**: This means the **default encoder is hardware-accelerated**
|
||||
(vaapih264 on Intel/AMD, nvh264 on NVIDIA). Greenfield defaults to x264
|
||||
(software, GPL).
|
||||
|
||||
## Canonical H.264 pipeline (Python equivalent)
|
||||
|
||||
Same GStreamer pipeline as Greenfield, ported to PyGObject:
|
||||
|
||||
```python
|
||||
# gst_encoder.py (clean-room, ~700 LOC)
|
||||
import gi
|
||||
gi.require_version("Gst", "1.0")
|
||||
gi.require_version("GstGL", "1.0")
|
||||
gi.require_version("GstEGL", "1.0") # if available
|
||||
from gi.repository import Gst, GstGL, GLib
|
||||
|
||||
class H264Encoder:
|
||||
def __init__(self, egl_display_ptr, egl_context_ptr, encoder_type="vaapih264"):
|
||||
# Create GLDisplay + GLContext from existing EGL context (zero-copy)
|
||||
self.gl_display = GstGL.GLDisplayEGL.new_with_egl_display(egl_display_ptr)
|
||||
self.gl_context = GstGL.GLContext.new_wrapped(
|
||||
self.gl_display,
|
||||
egl_context_ptr,
|
||||
GstGL.GLPlatform.EGL,
|
||||
GstGL.GLLAPI.GLES2,
|
||||
)
|
||||
|
||||
# Encoder pipeline
|
||||
encoder_element = {
|
||||
"vaapih264": "vaapih264enc",
|
||||
"nvh264": "nvh264enc",
|
||||
"openh264": "openh264enc",
|
||||
}[encoder_type]
|
||||
|
||||
self.pipeline = Gst.parse_launch(
|
||||
f"appsrc name=src ! "
|
||||
f"glupload ! glcolorconvert ! "
|
||||
f"glshader name=scale ! " # pads to macroblock multiple
|
||||
f"glcolorconvert ! video/x-raw(memory:GLMemory),format=NV12 ! "
|
||||
f"gldownload ! queue ! "
|
||||
f"{encoder_element} tune=zerolatency bitrate=12000 vbv-buf-capacity=160 "
|
||||
f"bframes=0 rc-lookahead=0 sliced-threads byte-stream=true ! "
|
||||
f"video/x-h264,profile=high,stream-format=byte-stream,alignment=au ! "
|
||||
f"appsink name=sink"
|
||||
)
|
||||
|
||||
# Wire up appsink callback (zero-copy external Buffers)
|
||||
sink = self.pipeline.get_by_name("sink")
|
||||
sink.emit("set-property", "emit-signals", True)
|
||||
sink.connect("new-sample", self._on_new_sample)
|
||||
|
||||
# Alpha pipeline (separate, for transparent windows)
|
||||
self.alpha_pipeline = self._build_alpha_pipeline(encoder_element)
|
||||
|
||||
# GLib MainLoop in separate thread (GStreamer requires it)
|
||||
self.glib_loop = GLib.MainLoop()
|
||||
self.glib_context = GLib.MainContext.new()
|
||||
GLib.MainContext.push_thread_default(self.glib_context)
|
||||
import threading
|
||||
self.glib_thread = threading.Thread(target=self.glib_loop.run, daemon=True)
|
||||
self.glib_thread.start()
|
||||
|
||||
def encode_frame(self, buffer_id, content_serial, creation_serial, dma_buf_fd, w, h):
|
||||
# Push buffer into appsrc (zero-copy via GstBuffer wrapping the dmabuf)
|
||||
# ... marshalled from asyncio via GLib.idle_add
|
||||
|
||||
def _on_new_sample(self, appsink):
|
||||
sample = appsink.emit("pull-sample")
|
||||
buffer = sample.get_buffer()
|
||||
# Marshal back to asyncio
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self._deliver_frame(buffer, self.current_buffer_id),
|
||||
self.asyncio_loop,
|
||||
)
|
||||
```
|
||||
|
||||
### Bitrates (same as Greenfield)
|
||||
|
||||
- Opaque: 12000 (vaapih264 / nvh264 / openh264)
|
||||
- Alpha: 1200 (10% of opaque)
|
||||
|
||||
### Adaptive bitrate (NEW — Greenfield lacks)
|
||||
|
||||
Consume `EncoderFeedback` data (refreshInterval + decode duration):
|
||||
|
||||
```python
|
||||
# adaptive_bitrate.py (clean-room, ~150 LOC)
|
||||
class AdaptiveBitrate:
|
||||
def __init__(self, encoder):
|
||||
self.encoder = encoder
|
||||
|
||||
def on_feedback(self, refresh_interval_ms, avg_decode_ms):
|
||||
# If decode is taking too long, reduce bitrate
|
||||
if avg_decode_ms > refresh_interval_ms * 0.8:
|
||||
new_bitrate = max(self.current_bitrate * 0.7, MIN_BITRATE)
|
||||
self.encoder.set_property("bitrate", new_bitrate)
|
||||
# If decode is fast and we have headroom, increase bitrate
|
||||
elif avg_decode_ms < refresh_interval_ms * 0.3:
|
||||
new_bitrate = min(self.current_bitrate * 1.3, MAX_BITRATE)
|
||||
self.encoder.set_property("bitrate", new_bitrate)
|
||||
```
|
||||
|
||||
## Wire format (REUSED from Greenfield)
|
||||
|
||||
Same binary format as Greenfield (see `../greenfield-analysis/09-hardware-accel.md`):
|
||||
|
||||
```
|
||||
struct packed props {
|
||||
u32 buffer_id, creation_serial, content_serial,
|
||||
u32 frame_encoding_type (0=h264, 1=png),
|
||||
u32 width, height, coded_width, coded_height
|
||||
}
|
||||
u32 opaque_length
|
||||
u8 opaque[opaque_length] // H.264 AU or PNG blob
|
||||
u32 alpha_length // 0 if no alpha split
|
||||
u8 alpha[alpha_length] // H.264 AU of alpha-only stream
|
||||
```
|
||||
|
||||
## Browser decode (clean-room TS)
|
||||
|
||||
### Preferred path: WebCodecs `VideoDecoder` (with HW decode enabled)
|
||||
|
||||
```typescript
|
||||
// WebCodecsFrameDecoder.ts (clean-room, ~250 LOC)
|
||||
export class WebCodecsFrameDecoder implements FrameDecoder {
|
||||
private opaqueDecoders: Map<SurfaceId, VideoDecoder> = new Map();
|
||||
private alphaDecoders: Map<SurfaceId, VideoDecoder> = new Map();
|
||||
|
||||
async isSupported(): Promise<boolean> {
|
||||
const config = {
|
||||
codec: 'avc1.64001f',
|
||||
optimizeForLatency: true,
|
||||
// NEW: enable hardware acceleration (Greenfield disables this)
|
||||
hardwareAcceleration: 'prefer-hardware',
|
||||
};
|
||||
const result = await VideoDecoder.isConfigSupported(config);
|
||||
return result.supported;
|
||||
}
|
||||
|
||||
async decode(surface: Surface, encodedFrame: EncodedFrame): Promise<DecodedFrame> {
|
||||
let opaqueDecoder = this.opaqueDecoders.get(surface.id);
|
||||
if (!opaqueDecoder) {
|
||||
opaqueDecoder = new VideoDecoder({
|
||||
output: (frame) => this.onDecoded(surface, frame),
|
||||
error: (e) => this.onError(surface, e),
|
||||
});
|
||||
opaqueDecoder.configure({
|
||||
codec: 'avc1.64001f',
|
||||
optimizeForLatency: true,
|
||||
hardwareAcceleration: 'prefer-hardware', // NEW
|
||||
});
|
||||
this.opaqueDecoders.set(surface.id, opaqueDecoder);
|
||||
}
|
||||
|
||||
const chunk = new EncodedVideoChunk({
|
||||
type: encodedFrame.isKeyframe ? 'key' : 'delta',
|
||||
data: encodedFrame.opaque,
|
||||
timestamp: encodedFrame.timestamp,
|
||||
});
|
||||
opaqueDecoder.decode(chunk);
|
||||
|
||||
// Alpha path (if present)
|
||||
if (encodedFrame.alpha) {
|
||||
// Same pattern for alphaDecoder
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key difference from Greenfield**: We use `hardwareAcceleration:
|
||||
'prefer-hardware'` (Greenfield uses `'prefer-software'` with a FIXME
|
||||
about "frame offset is wrong"). This needs testing, but should give
|
||||
significant CPU savings.
|
||||
|
||||
### Fallback path: OpenH264 BSD Wasm (NOT FFmpeg — FFmpeg is GPL)
|
||||
|
||||
```typescript
|
||||
// OpenH264FrameDecoder.ts (clean-room, ~250 LOC)
|
||||
// Uses OpenH264 BSD-2-Clause Wasm build (Cisco's binary release or self-built)
|
||||
// Available from: https://github.com/cisco/openh264 (BSD-2-Clause)
|
||||
```
|
||||
|
||||
OpenH264 is BSD-2-Clause — fully proprietary-compatible. Less featureful
|
||||
than FFmpeg (no VP9/AV1), but sufficient for H.264 fallback.
|
||||
|
||||
## WebGL2 upload (clean-room TS, modernized)
|
||||
|
||||
WebGL2 with `desynchronized:true` + OffscreenCanvas in Web Worker.
|
||||
|
||||
Per Scene (canvas/output), three GL helpers (same as Greenfield, but
|
||||
WebGL2 instead of WebGL1):
|
||||
|
||||
### `YUVA2RGBA`
|
||||
|
||||
FBO pass: uploads Y/U/V/alphaY as **4× LUMINANCE textures** (WebGL1
|
||||
compatible) OR **1× R8UI texture array** (WebGL2, more efficient),
|
||||
renders into the surface's RGBA texture via `YUVA2RGBAShader` (BT.601
|
||||
coefficients).
|
||||
|
||||
### `RGBXandA2RGBA`
|
||||
|
||||
FBO pass for the `DualPlaneRGBA*` WebCodecs outputs (opaque + alpha
|
||||
planes composited via `RGBandA2RGBAShader`).
|
||||
|
||||
### `SceneShader`
|
||||
|
||||
Final pass: draws every view's RGBA texture to the canvas with
|
||||
`gl.blendFunc(ONE, ONE_MINUS_SRC_ALPHA)`.
|
||||
|
||||
**NEW**: WebGL2 `texStorage2D` for immutable textures (faster upload).
|
||||
**NEW**: `VideoFrame` directly as a WebGL texture source (zero-copy from
|
||||
WebCodecs to GPU, no `copyTo` needed).
|
||||
|
||||
## End-to-end pipeline
|
||||
|
||||
```
|
||||
┌──────────────────── NATIVE CLIENT PROCESS ────────────────────┐
|
||||
│ App renders via GTK/Qt/raw EGL into │
|
||||
│ ┌────────────────────────────────────┐ │
|
||||
│ │ wl_shm buffer (ARGB8888/XRGB8888)│ ─── or ─────────┐ │
|
||||
│ │ zwp_linux_dmabuf_v1 (DMA-BUF) │ │ │
|
||||
│ │ wl_drm create_prime_buffer │ │ │
|
||||
│ └────────────────────────────────────┘ │ │
|
||||
└────────────────────────┬──────────────────────────────────┼────┘
|
||||
│ Wayland socket (UNIX domain) │
|
||||
▼ │
|
||||
┌────────────────────── PYTHON PROXY (aiohttp + pywayland + GStreamer)─┐
|
||||
│ pywayland (Apache-2.0) │
|
||||
│ └─ Display → Client → Resource (buffer) │
|
||||
│ └─ display.init_shm(), init_drm("/dev/dri/renderD128") │
|
||||
│ └─ Vendored westfield-egl.c (MIT, via cffi): │
|
||||
│ EGLDevice + EGLDisplay + EGLContext │
|
||||
│ │
|
||||
│ gst_encoder.py (~700 LOC Python, clean-room) │
|
||||
│ ┌─ frame_buffer {type: SHM|DMA, w, h, fmt} │
|
||||
│ │ │
|
||||
│ │ GLib MainLoop in separate thread (GLib.idle_add) │
|
||||
│ │ │ │
|
||||
│ │ ▼ │
|
||||
│ │ gst_parse_launch: │
|
||||
│ │ ┌─ Encoder selection (per-frame): │
|
||||
│ │ │ pixel_count ≤ 256×256 ─► PNG pipeline │
|
||||
│ │ │ > 256×256 + ARGB/XRGB/dmabuf ─► configured h264 │
|
||||
│ │ │ │
|
||||
│ │ │ H.264 pipelines (parallel, NO x264 — GPL): │
|
||||
│ │ │ opaque: appsrc → glupload → glcolorconvert → │
|
||||
│ │ │ glshader(scale→coded_size) → │
|
||||
│ │ │ glcolorconvert(NV12) → gldownload → │
|
||||
│ │ │ queue → vaapih264enc|nvh264enc|openh264enc │
|
||||
│ │ │ → byte-stream AU → appsink │
|
||||
│ │ │ alpha: same source, fragment shader │
|
||||
│ │ │ `vec4(pix.a,pix.a,pix.a,0.)` → │
|
||||
│ │ │ second H.264 stream │
|
||||
│ │ │ │
|
||||
│ │ │ PNG pipeline (single): │
|
||||
│ │ │ appsrc → glupload → glcolorconvert → glshader → │
|
||||
│ │ │ RGBA → gldownload → pngenc → appsink │
|
||||
│ │ │ │
|
||||
│ │ └─ asyncio.run_coroutine_threadsafe for cross-thread │
|
||||
│ │ │
|
||||
│ └─ surface_buffer_encoding.py │
|
||||
│ └─ frame_data_channel.send(buffer) │
|
||||
│ │
|
||||
│ channel.py: │
|
||||
│ ARQChannel (kcp PyPI package, MIT, over WebSocket) │
|
||||
│ SimpleChannel (raw WebSocket, for FEEDBACK only) │
|
||||
│ ▲ │
|
||||
│ │ frame_feedback.py: 16-bit rAF interval + 16-bit avg │
|
||||
│ │ decode duration → used to pace wl_frame.done AND │
|
||||
│ │ drive adaptive_bitrate.py (NEW vs Greenfield) │
|
||||
└───────┼───────────────────────────────────────────────────────┘
|
||||
│ 4 logical channels per client: PROTOCOL / FRAME / XWM / FEEDBACK
|
||||
▼
|
||||
┌────────────────────── BROWSER (Custom Compositor TS, clean-room) ──────┐
|
||||
│ WebSocket → ReconnectingWebSocket (vendored MIT) │
|
||||
│ ↓ │
|
||||
│ channel.ts: ARQChannel.input(kcp) → onMessage │
|
||||
│ ↓ │
|
||||
│ RemoteAppLauncher.setupFrameDataChannel → │
|
||||
│ BufferStream.deliverContentToBufferStream (parse serials) │
|
||||
│ ↓ │
|
||||
│ StreamingBuffer.getContents(surface, serial) │
|
||||
│ ↓ │
|
||||
│ Session.frameDecoder.decode(surface, encodedFrame) │
|
||||
│ │ │
|
||||
│ ├─ PNG path (both decoders): │
|
||||
│ │ new Blob([opaque], {type:'image/png'}) → │
|
||||
│ │ createImageBitmap(blob, sx, sy, w, h) // crop padding │
|
||||
│ │ → SinglePlane { bitmap: ImageBitmap } │
|
||||
│ │ │
|
||||
│ └─ H.264 path: │
|
||||
│ (preferred) WebCodecs VideoDecoder │
|
||||
│ config: avc1.64001f, prefer-hardware (NEW!), │
|
||||
│ optimizeForLatency │
|
||||
│ opaqueDecoder + alphaDecoder per surface │
|
||||
│ EncodedVideoChunk {type:key|delta, data:nal} │
|
||||
│ → VideoFrame.output → │
|
||||
│ if I420: copyTo → DualPlaneYUVAArrayBuffer │
|
||||
│ else: keep VideoFrame → DualPlaneRGBAVideoFrame │
|
||||
│ (fallback) OpenH264 BSD Wasm in 2 Web Workers │
|
||||
│ (NOT FFmpeg — GPL contamination) │
|
||||
│ worker.postMessage(nal, [nal.buffer]) // transferable │
|
||||
│ OpenH264._decode → I420 planes │
|
||||
│ postMessage back with [y.buffer, u.buffer, v.buffer] │
|
||||
│ → DualPlaneYUVASplitBuffer │
|
||||
│ │
|
||||
│ Renderer.render() (rAF-driven, in Web Worker via OffscreenCanvas)│
|
||||
│ ↓ │
|
||||
│ Scene.render(viewStack) │
|
||||
│ ├─ Scene['image/png'] → texImage2D(ImageBitmap) │
|
||||
│ ├─ Scene['image/bitmap'] (web-bitmapbuf path) → texImage2D │
|
||||
│ └─ Scene['video/h264']: │
|
||||
│ YUVA2RGBA.convertYUVASplitBufferInto / │
|
||||
│ convertYUVAArrayBufferInto: │
|
||||
│ Y/U/V/alphaY → 4× LUMINANCE textures (WebGL1 compat) │
|
||||
│ OR 1× R8UI texture array (WebGL2, more efficient) │
|
||||
│ bind FBO → renderState.texture (RGBA) as color attach │
|
||||
│ YUVA2RGBAShader (BT.601) → drawArrays → RGBA in tex │
|
||||
│ or RGBXandA2RGBA.convertInto (WebCodecs RGBA): │
|
||||
│ VideoFrame/ImageBitmap → texImage2D directly │
|
||||
│ ↓ │
|
||||
│ SceneShader.draw(): drawArrays with surface's RGBA texture, │
|
||||
│ blend gl.ONE / gl.ONE_MINUS_SRC_ALPHA, desynchronized canvas │
|
||||
│ ↓ │
|
||||
│ Browser composites canvas into the page │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
|
||||
|
||||
Same-frame web app fast path (web-bitmapbuf, REUSED):
|
||||
┌───────────────┐ MessagePort.postMessage([view, ...fds], ┌──────────────┐
|
||||
│ Web app │ ──► [imageBitmap]) ──────────────────────────► │ Compositor │
|
||||
│ (iframe) │ web_bitmapbuf_factory.create_buffer(id, fd) │ WebConnHandl │
|
||||
└───────────────┘ → wl_buffer → ImageBitmapBuffer └──────────────┘
|
||||
(no encoding, no WebSocket, no GStreamer)
|
||||
```
|
||||
|
||||
## Key answers (quick reference)
|
||||
|
||||
### Why H.264 over PNG?
|
||||
|
||||
H.264 at 12 Mbps × 1/60s ≈ 25 KB/frame for 1080p, vs PNG 100 KB – 3 MB.
|
||||
PNG wins for ≤256×256 because H.264 I-frame overhead and alpha-split
|
||||
doubling dominate at small sizes.
|
||||
|
||||
### WebCodecs?
|
||||
|
||||
Yes — preferred path. **Hardware decode enabled** (Greenfield disables
|
||||
with FIXME). OpenH264 BSD Wasm is the fallback (NOT FFmpeg — GPL).
|
||||
|
||||
### Trade-offs?
|
||||
|
||||
- H.264 = small wire / decode cost
|
||||
- PNG = medium wire / browser-native fast decode (`createImageBitmap`)
|
||||
- raw SHM = **not used** between proxy and browser — every surface commit
|
||||
is encoded.
|
||||
|
||||
### Per-surface policy?
|
||||
|
||||
No — per-frame deterministic. Only configurable knob is
|
||||
`--encoder=vaapih264|nvh264|openh264`. PNG is always automatic fallback.
|
||||
|
||||
### Cursors?
|
||||
|
||||
Always PNG (32×32 ≤ 256×256). Decoded to ImageBitmap and set as the CSS
|
||||
cursor via `setBrowserCursor` — never enters the WebGL compositor.
|
||||
|
||||
### Alpha pre-multiplication?
|
||||
|
||||
Implicit in shaders (same as Greenfield). Works acceptably for typical
|
||||
Wayland content.
|
||||
|
||||
### OffscreenCanvas / Web Workers?
|
||||
|
||||
**Yes** — Render Worker (WebGL2 + OffscreenCanvas) + Decode Worker
|
||||
(WebCodecs VideoDecoder). Greenfield uses only 2 workers for the WASM
|
||||
fallback path; our MIT approach uses Workers for ALL rendering.
|
||||
|
||||
### Adaptive bitrate (NEW)?
|
||||
|
||||
**Yes** — `adaptive_bitrate.py` consumes `EncoderFeedback` data and
|
||||
adjusts `vaapih264enc`/`nvh264enc`/`openh264enc` bitrate at runtime.
|
||||
Greenfield produces the feedback data but doesn't use it.
|
||||
|
||||
## Notable design trade-offs for the MIT approach
|
||||
|
||||
1. **Hardware-accelerated encoding by default** (vaapih264 on Intel/AMD,
|
||||
nvh264 on NVIDIA). Greenfield defaults to x264 (software, GPL).
|
||||
2. **Hardware-accelerated decoding by default** (WebCodecs
|
||||
`prefer-hardware`). Greenfield uses `prefer-software` with a FIXME.
|
||||
3. **No GPL anywhere in the encoding/decoding pipeline** (no x264, no
|
||||
FFmpeg Wasm).
|
||||
4. **Adaptive bitrate** consumes the feedback data Greenfield ignores.
|
||||
5. **OpenH264 BSD Wasm** as fallback (Cisco's binary release or
|
||||
self-built). Sufficient for H.264 fallback when WebCodecs is
|
||||
unavailable.
|
||||
6. **OffscreenCanvas + Web Workers** for all rendering (Greenfield is
|
||||
main-thread).
|
||||
7. **WebGL2** instead of WebGL1 (uniform buffer objects, simplified
|
||||
shaders).
|
||||
8. **VideoFrame directly as WebGL texture source** (zero-copy from
|
||||
WebCodecs to GPU, no `copyTo` needed).
|
||||
9. **Same alpha-split H.264 approach** as Greenfield (doubles cost for
|
||||
transparent windows) — could be replaced with AV1 with native alpha
|
||||
support in the future.
|
||||
10. **Future: AV1/VP9 support** via WebCodecs (when browser support is
|
||||
widespread) — sidesteps H.264 MPEG-LA patent royalties entirely.
|
||||
|
||||
## File reference
|
||||
|
||||
### Browser side (custom, clean-room TS)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/transport/decoders/WebCodecsFrameDecoder.ts` | WebCodecs VideoDecoder (HW-accelerated) |
|
||||
| `packages/compositor/src/transport/decoders/OpenH264FrameDecoder.ts` | OpenH264 BSD Wasm fallback |
|
||||
| `packages/compositor/src/transport/decoders/FrameDecoder.ts` | Decoder interface |
|
||||
| `packages/compositor/src/transport/EncodedFrame.ts` | Wire format parser for encoded frames |
|
||||
| `packages/compositor/src/transport/EncoderFeedback.ts` | Sends 4-byte decode feedback at 1 Hz |
|
||||
| `packages/compositor/src/transport/StreamingBuffer.ts` | Per-surface encoded-frame buffer |
|
||||
| `packages/compositor/src/transport/BufferStream.ts` | Manages serial → frame promise resolution |
|
||||
| `packages/compositor/src/render/Renderer.ts` | 4-phase render pipeline (WebGL2 + OffscreenCanvas) |
|
||||
| `packages/compositor/src/render/Scene.ts` | One per output, manages view stack + textures |
|
||||
| `packages/compositor/src/render/YUVA2RGBA.ts` | YUVA→RGBA FBO pass |
|
||||
| `packages/compositor/src/render/YUVA2RGBAShader.ts` | BT.601 YUVA→RGBA shader |
|
||||
| `packages/compositor/src/render/RGBXandA2RGBA.ts` | RGB+Alpha → RGBA FBO pass (WebCodecs output) |
|
||||
|
||||
### Browser side (REUSED MIT)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `libs/compositor-wasm/src/index.ts` | pixman + xkbcommon WASM (region math + keymap) |
|
||||
|
||||
### Python side (custom, clean-room)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `proxy/encoding/gst_encoder.py` | GStreamer H.264/PNG encoder (~700 LOC) |
|
||||
| `proxy/encoding/adaptive_bitrate.py` | Adaptive bitrate controller (NEW) |
|
||||
| `proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration |
|
||||
| `proxy/proxy_buffer.py` | Proxy-side buffer representation |
|
||||
| `proxy/frame_feedback.py` | Receives 4-byte decode feedback |
|
||||
| `proxy/encoding/glib_thread.py` | GLib MainLoop in separate thread + asyncio marshalling |
|
||||
|
||||
### Vendored Greenfield C files (MIT, used via cffi)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `native/westfield-egl.c` | EGL zero-copy setup |
|
||||
| `native/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM |
|
||||
| `native/wlr_drm.c` | `wl_drm` global impl |
|
||||
| `native/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl |
|
||||
| `native/drm_format_set.c` | DRM format helpers |
|
||||
|
||||
### Reused (LGPL/BSD/MIT)
|
||||
|
||||
| Component | Role |
|
||||
|---|---|
|
||||
| PyGObject (`gi.repository.Gst`) | GStreamer Python bindings (LGPL-2.1+) |
|
||||
| GStreamer core | Media pipeline framework (LGPL-2.0+) |
|
||||
| gst-plugins-base, gst-plugins-good | PNG, appsrc, appsink, videoconvert (LGPL) |
|
||||
| gst-vaapi | `vaapih264enc` for Intel/AMD (LGPL + MIT libva) |
|
||||
| gst-plugins-bad (BSD subset) | `nvh264enc` for NVIDIA, `openh264enc` for CPU fallback |
|
||||
| libopenh264 | OpenH264 BSD Wasm fallback decoder (BSD-2-Clause) |
|
||||
@@ -0,0 +1,659 @@
|
||||
# 10 — Odoo 20 Integration (OWL Shell + Backend Addon + JWT + Nginx + Docker)
|
||||
|
||||
This chapter covers the Odoo 20 integration layer in the MIT approach.
|
||||
**Critical finding**: Odoo's built-in WebSocket server is **unsuitable**
|
||||
for compositor-scale traffic. We need a separate aiohttp Python process
|
||||
for the compositor proxy.
|
||||
|
||||
## Why Odoo's built-in WebSocket server is unsuitable
|
||||
|
||||
Odoo does **NOT** use Werkzeug's WebSocket support (Werkzeug has no
|
||||
first-party WS). Instead, Odoo ships its own pure-Python RFC 6455
|
||||
implementation in `addons/bus/websocket.py` (~900 LOC) running in a
|
||||
**separate gevent worker process** on port 8072.
|
||||
|
||||
### Hard limits that disqualify it for compositor use
|
||||
|
||||
- **`MESSAGE_MAX_SIZE = 2**20 = 1 MB`** per frame (cannot be configured)
|
||||
— too small for H.264 video frames + multiplexed channels
|
||||
- **Shared PostgreSQL connection pool** — sustained WS streaming starves
|
||||
other Odoo requests
|
||||
- **Single gevent process** — one Python GIL for ALL Discuss/chat/presence
|
||||
traffic
|
||||
- `ir.websocket` only dispatches two events: `subscribe` and
|
||||
`update_presence`. New event types require forking
|
||||
`addons/bus/websocket.py`.
|
||||
- Odoo.sh user confirmed edits to `ir.websocket` need gevent-worker restart
|
||||
|
||||
### Conclusion
|
||||
|
||||
Run a **separate aiohttp Python process** (port 8081) for the compositor
|
||||
proxy. Route via Nginx for single-origin HTTPS.
|
||||
|
||||
## Recommended architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ User's Browser │
|
||||
│ ┌─────────────────────────────────────────┐ │
|
||||
│ │ Odoo web client + OWL Compositor Shell │ │
|
||||
│ │ (ir.actions.client, target=fullscreen) │ │
|
||||
│ │ ┌──────────────────────────────────┐ │ │
|
||||
│ │ │ <canvas> + WebGL compositor lib │ │ │
|
||||
│ │ │ (MIT-licensed browser compositor) │ │ │
|
||||
│ │ └──────────────────────────────────┘ │ │
|
||||
│ └─────────────────────────────────────────┘ │
|
||||
└────────────────────┬────────────────────────┘
|
||||
│
|
||||
HTTPS (single origin: https://your-odoo.example.com)
|
||||
session_id cookie + JWT (in JS payload)
|
||||
│
|
||||
┌────────────────────▼────────────────────────┐
|
||||
│ Nginx (TLS termination) │
|
||||
│ / → Odoo :8069 (HTTP workers) │
|
||||
│ /websocket → Odoo :8072 (gevent worker) │
|
||||
│ /compositor/ → Proxy :8081 (aiohttp WS) │
|
||||
└─────────┬──────────────────┬─────────────────┘
|
||||
│ │
|
||||
┌────────────────┴─────────┐ ┌────▼───────────────────┐
|
||||
│ Odoo 20 (HTTP) │ │ Odoo 20 (geevent) │
|
||||
│ N preforked workers │ │ 1 process │
|
||||
│ │ │ /websocket (Discuss, │
|
||||
│ Controllers: │ │ chatter, presence) │
|
||||
│ /web/compositor/token │ │ bus.bus notifications│
|
||||
│ /web/compositor/apps │ │ (NOT for compositor!)│
|
||||
│ /web/compositor/launch │ │ 1 MB cap, pgsql pool │
|
||||
│ /web/compositor/terminate│ └───────────────────────┘
|
||||
│ /web/compositor/audit │
|
||||
│ Models: │
|
||||
│ compositor.app │
|
||||
│ compositor.session │
|
||||
│ (mail.thread mixin) │
|
||||
│ compositor.audit.log │
|
||||
│ (append-only) │
|
||||
│ JWT issuer (HS256 secret│
|
||||
│ in ir.config_parameter)│
|
||||
└────────────┬─────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────────────────┐
|
||||
│ PostgreSQL (Odoo schema) │
|
||||
└────────────────────────────────────────────────────┘
|
||||
▲
|
||||
┌────────────────────────────────────────────────┴──┐
|
||||
│ Compositor Proxy (aiohttp, :8081) │
|
||||
│ Async Python (asyncio + aiohttp) │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ WS server (aiohttp.web.Application) │ │
|
||||
│ │ /signal, /channel, /mkfifo, /mkstemp-mmap,│ │
|
||||
│ │ /read-fd, /write-fd-as-stream, │ │
|
||||
│ │ /request-keyframe │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ JWT validator (pyjwt, HS256, secret env) │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Per-session fork: pywayland + bwrap + Gst │ │
|
||||
│ │ WAYLAND_DISPLAY=wayland-N, /dev/dri │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ Audit callback → Odoo /web/compositor/audit│ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Auth bridging — JWT (HS256) is recommended
|
||||
|
||||
### Why JWT (not session cookie or OAuth2)
|
||||
|
||||
- Odoo's `session_id` cookie reuse pattern is too fragile and adds Odoo round-trips per WS message
|
||||
- OAuth2 is overkill — we control both Odoo and the proxy
|
||||
- JWT (HS256, symmetric) is stateless — no DB lookup per frame, fast validation
|
||||
|
||||
### JWT payload
|
||||
|
||||
```json
|
||||
{
|
||||
"iss": "odoo.example.com",
|
||||
"aud": "compositor-proxy",
|
||||
"sub": 42, // Odoo user ID
|
||||
"db": "production", // Odoo database name
|
||||
"company_ids": [1, 3],
|
||||
"scope": "compositor",
|
||||
"compositor_session_id": "uuid-here",
|
||||
"iat": 1736000000,
|
||||
"exp": 1736000900
|
||||
}
|
||||
```
|
||||
|
||||
- 15-minute expiry
|
||||
- Signed with `compositor.jwt_secret` from Odoo's `ir.config_parameter`
|
||||
(shared via env var at proxy startup)
|
||||
- Validated by the proxy using `pyjwt` (MIT license)
|
||||
|
||||
### Issuance flow
|
||||
|
||||
1. OWL shell calls `/web/compositor/token` (Odoo controller)
|
||||
2. Odoo backend signs JWT with the shared secret
|
||||
3. Returns `{ jwt, proxy_url, expires_at }` to the OWL shell
|
||||
4. OWL shell passes JWT to the compositor library
|
||||
5. Compositor library includes `Authorization: Bearer <jwt>` on every WebSocket connection
|
||||
|
||||
## Multi-tenancy
|
||||
|
||||
### Recommended: one compositor proxy per Odoo database
|
||||
|
||||
- `db` claim in JWT scopes every WS to one Odoo database
|
||||
- Per-session wayland socket (`$XDG_RUNTIME_DIR/wayland-N`) isolates Wayland displays
|
||||
- Per-session forked worker process (matches Greenfield's existing model)
|
||||
- For SaaS with strict SLA: Docker container per tenant + bubblewrap per session
|
||||
|
||||
### Optional: per-tenant Docker containers
|
||||
|
||||
For strict isolation between tenants:
|
||||
- One Docker container per Odoo database, each running its own compositor proxy
|
||||
- Nginx routes based on the JWT's `db` claim
|
||||
- `--device /dev/dri:/dev/dri` for GPU passthrough
|
||||
- `--cpus` and `--memory` limits per container
|
||||
|
||||
## OWL shell integration — `ir.actions.client` with `target=fullscreen`
|
||||
|
||||
### Critical Odoo 20 / OWL 3.x gotcha
|
||||
|
||||
`useState()` is **REMOVED** in Odoo 20. Must use `proxy()` from `@odoo/owl`
|
||||
instead. This is a breaking change from Odoo 19, verified by Cybrosys
|
||||
Sep 2026 blog + OWL migration guide gist.
|
||||
|
||||
### OWL shell component (pseudocode)
|
||||
|
||||
```typescript
|
||||
// compositor_shell/static/src/components/compositor_shell.js
|
||||
import { Component } from "@odoo/owl";
|
||||
import { useService } from "@web/core/utils/hooks";
|
||||
import { createCompositorSession } from "@odoo/compositor/compositor_lib";
|
||||
|
||||
export class CompositorShell extends Component {
|
||||
setup() {
|
||||
this.rpc = useService("rpc");
|
||||
this.notification = useService("notification");
|
||||
this.state = proxy({
|
||||
surfaces: new Map(), // id → { title, appId, active }
|
||||
clients: new Map(),
|
||||
});
|
||||
this.canvasRef = useRef("canvas");
|
||||
|
||||
onMounted(async () => {
|
||||
// 1. Get JWT from Odoo backend
|
||||
const { jwt, proxy_url, expires_at } = await this.rpc("/web/compositor/token");
|
||||
this.jwt = jwt;
|
||||
this.proxyUrl = proxy_url;
|
||||
|
||||
// 2. Initialize compositor session
|
||||
const session = createCompositorSession({
|
||||
jwt,
|
||||
proxyUrl: proxy_url,
|
||||
});
|
||||
this.session = session;
|
||||
|
||||
// 3. Wire UserShellApi events to OWL reactive state
|
||||
session.userShell.events.clientCreated = (client) => {
|
||||
this.state.clients.set(client.id, { id: client.id });
|
||||
};
|
||||
session.userShell.events.clientDestroyed = (client) => {
|
||||
this.state.clients.delete(client.id);
|
||||
};
|
||||
session.userShell.events.surfaceCreated = (surface) => {
|
||||
this.state.surfaces.set(surface.id, {
|
||||
title: "", appId: "", active: false,
|
||||
});
|
||||
};
|
||||
session.userShell.events.surfaceDestroyed = (surface) => {
|
||||
this.state.surfaces.delete(surface.id);
|
||||
};
|
||||
session.userShell.events.surfaceTitleUpdated = (surface, title) => {
|
||||
const s = this.state.surfaces.get(surface.id);
|
||||
if (s) s.title = title;
|
||||
};
|
||||
session.userShell.events.surfaceActivationUpdated = (surface, active) => {
|
||||
// Deactivate all, then activate the new one
|
||||
for (const s of this.state.surfaces.values()) {
|
||||
s.active = false;
|
||||
}
|
||||
const s = this.state.surfaces.get(surface.id);
|
||||
if (s) s.active = active;
|
||||
};
|
||||
// NEW vs Greenfield: surface state changes
|
||||
session.userShell.events.surfaceStateChanged = (surface, state) => {
|
||||
const s = this.state.surfaces.get(surface.id);
|
||||
if (s) Object.assign(s, state);
|
||||
};
|
||||
|
||||
// 4. Initialize the canvas (OWL owns the lifecycle)
|
||||
const canvas = this.canvasRef.el;
|
||||
session.userShell.actions.initScene(() => ({
|
||||
canvas,
|
||||
width: canvas.clientWidth,
|
||||
height: canvas.clientHeight,
|
||||
}));
|
||||
});
|
||||
}
|
||||
|
||||
// User clicks app in launcher
|
||||
async launchApp(appId) {
|
||||
try {
|
||||
const response = await this.rpc("/web/compositor/launch", {
|
||||
app_id: appId,
|
||||
});
|
||||
// Backend returns { signal_url, pid, name }
|
||||
// The compositor library handles the rest (opens WS, etc.)
|
||||
} catch (e) {
|
||||
this.notification.add("Failed to launch app: " + e.message, { type: "danger" });
|
||||
}
|
||||
}
|
||||
|
||||
// User clicks taskbar entry
|
||||
activateSurface(surfaceId) {
|
||||
this.session.userShell.actions.activateSurface({ id: surfaceId });
|
||||
}
|
||||
|
||||
// User closes window
|
||||
closeSurface(surfaceId) {
|
||||
this.session.userShell.actions.closeSurface({ id: surfaceId });
|
||||
}
|
||||
}
|
||||
|
||||
CompositorShell.template = "compositor_shell.CompositorShell";
|
||||
CompositorShell.target = "fullscreen"; // Odoo ir.actions.client target
|
||||
```
|
||||
|
||||
### Odoo action registration
|
||||
|
||||
```xml
|
||||
<!-- compositor_shell/views/ir_actions_client_views.xml -->
|
||||
<record id="action_compositor_shell" model="ir.actions.client">
|
||||
<field name="name">Compositor Shell</field>
|
||||
<field name="tag">compositor_shell</field>
|
||||
<field name="target">fullscreen</field>
|
||||
</record>
|
||||
```
|
||||
|
||||
## Backend addon structure
|
||||
|
||||
### Models
|
||||
|
||||
```python
|
||||
# compositor_shell/models/compositor_app.py
|
||||
from odoo import models, fields, api
|
||||
|
||||
class CompositorApp(models.Model):
|
||||
_name = "compositor.app"
|
||||
_description = "Compositor Application"
|
||||
|
||||
name = fields.Char(required=True, translate=True)
|
||||
app_id = fields.Char(required=True, help="Unique identifier, e.g. 'gnome-calculator'")
|
||||
executable = fields.Char(required=True, help="Binary path, e.g. '/usr/bin/gnome-calculator'")
|
||||
icon = fields.Image()
|
||||
description = fields.Text()
|
||||
|
||||
allowed_group_ids = fields.Many2many("res.groups", string="Allowed Groups")
|
||||
memory_limit_mb = fields.Integer(default=1024, help="Max memory per app instance (MB)")
|
||||
cpu_quota_percent = fields.Integer(default=200, help="CPU quota (100% = 1 core)")
|
||||
|
||||
active = fields.Boolean(default=True)
|
||||
```
|
||||
|
||||
```python
|
||||
# compositor_shell/models/compositor_session.py
|
||||
from odoo import models, fields, api
|
||||
|
||||
class CompositorSession(models.Model):
|
||||
_name = "compositor.session"
|
||||
_description = "Compositor Session"
|
||||
_inherit = ["mail.thread"]
|
||||
|
||||
user_id = fields.Many2one("res.users", required=True, tracking=True)
|
||||
compositor_session_id = fields.Char(required=True, help="UUID")
|
||||
|
||||
app_ids = fields.One2many("compositor.app.instance", "session_id", string="Apps")
|
||||
app_count = fields.Integer(compute="_compute_app_count")
|
||||
|
||||
state = fields.Selection([
|
||||
("active", "Active"),
|
||||
("disconnected", "Disconnected"),
|
||||
("closed", "Closed"),
|
||||
], default="active", tracking=True)
|
||||
|
||||
started_at = fields.Datetime(default=fields.Datetime.now)
|
||||
ended_at = fields.Datetime()
|
||||
|
||||
@api.depends("app_ids")
|
||||
def _compute_app_count(self):
|
||||
for record in self:
|
||||
record.app_count = len(record.app_ids)
|
||||
```
|
||||
|
||||
```python
|
||||
# compositor_shell/models/compositor_audit_log.py
|
||||
from odoo import models, fields
|
||||
|
||||
class CompositorAuditLog(models.Model):
|
||||
_name = "compositor.audit.log"
|
||||
_description = "Compositor Audit Log (append-only)"
|
||||
_order = "create_date desc"
|
||||
|
||||
user_id = fields.Many2one("res.users", required=True)
|
||||
app_id = fields.Char(required=True)
|
||||
action = fields.Selection([
|
||||
("launch", "App Launched"),
|
||||
("terminate", "App Terminated"),
|
||||
("crash", "App Crashed"),
|
||||
], required=True)
|
||||
pid = fields.Integer()
|
||||
exit_code = fields.Integer()
|
||||
timestamp = fields.Datetime(default=fields.Datetime.now)
|
||||
|
||||
def write(self, vals):
|
||||
raise UserError("Audit logs are append-only and cannot be modified.")
|
||||
|
||||
def unlink(self):
|
||||
raise UserError("Audit logs are append-only and cannot be deleted.")
|
||||
```
|
||||
|
||||
### Controllers
|
||||
|
||||
```python
|
||||
# compositor_shell/controllers/main.py
|
||||
from odoo import http
|
||||
from odoo.http import request
|
||||
import jwt, datetime, uuid
|
||||
|
||||
class CompositorController(http.Controller):
|
||||
|
||||
@http.route("/web/compositor/token", type="json", auth="user")
|
||||
def get_token(self):
|
||||
user = request.env.user
|
||||
secret = request.env["ir.config_parameter"].sudo().get_param("compositor.jwt_secret")
|
||||
if not secret:
|
||||
raise UserError("Compositor JWT secret not configured")
|
||||
|
||||
compositor_session_id = str(uuid.uuid4())
|
||||
payload = {
|
||||
"iss": request.http.host,
|
||||
"aud": "compositor-proxy",
|
||||
"sub": user.id,
|
||||
"db": request.db,
|
||||
"company_ids": user.company_ids.ids,
|
||||
"scope": "compositor",
|
||||
"compositor_session_id": compositor_session_id,
|
||||
"iat": datetime.datetime.utcnow(),
|
||||
"exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=15),
|
||||
}
|
||||
token = jwt.encode(payload, secret, algorithm="HS256")
|
||||
|
||||
# Create session record
|
||||
request.env["compositor.session"].sudo().create({
|
||||
"user_id": user.id,
|
||||
"compositor_session_id": compositor_session_id,
|
||||
})
|
||||
|
||||
proxy_url = request.env["ir.config_parameter"].sudo().get_param(
|
||||
"compositor.proxy_url", default="http://localhost:8081"
|
||||
)
|
||||
return {
|
||||
"jwt": token,
|
||||
"proxy_url": proxy_url,
|
||||
"expires_at": payload["exp"].isoformat(),
|
||||
"compositor_session_id": compositor_session_id,
|
||||
}
|
||||
|
||||
@http.route("/web/compositor/apps", type="json", auth="user")
|
||||
def list_apps(self):
|
||||
user = request.env.user
|
||||
apps = request.env["compositor.app"].search_read(
|
||||
[("allowed_group_ids", "in", user.groups_id.ids), ("active", "=", True)],
|
||||
["name", "app_id", "icon", "description"],
|
||||
)
|
||||
return apps
|
||||
|
||||
@http.route("/web/compositor/launch", type="json", auth="user")
|
||||
def launch_app(self, app_id):
|
||||
user = request.env.user
|
||||
app = request.env["compositor.app"].search([
|
||||
("app_id", "=", app_id),
|
||||
("allowed_group_ids", "in", user.groups_id.ids),
|
||||
], limit=1)
|
||||
if not app:
|
||||
raise UserError("App not found or not allowed")
|
||||
|
||||
# Get JWT (re-issued here for the proxy call)
|
||||
# ... JWT issuance ...
|
||||
|
||||
# Tell the compositor proxy to spawn the app
|
||||
# ... HTTP call to proxy ...
|
||||
|
||||
# Audit log
|
||||
request.env["compositor.audit.log"].sudo().create({
|
||||
"user_id": user.id,
|
||||
"app_id": app_id,
|
||||
"action": "launch",
|
||||
})
|
||||
|
||||
return {"signal_url": "...", "pid": 12345, "name": app.name}
|
||||
|
||||
@http.route("/web/compositor/audit", type="json", auth="user")
|
||||
def audit_callback(self, **kwargs):
|
||||
"""Called by the compositor proxy to report events."""
|
||||
# Verify this is from the proxy (shared secret or JWT)
|
||||
# ...
|
||||
request.env["compositor.audit.log"].sudo().create({
|
||||
"user_id": kwargs.get("user_id"),
|
||||
"app_id": kwargs.get("app_id"),
|
||||
"action": kwargs.get("action"),
|
||||
"pid": kwargs.get("pid"),
|
||||
"exit_code": kwargs.get("exit_code"),
|
||||
})
|
||||
return {"ok": True}
|
||||
```
|
||||
|
||||
### Security
|
||||
|
||||
```xml
|
||||
<!-- compositor_shell/security/compositor_security.xml -->
|
||||
<record id="compositor_app_group_user" model="res.groups">
|
||||
<field name="name">Compositor / User</field>
|
||||
<field name="category_id" ref="module_category_compositor"/>
|
||||
</record>
|
||||
|
||||
<record id="compositor_app_group_manager" model="res.groups">
|
||||
<field name="name">Compositor / Manager</field>
|
||||
<field name="category_id" ref="module_category_compositor"/>
|
||||
<field name="implied_ids" eval="[(4, ref('compositor_app_group_user'))]"/>
|
||||
</record>
|
||||
|
||||
<record id="compositor_app_rule_user" model="ir.rule">
|
||||
<field name="name">Compositor App: User sees allowed apps only</field>
|
||||
<field name="model_id" ref="model_compositor_app"/>
|
||||
<field name="domain_force">["|", ("allowed_group_ids", "=", False), ("allowed_group_ids", "in", [g.id for g in user.groups_id])]</field>
|
||||
<field name="groups" eval="[(4, ref('compositor_app_group_user'))]"/>
|
||||
</record>
|
||||
```
|
||||
|
||||
## Nginx configuration
|
||||
|
||||
```nginx
|
||||
# /etc/nginx/sites-available/odoo-compositor
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name your-odoo.example.com;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/your-odoo.example.com/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/your-odoo.example.com/privkey.pem;
|
||||
|
||||
# Odoo HTTP workers
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8069;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
# Odoo gevent worker (Discuss, chat, presence — NOT compositor!)
|
||||
location /websocket {
|
||||
proxy_pass http://127.0.0.1:8072;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
}
|
||||
|
||||
# Compositor proxy (aiohttp, high-throughput WS)
|
||||
location /compositor/ {
|
||||
proxy_pass http://127.0.0.1:8081;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
|
||||
# CRITICAL: disable buffering for low-latency video streaming
|
||||
proxy_buffering off;
|
||||
proxy_request_buffering off;
|
||||
|
||||
# WebSocket keepalive
|
||||
proxy_read_timeout 86400s;
|
||||
proxy_send_timeout 86400s;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Docker Compose
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
version: "3.8"
|
||||
|
||||
services:
|
||||
odoo:
|
||||
image: odoo:20
|
||||
ports:
|
||||
- "8069:8069"
|
||||
- "8072:8072" # gevent
|
||||
environment:
|
||||
- HOST=db
|
||||
- USER=odoo
|
||||
- PASSWORD=odoo
|
||||
volumes:
|
||||
- odoo-data:/var/lib/odoo
|
||||
- ./addons:/mnt/extra-addons
|
||||
depends_on:
|
||||
- db
|
||||
|
||||
db:
|
||||
image: postgres:16
|
||||
environment:
|
||||
- POSTGRES_DB=postgres
|
||||
- POSTGRES_USER=odoo
|
||||
- POSTGRES_PASSWORD=odoo
|
||||
volumes:
|
||||
- db-data:/var/lib/postgresql/data
|
||||
|
||||
compositor-proxy:
|
||||
build: ./compositor-proxy
|
||||
ports:
|
||||
- "8081:8081"
|
||||
environment:
|
||||
- COMPOSITOR_JWT_SECRET=${COMPOSITOR_JWT_SECRET}
|
||||
- COMPOSITOR_BIND_IP=0.0.0.0
|
||||
- COMPOSITOR_BIND_PORT=8081
|
||||
- COMPOSITOR_ENCODER=vaapih264 # or nvh264 or openh264
|
||||
- COMPOSITOR_RENDER_DEVICE=/dev/dri/renderD128
|
||||
devices:
|
||||
- /dev/dri:/dev/dri # GPU passthrough
|
||||
volumes:
|
||||
- /tmp/.X11-unix:/tmp/.X11-unix # XWayland (if enabled)
|
||||
- compositor-runtime:/run/user/1000 # XDG_RUNTIME_DIR
|
||||
depends_on:
|
||||
- odoo
|
||||
|
||||
nginx:
|
||||
image: nginx:latest
|
||||
ports:
|
||||
- "443:443"
|
||||
volumes:
|
||||
- ./nginx.conf:/etc/nginx/nginx.conf:ro
|
||||
- ./certs:/etc/nginx/certs:ro
|
||||
depends_on:
|
||||
- odoo
|
||||
- compositor-proxy
|
||||
|
||||
volumes:
|
||||
odoo-data:
|
||||
db-data:
|
||||
compositor-runtime:
|
||||
```
|
||||
|
||||
## Effort estimates
|
||||
|
||||
| Component | Days | Notes |
|
||||
|---|---|---|
|
||||
| Odoo addon skeleton (models + controllers + views + security) | 2 | compositor.app, compositor.session, compositor.audit.log |
|
||||
| JWT issuance + audit log model | 1.5 | HS256, 15-min expiry, pyjwt |
|
||||
| OWL 3 fullscreen client action + canvas + WS wiring | 2.5 | ir.actions.client with target=fullscreen, **must use proxy() not useState()** |
|
||||
| Browser compositor library integration | 2 | userShell.actions.initScene(canvasCreator) |
|
||||
| Compositor proxy JWT middleware + WS handshake | 1.5 | pyjwt validation |
|
||||
| Audit POSTback + Nginx + Docker Compose + GPU passthrough | 2 | |
|
||||
| Multi-tenant: per-DB proxy spawning + db claim scoping | 1.5 | |
|
||||
| Tests (HttpCase + TransactionCase) | 2 | |
|
||||
| Documentation + buffer | 3 | |
|
||||
| **Total** | **~18 days** | (parallelizable to ~10 with 2 engineers) |
|
||||
|
||||
Critical path: **~13 working days** for single-tenant MVP, **+5 days** for
|
||||
multi-tenant + bubblewrap isolation.
|
||||
|
||||
## File reference
|
||||
|
||||
### Odoo 20 backend addon
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `compositor_shell/__manifest__.py` | Addon manifest |
|
||||
| `compositor_shell/models/compositor_app.py` | App catalog model |
|
||||
| `compositor_shell/models/compositor_session.py` | Session model (mail.thread) |
|
||||
| `compositor_shell/models/compositor_audit_log.py` | Append-only audit log |
|
||||
| `compositor_shell/controllers/main.py` | HTTP controllers (token, apps, launch, audit) |
|
||||
| `compositor_shell/security/compositor_security.xml` | Groups + rules |
|
||||
| `compositor_shell/security/ir.model.access.csv` | ACLs |
|
||||
| `compositor_shell/views/compositor_app_views.xml` | App management views |
|
||||
| `compositor_shell/views/compositor_session_views.xml` | Session views |
|
||||
| `compositor_shell/views/ir_actions_client_views.xml` | OWL action registration |
|
||||
| `compositor_shell/static/src/components/compositor_shell.js` | OWL shell component |
|
||||
| `compositor_shell/static/src/components/compositor_shell.xml` | OWL template |
|
||||
| `compositor_shell/static/src/components/compositor_shell.scss` | Styles |
|
||||
| `compositor_shell/static/src/lib/compositor_lib/` | Custom browser compositor library (clean-room TS) |
|
||||
|
||||
### Python compositor proxy
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `compositor_proxy/main.py` | HTTP + WS hub, forks per session |
|
||||
| `compositor_proxy/session.py` | Top-level proxy session |
|
||||
| `compositor_proxy/session_controller.py` | aiohttp WS server with 9 endpoint paths |
|
||||
| `compositor_proxy/jwt_auth.py` | JWT validation (pyjwt) |
|
||||
| `compositor_proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn |
|
||||
| ... | (see `07-native-app-bridge.md` for full file list) |
|
||||
|
||||
### Nginx config
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `nginx/sites-available/odoo-compositor` | Nginx reverse proxy with WS upgrade + proxy_buffering off |
|
||||
|
||||
### Docker
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `docker-compose.yml` | Odoo + PostgreSQL + compositor-proxy + Nginx |
|
||||
| `compositor-proxy/Dockerfile` | Python proxy image with GPU passthrough |
|
||||
+392
@@ -0,0 +1,392 @@
|
||||
# 11 — Pitfalls, GPL Contaminants, Clean-Room Requirements
|
||||
|
||||
This is the prominent pitfalls chapter for the MIT approach. It documents:
|
||||
|
||||
1. **3 GPL contaminants found** (must be replaced)
|
||||
2. **Clean-room legal requirements** (cannot read AGPL source and reimplement)
|
||||
3. **Engineering pitfalls of building from scratch**
|
||||
4. **Risk analysis**
|
||||
|
||||
## Critical finding: 3 GPL contaminants
|
||||
|
||||
The user's #1 priority is **license purity** — no GPL/AGPL contamination of
|
||||
the shipping product. The license audit (Agent T) found **3 GPL
|
||||
contaminants** that MUST be replaced:
|
||||
|
||||
### Contaminant 1: `@gfld/compositor-ffmpeg-h264` is GPL-3.0, not Apache-2.0
|
||||
|
||||
**What Greenfield claims**: `package.json` declares `"license":
|
||||
"Apache-2.0"`.
|
||||
|
||||
**What the LICENSE file actually says**: Full GPL v3 text.
|
||||
|
||||
**Why this happened**: The TypeScript wrapper (`H264Decoder.ts`,
|
||||
`H264Worker.ts`, `index.ts`) is genuinely Apache-2.0. But the
|
||||
`build_wasm.sh` script compiles FFmpeg with `--enable-gpl
|
||||
--enable-version3 --enable-nonfree`, which makes the resulting WASM
|
||||
binary GPL-3.0. The 1.8 MB compiled `libav-h264.js` contains embedded
|
||||
GPL/LGPL license strings.
|
||||
|
||||
**Impact on the user**: Using `@gfld/compositor-ffmpeg-h264` would force
|
||||
the entire Odoo fork to be GPL-3.0.
|
||||
|
||||
**Replacement**: WebCodecs `VideoDecoder` API (browser-native,
|
||||
hardware-accelerated, W3C standard). Fallback: OpenH264 BSD Wasm (NOT
|
||||
FFmpeg).
|
||||
|
||||
### Contaminant 2: `gstreamer1.0-plugins-ugly` (provides `x264enc`) is GPL-2.0+
|
||||
|
||||
**What it provides**: The `x264enc` GStreamer element, which is the
|
||||
software H.264 encoder Greenfield uses by default.
|
||||
|
||||
**Why it's GPL**: `libx264.so.164` is GPL-2.0+. The plugin that wraps it
|
||||
(`gst-plugins-ugly`) inherits the GPL. Loading this plugin into the
|
||||
Python process = GPL contamination under FSF's strict interpretation.
|
||||
|
||||
**Verified**: `ldd libgstx264.so` → links `libx264.so.164` (GPL-2.0+).
|
||||
x264 has NO dynamic-linking exception.
|
||||
|
||||
**Impact**: Using `x264enc` would force the entire Python proxy to be
|
||||
GPL-2.0+.
|
||||
|
||||
**Replacement**: Use `vaapih264enc` (Intel/AMD, LGPL plugin + MIT libva),
|
||||
`nvh264enc` (NVIDIA, BSD plugin + proprietary NVENC blob), or
|
||||
`openh264enc` (CPU fallback, BSD libopenh264).
|
||||
|
||||
### Contaminant 3: `gstreamer1.0-libav` on Debian is GPL (HIDDEN TRAP)
|
||||
|
||||
**What it provides**: The `gst-libav` plugin set, which wraps FFmpeg's
|
||||
libavcodec for various codecs.
|
||||
|
||||
**Why it's a trap**: The gst-libav plugin SOURCE CODE is LGPL-2.1+.
|
||||
**BUT** Debian's `libavcodec.so.61` is built with `--enable-gpl` and
|
||||
directly links `libx264.so.164` + `libx265.so.215`.
|
||||
|
||||
**Verified**: `ldd libgstlibav.so` → `libavcodec.so.61` → `libx264.so.164`
|
||||
+ `libx265.so.215`.
|
||||
|
||||
**Impact**: Even if the user might never use `avenc_h264`, the mere
|
||||
presence of `libgstlibav.so` triggers GStreamer's plugin auto-discovery,
|
||||
which dlopens it, which loads the GPL'd FFmpeg into the Python process's
|
||||
address space.
|
||||
|
||||
**Replacement**: Don't install `gstreamer1.0-libav`. Use only
|
||||
`gst-plugins-base` + `gst-plugins-good` + `gst-vaapi` + BSD subset of
|
||||
`gst-plugins-bad`.
|
||||
|
||||
## Clean-room legal requirements (CRITICAL)
|
||||
|
||||
### The problem
|
||||
|
||||
**You CANNOT read Greenfield's AGPL `@gfld/compositor` source and
|
||||
reimplement it as "inspired-by" code.** Even non-literal similarity is
|
||||
sufficient to find derivative-work infringement.
|
||||
|
||||
### Reasons
|
||||
|
||||
1. **AFC test risk**: Under *Computer Associates v. Altai*'s
|
||||
Abstraction-Filtration-Comparison test, even non-literal similarity
|
||||
(same class names, method names, code structure) is sufficient to
|
||||
find derivative-work infringement. A single developer who reads AGPL
|
||||
source then writes a TS replacement will almost certainly reproduce
|
||||
the SSO (structure, sequence, organization) subconsciously.
|
||||
|
||||
2. **AGPL §13 network-use clause**: This is the killer for an Odoo fork.
|
||||
AGPL-3.0 §13 requires offering the source of any modified AGPL work
|
||||
to all users interacting with it over a network. If a court finds the
|
||||
custom compositor is an AGPL derivative, the entire Odoo fork (which
|
||||
links to the compositor) must be open-sourced to all users —
|
||||
destroying the proprietary value.
|
||||
|
||||
3. **Westfield doesn't help**: Greenfield's predecessor repo at
|
||||
https://github.com/udevbe/westfield IS MIT-licensed, but it contains
|
||||
ONLY the protocol infrastructure (the same `@gfld/common` +
|
||||
`@gfld/compositor-protocol` + `@gfld/compositor-wasm` we already have).
|
||||
It NEVER had the in-browser compositor logic (Surface/Renderer/Seat/etc.).
|
||||
The compositor code was always in the Greenfield monorepo, always AGPL.
|
||||
|
||||
### Right approach: True clean-room
|
||||
|
||||
1. **Person A** (spec author, MAY read AGPL source) writes a behavioral
|
||||
spec (inputs/outputs/edge cases, NO code).
|
||||
2. **Person B** (implementer, MUST NOT open `packages/compositor/src/`)
|
||||
implements from spec + MIT references only:
|
||||
- Wayland protocol spec (https://wayland.freedesktop.org/docs/html/)
|
||||
- wlroots C source (MIT, https://gitlab.freedesktop.org/wlroots/wlroots)
|
||||
- Weston C source (MIT, https://gitlab.freedesktop.org/wayland/weston)
|
||||
- Smithay Rust source (MIT, https://github.com/Smithay/smithay)
|
||||
- WebGL2/WebCodecs/KCP MDN docs
|
||||
|
||||
### Legal-defensibility artifacts
|
||||
|
||||
Maintain these as legal-defensibility artifacts:
|
||||
|
||||
- `SPEC.md` — the behavioral spec (written by Person A)
|
||||
- `CLEAN_ROOM_LOG.md` — who read AGPL, who didn't, when, what they referenced
|
||||
|
||||
### What can Person B reference?
|
||||
|
||||
| Reference | License | OK to read for implementation? |
|
||||
|---|---|---|
|
||||
| Greenfield's `packages/compositor/src/` | AGPL-3.0 | ❌ NO (Person B forbidden) |
|
||||
| Greenfield's `libs/` (MIT) | MIT | ✅ Yes (already reused) |
|
||||
| Greenfield's `protocol/` XML | MIT | ✅ Yes |
|
||||
| Greenfield's `docs/` | MIT | ✅ Yes |
|
||||
| Wayland protocol spec | MIT | ✅ Yes |
|
||||
| wlroots C source | MIT | ✅ Yes |
|
||||
| Weston C source | MIT | ✅ Yes |
|
||||
| Smithay Rust source | MIT | ✅ Yes |
|
||||
| MDN docs (WebGL2, WebCodecs, etc.) | CC-BY-SA | ✅ Yes |
|
||||
| pywayland source | Apache-2.0 | ✅ Yes |
|
||||
| qtile source | GPL-2.0 | ❌ NO (architectural patterns OK, code not) |
|
||||
|
||||
## License audit table (full stack)
|
||||
|
||||
### Browser side
|
||||
|
||||
| Component | License | Compatible with proprietary? |
|
||||
|---|---|---|
|
||||
| @gfld/common | MIT | ✅ |
|
||||
| @gfld/client-protocol | MIT | ✅ |
|
||||
| @gfld/compositor-protocol | MIT | ✅ (verified NO @gfld/compositor import) |
|
||||
| @gfld/client-generator | MIT | ✅ |
|
||||
| @gfld/compositor-generator | MIT | ✅ |
|
||||
| @gfld/compositor-wasm (pixman+xkbcommon) | MIT | ✅ |
|
||||
| @gfld/xtsb (X11 bindings, optional) | MIT | ✅ (only if XWayland needed) |
|
||||
| ~~@gfld/compositor-ffmpeg-h264~~ | ~~GPL-3.0~~ | ❌ **REPLACE with WebCodecs** |
|
||||
| Custom browser compositor (your code) | PROPRIETARY | ✅ (clean-room) |
|
||||
| WebCodecs API | W3C standard | ✅ |
|
||||
| WebGL2 | W3C standard | ✅ |
|
||||
| OffscreenCanvas | W3C standard | ✅ |
|
||||
| OpenH264 BSD Wasm (fallback) | BSD-2-Clause | ✅ |
|
||||
|
||||
### Python backend
|
||||
|
||||
| Component | License | Compatible with proprietary? |
|
||||
|---|---|---|
|
||||
| pywayland | Apache-2.0 | ✅ |
|
||||
| python-xkbcommon (`xkbcommon` PyPI) | MIT | ✅ |
|
||||
| PyGObject (gi.repository.Gst) | LGPL-2.1+ | ✅ (dynamic linking) |
|
||||
| GLib | LGPL-2.1+ | ✅ (dynamic linking) |
|
||||
| GStreamer core | LGPL-2.0+ | ✅ (dynamic linking) |
|
||||
| gst-plugins-base | LGPL | ✅ |
|
||||
| gst-plugins-good (pngenc) | LGPL | ✅ |
|
||||
| gst-vaapi (vaapih264enc) | LGPL + MIT (libva) | ✅ |
|
||||
| gst-plugins-bad (nvh264enc, openh264enc) | BSD subset | ✅ |
|
||||
| ~~gst-plugins-ugly (x264enc)~~ | ~~GPL-2.0+~~ | ❌ **DROP** |
|
||||
| ~~gst-libav (libavcodec)~~ | ~~LGPL source, GPL binary on Debian~~ | ❌ **DROP** |
|
||||
| libwayland | MIT | ✅ |
|
||||
| libdrm | MIT | ✅ |
|
||||
| mesa (libGL, libEGL, libgbm) | MIT | ✅ |
|
||||
| aiohttp | Apache-2.0 | ✅ |
|
||||
| kcp (PyPI, RealistikDash) | MIT | ✅ |
|
||||
| PyJWT | MIT | ✅ |
|
||||
| bubblewrap (bwrap) | LGPL-2.0+ | ✅ (dynamic linking) |
|
||||
| libseccomp | LGPL-2.1 | ✅ |
|
||||
|
||||
### Odoo integration
|
||||
|
||||
| Component | License | Compatible? |
|
||||
|---|---|---|
|
||||
| Odoo 20 Community | LGPL-3.0 | ✅ (separate process, no contamination) |
|
||||
| Nginx | BSD-2-Clause | ✅ |
|
||||
| Docker | Apache-2.0 | ✅ |
|
||||
|
||||
### Vendored Greenfield C files (MIT, reused verbatim)
|
||||
|
||||
| File | License | Reuse? |
|
||||
|---|---|---|
|
||||
| westfield-buffer.c | MIT (Greenfield) | ✅ |
|
||||
| westfield-dmabuf.c | MIT (Greenfield) | ✅ |
|
||||
| westfield-egl.c | MIT (Greenfield) | ✅ |
|
||||
| westfield-xwayland.c | MIT (Greenfield) | ✅ |
|
||||
| wlr_drm.c | MIT (vendored from wlroots) | ✅ |
|
||||
| wlr_linux_dmabuf_v1.c | MIT (vendored from wlroots) | ✅ |
|
||||
| drm_format_set.c | MIT (vendored from wlroots) | ✅ |
|
||||
| addon.c | MIT (vendored from wlroots) | ✅ |
|
||||
| changes.diff | MIT (Greenfield patches to libwayland) | ✅ (optional, for fast-sync) |
|
||||
|
||||
## CI/CD license hygiene gates (recommended)
|
||||
|
||||
Add these to your CI pipeline to catch GPL contamination early:
|
||||
|
||||
### Docker image audit
|
||||
|
||||
```bash
|
||||
# Must return zero matches
|
||||
dpkg -l | grep -E "libx264|libx265|gstreamer1.0-plugins-ugly|gstreamer1.0-libav"
|
||||
```
|
||||
|
||||
### npm license check
|
||||
|
||||
```bash
|
||||
# Must pass (no GPL/AGPL)
|
||||
npx license-checker --production --failOn "GPL-3.0;AGPL-3.0;GPL-2.0;GPL-2.0+;GPL-3.0+"
|
||||
```
|
||||
|
||||
### Python license check
|
||||
|
||||
```bash
|
||||
# Must pass (no GPL/AGPL)
|
||||
pip-licenses --fail-on "GPL.*;AGPL.*" --ignore-builtin
|
||||
```
|
||||
|
||||
### Runtime GStreamer plugin check
|
||||
|
||||
```bash
|
||||
# Must return zero matches
|
||||
gst-inspect-1.0 --print-plugin-auto-install-info | grep -E "x264|x265|libav"
|
||||
```
|
||||
|
||||
## Engineering pitfalls (beyond licensing)
|
||||
|
||||
### 1. Browser compositor is the biggest effort
|
||||
|
||||
The custom browser compositor is ~13,400 LOC TypeScript for MVP, ~25
|
||||
dev-weeks (1 senior dev). This is the critical path.
|
||||
|
||||
**Mitigation**: Hire 2 senior engineers to parallelize. Use Smithay (Rust,
|
||||
MIT) as a reference for Wayland semantics. Use wlroots (C, MIT) as a
|
||||
reference for compositor patterns.
|
||||
|
||||
### 2. xdg-shell state machine is the highest-risk component
|
||||
|
||||
The configure/ack-configure round-trip, popup positioner math, and
|
||||
subsurface sync/desync cascades are subtle. Bugs here can cause apps to
|
||||
hang or crash.
|
||||
|
||||
**Mitigation**: Write extensive unit tests using Weston's test clients
|
||||
(ported to TypeScript). Test with real GTK4/Qt6 apps early and often.
|
||||
|
||||
### 3. asyncio + libwayland integration is under-documented
|
||||
|
||||
The 5-line cffi shim for `wl_event_loop_get_fd` is verified working, but
|
||||
the pattern is not widely documented. Bugs here could cause Wayland
|
||||
events to be dropped or duplicated.
|
||||
|
||||
**Mitigation**: Spend 1 day on a standalone prototype before committing.
|
||||
Reference qtile's Wayland backend (architectural patterns OK, code is
|
||||
GPL — don't copy).
|
||||
|
||||
### 4. GStreamer threading is non-trivial
|
||||
|
||||
GLib MainLoop runs in a separate thread; results must be marshalled back
|
||||
to asyncio via `asyncio.run_coroutine_threadsafe`. Bugs here can cause
|
||||
race conditions or deadlocks.
|
||||
|
||||
**Mitigation**: Reuse Greenfield's MIT C files via cffi — only the
|
||||
marshalling needs Python. Test with high-frame-rate content (e.g. glxgears).
|
||||
|
||||
### 5. XWayland is a major time sink (~8 weeks)
|
||||
|
||||
If you need X11 app support (e.g. older apps that don't speak Wayland
|
||||
natively), the XWayland bridge is ~5,000 LOC TypeScript + ~465 LOC C.
|
||||
It uses `@gfld/xtsb` (MIT, REUSED) for X11 protocol bindings.
|
||||
|
||||
**Mitigation**: Skip XWayland in MVP. Most modern GTK4/Qt6 apps are
|
||||
Wayland-native. Add XWayland only if a specific X11-only app is needed.
|
||||
|
||||
### 6. KCP Python port may have performance issues
|
||||
|
||||
The `kcp` PyPI package (RealistikDash, MIT) is a Cython wrapper over
|
||||
upstream `ikcp.c`. Performance should be adequate, but if it's not,
|
||||
porting Greenfield's TypeScript KCP (~1100 lines) to Python would be
|
||||
~1500 lines of pure Python (slower).
|
||||
|
||||
**Mitigation**: Benchmark early. If Cython wrapper is too slow, write a
|
||||
C extension via cffi.
|
||||
|
||||
### 7. WebGL2 + OffscreenCanvas + Workers is complex
|
||||
|
||||
Multi-threaded rendering with OffscreenCanvas requires careful message
|
||||
passing between workers. Debugging is harder than main-thread code.
|
||||
|
||||
**Mitigation**: Start with main-thread WebGL2 (simpler), then move to
|
||||
OffscreenCanvas in Phase 5 (polish).
|
||||
|
||||
### 8. WebCodecs hardware decode may have bugs
|
||||
|
||||
Greenfield explicitly disables hardware decode with `// FIXME hardware
|
||||
decoding frame offset is wrong`. We're enabling it (`prefer-hardware`),
|
||||
but the bug Greenfield hit may still affect us.
|
||||
|
||||
**Mitigation**: Test extensively on Intel/AMD/NVIDIA GPUs. Fall back to
|
||||
`prefer-software` if hardware decode is broken on specific hardware.
|
||||
|
||||
### 9. Bubblewrap + seccomp may break some apps
|
||||
|
||||
Some GTK apps may need additional filesystem access (e.g. ~/.config for
|
||||
settings, ~/.cache for thumbnails). The default-deny approach may break
|
||||
them.
|
||||
|
||||
**Mitigation**: Test with the apps you want to support. Add per-app
|
||||
overrides in `compositor.app` model (e.g. extra filesystem binds).
|
||||
|
||||
### 10. JWT secret rotation
|
||||
|
||||
The JWT secret is shared between Odoo and the compositor proxy via env
|
||||
var. If the secret leaks, attackers can forge JWTs and run apps on
|
||||
behalf of any user.
|
||||
|
||||
**Mitigation**: Rotate the secret periodically (e.g. monthly). Use a
|
||||
secrets manager (Vault, AWS Secrets Manager). Log all JWT validations.
|
||||
|
||||
## Risk analysis summary
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|---|---|---|---|
|
||||
| GPL contamination in shipping product | Medium (3 contaminants found) | **CRITICAL** (forces open-source) | 3 substitutions + CI/CD license gates |
|
||||
| AGPL derivative-work lawsuit | Low (clean-room approach) | **CRITICAL** (forces open-source) | True clean-room with SPEC.md + CLEAN_ROOM_LOG.md |
|
||||
| Browser compositor takes >6 months | Medium | High (delays MVP) | 2 senior engineers in parallel |
|
||||
| xdg-shell state machine bugs | High | Medium (app crashes) | Extensive testing with Weston test clients |
|
||||
| asyncio + libwayland integration bugs | Medium | High (events dropped) | 1-day prototype before committing |
|
||||
| GStreamer threading deadlocks | Medium | Medium (encoder stalls) | Reuse Greenfield's MIT C files via cffi |
|
||||
| XWayland scope creep | High | High (+8 weeks) | Skip XWayland in MVP |
|
||||
| KCP Python performance | Low | Medium (latency) | Benchmark early, fall back to cffi C extension |
|
||||
| WebGL2 + Workers complexity | Medium | Medium (debugging pain) | Start with main-thread, move to Workers in Phase 5 |
|
||||
| WebCodecs hardware decode bugs | Medium | Medium (CPU usage) | Test on Intel/AMD/NVIDIA, fall back to software |
|
||||
| Bubblewrap breaks apps | Medium | Low (per-app overrides) | Test with target apps, add per-app overrides |
|
||||
| JWT secret leak | Low | **CRITICAL** (auth bypass) | Rotate periodically, use secrets manager |
|
||||
|
||||
## Design wins to copy from Greenfield
|
||||
|
||||
These Greenfield design choices are worth copying in the MIT approach
|
||||
(but implemented clean-room):
|
||||
|
||||
1. **Generator-driven protocol stubs** — single XML source for both
|
||||
sides. REUSED from Greenfield's MIT libs.
|
||||
2. **Dual-buffer commit semantics** — Surface state vs pendingState,
|
||||
atomic merge. Clean-room from spec.
|
||||
3. **UserShellApi decoupling** — shell sees only `{events, actions}`,
|
||||
never Wayland resources. Clean-room from spec.
|
||||
4. **Asynchronous browser compositor** — per-client channels, never
|
||||
blocks on a slow client. Clean-room from spec.
|
||||
5. **KCP-over-WebSocket** for UDP-like latency over TCP. Use `kcp` PyPI
|
||||
package (MIT).
|
||||
6. **WebCodecs primary + fallback** — capability probe at session
|
||||
startup. Clean-room, but enable hardware decode (Greenfield doesn't).
|
||||
7. **Zero-copy dmabuf → encoder** via `linux-dmabuf-unstable-v1`.
|
||||
REUSED vendored Greenfield C files (MIT).
|
||||
8. **Per-surface encoder feedback channel** — Greenfield produces it
|
||||
but doesn't use it. **Our MIT approach consumes it for adaptive
|
||||
bitrate** (NEW).
|
||||
9. **`ReconnectingWebSocket`** with exponential backoff. Vendored from
|
||||
pladaria (MIT).
|
||||
10. **Honest docs** — Greenfield's `docs/pages/design/` openly admits
|
||||
what's wrong. Use as reference for trade-off analysis.
|
||||
|
||||
## Top-5 recommendations
|
||||
|
||||
1. **Replace the 3 GPL contaminants IMMEDIATELY**: WebCodecs (not FFmpeg
|
||||
Wasm), vaapih264enc/nvh264enc/openh264enc (not x264enc), don't install
|
||||
gst-libav.
|
||||
2. **Adopt true clean-room discipline** for the browser compositor. One
|
||||
spec author (may read AGPL), one implementer (must NOT). Maintain
|
||||
`SPEC.md` + `CLEAN_ROOM_LOG.md` as legal-defensibility artifacts.
|
||||
3. **Skip XWayland in MVP** — add 8 weeks only if legacy X11 apps are
|
||||
required.
|
||||
4. **Use the kcp PyPI package** — don't port Greenfield's TS KCP to
|
||||
Python (would be slower).
|
||||
5. **Add CI/CD license hygiene gates** — Docker image audit, npm
|
||||
license-checker, pip-licenses, gst-inspect check. Catch GPL
|
||||
contamination early.
|
||||
+339
@@ -0,0 +1,339 @@
|
||||
# 12 — Appendix: Glossary, File Index, References
|
||||
|
||||
## Glossary
|
||||
|
||||
### Wayland concepts
|
||||
|
||||
Same as Greenfield — see `../greenfield-analysis/12-appendix.md` for
|
||||
the full Wayland glossary. Key terms: `wl_display`, `wl_registry`,
|
||||
`wl_compositor`, `wl_surface`, `wl_seat`, `wl_keyboard`, `wl_pointer`,
|
||||
`wl_touch`, `wl_output`, `wl_shm`, `wl_buffer`, `wl_callback`,
|
||||
`wl_data_device_manager`, `wl_data_device`, `wl_data_source`,
|
||||
`wl_data_offer`, `wl_drm`, `wl_subcompositor`, `wl_subsurface`,
|
||||
`wl_region`.
|
||||
|
||||
### xdg-shell concepts
|
||||
|
||||
Same as Greenfield — `xdg_wm_base`, `xdg_surface`, `xdg_toplevel`,
|
||||
`xdg_popup`, `xdg_positioner`, `configure`, `ack_configure`.
|
||||
|
||||
### MIT-approach-specific terms
|
||||
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| **Clean-room implementation** | Software written by an engineer who has NOT read the source code of the original (AGPL) implementation. The engineer writes from a behavioral spec produced by a separate person who MAY have read the AGPL source. This protects against derivative-work claims. |
|
||||
| **Person A (spec author)** | The engineer who writes the behavioral spec. MAY read AGPL source for reference. Does NOT write implementation code. |
|
||||
| **Person B (implementer)** | The engineer who writes the implementation. MUST NOT read AGPL source. Implements from spec + MIT references only. |
|
||||
| **SPEC.md** | The behavioral spec artifact (inputs/outputs/edge cases, NO code). Written by Person A. |
|
||||
| **CLEAN_ROOM_LOG.md** | Legal-defensibility artifact documenting who read AGPL source, who didn't, when, what they referenced. |
|
||||
| **AFC test** | Abstraction-Filtration-Comparison test, from *Computer Associates v. Altai* (1992). The legal test for derivative-work infringement in software. Even non-literal similarity (SSO: structure, sequence, organization) can be infringing. |
|
||||
| **AGPL §13** | The network-use clause of AGPL-3.0. Requires offering source to all users interacting with the modified work over a network. The killer clause for proprietary web services. |
|
||||
| **GPL dynamic-linking exception** | GPL applies to the entire combined work when statically linked. Dynamic linking is more permissive (LGPL allows it from proprietary code). FSF's strict interpretation says loading a GPL .so into a proprietary process contaminates it. |
|
||||
| **ProxyFD** | JSON descriptor for a file descriptor sent over WebSocket: `{handle, type, host}`. Same as Greenfield — see `../greenfield-analysis/04-transport.md`. |
|
||||
|
||||
### Python-specific terms
|
||||
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| **pywayland** | Python bindings to libwayland-server, via cffi. Apache-2.0 licensed. |
|
||||
| **pywlroots** | Python bindings to libwlroots. NCSA/UIUC license (BSD-like). NOT RECOMMENDED — operates above wire-dispatch layer, incompatible with Greenfield's architecture. |
|
||||
| **python-xkbcommon** | The `xkbcommon` PyPI package. Python bindings to libxkbcommon. MIT licensed. |
|
||||
| **PyGObject** | Python bindings to GLib/GObject. LGPL-2.1+. Provides `gi.repository.Gst` for GStreamer. |
|
||||
| **cffi** | C Foreign Function Interface for Python. Used to call C functions from Python without writing a C extension. |
|
||||
| **asyncio** | Python's async I/O framework, built into the standard library. |
|
||||
| **aiohttp** | Async HTTP + WebSocket server framework for Python. Apache-2.0. |
|
||||
| **multiprocessing.Process** | Python's process-spawning primitive. Used to fork one child per Odoo session. |
|
||||
| **multiprocessing.reduction.send_handle** | Function to pass an open file descriptor from one Python process to another (uses SCM_RIGHTS on Linux). |
|
||||
| **GLib MainLoop** | GLib's event loop, required by GStreamer. Runs in a separate Python thread. |
|
||||
| **asyncio.run_coroutine_threadsafe** | Function to schedule a coroutine on an asyncio event loop from a different thread. Used to marshal GStreamer results back to asyncio. |
|
||||
| **bubblewrap (bwrap)** | Unprivileged sandboxing tool. LGPL-2.0+. Used by Flatpak. |
|
||||
| **seccomp BPF** | Linux kernel feature for filtering syscalls. Compiled to BPF bytecode, loaded via `bwrap --seccomp <memfd>`. |
|
||||
| **systemd-run --scope** | systemd command to run a process in a transient cgroup. Used for memory/CPU/PID limits. |
|
||||
| **memfd_create** | Linux syscall to create an anonymous in-memory file. Python 3.8+ has `os.memfd_create()`. Replaces Greenfield's `createMemoryMappedFile` native addon. |
|
||||
|
||||
### Browser-specific terms
|
||||
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| **WebGL2** | Next-gen WebGL API. Uniform buffer objects, transform feedback, simplified shaders. 97%+ browser support. |
|
||||
| **WebCodecs** | W3C standard for native video decode/encode in the browser. `VideoDecoder`, `VideoEncoder`, `EncodedVideoChunk`, `VideoFrame`. Replaces FFmpeg Wasm. |
|
||||
| **OffscreenCanvas** | W3C standard for canvas rendering in a Web Worker. Allows WebGL to run off the main thread. |
|
||||
| **Web Workers** | W3C standard for running JavaScript in background threads. |
|
||||
| **SharedArrayBuffer + Atomics** | W3C standard for zero-copy shared memory between workers. Needs COOP/COEP headers. |
|
||||
| **ImageBitmap** | W3C standard for a GPU-backed bitmap. Transferable between workers. Used by the `web-bitmapbuf` fast path. |
|
||||
| **WebTransport** | W3C standard (in progress) for UDP-over-QUIC transport. Future replacement for WebSocket+KCP. |
|
||||
| **OpenH264** | BSD-2-Clause H.264 codec. Cisco's binary release. Used as fallback when WebCodecs is unavailable. |
|
||||
|
||||
## File-to-feature index
|
||||
|
||||
### Custom browser compositor (PROPRIETARY, clean-room TS)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `packages/compositor/src/index.ts` | Public API: `createCompositorSession()` |
|
||||
| `packages/compositor/src/core/Session.ts` | Top-level session |
|
||||
| `packages/compositor/src/core/Compositor.ts` | `wl_compositor` global |
|
||||
| `packages/compositor/src/core/Globals.ts` | Wayland globals registry |
|
||||
| `packages/compositor/src/surface/Surface.ts` | `wl_surface`, double-buffered state |
|
||||
| `packages/compositor/src/surface/SurfaceRole.ts` | Role abstraction |
|
||||
| `packages/compositor/src/surface/Subsurface.ts` | `wl_subsurface` |
|
||||
| `packages/compositor/src/surface/Subcompositor.ts` | `wl_subcompositor` global |
|
||||
| `packages/compositor/src/surface/Region.ts` | `wl_region` (pixman-WASM) |
|
||||
| `packages/compositor/src/surface/FrameCallback.ts` | `wl_callback` |
|
||||
| `packages/compositor/src/render/Renderer.ts` | 4-phase render pipeline (WebGL2 + OffscreenCanvas) |
|
||||
| `packages/compositor/src/render/Scene.ts` | One per output |
|
||||
| `packages/compositor/src/render/View.ts` | Positioned surface |
|
||||
| `packages/compositor/src/render/Output.ts` | `wl_output`, OffscreenCanvas binding |
|
||||
| `packages/compositor/src/render/RenderState.ts` | Per-view-per-scene texture + transform |
|
||||
| `packages/compositor/src/render/Texture.ts` | WebGL2 texture wrapper |
|
||||
| `packages/compositor/src/render/Loop.ts` | requestAnimationFrame integration |
|
||||
| `packages/compositor/src/render/shaders/YUVA2RGBA.glsl` | BT.601 YUVA → RGBA |
|
||||
| `packages/compositor/src/render/shaders/RGBandA2RGBA.glsl` | RGB+Alpha → RGBA |
|
||||
| `packages/compositor/src/render/shaders/SceneShader.glsl` | Final pass: textured quads |
|
||||
| `packages/compositor/src/buffer/Shm.ts` | `wl_shm`, BMP-header hack |
|
||||
| `packages/compositor/src/buffer/BufferContents.ts` | Buffer abstraction (mimeType) |
|
||||
| `packages/compositor/src/buffer/ImageBitmapBuffer.ts` | web-bitmapbuf fast path |
|
||||
| `packages/compositor/src/buffer/EncodedFrameBuffer.ts` | H.264/PNG from proxy |
|
||||
| `packages/compositor/src/transport/Channel.ts` | ARQChannel + SimpleChannel |
|
||||
| `packages/compositor/src/transport/kcp.ts` | Ported fresh from MIT C ref (skywind3000/kcp) |
|
||||
| `packages/compositor/src/transport/ReconnectingWebSocket.ts` | Vendored from pladaria (MIT) |
|
||||
| `packages/compositor/src/transport/BufferStream.ts` | Serial → frame promise resolution |
|
||||
| `packages/compositor/src/transport/StreamingBuffer.ts` | Per-surface encoded-frame buffer |
|
||||
| `packages/compositor/src/transport/EncodedFrame.ts` | Wire format parser |
|
||||
| `packages/compositor/src/transport/EncoderFeedback.ts` | 4-byte decode feedback, 1 Hz |
|
||||
| `packages/compositor/src/transport/RemoteInputOutput.ts` | FD-over-WebSocket for native apps |
|
||||
| `packages/compositor/src/transport/RemoteAppLauncher.ts` | Channel setup per app |
|
||||
| `packages/compositor/src/transport/decoders/WebCodecsFrameDecoder.ts` | WebCodecs VideoDecoder (HW-accelerated) |
|
||||
| `packages/compositor/src/transport/decoders/OpenH264FrameDecoder.ts` | OpenH264 BSD Wasm fallback |
|
||||
| `packages/compositor/src/input/Seat.ts` | `wl_seat` global |
|
||||
| `packages/compositor/src/input/Keyboard.ts` | `wl_keyboard` |
|
||||
| `packages/compositor/src/input/Pointer.ts` | `wl_pointer` |
|
||||
| `packages/compositor/src/input/Touch.ts` | `wl_touch` (REAL implementation, not stub) |
|
||||
| `packages/compositor/src/input/Xkb.ts` | @gfld/compositor-wasm wrapper (REUSED MIT) |
|
||||
| `packages/compositor/src/input/InputQueue.ts` | Event coalescing |
|
||||
| `packages/compositor/src/input/InputOutput.ts` | FD abstraction |
|
||||
| `packages/compositor/src/roles/XdgWmBase.ts` | `xdg_wm_base` global |
|
||||
| `packages/compositor/src/roles/XdgSurface.ts` | `xdg_surface` role |
|
||||
| `packages/compositor/src/roles/XdgToplevel.ts` | `xdg_toplevel` role |
|
||||
| `packages/compositor/src/roles/XdgPopup.ts` | `xdg_popup` role |
|
||||
| `packages/compositor/src/roles/XdgPositioner.ts` | `xdg_positioner` |
|
||||
| `packages/compositor/src/roles/Shell.ts` | `wl_shell` (legacy, optional) |
|
||||
| `packages/compositor/src/roles/ShellSurface.ts` | `wl_shell_surface` (legacy, optional) |
|
||||
| `packages/compositor/src/data/DataDeviceManager.ts` | `wl_data_device_manager` |
|
||||
| `packages/compositor/src/data/DataSource.ts` | `wl_data_source` |
|
||||
| `packages/compositor/src/data/DataOffer.ts` | `wl_data_offer` |
|
||||
| `packages/compositor/src/data/PrimarySelectionDeviceManager.ts` | **NEW: zwp_primary_selection_v1** |
|
||||
| `packages/compositor/src/desktop/Desktop.ts` | Floating vs AlwaysFullScreen |
|
||||
| `packages/compositor/src/desktop/FloatingDesktopSurface.ts` | Default floating mode |
|
||||
| `packages/compositor/src/userapi/UserShellApi.ts` | OWL shell integration seam |
|
||||
| `packages/compositor/src/browser/input.ts` | DOM keyboard events |
|
||||
| `packages/compositor/src/browser/pointer.ts` | DOM pointer events |
|
||||
| `packages/compositor/src/browser/touch.ts` | DOM touch events (NEW) |
|
||||
| `packages/compositor/src/browser/dnd.ts` | DnD bridge (NEW: native→DOM DnD) |
|
||||
| `packages/compositor/src/browser/selection.ts` | Clipboard bridge |
|
||||
| `packages/compositor/src/math/Mat4.ts` | 4×4 matrix |
|
||||
| `packages/compositor/src/math/Rect.ts` | Rectangle math |
|
||||
|
||||
### Custom Python proxy (PROPRIETARY, ~2880 LOC)
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `compositor_proxy/main.py` | HTTP + WS hub, forks per session |
|
||||
| `compositor_proxy/session.py` | Top-level proxy session |
|
||||
| `compositor_proxy/session_controller.py` | aiohttp WS server with 9 endpoints |
|
||||
| `compositor_proxy/jwt_auth.py` | JWT validation (pyjwt) |
|
||||
| `compositor_proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn |
|
||||
| `compositor_proxy/native_wayland_client_session.py` | Per-client message dispatcher |
|
||||
| `compositor_proxy/native_wayland_compositor_session.py` | Display creation, initShm, initDrm |
|
||||
| `compositor_proxy/channel.py` | ARQChannel + SimpleChannel (clean-room) |
|
||||
| `compositor_proxy/kcp_adapter.py` | kcp PyPI package integration |
|
||||
| `compositor_proxy/frame_feedback.py` | Receives 4-byte decode feedback |
|
||||
| `compositor_proxy/proxy_buffer.py` | Proxy-side buffer representation |
|
||||
| `compositor_proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration |
|
||||
| `compositor_proxy/xwayland_session.py` | XWayland spawn + bridge |
|
||||
| `compositor_proxy/wayland_server.py` | pywayland wrapper + cffi shim |
|
||||
| `compositor_proxy/io/` | WebSocket server glue |
|
||||
| `compositor_proxy/encoding/gst_encoder.py` | GStreamer H.264/PNG encoder (~700 LOC) |
|
||||
| `compositor_proxy/encoding/adaptive_bitrate.py` | Adaptive bitrate controller (NEW) |
|
||||
| `compositor_proxy/encoding/glib_thread.py` | GLib MainLoop in separate thread |
|
||||
| `compositor_proxy/sandbox.py` | bubblewrap + seccomp + systemd-run wrappers |
|
||||
|
||||
### Reused from Greenfield (MIT/Apache)
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `greenfield/protocol/wayland.xml` | Upstream Wayland core (MIT) |
|
||||
| `greenfield/protocol/xdg-shell.xml` | Stable xdg-shell (MIT) |
|
||||
| `greenfield/protocol/wayland-drm.xml` | Legacy wl_drm (MIT) |
|
||||
| `greenfield/protocol/web-bitmapbuf-unstable-v1.xml` | Greenfield's custom extension (MIT) |
|
||||
| `greenfield/libs/common/src/Connection.ts` | Wire format codec (REUSED) |
|
||||
| `greenfield/libs/compositor-protocol/src/` | Server-side runtime + generated bindings (REUSED) |
|
||||
| `greenfield/libs/client-protocol/src/` | Client-side runtime + generated bindings (REUSED) |
|
||||
| `greenfield/libs/compositor-wasm/src/` | pixman + xkbcommon WASM (REUSED) |
|
||||
| `greenfield/libs/xtsb/` | X11 TypeScript bindings (REUSED, only if XWayland) |
|
||||
| `greenfield/libs/client-generator/` | Browser client bindings generator (REUSED) |
|
||||
| `greenfield/libs/compositor-generator/` | Browser server bindings generator (REUSED) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-buffer.c` | Buffer helpers (REUSED, MIT) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-dmabuf.c` | DMA-BUF import (REUSED, MIT) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-egl.c` | EGL zero-copy setup (REUSED, MIT) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/westfield-xwayland.c` | XWayland bridge (REUSED as reference, ported to Python) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/wlr_drm.c` | `wl_drm` global impl (REUSED, MIT, vendored from wlroots) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl (REUSED, MIT, vendored from wlroots) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/drm_format_set.c` | DRM format helpers (REUSED, MIT, vendored from wlroots) |
|
||||
| `greenfield/packages/compositor-proxy/native/wayland/src/wayland-server/changes.diff` | Patches to libwayland-server (REUSED, MIT, optional for fast-sync) |
|
||||
|
||||
### Odoo 20 backend addon
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `compositor_shell/__manifest__.py` | Addon manifest |
|
||||
| `compositor_shell/models/compositor_app.py` | App catalog model |
|
||||
| `compositor_shell/models/compositor_session.py` | Session model (mail.thread) |
|
||||
| `compositor_shell/models/compositor_audit_log.py` | Append-only audit log |
|
||||
| `compositor_shell/controllers/main.py` | HTTP controllers (token, apps, launch, audit) |
|
||||
| `compositor_shell/security/compositor_security.xml` | Groups + rules |
|
||||
| `compositor_shell/security/ir.model.access.csv` | ACLs |
|
||||
| `compositor_shell/views/compositor_app_views.xml` | App management views |
|
||||
| `compositor_shell/views/compositor_session_views.xml` | Session views |
|
||||
| `compositor_shell/views/ir_actions_client_views.xml` | OWL action registration |
|
||||
| `compositor_shell/static/src/components/compositor_shell.js` | OWL shell component |
|
||||
| `compositor_shell/static/src/components/compositor_shell.xml` | OWL template |
|
||||
| `compositor_shell/static/src/components/compositor_shell.scss` | Styles |
|
||||
| `compositor_shell/static/src/lib/compositor_lib/` | Custom browser compositor library (clean-room TS) |
|
||||
|
||||
## External references
|
||||
|
||||
### Wayland protocol
|
||||
|
||||
- **Wayland documentation**: https://wayland.freedesktop.org/docs/html/
|
||||
- **Wayland protocol XML**: https://gitlab.freedesktop.org/wayland/wayland/-/blob/main/protocol/wayland.xml
|
||||
- **xdg-shell protocol**: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/stable/xdg-shell/xdg-shell.xml
|
||||
- **wayland-drm protocol**: https://gitlab.freedesktop.org/wayland/wayland/-/blob/main/protocol/wayland.xml
|
||||
- **linux-dmabuf-unstable-v1**: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/main/unstable/linux-dmabuf/linux-dmabuf-unstable-v1.xml
|
||||
|
||||
### libwayland + pywayland
|
||||
|
||||
- **libwayland source**: https://gitlab.freedesktop.org/wayland/wayland
|
||||
- **Wayland book**: https://wayland-book.com/
|
||||
- **pywayland docs**: https://pywayland.readthedocs.io/
|
||||
- **pywayland source**: https://github.com/flacjacket/pywayland
|
||||
- **pywayland on PyPI**: https://pypi.org/project/pywayland/
|
||||
|
||||
### wlroots + pywlroots
|
||||
|
||||
- **wlroots source**: https://gitlab.freedesktop.org/wlroots/wlroots
|
||||
- **wlroots mirror**: https://github.com/swaywm/wlroots
|
||||
- **pywlroots source**: https://github.com/flacjacket/pywlroots
|
||||
- **Smithay (Rust alternative)**: https://github.com/Smithay/smithay
|
||||
|
||||
### xkbcommon + python-xkbcommon
|
||||
|
||||
- **xkbcommon homepage**: https://xkbcommon.org/
|
||||
- **xkbcommon source**: https://github.com/xkbcommon/libxkbcommon
|
||||
- **xkbcommon on PyPI**: https://pypi.org/project/xkbcommon/
|
||||
|
||||
### GStreamer + PyGObject
|
||||
|
||||
- **GStreamer homepage**: https://gstreamer.freedesktop.org/
|
||||
- **GStreamer Python bindings (gst-python)**: https://github.com/GStreamer/gst-python
|
||||
- **PyGObject docs**: https://pygobject.readthedocs.io/
|
||||
- **GStreamer tutorials**: https://gstreamer.freedesktop.org/documentation/tutorials/index.html
|
||||
- **GStreamer GL plugins**: https://gstreamer.freedesktop.org/documentation/opengl/index.html
|
||||
- **vaapih264enc docs**: https://gstreamer.freedesktop.org/documentation/vaapi/index.html
|
||||
- **nvh264enc docs**: https://gstreamer.freedesktop.org/documentation/nvcodec/index.html
|
||||
- **openh264enc docs**: https://gstreamer.freedesktop.org/documentation/openh264/index.html
|
||||
|
||||
### OpenH264 (BSD fallback decoder)
|
||||
|
||||
- **OpenH264 source**: https://github.com/cisco/openh264
|
||||
- **Cisco binary releases**: https://github.com/cisco/openh264/releases
|
||||
|
||||
### WebCodecs + WebGL2 + OffscreenCanvas
|
||||
|
||||
- **WebCodecs API (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API
|
||||
- **VideoDecoder (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder
|
||||
- **WebGL2 (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebGL2RenderingContext
|
||||
- **OffscreenCanvas (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas
|
||||
- **Web Workers (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API
|
||||
- **SharedArrayBuffer (MDN)**: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer
|
||||
- **WebTransport (MDN)**: https://developer.mozilla.org/en-US/docs/Web/API/WebTransport
|
||||
- **WebCodecs samples**: https://w3c.github.io/webcodecs/samples/
|
||||
|
||||
### KCP
|
||||
|
||||
- **KCP protocol (skywind3000)**: https://github.com/skywind3000/kcp
|
||||
- **KCP documentation**: https://skywind3000.github.io/kcp/
|
||||
- **kcp on PyPI (RealistikDash)**: https://pypi.org/project/kcp/
|
||||
|
||||
### asyncio + aiohttp
|
||||
|
||||
- **asyncio docs**: https://docs.python.org/3/library/asyncio.html
|
||||
- **aiohttp docs**: https://docs.aiohttp.org/
|
||||
- **aiohttp WebSocket**: https://docs.aiohttp.org/en/stable/web_advanced.html#websockets
|
||||
|
||||
### JWT
|
||||
|
||||
- **PyJWT docs**: https://pyjwt.readthedocs.io/
|
||||
- **JWT.io**: https://jwt.io/
|
||||
- **RFC 7519 (JWT)**: https://datatracker.ietf.org/doc/html/rfc7519
|
||||
|
||||
### Sandboxing
|
||||
|
||||
- **bubblewrap docs**: https://github.com/containers/bubblewrap
|
||||
- **libseccomp**: https://github.com/seccomp/libseccomp
|
||||
- **Flatpak sandboxing**: https://docs.flatpak.org/en/latest/sandbox-permissions.html
|
||||
- **cgroups v2**: https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html
|
||||
- **systemd-run**: https://www.freedesktop.org/software/systemd/man/systemd-run.html
|
||||
|
||||
### Odoo 20
|
||||
|
||||
- **Odoo 20 documentation**: https://www.odoo.com/documentation/20.0/
|
||||
- **Odoo OWL framework**: https://github.com/odoo/owl
|
||||
- **Odoo OWL migration guide (Odoo 19→20)**: https://www.odoo.com/documentation/20.0/developer/howtos/owl_migration.html
|
||||
- **Odoo controller patterns**: https://www.odoo.com/documentation/20.0/developer/howtos/routing.html
|
||||
- **Odoo security**: https://www.odoo.com/documentation/20.0/developer/howtos/security.html
|
||||
|
||||
### Greenfield (for reference, NOT for clean-room implementers)
|
||||
|
||||
- **Greenfield homepage**: https://www.greenfield.app
|
||||
- **Greenfield source**: https://github.com/udevbe/greenfield
|
||||
- **Greenfield design docs**: https://github.com/udevbe/greenfield/blob/master/docs/pages/design/index.md
|
||||
- **Greenfield MIT libs (REUSABLE)**: https://github.com/udevbe/greenfield/tree/master/libs
|
||||
- **Greenfield AGPL packages (DO NOT READ for clean-room)**: https://github.com/udevbe/greenfield/tree/master/packages
|
||||
|
||||
### Westfield (Greenfield's predecessor, MIT)
|
||||
|
||||
- **Westfield source**: https://github.com/udevbe/westfield
|
||||
- **Note**: Contains ONLY the protocol infrastructure (same as Greenfield's MIT libs). NEVER had the in-browser compositor logic.
|
||||
|
||||
### Reference Wayland compositors (MIT, for clean-room spec authors)
|
||||
|
||||
- **wlroots (C, MIT)**: https://gitlab.freedesktop.org/wlroots/wlroots — reference for compositor patterns
|
||||
- **Weston (C, MIT)**: https://gitlab.freedesktop.org/wayland/weston — reference Wayland compositor
|
||||
- **Smithay (Rust, MIT)**: https://github.com/Smithay/smithay — Rust compositor library
|
||||
- **wprs (Rust, MIT)**: https://github.com/Smithay/wprs — Rust wayland-xpra, mirrors Greenfield's architecture almost exactly
|
||||
|
||||
## Pointers to the user's next steps
|
||||
|
||||
1. **Read the comparison document**: `../Greenfield-vs-MIT-Comparison.md`
|
||||
for the head-to-head analysis and final recommendation.
|
||||
|
||||
2. **If you choose the MIT approach** (recommended):
|
||||
- Phase 0: Spike (1 week) — validate libwayland ↔ asyncio integration
|
||||
- Phase 1: Browser compositor MVP (25 weeks, clean-room TS)
|
||||
- Phase 2: Python proxy MVP (3.5 weeks, aiohttp + pywayland + GStreamer)
|
||||
- Phase 3: Odoo integration (3.5 weeks, OWL shell + backend addon + JWT)
|
||||
- Phase 4: Sandboxing v1 (1.5 weeks, bubblewrap + seccomp)
|
||||
- Phase 5: Polish & testing (3 weeks)
|
||||
|
||||
3. **Critical first step**: Set up the clean-room process.
|
||||
- Designate Person A (spec author) and Person B (implementer)
|
||||
- Person A reads Greenfield AGPL source + writes `SPEC.md`
|
||||
- Person B implements from `SPEC.md` + MIT references only
|
||||
- Maintain `CLEAN_ROOM_LOG.md` as a legal-defensibility artifact
|
||||
|
||||
4. **CI/CD license hygiene gates**: Add the 4 license checks to your CI
|
||||
pipeline from day one. See `11-pitfalls.md` for the commands.
|
||||
|
||||
5. **Engage legal counsel** before starting the clean-room process. This
|
||||
bundle is engineering analysis, not legal advice.
|
||||
@@ -0,0 +1,282 @@
|
||||
# MIT-Licensed Wayland-to-HTML5 Compositor — Architecture & Build Plan
|
||||
|
||||
> **Purpose.** This bundle documents the architecture, components, and
|
||||
> implementation plan for building a **fully proprietary** Wayland-to-HTML5
|
||||
> compositor that integrates with an Odoo 20 fork. It is the alternative to
|
||||
> the AGPL-licensed Greenfield codebase (`../greenfield-analysis/`).
|
||||
>
|
||||
> **Goal.** Run real Linux applications (GTK3/4, Qt, xterm, X11 via XWayland)
|
||||
> **natively inside the Odoo web client**, as if they were Odoo-native apps —
|
||||
> while a real Linux machine under the hood actually executes them.
|
||||
>
|
||||
> **License strategy.** Use every MIT/Apache-2.0/BSD-licensed Greenfield
|
||||
> library as a building block. Replace the AGPL-licensed packages
|
||||
> (`@gfld/compositor`, `@gfld/compositor-proxy`, `@gfld/compositor-shell`,
|
||||
> `@gfld/compositor-proxy-cli`) with **clean-room reimplementations** in
|
||||
> TypeScript (browser) and Python (Linux host). Ship the result as a
|
||||
> proprietary addon for the Odoo fork.
|
||||
|
||||
## How to read this bundle
|
||||
|
||||
The bundle is split into 13 Markdown files mirroring the Greenfield
|
||||
analysis structure (so you can compare them side-by-side). Files are
|
||||
ordered from the highest-level view down to implementation details,
|
||||
then pitfalls, then appendix.
|
||||
|
||||
| # | File | What you'll find inside |
|
||||
|---|------|-------------------------|
|
||||
| 0 | `00-overview.md` | The MIT approach at a glance, component inventory, license summary, architecture diagram |
|
||||
| 1 | `01-architecture.md` | Layered map of the proposed architecture, side-by-side with Greenfield's layers |
|
||||
| 2 | `02-wayland-protocol.md` | How to handle the Wayland protocol layer with MIT libs only (pywayland + Greenfield's MIT @gfld libs) |
|
||||
| 3 | `03-html5-rendering.md` | Custom WebGL2 + Web Workers + OffscreenCanvas browser compositor (replacing @gfld/compositor) |
|
||||
| 4 | `04-transport.md` | Python aiohttp WebSocket + KCP, replacing Node's ws + KCP TS |
|
||||
| 5 | `05-input-handling.md` | Python xkbcommon for keymap compilation, browser-side xkbcommon WASM for state |
|
||||
| 6 | `06-data-clipboard.md` | Clipboard + DnD bridge, with primary selection (which Greenfield lacks) |
|
||||
| 7 | `07-native-app-bridge.md` | pywayland + bubblewrap + seccomp + systemd-run, with proper sandboxing (Greenfield has none) |
|
||||
| 8 | `08-app-lifecycle.md` | xdg-shell state machine — what to implement, with a clean-room spec |
|
||||
| 9 | `09-hardware-accel.md` | GStreamer Python + WebCodecs (no GPL x264, no GPL FFmpeg WASM) |
|
||||
| 10 | `10-odoo-integration.md` | Odoo 20 OWL shell + backend addon, JWT auth, multi-tenancy |
|
||||
| 11 | `11-pitfalls.md` | Pitfalls of the MIT approach + risk analysis + the 3 GPL contaminants found |
|
||||
| 12 | `12-appendix.md` | Glossary, file index, references |
|
||||
| — | `../Greenfield-vs-MIT-Comparison.md` | Top-level comparison document with person-week estimates and strong recommendation |
|
||||
|
||||
## TL;DR — can we really do this?
|
||||
|
||||
**YES — with three mandatory substitutions.**
|
||||
|
||||
The 9 MIT/Apache/BSD Greenfield libraries are reusable as-is, with
|
||||
**zero AGPL contamination** (verified by grep'ing every `import` statement
|
||||
in `libs/`). But the audit found three GPL contaminants that must be
|
||||
replaced:
|
||||
|
||||
1. **`@gfld/compositor-ffmpeg-h264` is GPL-3.0, not Apache-2.0** as its
|
||||
`package.json` claims. The TypeScript wrapper is Apache-2.0, but the
|
||||
compiled FFmpeg WASM binary is GPL-3.0 (built with `--enable-gpl
|
||||
--enable-version3 --enable-nonfree`). **Replace with WebCodecs
|
||||
`VideoDecoder` API** (browser-native, MIT/W3C standard).
|
||||
|
||||
2. **`x264enc` GStreamer plugin is GPL-2.0+**. Using it in a proprietary
|
||||
product forces GPL on the entire combined work. **Replace with
|
||||
`vaapih264enc` (Intel/AMD), `nvh264enc` (NVIDIA), or `openh264enc`
|
||||
(BSD libopenh264, CPU fallback)**.
|
||||
|
||||
3. **`gstreamer1.0-libav` on Debian is GPL** — the source is LGPL-2.1+,
|
||||
but Debian's `libavcodec.so.61` is built with `--enable-gpl` and
|
||||
directly links `libx264.so.164` + `libx265.so.215`. **Don't install
|
||||
this package** — use only gst-plugins-base + gst-plugins-good +
|
||||
gst-vaapi + BSD subset of gst-plugins-bad.
|
||||
|
||||
With these substitutions, **the proprietary strategy is unambiguously
|
||||
viable**. See `11-pitfalls.md` for the full audit table and
|
||||
`../Greenfield-vs-MIT-Comparison.md` for the final recommendation.
|
||||
|
||||
## Architecture in one diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ BROWSER (Odoo 20 web client) │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Odoo 20 OWL Shell addon (PROPRIETARY — your code) │ │
|
||||
│ │ - Application launcher, window manager, keyboard picker │ │
|
||||
│ │ - Per-output <canvas> element │ │
|
||||
│ │ - Implements UserShellApi.events.* callbacks (clean-room spec) │ │
|
||||
│ │ - Calls userShell.actions.* (initScene, activateSurface, ...) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ userShell.events + actions (clean-room API) │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Custom Browser Compositor Library (PROPRIETARY — your code, clean-room) │ │
|
||||
│ │ - Surface.ts, View.ts, Output.ts, Seat.ts, Keyboard.ts, Pointer.ts, ... │ │
|
||||
│ │ - render/ (WebGL2 + OffscreenCanvas + Web Workers) │ │
|
||||
│ │ - transport/ (WebSocket + KCP + ReconnectingWebSocket) │ │
|
||||
│ │ - roles/ (xdg-shell state machine, clean-room from spec) │ │
|
||||
│ │ - desktop/, input/, data/, math/ │ │
|
||||
│ │ ~13,400 LOC TypeScript, ~25 dev-weeks for MVP │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ Reuses MIT libs as npm deps: │
|
||||
│ │ - @gfld/common (MIT, wire format) │
|
||||
│ │ - @gfld/compositor-protocol (MIT, server stubs)│
|
||||
│ │ - @gfld/client-protocol (MIT, browser apps) │
|
||||
│ │ - @gfld/compositor-wasm (MIT, pixman+xkbcommon)│
|
||||
│ │ - @gfld/xtsb (MIT, X11 — only if XWayland) │
|
||||
│ │ - WebCodecs API (W3C standard, replaces GPL │
|
||||
│ │ FFmpeg WASM) │
|
||||
└──────────────────────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ WebSocket + KCP-over-WS
|
||||
│ 9 endpoint paths per session
|
||||
│ JWT (HS256) auth
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ PYTHON HOST (Linux server) │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Odoo 20 backend addon (PROPRIETARY — your code) │ │
|
||||
│ │ - /web/compositor/token → JWT issuance (HS256, 15-min expiry) │ │
|
||||
│ │ - /web/compositor/apps → per-user allowed apps list │ │
|
||||
│ │ - /web/compositor/launch → triggers proxy to spawn app │ │
|
||||
│ │ - /web/compositor/audit → audit log intake │ │
|
||||
│ │ - Models: compositor.app, compositor.session, compositor.audit.log │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ subprocess / IPC │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Python Compositor Proxy (PROPRIETARY — your code, ~2880 LOC) │ │
|
||||
│ │ - aiohttp WebSocket server (9 endpoints) on :8081 │ │
|
||||
│ │ - asyncio event loop integrating libwayland FD (5-line cffi shim) │ │
|
||||
│ │ - pywayland (Apache-2.0) for Wayland protocol dispatch │ │
|
||||
│ │ - PyGObject+GStreamer (LGPL) for H.264/PNG encoding │ │
|
||||
│ │ - Uses vaapih264enc / nvh264enc / openh264enc (NOT x264enc — GPL) │ │
|
||||
│ │ - python-xkbcommon (MIT) for keymap │ │
|
||||
│ │ - kcp PyPI package (MIT, Cython wrapper over ikcp.c) │ │
|
||||
│ │ - bubblewrap + seccomp + systemd-run for app sandboxing │ │
|
||||
│ │ - multiprocessing.Process per Odoo session (fork) │ │
|
||||
│ │ - JWT validation (pyjwt, MIT) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ │ Reuses Greenfield MIT C files as vendored src:│
|
||||
│ │ - westfield-buffer.c, westfield-dmabuf.c, │
|
||||
│ │ westfield-egl.c, westfield-xwayland.c │
|
||||
│ │ - wlr_drm.c, wlr_linux_dmabuf_v1.c (all MIT) │
|
||||
│ │ - Greenfield's changes.diff (patched │
|
||||
│ │ libwayland-server, MIT) │
|
||||
│ │ │
|
||||
│ │ WAYLAND_DISPLAY=wayland-N │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────────────────────────────────────────────────────────────┐ │
|
||||
│ │ Native Linux apps (GTK3/4, Qt, xterm, X11 via XWayland) │ │
|
||||
│ │ - Spawned via bwrap + seccomp + systemd-run --scope │ │
|
||||
│ │ - cgroup limits: MemoryMax=1G, CPUQuota=200%, TasksMax=256 │ │
|
||||
│ │ - Filesystem: /usr ro-bind, minimal /etc, $HOME overlay (discarded) │ │
|
||||
│ │ - GPU: /dev/dri/renderD128 only (no card0) │ │
|
||||
│ │ - Network: --unshare-net by default (most apps don't need it) │ │
|
||||
│ │ - UID remap to 100000-165535 (per-app, /etc/subuid) │ │
|
||||
│ └────────────────────────────────────────────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Effort summary (single engineer, sequential)
|
||||
|
||||
| Phase | Weeks | Deliverable |
|
||||
|---|---|---|
|
||||
| **Phase 0: Spike** | 1 | Validate libwayland ↔ asyncio integration (5-line cffi shim), validate kcp Python package works |
|
||||
| **Phase 1: Browser compositor MVP** | 25 | Clean-room TypeScript compositor library, ~13,400 LOC, runs a single Linux app in `<canvas>` |
|
||||
| **Phase 2: Python proxy MVP** | 3.5 (~17 dev-days) | aiohttp + pywayland + GStreamer, ~2880 LOC Python |
|
||||
| **Phase 3: Odoo integration** | 3.5 (~18 dev-days) | OWL shell + backend addon + JWT auth + Nginx + Docker |
|
||||
| **Phase 4: Sandboxing** | 1.5 (~7-9 dev-days) | bubblewrap + seccomp + systemd-run, v1 shippable |
|
||||
| **Phase 5: Polish & testing** | 3 | Tests, hardening, docs, XWayland (optional +8 weeks) |
|
||||
| **TOTAL (MVP)** | **~37 weeks** | Single-tenant, single-app, fully proprietary, sandboxed |
|
||||
| **TOTAL (with XWayland + multi-tenant)** | **~46 weeks** | Production-ready |
|
||||
|
||||
With 2 senior engineers in parallel: **~5-6 months to MVP**, **~7-9 months to production**.
|
||||
|
||||
See `../Greenfield-vs-MIT-Comparison.md` for the detailed comparison and
|
||||
recommendation.
|
||||
|
||||
## Key findings to remember
|
||||
|
||||
1. **License is clean** — with 3 substitutions (no GPL x264, no GPL FFmpeg WASM,
|
||||
no Debian gstreamer1.0-libav), the entire stack is MIT/Apache-2.0/BSD/LGPL.
|
||||
Fully proprietary product is shippable. (See `11-pitfalls.md`.)
|
||||
|
||||
2. **Clean-room is mandatory** — you CANNOT read Greenfield's AGPL
|
||||
`@gfld/compositor` source and reimplement it as "inspired-by" code.
|
||||
Under the *Computer Associates v. Altai* Abstraction-Filtration-Comparison
|
||||
test, even non-literal similarity (class names, method names, code
|
||||
structure) is sufficient to find derivative-work infringement. AGPL §13
|
||||
network-use clause would then force you to open-source your entire Odoo
|
||||
fork. **One person writes the spec (may read AGPL), another implements
|
||||
(must NOT read AGPL).** See `11-pitfalls.md` §Clean-room.
|
||||
|
||||
3. **pywayland is Apache-2.0, not MIT** — but Apache-2.0 is permissive
|
||||
(allows proprietary use, modification, distribution). The user's
|
||||
"MIT-only" preference is satisfied in spirit; the only obligation is
|
||||
to keep the LICENSE + NOTICE files. (See `02-wayland-protocol.md`.)
|
||||
|
||||
4. **pywlroots is NOT recommended** — it operates ABOVE the wire-dispatch
|
||||
layer (wlroots handles each request locally), incompatible with
|
||||
Greenfield's wire-forwarding architecture. Instead: pywayland + 5-line
|
||||
cffi shim + vendored Greenfield C files (MIT) for dmabuf/egl/xwayland.
|
||||
(See `07-native-app-bridge.md`.)
|
||||
|
||||
5. **Odoo's built-in WebSocket server is unsuitable** — `addons/bus/websocket.py`
|
||||
has a 1 MB frame cap, shared PostgreSQL pool, single gevent process.
|
||||
Run a separate aiohttp Python process on :8081 and route via Nginx.
|
||||
(See `10-odoo-integration.md`.)
|
||||
|
||||
6. **Sandboxing is critical and achievable in 7-9 days** — Greenfield has
|
||||
zero sandboxing (pitfall M). With bubblewrap + seccomp + systemd-run
|
||||
+ cgroups v2, v1 shippable in 7-9 engineer-days. (See
|
||||
`07-native-app-bridge.md`.)
|
||||
|
||||
7. **The browser compositor is the biggest effort** — ~25 dev-weeks for
|
||||
MVP, ~30-36 weeks total with XWayland. This is the bottleneck. See
|
||||
`03-html5-rendering.md` for the file structure and component-by-component
|
||||
estimate.
|
||||
|
||||
## How this analysis was produced
|
||||
|
||||
This bundle synthesizes findings from 10 parallel research agents (one
|
||||
failed with a context deadline, but its scope was covered by other agents):
|
||||
|
||||
- **Agent K** — pywayland deep dive (API surface, asyncio integration,
|
||||
license verification)
|
||||
- **Agent L** — wlroots + pywlroots + Smithay comparison, recommendation
|
||||
to NOT use pywlroots
|
||||
- **Agent M** — Greenfield MIT libs audit (per-library license verification,
|
||||
found GPL contaminant in compositor-ffmpeg-h264)
|
||||
- **Agent N** — GStreamer Python bindings feasibility, found x264enc GPL
|
||||
issue
|
||||
- **Agent P** — Custom browser compositor library design, clean-room
|
||||
legal analysis, file structure, effort estimate
|
||||
- **Agent Q** — Python WebSocket + KCP transport, recommended aiohttp +
|
||||
kcp PyPI package
|
||||
- **Agent R** — Odoo 20 integration patterns, JWT auth, multi-tenancy
|
||||
- **Agent S** — App sandboxing with bubblewrap + seccomp + systemd-run
|
||||
- **Agent T** — Comprehensive license audit, found 3 GPL contaminants,
|
||||
verified all other components are clean
|
||||
|
||||
Each agent read the worklog (`/home/z/my-project/worklog.md`), researched
|
||||
its assigned area, and appended findings. This bundle is the synthesis.
|
||||
|
||||
## License summary table
|
||||
|
||||
| Component | License | Compatible with proprietary? |
|
||||
|---|---|---|
|
||||
| **Browser side** | | |
|
||||
| @gfld/common | MIT | ✅ |
|
||||
| @gfld/client-protocol | MIT | ✅ |
|
||||
| @gfld/compositor-protocol | MIT | ✅ (verified NO @gfld/compositor import) |
|
||||
| @gfld/client-generator | MIT | ✅ |
|
||||
| @gfld/compositor-generator | MIT | ✅ |
|
||||
| @gfld/compositor-wasm (pixman+xkbcommon) | MIT | ✅ |
|
||||
| @gfld/xtsb (X11 bindings) | MIT | ✅ (only if XWayland needed) |
|
||||
| ~~@gfld/compositor-ffmpeg-h264~~ | ~~GPL-3.0~~ | ❌ REPLACE with WebCodecs |
|
||||
| Custom browser compositor (your code) | PROPRIETARY | ✅ (clean-room) |
|
||||
| **Python backend** | | |
|
||||
| pywayland | Apache-2.0 | ✅ |
|
||||
| python-xkbcommon (`xkbcommon` PyPI) | MIT | ✅ |
|
||||
| PyGObject (gi.repository.Gst) | LGPL-2.1+ | ✅ (dynamic linking) |
|
||||
| GLib | LGPL-2.1+ | ✅ (dynamic linking) |
|
||||
| GStreamer core | LGPL-2.0+ | ✅ (dynamic linking) |
|
||||
| gst-plugins-base, gst-plugins-good | LGPL | ✅ |
|
||||
| gst-vaapi (vaapih264enc) | LGPL + MIT (libva) | ✅ |
|
||||
| gst-plugins-bad (nvh264enc, openh264enc) | BSD subset | ✅ |
|
||||
| ~~gst-plugins-ugly (x264enc)~~ | ~~GPL-2.0+~~ | ❌ DROP |
|
||||
| ~~gst-libav (libavcodec)~~ | ~~LGPL source, GPL binary on Debian~~ | ❌ DROP |
|
||||
| libwayland | MIT | ✅ |
|
||||
| libdrm | MIT | ✅ |
|
||||
| mesa (libGL, libEGL, libgbm) | MIT | ✅ |
|
||||
| aiohttp | Apache-2.0 | ✅ |
|
||||
| kcp (PyPI, RealistikDash) | MIT (Cython wrapper over ikcp.c) | ✅ |
|
||||
| PyJWT | MIT | ✅ |
|
||||
| bubblewrap (bwrap) | LGPL-2.0+ | ✅ (dynamic linking) |
|
||||
| libseccomp | LGPL-2.1 | ✅ |
|
||||
| **Odoo integration** | | |
|
||||
| Odoo 20 Community | LGPL-3.0 | ✅ (separate process, no contamination) |
|
||||
| Nginx | BSD-2-Clause | ✅ |
|
||||
| Docker | Apache-2.0 | ✅ |
|
||||
Reference in new issue
Block a user