Files

424 lines
26 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 07 — Native App Bridge (pywayland + bubblewrap + seccomp + systemd-run)
This chapter covers the native app bridge in the MIT approach: how real
Linux applications get launched, sandboxed, connected to the Python proxy,
and routed to the browser. **This is the most important chapter for the
user** — they want to run real Linux apps in their Odoo 20 web app.
## Process model — three layers (same as Greenfield, but in Python)
### Layer 1: Main process (HTTP + WS hub)
Single Python process (aiohttp) listens on `:8081`. Per Odoo session, it
forks one `SessionProcess` child — one child per `compositor_session_id`,
**not** per app. Multiple apps in the same browser tab share one child.
WebSocket upgrades are handed off to the matching child via
`multiprocessing.reduction.send_handle` (uses SCM_RIGHTS on Linux).
### Layer 2: Session process
The forked child. Receives IPC messages:
- `start` — creates `Session` (Display, initShm, initDrm) + WS dispatcher
- `launch_app` — spawns the actual app binary via `subprocess.Popen` +
`bubblewrap` + `seccomp` + `systemd-run --scope`
- `ws_upgrade` — hands WS to the dispatcher
On `session.close()` the process exits.
### Layer 3: Per-client session
For each new `wl_client` (one per Wayland connection), creates a
`PROTOCOL` ARQ channel + `FRAME` ARQ channel. Marshalls FDs as ProxyFD
JSON. Routes requests via per-resource dispatchers.
## App launch flow (definitive answer)
1. Browser sends `GET http://proxy:8081/<path>` with
`Authorization: Bearer <jwt>` header.
2. Main process validates JWT (HS256, via `pyjwt`).
3. Main process forks a `SessionProcess` if none exists for that
`compositor_session_id`, sends `start` IPC.
4. Main sends `launch_app` IPC with `{name, executable, args, env, serial}`.
5. `SessionProcess` calls `launch_application()`:
- `appEnv = {...os.environ, ...env, WAYLAND_DISPLAY: session.wayland_display}`
- `subprocess.Popen(['systemd-run', '--user', '--scope', '-p',
'MemoryMax=1G', '-p', 'CPUQuota=200%', '-p', 'TasksMax=256', '--',
'bwrap', '--unshare-all', '--clearenv', '--seccomp', '<memfd>',
'--ro-bind', '/usr', '/usr', '--dev-bind', '/dev/dri/renderD128',
'/dev/dri/renderD128', '--ro-bind',
f'{host_xdg}/wayland-N', f'/run/user/{uid}/wayland-N',
'--uid', str(uid), '--gid', str(gid),
'--new-session', '--die-with-parent', executable, *args],
env=appEnv)`
6. On `spawn` event, captures `pid`, creates `NativeAppContext`, registers
`exit` listener.
7. Replies to main with `{pid, key}` → main returns HTTP 201 with
`{baseURL, signalURL, key, pid, name}`.
8. Browser opens `signalURL` as WebSocket → control channel established.
9. App connects to `$XDG_RUNTIME_DIR/wayland-N` → libwayland fires
`wl_client` creation → pywayland's per-resource dispatchers route
requests to `{native, browser}`.
10. For each new `wl_client`, creates a `PROTOCOL` ARQ channel and sends
`CONNECT_CHANNEL` over the signaling WS; browser opens a new WS at
`/channel?id=…`.
11. Wayland wire messages now flow: app → libwayland → pywayland
dispatcher → routed to `{native, browser}`.
## Sandboxing (NEW — Greenfield has none)
This is the **critical gap** in Greenfield (pitfall M). The MIT approach
ships with proper sandboxing from day one.
### Recommended stack
- **bubblewrap (bwrap)** — LGPL-2.0+, unprivileged, setuid-free, Flatpak-grade
- **seccomp BPF** — libseccomp LGPL-2.1, Docker's default profile + allow `CLONE_NEWUSER`
- **systemd-run --scope** — native cgroup v2 integration (MemoryMax, CPUQuota, TasksMax)
- **Linux user namespaces** — UID remap to 100000-165535 via `/etc/subuid`/`/etc/subgid`
- **cgroups v2** — memory, CPU, PID limits per app
### Reference bwrap command line (GTK4 app, GPU, no network)
```bash
systemd-run --user --scope --unit=app-odoo-calculator-$(uuid) \
-p MemoryMax=256M -p MemorySwapMax=0 -p CPUQuota=50% -p TasksMax=32 -p Delegate=yes -- \
bwrap \
--clearenv --unshare-all --hostname sandbox --new-session --die-with-parent \
--symlink usr/lib /lib --symlink usr/lib64 /lib64 \
--symlink usr/bin /bin --symlink usr/bin /sbin \
--ro-bind /usr /usr --ro-bind /opt /opt \
--ro-bind /etc/passwd /etc/passwd --ro-bind /etc/group /etc/group \
--ro-bind /etc/hosts /etc/hosts --ro-bind /etc/localtime /etc/localtime \
--ro-bind /etc/locale.conf /etc/locale.conf --ro-bind /etc/machine-id /etc/machine-id \
--ro-bind /etc/fonts /etc/fonts --ro-bind /etc/ssl /etc/ssl \
--ro-bind /etc/ca-certificates /etc/ca-certificates \
--proc /proc --dev /dev \
--dev-bind-try /dev/dri/renderD128 /dev/dri/renderD128 \
--tmpfs /tmp --tmpfs /var/tmp --tmpfs /run \
--dir /run/user/100000 \
--ro-bind "$HOST_XDG/wayland-0" /run/user/100000/wayland-0 \
--overlay /usr/share/compositor-proxy/skel/gnome-calculator:/var/lib/compositor-proxy/tenants/$TENANT/appdata/gnome-calculator-upper /home/appuser \
--setenv HOME /home/appuser \
--setenv XDG_RUNTIME_DIR /run/user/100000 \
--setenv WAYLAND_DISPLAY wayland-0 \
--setenv XDG_DATA_DIRS /usr/share --setenv PATH /usr/bin --setenv LANG en_US.UTF-8 \
--uid 100000 --gid 100000 \
--seccomp /proc/self/fd/3 \
/usr/bin/gnome-calculator
```
### Key design decisions
| Concern | Decision | Rationale |
|---|---|---|
| Sandbox runner | **bubblewrap** | MIT license, unprivileged, setuid-free, Flatpak-grade |
| cgroup limits | **`systemd-run --user --scope` wrapping bwrap** | Native cgroup v2 integration; no manual cgroupfs writes |
| Wayland socket | **Single-file `--ro-bind` of `wayland-N`** into fresh `/run/user/<UID>` tmpfs | Critical: prevents access to PulseAudio, pipewire, D-Bus, keyring, and sibling tenants' sockets |
| GPU | **`--dev-bind /dev/dri/renderD128`** (single render node, NOT `card0`) | Kernel render-node access control already restricts to non-display ioctls |
| Network | **`--unshare-net` by default** (loopback-down only); per-app opt-in to veth+nftables filter | Most GTK apps need no network. Firefox/Odoo clients opt in. |
| Seccomp | **Docker's default profile** (allowlist of ~250 syscalls, blocking ~44) + allow `CLONE_NEWUSER` for nested sandboxes | Compile to BPF at proxy startup; pass via `bwrap --seccomp <memfd>` |
| Filesystem | **`/usr` ro-bind, minimal `/etc` whitelist, `/tmp` tmpfs, `$HOME` overlay (changes discarded)** | No host write access; per-app `$HOME` reset on each launch kills malware persistence |
| User ns | **UID remap to 100000–165535 via `/etc/subuid`/`/etc/subgid`** | Even on kernel escape the app is unprivileged on host; per-app UID enables nftables filtering and audit |
| XWayland | **v1: unsandboxed (documented limitation); v2: per-Xwayland bwrap sandbox** | Hardest piece (3–4 days). v1 ships like Flatpak's old `socket=x11`. |
| `$XDG_RUNTIME_DIR` | **`--clearenv` then explicit `--setenv` whitelist** | No host env leak (DB password, Odoo session cookies, AWS keys, etc.) |
## Native addon architecture (NO native Node addons — pure Python)
**No N-API addons, no `.node` files, no CMake/Ninja build chain.** The
entire proxy is pure Python.
### Replaces Greenfield's 3 native C addons
| Greenfield native | Python replacement | Notes |
|---|---|---|
| `wayland-server-addon.node` (1231 lines C, patched libwayland) | **pywayland** (Apache-2.0) + 5-line cffi shim | Vanilla upstream libwayland 1.26.0 bundled in pywayland's wheel. Apply Greenfield's `changes.diff` (MIT patches) optionally for fast-sync optimization. |
| `proxy-poll-addon.node` (127 lines C, libuv `uv_poll_t`) | `asyncio.get_running_loop().add_reader(wl_fd, dispatch_fn)` | Built-in asyncio. NO addon needed. |
| `proxy-encoding-addon.node` (290 lines C) + `libproxy-encoding.so` (~1400 lines C, GStreamer) | **PyGObject + GStreamer Python bindings** (LGPL) | Same GStreamer, Python wrapper. ~700 LOC Python replaces ~1700 LOC C. |
| `westfield-*.c` (EGL, dmabuf, xwayland helpers, ~2000 lines C) | **Vendored Greenfield C files via cffi** (MIT) | Reuse verbatim. ~3000 LOC of MIT C cherry-picked from Greenfield. |
### Vendored Greenfield C files (MIT, reused as-is)
These files from `greenfield/packages/compositor-proxy/native/wayland/src/`
are MIT-licensed (Greenfield's own work or vendored from wlroots which is
also MIT) and can be **reused verbatim** in our Python proxy:
| File | Lines | Purpose |
|---|---|---|
| `westfield-buffer.c` | ~150 | Buffer helpers |
| `westfield-dmabuf.c` | ~200 | DMA-BUF import via EGL/GBM |
| `westfield-egl.c` | ~900 | EGL zero-copy setup |
| `westfield-xwayland.c` | ~465 | XWayland spawn + bridge |
| `wlr_drm.c` | ~244 | `wl_drm` global impl (vendored from wlroots) |
| `wlr_linux_dmabuf_v1.c` | ~1,110 | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) |
| `drm_format_set.c` | ~239 | DRM format helpers (vendored from wlroots) |
| `addon.c` | ~73 | wlroots addon system helper |
| **TOTAL** | **~3,374 lines C** | All MIT, reused verbatim |
These compile into a single `libwestfield.so` shared library, loaded by
the Python proxy via cffi at runtime.
## XWayland bridge
### Native side (Python equivalent of westfield-xwayland.c)
In Python:
```python
import subprocess, os, socket
def spawn_xwayland(display_num, wl_client_fd, wm_fd, display_fd):
# Create X11 socket
x11_socket = f"/tmp/.X11-unix/X{display_num}"
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
listener.bind(x11_socket)
listener.listen(1)
# Set DISPLAY env
os.environ["DISPLAY"] = f":{display_num}"
# Fork + exec Xwayland
subprocess.Popen([
"Xwayland", f":{display_num}", "-ac", "-rootless",
"-listen", str(listener.fileno()),
"-displayfd", str(display_fd),
"-wm", str(wm_fd),
], pass_fds=[listener.fileno(), display_fd, wm_fd])
# Xwayland itself becomes a Wayland client of the proxy
# (via wl_client_create(wl_display, wl_client_fd))
```
### xtsb (REUSED MIT, browser side)
`@gfld/xtsb` (MIT) — pure-TS X11 client, no libxcb. Dual-environment
(Node `net.Socket` + browser WebSocket). Auto-generated from XCB XML
protocol files via `xcbgen/ts_client.py`. The browser-side xtsb
implements the actual XWM logic (translates X `CreateNotify` events →
`xdg_toplevel` Wayland surfaces).
**Optional**: skip XWayland in MVP (8 weeks of effort). Add only if you
need to run X11-only apps (most modern GTK/Qt apps are Wayland-native).
## SDK misconception corrected (same as Greenfield)
The `sdk/` directory is **NOT for native apps**. It's exclusively for
**WASM apps** that run inside the browser via Emscripten. The
`sysrootlibs/` cross-compiles wayland/gtk/cairo/etc. to wasm32. Native
Linux apps on the proxy use the host's own system libraries.
## ASCII diagram — native app launch & connection
```
Browser (Odoo OWL shell) Python proxy host (Linux) App (sandboxed in bwrap)
========================== ==================== ==============================
[1] User clicks app in shell $ python -m compositor_proxy.cli \
│ └─── main.py listens on :8081
│ sessionProcesses: Dict[id, Process]
│
│ HTTP GET /gtk4-demo
│ Authorization: Bearer <jwt>
├──────────────────────────────► │
│ │ validate JWT (HS256, pyjwt)
│ │ if no child for <session_id>:
│ │ multiprocessing.Process(target=session_main)
│ │ IPC: {type:'start', payload:{config, session_id}}
│ │ └── create_session() → Display
│ │ ├── Display()
│ │ ├── display.add_socket() → "wayland-N"
│ │ │ (becomes WAYLAND_DISPLAY)
│ │ ├── display.init_shm() + YUV420/NV12/...
│ │ ├── init_drm("/dev/dri/renderD128")
│ │ │ (uses vendored westfield-egl.c via cffi)
│ │ │ (registers zwp_linux_dmabuf_v1 + wl_drm globals)
│ │ └── asyncio.add_reader(wl_fd, dispatch_wl)
│ │ (5-line cffi shim for wl_event_loop_get_fd)
│ │
│ │ IPC: {type:'launch_app', payload:{name, executable, args, env, serial}}
│ │ └── launch_application() (NativeAppContext.py)
│ │ ├── appEnv = {...os.environ, ...env,
│ │ │ WAYLAND_DISPLAY: "wayland-N"}
│ │ ├── subprocess.Popen([
│ │ │ 'systemd-run', '--user', '--scope',
│ │ │ '-p', 'MemoryMax=1G',
│ │ │ '-p', 'CPUQuota=200%',
│ │ │ '-p', 'TasksMax=256', '--',
│ │ │ 'bwrap',
│ │ │ '--unshare-all', '--clearenv',
│ │ │ '--seccomp', '<memfd>',
│ │ │ '--ro-bind', '/usr', '/usr',
│ │ │ '--dev-bind', '/dev/dri/renderD128', '/dev/dri/renderD128',
│ │ │ '--ro-bind', '$XDG/wayland-N', '/run/user/100000/wayland-N',
│ │ │ '--uid', '100000', '--gid', '100000',
│ │ │ '--new-session', '--die-with-parent',
│ │ │ '/usr/bin/gtk4-demo'
│ │ │ ], env=appEnv)
│ │ │ ◄── systemd-run + bwrap, NO chroot, NO container
│ │ │ ──────────────────────────────► [2] gtk4-demo (sandboxed)
│ │ │ │
│ │ │ │ libwayland-client.so
│ │ │ │ connects to
│ │ │ │ $XDG_RUNTIME_DIR/wayland-N
│ │ │ │
│ │ │ ▼
│ │ │ ┌──────────────────────────┐
│ │ │ │ wl_display event loop │
│ │ │ │ (in proxy, libwayland- │
│ │ │ │ server.so.0 — vanilla, │
│ │ │ │ NO patches needed) │
│ │ │ │ │
│ │ │ ◄── asyncio fires, │ wl_client created → │
│ │ │ dispatch_wl() called │ Client.get_credentials │
│ │ │ wl_event_loop_dispatch() │ → (pid, uid, gid) │
│ │ │ │ find_matching_app_ctx │
│ │ │ │ (pid) by PPID walk │
│ │ │ │ │
│ │ │ │ Per-resource dispatchers│
│ │ │ │ (auto-generated by │
│ │ │ │ pywayland scanner) │
│ │ │ │ route to {native, │
│ │ │ │ browser} │
│ │ │ └──────────────────────────┘
│ │ │
│ │ ├── child.on('spawn') → capture pid
│ │ │ create_native_app_context(pid, name, sandbox=True)
│ │ │ native_app_context.on_disconnect() # start 10-min SIGHUP timer
│ │ │
│ │ └── child.on('exit') → native_app_context.on_exit({exit_code|signal})
│ │
│ │ IPC reply: {type:'launch_app_success', payload:{pid, key}}
│
│ HTTP 201 Created
│ Body: { baseURL, signalURL:"ws://…/signal?key=…", key, pid, name, sandboxed:true }
◄────────────────────────────── │
│ │
│ Open WS: ws://…/signal?key=<16-hex>
│ Authorization: Bearer <jwt>
├──────────────────────────────► │
│ │ aiohttp demux /signal → native_app_context.on_connect(ws)
│ │ clears 10-min SIGHUP timer
│ │
│ │ For each new wl_client (one per wayland connection):
│ │ create_protocol_channel() → ARQChannel (kcp over WS)
│ │ sends CONNECT_CHANNEL {url:"ws://…/channel?id=…&key=…", desc}
│ ◄──── CONNECT_CHANNEL JSON ───┤
│ │
│ Open WS: ws://…/channel?id=…&key=…
│ Authorization: Bearer <jwt>
├──────────────────────────────► │ binds WS to the ARQChannel
│ │
│ Wayland wire protocol flows over the ARQ channel (KCP reliable, 100ms interval):
│
│ App → wl_surface.commit ──── proxy intercepts via dispatcher ────► browser compositor
│ App ← wl_surface.frame ──── proxy intercepts E1 (frame callback) ◄── FrameFeedback paces callback
│ App → wl_display.sync ────── proxy fast-path sync (no roundtrip if no reply expected)
│ App ← wl_keyboard.enter ─── browser → proxy → libwayland-server sends via SCM_RIGHTS
│
│ Buffer path (per wl_surface):
│ - App submits wl_buffer (dmabuf or shm)
│ - setBufferCreatedCallback → ProxyBuffer with creationSerial
│ - On commit: dispatcher → Encoder.encodeFrame()
│ └── gst_parse_launch("appsrc ! glupload ! glcolorconvert ! glshader ! "
│ "glcolorconvert ! gldownload ! queue ! vaapih264enc ! appsink")
│ (vaapih264enc — LGPL, NOT x264enc which is GPL)
│ └── GLib MainLoop in separate thread
│ └── encoded H.264 sample → asyncio.run_coroutine_threadsafe → main thread
│ └── sent over the surface's FRAME data channel (separate WS)
│
│ X11 path (xterm etc., IF XWayland enabled):
│ - App connects to /tmp/.X11-unix/X1 (DISPLAY=:1, set by spawn_xwayland())
│ - First X connect → spawn_xwayland() forks Xwayland: execlp("Xwayland", …, "-rootless", "-wm", wm_fd)
│ - Xwayland itself becomes a Wayland client of the proxy (wl_client_create)
│ - When Xwayland ready (displayFd readable):
│ └── xtsb (REUSED MIT, browser side) speaks X11 → translates X windows → xdg_toplevel surfaces
│
│ Lifecycle:
│ - User closes browser tab → /signal WS closes with code 1001 → kill('SIGHUP')
│ - User clicks "force quit" in shell → /signal WS sends KILL_APP {signal:'SIGTERM'} → kill('SIGTERM')
│ └── if not exited in 10s → SIGKILL
│ - App crashes → child 'exit' event → native_app_context.on_exit() → APP_TERMINATED msg → browser UI updates
│ - All clients gone for >1s → session.close() → process.exit() in SessionProcess.py → main deletes entry
│
│ Sandbox (NEW vs Greenfield):
│ - bwrap --unshare-all --clearenv --seccomp <memfd>
│ - --ro-bind /usr /usr, minimal /etc whitelist
│ - --dev-bind /dev/dri/renderD128 (single GPU render node, no card0)
│ - --ro-bind $HOST_XDG/wayland-N /run/user/100000/wayland-N (single socket only)
│ - --uid 100000-165535 per-app (--gid same)
│ - --unshare-net by default
│ - cgroup: MemoryMax=1G, CPUQuota=200%, TasksMax=256
│ - $HOME overlay (changes discarded on app exit)
```
## Effort estimates
| Phase | Days | Deliverable |
|---|---|---|
| **v1 shippable** | **7-9 days** | bubblewrap integration + seccomp + cgroup limits + Wayland socket isolation + GPU render-node passthrough + default-deny net. XWayland unsandboxed (documented). |
| v2 full | +4-9 days | + XWayland sandbox + veth+nftables networking + per-tenant hierarchical cgroups + XDG Desktop Portal |
### v1 breakdown
| Component | Days |
|---|---|
| bwrap integration | 2-3 |
| seccomp profile + compile | 1-2 |
| systemd-run cgroup wrapper | 1 |
| Wayland socket ownership fix | 0.5 |
| GPU passthrough testing | 1 |
| Schema + docs | 1 |
| **TOTAL** | **7-9** |
## File reference
### Python proxy (custom, ~2880 LOC)
| File | Role |
|---|---|
| `proxy/main.py` | HTTP + WS hub, forks per session |
| `proxy/session.py` | Top-level proxy session |
| `proxy/session_controller.py` | aiohttp WS server with 9 endpoint paths |
| `proxy/native_app_context.py` | Per-app state, signaling WS, kill timers, bwrap spawn |
| `proxy/native_wayland_client_session.py` | Per-client message dispatcher |
| `proxy/native_wayland_compositor_session.py` | Display creation, initShm, initDrm |
| `proxy/channel.py` | ARQChannel + SimpleChannel (clean-room) |
| `proxy/kcp_adapter.py` | kcp PyPI package integration |
| `proxy/frame_feedback.py` | Receives 4-byte decode feedback, paces wl_frame.done |
| `proxy/proxy_buffer.py` | Proxy-side buffer representation |
| `proxy/surface_buffer_encoding.py` | Encoding pipeline orchestration |
| `proxy/xwayland_session.py` | XWayland spawn + bridge (Python equivalent of westfield-xwayland.c) |
| `proxy/wayland_server.py` | pywayland wrapper + cffi shim |
| `proxy/io/` | WebSocket server glue |
| `proxy/encoding/` | GStreamer encoder orchestration via PyGObject |
| `proxy/protocol/` | pywayland-generated dispatchers (auto) |
| `proxy/jwt_auth.py` | JWT validation (pyjwt) |
| `proxy/sandbox.py` | bubblewrap + seccomp + systemd-run wrappers |
### Vendored Greenfield C files (MIT, reused as-is)
| Path | Role |
|---|---|
| `native/westfield-buffer.c` | Buffer helpers |
| `native/westfield-dmabuf.c` | DMA-BUF import via EGL/GBM |
| `native/westfield-egl.c` | EGL zero-copy setup |
| `native/westfield-xwayland.c` | XWayland spawn + bridge (used as reference, ported to Python) |
| `native/wlr_drm.c` | `wl_drm` global impl (vendored from wlroots) |
| `native/wlr_linux_dmabuf_v1.c` | `zwp_linux_dmabuf_v1` global impl (vendored from wlroots) |
| `native/drm_format_set.c` | DRM format helpers (vendored from wlroots) |
| `native/addon.c` | wlroots addon system helper |
| `native/changes.diff` | Patches to libwayland-server (optional, for fast-sync) |
### Reused (MIT/Apache)
| Component | Role |
|---|---|
| pywayland | Python bindings to libwayland-server |
| `xkbcommon` PyPI package | Python bindings to libxkbcommon (optional, for XWayland) |
| PyGObject (`gi.repository.Gst`) | GStreamer Python bindings |
| `kcp` PyPI package | Cython wrapper over upstream `ikcp.c` |
| aiohttp | Async HTTP + WebSocket server |
| PyJWT | JWT validation |
| bubblewrap | Unprivileged sandboxing |
| libseccomp | Seccomp BPF filter compilation |