MCP demo v2: examples/demo/enclosure_ui_mcp.py builds the enclosure through the UI like a person (toolbar clicks, real dialogs, sketches on the model's faces, Look At and back out, orbits between features) with the parameters and their dimensions over MCP; record_v2.sh and overlay.py record it with captions (UI click / MCP call) and a drawn pointer with click ripples. Toolbar buttons, drop-down items and panel labels publish handles (toolbar:<id>, panel:<tab>:<panel>); control method ui.handles lists them

Co-Authored-By: Claude Opus 5.5 <[email protected]>

.gitignore

@@ -6,3 +6,4 @@
 *.swp
 /.claude/worktrees/
 /target-snapshot
+__pycache__/

crates/ui-egui/src/control.rs

@@ -252,6 +252,11 @@ pub fn handle(app: &mut SolveApp, ctx: &egui::Context, req: &ControlRequest) ->
                 _ => err("no dialog input with that index"),
             }
         }
+        "ui.handles" => {
+            // The handles drawn last frame (toolbar buttons `toolbar:<id>`, panels, browser rows…).
+            let list = crate::scenario::handles(s("prefix").unwrap_or(""));
+            ok(json!(list.into_iter().map(|(n, p)| json!({"name": n, "x": p.x, "y": p.y})).collect::<Vec<_>>()))
+        }
         "ui.at" => {
             // A scenario point: {world: [x,y,z]} | {sketch: [x,y]} | {plane: "XY"} (the middle of an
             // origin plane's square) → screen point.

crates/ui-egui/src/scenario.rs

@@ -62,6 +62,11 @@ pub fn handle_at(name: &str) -> Option<egui::Pos2> {
     HANDLES.with(|h| h.borrow().iter().rfind(|(n, _)| n == name).map(|x| x.1))
 }
 
+/// Every handle drawn last frame whose name starts with `prefix`.
+pub fn handles(prefix: &str) -> Vec<(String, egui::Pos2)> {
+    HANDLES.with(|h| h.borrow().iter().filter(|(n, _)| n.starts_with(prefix)).cloned().collect())
+}
+
 /// A painted part's rect this frame, by name ("toolbar", "workspace_switcher").
 pub fn publish_rect(name: &str, r: egui::Rect) {
     RECTS.with(|h| h.borrow_mut().push((name.to_string(), r)));

crates/ui-egui/src/toolbar.rs

@@ -297,6 +297,7 @@ pub fn toolbar(app: &mut SolveApp, ui: &mut egui::Ui) {
                 let br = Rect::from_min_size(pos2(px + 4.0 + i as f32 * 40.0, top + 2.0), vec2(38.0, 38.0));
                 let info = c.info(&app.session);
                 let resp = ui.interact(br, ui.id().with(("cmd", c.id)), Sense::click());
+                crate::scenario::publish_handle(&format!("toolbar:{}", c.id), br.center());
                 if resp.hovered() && info.enabled {
                     painter.rect_filled(br, 4.0, t.hover);
                 }
@@ -324,6 +325,7 @@ pub fn toolbar(app: &mut SolveApp, ui: &mut egui::Ui) {
             let lr = Rect::from_min_size(pos2(px, top + 42.0), vec2(width, 18.0));
             let label = panel.to_string();
             let lresp = ui.interact(lr, ui.id().with(("panel", panel)), Sense::click());
+            crate::scenario::publish_handle(&format!("panel:{tab}:{panel}"), lr.center());
             if lresp.hovered() && enabled_panel {
                 painter.rect_filled(lr, 3.0, t.hover);
             }
@@ -360,6 +362,7 @@ pub fn toolbar(app: &mut SolveApp, ui: &mut egui::Ui) {
                                 let (ir, _) = ui.allocate_exact_size(vec2(18.0, 18.0), Sense::hover());
                                 icons::paint(ui.painter(), ir, c.icon, t.icon, t.icon_fill, t.accent);
                                 let b = ui.add_enabled(info.enabled, egui::Button::new(c.label).frame(false).shortcut_text(c.shortcut.unwrap_or("")));
+                                crate::scenario::publish_handle(&format!("toolbar:{}", c.id), b.rect.center());
                                 if b.clicked() {
                                     let id = c.id;
                                     app.start(id);

docs/control-protocol.md

@@ -37,6 +37,7 @@ Commands never open dialogs when run this way. `solvecraft-cli commands` prints
 | `ui.sketchToScreen` | `points`: `[[x, y]…]` in active-sketch coordinates | their screen points (null when behind the camera) |
 | `ui.dialogInput` | `index` | what clicking an input's Select chip does: picks go to that input |
 | `ui.at` | `world: [x,y,z]` \| `sketch: [x,y]` \| `plane: "XY"` \| `axis: "Z"` \| `dimension: "d1"` (its value text) \| `handle: "arrow"` (a manipulator handle, see `scenario::publish_handle`) | the screen point (a scenario step's target; see `crates/ui-egui/src/scenario.rs`) |
+| `ui.handles` | `prefix?` | the handles drawn last frame, `[{name, x, y}]`: toolbar buttons and drop-down items `toolbar:<command id>`, panel labels `panel:<tab>:<panel>`, browser rows `row:<label>` and fold arrows `fold:<label>`, manipulator handles |
 | `ui.editFeature` | `feature` (id or name) | what double-clicking a timeline item does: rolls back to it and opens its dialog filled in (sketches open in sketch mode) |
 | `ui.scroll` | `x`, `y`, `delta?` | wheel zoom at the cursor |
 | `ui.key` | `key` (egui key name, `Enter`, `Escape`…), `cmd?`, `shift?` | |

docs/mcp.md

@@ -81,6 +81,12 @@ USB port cut through a side wall, four screw bosses, and then a change of the `w
 parameter that the timeline rebuilds. `examples/demo/record.sh` records the run with ffmpeg,
 with each step's caption burnt in.
 
+`examples/demo/enclosure_ui_mcp.py` (v2) builds the same part the way a person would: the
+modelling goes through the app's UI over its control channel. The pointer glides to toolbar
+buttons and faces and clicks them, the real dialogs open, the sketches go on the model's faces,
+and the camera follows. The parameters, the sketch dimensions bound to them and two extrudes go
+over MCP. `record_v2.sh` records it and draws the pointer and the captions.
+
 ## A typical session
 
 ```text

examples/demo/enclosure_ui_mcp.py

@@ -0,0 +1,251 @@
+#!/usr/bin/env python3
+"""Demo v2: build an electronics enclosure in a running SolveCraft the way a person would, with
+the parametric steps sent over MCP.
+
+    solvecraft --control 7878 &
+    python3 examples/demo/enclosure_ui_mcp.py --port 7878 --cli solvecraft-cli
+
+Modelling steps go through the app's UI over its control channel: the pointer glides to a
+toolbar button and clicks it (the real dialog opens), glides to the face or point it picks and
+clicks, and values are typed into the dialog, as by hand. Sketches go on the model's faces and
+the camera follows like a user's would (Look At on a new sketch, back out to a 3/4 view on
+Finish Sketch, a little orbit between features). The parameters and the dimensions that use them
+are MCP `execute` calls through `solvecraft-cli mcp --connect` (enclosure_mcp.py's client).
+
+--log writes one JSON line per caption and per pointer event, which record.sh turns into burnt-in
+captions and a drawn cursor (the app gets synthetic pointer events, so the screen's own pointer
+never moves).
+"""
+
+import argparse
+import json
+import math
+import os
+import sys
+import time
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from enclosure_mcp import Mcp, control  # noqa: E402
+
+
+class Demo:
+    def __init__(self, port, mcp, log, speed):
+        self.port, self.mcp, self.log, self.speed = port, mcp, log, speed
+        self.pos = None
+        self.stop = 10**6
+
+    # -- logging -------------------------------------------------------------------------------
+    def mark(self, **k):
+        if self.log:
+            self.log.write(json.dumps({"t": time.time(), **k}) + "\n")
+            self.log.flush()
+
+    def caption(self, text, via=""):
+        self.n = getattr(self, "n", 0) + 1
+        if self.n > self.stop:
+            raise StopIteration
+        print(f"  {text}  [{via}]", file=sys.stderr)
+        self.mark(caption=text, via=via)
+
+    def wait(self, s):
+        time.sleep(s * self.speed)
+
+    # -- the app -------------------------------------------------------------------------------
+    def ui(self, method, params=None):
+        r = control(self.port, method, params)
+        if not r.get("ok", False):
+            raise RuntimeError(f"{method} {params}: {r.get('error')}")
+        return r.get("result")
+
+    def at(self, **where):
+        p = self.ui("ui.at", where)
+        return (p[0], p[1])
+
+    def glide(self, to, dur=0.35):
+        """Move the pointer to `to` along an eased path (the app sees every step: hover
+        highlights follow it)."""
+        frm = self.pos or (1150.0, 620.0)
+        n = max(2, int(dur * 40))
+        for i in range(1, n + 1):
+            t = i / n
+            e = t * t * (3 - 2 * t)
+            x, y = frm[0] + (to[0] - frm[0]) * e, frm[1] + (to[1] - frm[1]) * e
+            self.ui("ui.move", {"x": x, "y": y})
+            self.mark(cursor=[x, y])
+            time.sleep(dur / n)
+        self.pos = to
+
+    def click(self, to, dur=0.35, settle=0.25):
+        self.glide(to, dur)
+        self.ui("ui.click", {"x": to[0], "y": to[1]})
+        self.mark(cursor=list(to), click=True)
+        self.wait(settle)
+
+    def tool(self, command, panel=None):
+        """Click a toolbar button (opening its panel's drop-down first when it's in one)."""
+        if panel:
+            self.click(self.at(handle=panel), settle=0.3)
+        self.click(self.at(handle=f"toolbar:{command}"), settle=0.35)
+
+    def type(self, text, enter=True):
+        self.ui("ui.text", {"text": text})
+        self.wait(0.35)
+        if enter:
+            self.ui("ui.key", {"key": "Enter"})
+            self.wait(0.5)
+
+    def key(self, k):
+        self.ui("ui.key", {"key": k})
+        self.wait(0.15)
+
+    def view(self, v, settle=0.7):
+        self.ui("ui.view", {"view": v, "animate": True})
+        self.wait(settle)
+
+    def orbit(self, dx, dy=0, steps=40):
+        """A right-button drag in the viewport, as a user orbits."""
+        vp = self.ui("ui.inspect").get("viewport") or [250, 126, 1350, 800]
+        cx, cy = vp[0] + vp[2] * 0.62, vp[1] + vp[3] * 0.55
+        self.glide((cx, cy), 0.25)
+        self.mark(drag_start=[cx, cy])
+        self.ui("ui.drag", {"x0": cx, "y0": cy, "x1": cx + dx, "y1": cy + dy, "button": "right", "steps": steps})
+        self.settle_camera()
+        # The pointer moved with the camera (the app plays the drag one event a frame).
+        self.mark(drag_end=[cx + dx, cy + dy])
+        self.pos = (cx + dx, cy + dy)
+
+    def settle_camera(self):
+        """Wait until the app has played queued pointer events (the camera stops moving)."""
+        last, still = None, 0
+        for _ in range(80):
+            time.sleep(0.1)
+            cam = json.dumps(self.ui("ui.inspect").get("camera"))
+            still = still + 1 if cam == last else 0
+            last = cam
+            if still >= 2:
+                return
+
+    def execute(self, command, params):
+        self.mcp.tool("execute", {"command": command, "params": params})
+
+
+def run(d):
+    d.caption("Parameters over MCP: width 80, depth 60, height 30, wall 2", "MCP tools/call execute parameters.change")
+    for name, expr in [("width", "80 mm"), ("depth", "60 mm"), ("height", "30 mm"), ("wall", "2 mm")]:
+        d.execute("parameters.change", {"name": name, "expression": expr})
+        d.wait(0.25)
+    d.wait(0.6)
+
+    d.caption("Create Sketch on the XY plane", "UI click")
+    d.tool("sketch.create")
+    d.click(d.at(plane="XY"), settle=0.8)
+
+    d.caption("Center rectangle from the origin", "UI click")
+    d.tool("sketch.rectangle.center")
+    d.click(d.at(sketch=[0, 0]), settle=0.15)
+    d.click(d.at(sketch=[40, 30]), dur=0.45, settle=0.2)
+    d.key("Escape")
+
+    d.caption("Constraints over MCP: centred, width × depth", "MCP tools/call execute sketch.dimension")
+    sk = json.loads(d.mcp.tool("execute", {"command": "sketch.inspect", "params": {}}))
+    sk = sk.get("result", sk)
+    centre = next((p["id"] for p in sk.get("points", []) if p["id"] != "origin" and abs(p["at"][0]) + abs(p["at"][1]) < 1e-6), None)
+    if centre:
+        d.execute("sketch.constraint.coincident", {"a": centre, "b": "origin"})
+    d.execute("sketch.dimension", {"entities": ["l1"], "value": "width"})
+    d.wait(0.3)
+    d.execute("sketch.dimension", {"entities": ["l2"], "value": "depth"})
+    d.wait(0.8)
+
+    d.caption("Finish Sketch", "UI click")
+    d.tool("sketch.finish")
+    # The origin planes are in the way of picking faces from here on.
+    d.ui("ui.set", {"showOrigin": False})
+    d.view("home", 0.6)
+
+    d.caption("Extrude: height", "UI click")
+    d.tool("solid.extrude")
+    d.wait(0.3)
+    d.type("height")
+
+    d.caption("Fillet the four vertical edges: 6 mm", "UI click")
+    d.tool("solid.fillet")
+    for p in ([40, -30, 15], [-40, -30, 15], [40, 30, 15]):
+        d.click(d.at(world=p), settle=0.15)
+    d.orbit(-260, 20)
+    d.click(d.at(world=[-40, 30, 15]), settle=0.2)
+    d.type("6")
+    d.view("home", 0.5)
+
+    d.caption("Shell: open the top face, wall thickness", "UI click")
+    d.tool("solid.shell", panel="panel:SOLID:MODIFY")
+    d.click(d.at(world=[0, 0, 30]), settle=0.3)
+    d.type("wall")
+
+    d.caption("Sketch on the side face for a USB port", "UI click")
+    d.tool("sketch.create")
+    d.click(d.at(world=[40, 10, 20]), settle=0.9)
+    d.tool("sketch.rectangle.center")
+    d.click(d.at(world=[40, 0, 12]), settle=0.15)
+    d.click(d.at(world=[40, 6, 15.5]), dur=0.4, settle=0.2)
+    d.key("Escape")
+    d.tool("sketch.finish")
+    d.view("home", 0.5)
+
+    # Over MCP with the profile named by a point inside it: a profile picked in the Extrude
+    # dialog is kept by its index, which a face sketch renumbers when the face changes shape.
+    d.caption("Cut the port through the wall", "MCP tools/call execute solid.extrude")
+    d.execute("solid.extrude", {"profiles": [{"point": [0, 12]}], "distance": "-wall", "operation": "cut"})
+    d.wait(0.6)
+    d.orbit(-140, 30)
+
+    d.caption("Sketch on the inner floor: four standoffs", "UI click")
+    d.tool("sketch.create")
+    d.click(d.at(world=[6, -9, 2]), settle=0.9)
+    d.tool("sketch.circle.center")
+    for x, y in [(-30, -20), (30, -20), (30, 20), (-30, 20)]:
+        d.click(d.at(world=[x, y, 2]), dur=0.3, settle=0.1)
+        d.click(d.at(world=[x + 3.5, y, 2]), dur=0.2, settle=0.15)
+    d.key("Escape")
+    d.tool("sketch.finish")
+
+    d.view("home", 0.5)
+
+    d.caption("Extrude the standoffs: 12 mm", "MCP tools/call execute solid.extrude")
+    d.execute("solid.extrude", {"profiles": [{"point": [x, y]} for x, y in [(-30, -20), (30, -20), (30, 20), (-30, 20)]], "distance": 12, "operation": "join"})
+    d.wait(0.4)
+
+    d.caption("Change one parameter: width 80 → 100 mm", "MCP tools/call execute parameters.change")
+    d.execute("parameters.change", {"name": "width", "expression": "100 mm"})
+    d.wait(1.8)
+
+    d.caption("", "")
+    d.orbit(420, 0, steps=90)
+    d.view("home", 1.5)
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--port", type=int, default=7878, help="the app's --control port")
+    ap.add_argument("--cli", default="solvecraft-cli", help="path to solvecraft-cli")
+    ap.add_argument("--log", help="write a JSON line per caption and pointer event")
+    ap.add_argument("--until", type=int, default=10**6, help="stop before step N (trying the script out)")
+    ap.add_argument("--speed", type=float, default=1.0, help="scale the pauses (2 = twice as long)")
+    a = ap.parse_args()
+    mcp = Mcp([a.cli, "mcp", "--connect", f"127.0.0.1:{a.port}"])
+    mcp.send("initialize", {"protocolVersion": "2025-06-18", "capabilities": {}, "clientInfo": {"name": "enclosure-demo-v2", "version": "2"}})
+    mcp.send("notifications/initialized", notify=True)
+    d = Demo(a.port, mcp, open(a.log, "w") if a.log else None, a.speed)
+    d.stop = a.until - 1
+    t0 = time.time()
+    try:
+        run(d)
+    except StopIteration:
+        pass
+    d.mark(caption="", via="")
+    mcp.close()
+    print(f"done in {time.time() - t0:.1f}s", file=sys.stderr)
+
+
+if __name__ == "__main__":
+    main()

examples/demo/overlay.py

@@ -0,0 +1,145 @@
+#!/usr/bin/env python3
+"""overlay.py LOG START OUTDIR: from enclosure_ui_mcp.py's --log (captions and pointer events)
+and the recording's start (epoch seconds), write OUTDIR/captions.ass, OUTDIR/cursor.cmd
+(ffmpeg sendcmd moves for the drawn pointer and the click ripple) and the two sprites
+(cursor.png, ripple.png). record_v2.sh composites them onto the screen recording."""
+
+import json
+import math
+import os
+import struct
+import sys
+import zlib
+
+
+def png(path, w, h, pixel):
+    """A small RGBA PNG, pixel(x, y) -> (r, g, b, a)."""
+    raw = b"".join(b"\0" + b"".join(bytes(pixel(x, y)) for x in range(w)) for y in range(h))
+
+    def chunk(t, d):
+        return struct.pack(">I", len(d)) + t + d + struct.pack(">I", zlib.crc32(t + d) & 0xFFFFFFFF)
+
+    data = b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 6, 0, 0, 0)) + chunk(b"IDAT", zlib.compress(raw, 9)) + chunk(b"IEND", b"")
+    open(path, "wb").write(data)
+
+
+def inside(px, py, poly):
+    c = False
+    for i in range(len(poly)):
+        (x1, y1), (x2, y2) = poly[i], poly[i - 1]
+        if (y1 > py) != (y2 > py) and px < (x2 - x1) * (py - y1) / (y2 - y1) + x1:
+            c = not c
+    return c
+
+
+def cursor_sprite(path):
+    # An arrow pointer (tip at 1,1): white with a dark outline, drawn 2x and box-filtered.
+    arrow = [(1, 1), (1, 23), (6.5, 18), (10, 26), (13.5, 24.5), (10, 17), (17, 17)]
+    s = 4
+
+    def cover(x, y, poly, grow):
+        n = 0
+        for i in range(s):
+            for j in range(s):
+                px, py = x + (i + 0.5) / s, y + (j + 0.5) / s
+                if grow:
+                    n += any(inside(px + dx, py + dy, poly) for dx, dy in ((-1.3, 0), (1.3, 0), (0, -1.3), (0, 1.3), (0, 0)))
+                else:
+                    n += inside(px, py, poly)
+        return n / (s * s)
+
+    def pixel(x, y):
+        fill, edge = cover(x, y, arrow, False), cover(x, y, arrow, True)
+        a = max(fill, edge)
+        if a == 0:
+            return (0, 0, 0, 0)
+        v = int(255 * fill / a) if a else 0
+        return (v, v, v, int(255 * a))
+
+    png(path, 22, 30, pixel)
+
+
+def ripple_sprite(path):
+    r = 22
+
+    def pixel(x, y):
+        d = math.hypot(x + 0.5 - r, y + 0.5 - r)
+        ring = max(0.0, 1 - abs(d - (r - 5)) / 2.2)
+        disc = 0.25 if d < r - 6 else 0.0
+        a = max(ring, disc)
+        return (90, 170, 255, int(255 * a))
+
+    png(path, 2 * r, 2 * r, pixel)
+
+
+def ts(t):
+    t = max(t, 0.0)
+    return f"{int(t // 3600)}:{int(t % 3600 // 60):02}:{t % 60:05.2f}"
+
+
+def main():
+    log, start, out = sys.argv[1], float(sys.argv[2]), sys.argv[3]
+    marks = [json.loads(line) for line in open(log) if line.strip()]
+    cursor_sprite(os.path.join(out, "cursor.png"))
+    ripple_sprite(os.path.join(out, "ripple.png"))
+
+    # Captions: the step, and under it how it was sent (UI click / MCP call) in its own colour.
+    caps = [m for m in marks if "caption" in m]
+    head = """[Script Info]
+ScriptType: v4.00+
+PlayResX: 1920
+PlayResY: 1080
+
+[V4+ Styles]
+Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
+Style: Step,DejaVu Sans,36,&H00FFFFFF,&H00FFFFFF,&H00000000,&H90000000,1,0,0,0,100,100,0,0,3,10,0,8,0,0,150,1
+Style: Via,DejaVu Sans Mono,24,&H00FFFFFF,&H00FFFFFF,&H00000000,&H90000000,0,0,0,0,100,100,0,0,3,8,0,8,0,0,210,1
+
+[Events]
+Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
+"""
+    lines = []
+    for i, m in enumerate(caps):
+        if not m["caption"]:
+            continue
+        a = m["t"] - start
+        b = (caps[i + 1]["t"] - start) if i + 1 < len(caps) else a + 3
+        lines.append(f"Dialogue: 0,{ts(a)},{ts(b)},Step,,0,0,0,,{m['caption']}")
+        via = m.get("via", "")
+        if via:
+            colour = "&H0090E8A0&" if via.startswith("UI") else "&H0050C8FF&"
+            lines.append(f"Dialogue: 0,{ts(a)},{ts(b)},Via,,0,0,0,,{{\\c{colour}}}{via}")
+    open(os.path.join(out, "captions.ass"), "w").write(head + "\n".join(lines) + "\n")
+
+    # Pointer: every move as logged; a drag's path spread over the time the app took to play it.
+    pts = []
+    for i, m in enumerate(marks):
+        t = m["t"] - start
+        if "cursor" in m:
+            pts.append((t, m["cursor"][0], m["cursor"][1]))
+        elif "drag_start" in m:
+            end = next((n for n in marks[i + 1:] if "drag_end" in n), None)
+            if end:
+                (x0, y0), (x1, y1), t1 = m["drag_start"], end["drag_end"], end["t"] - start
+                n = max(2, int((t1 - t) * 30))
+                for k in range(n + 1):
+                    u = k / n
+                    pts.append((t + (t1 - t) * u, x0 + (x1 - x0) * u, y0 + (y1 - y0) * u))
+    pts.sort()
+    cmds = []
+    for t, x, y in pts:
+        cmds.append(f"{t:.3f} [enter] overlay@cur x {x:.1f}, [enter] overlay@cur y {y:.1f};")
+    for m in marks:
+        if m.get("click"):
+            t = m["t"] - start
+            x, y = m["cursor"]
+            cmds.append(f"{t:.3f} [enter] overlay@rip x {x - 22:.1f}, [enter] overlay@rip y {y - 22:.1f};")
+    open(os.path.join(out, "cursor.cmd"), "w").write("\n".join(sorted(cmds, key=lambda c: float(c.split()[0]))) + "\n")
+    clicks = [m["t"] - start for m in marks if m.get("click")]
+    open(os.path.join(out, "ripple.enable"), "w").write("+".join(f"between(t,{c:.3f},{c + 0.28:.3f})" for c in clicks) or "0")
+    first = pts[0] if pts else (0, -100, -100)
+    open(os.path.join(out, "cursor.start"), "w").write(f"{first[1]:.1f} {first[2]:.1f}\n")
+
+
+if __name__ == "__main__":
+    main()

examples/demo/record_v2.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+# record_v2.sh DISPLAY PORT OUTDIR: record enclosure_ui_mcp.py driving the app on X display
+# DISPLAY (1920x1080, started with --control PORT) to OUTDIR/enclosure-demo-v2.mp4 (H.264) and
+# .gif, with captions and a drawn pointer (the app gets synthetic pointer events, so the
+# screen's own pointer never moves). Needs ffmpeg with x11grab and libass.
+set -e
+DISP=${1:-:0}; PORT=${2:-7878}; OUT=$(cd "${3:-.}" && pwd)
+HERE=$(cd "$(dirname "$0")" && pwd)
+CLI=${SOLVECRAFT_CLI:-solvecraft-cli}
+# x11grab stamps frames with the wall clock: its "start:" is when the first frame was taken.
+ffmpeg -loglevel info -nostats -y -f x11grab -draw_mouse 0 -video_size 1920x1080 -framerate 30 -i "$DISP" \
+  -c:v libx264 -preset veryfast -crf 16 -pix_fmt yuv420p "$OUT/raw.mp4" 2> "$OUT/ffmpeg.log" &
+FF=$!
+sleep 1.5
+python3 "$HERE/enclosure_ui_mcp.py" --port "$PORT" --cli "$CLI" --log "$OUT/steps.log"
+sleep 1.5
+kill -INT $FF; wait $FF || true
+START=$(grep -o "start: [0-9.]*" "$OUT/ffmpeg.log" | head -1 | cut -d" " -f2)
+python3 "$HERE/overlay.py" "$OUT/steps.log" "$START" "$OUT"
+read CX CY < "$OUT/cursor.start"
+ffmpeg -loglevel error -y -i "$OUT/raw.mp4" -loop 1 -i "$OUT/cursor.png" -loop 1 -i "$OUT/ripple.png" -filter_complex \
+  "[0:v]ass=$OUT/captions.ass[v0];[v0][2:v]overlay@rip=x=-100:y=-100:enable='$(cat "$OUT/ripple.enable")':shortest=1[v1];[v1][1:v]overlay@cur=x=$CX:y=$CY:shortest=1,sendcmd=f=$OUT/cursor.cmd[v]" \
+  -map "[v]" -c:v libx264 -preset slow -crf 20 -pix_fmt yuv420p -movflags +faststart "$OUT/enclosure-demo-v2.mp4"
+ffmpeg -loglevel error -y -i "$OUT/enclosure-demo-v2.mp4" \
+  -vf "fps=10,scale=800:-1:flags=lanczos,split[a][b];[a]palettegen=max_colors=96:stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=4" \
+  "$OUT/enclosure-demo-v2.gif"
+ls -la "$OUT"/enclosure-demo-v2.*