Add audited, versioned MCP save workflow (#1452)

- `--export ... --target-version`, and explicit target_format /
  target_version on automation and protocol saves; a conversion that
  would drop unsupported records is refused unless allow_lossy is set.
- `audit` read op: dangling layers/blocks, duplicate handles, non-finite
  bounds, serialization, lossy records and the source DXF handle graph.
- `save_verified`: writes the requested version from the same snapshot a
  normal save writes, reopens it, checks the version and the entity
  manifest, audits the ASCII DXF handle graph and returns a SHA-256.
- ZOOM EXTENTS includes SDF text glyphs.
- Live MCP acceptance script and an optional reference-console audit
  runner under docs/automation.

Squashed from the PR with review fixes: merged with the DWT save path,
the save snapshot, the tool lists added since, and vendor-neutral
harness names; the dated verification record is left out.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Luis MartinezandClaude Opus 5.5 authored and Hakan Seven committed 2026-09-24 16:49:11 +03:00
1 parent 0f99b77004
commit dddf60d79e
16 files changed
+1102 -22

No files matched your search

+49
View File
@@ -20,6 +20,19 @@ The server provides four tools:
- `ocs_execute` performs one operation, an atomic record update, or a sequential batch against the real editor.
- `ocs_capture` returns a bounded PNG of the drawing viewport or complete window.
Run `python docs/automation/mcp_acceptance.py target/debug/OpenCADStudio.exe`
for a repeatable end-to-end acceptance. It creates visible geometry through
MCP, audits and verified-saves DWG 2000/2013/2018 plus DXF 2000, and records a
viewport PNG and JSON report below `target/acceptance/`.
On a Windows workstation with a reference CAD application's command-line
console, run
`docs/automation/reference_acceptance.ps1 -Directory target/acceptance/TIMESTAMP -ConsolePath <console.exe>`.
It opens the three generated DWGs read-only in isolated console profiles, runs
AUDIT without repairs, and keeps one log per version. A console that stays
alive after `QUIT` is terminated after the timeout (only the process the runner
launched), and `ForcedTermination` is recorded separately from the audit result.
Native builds also accept a second, headless entry point (`OpenCADStudio --serve`, one
JSON request per line) for scripts and CI; the two native channels, their startup dialogs
and their limits are described in [native.md](native.md).
@@ -40,6 +53,42 @@ The normal flow is to call `ocs_sessions`, pass its returned `session_id` as `oc
Call `ocs_read` with `op: "capabilities"` before unfamiliar work. It reports the command, geometry, transaction, capture, and database facilities supported by the running build. The `records.collections` list gives every available database collection, its record count, and whether it can be edited.
Before delivery, call `ocs_read` with `op: "audit"` and the intended
`target_format` / `target_version`. The report includes entity manifests,
layers, blocks, finite bounds, duplicate entity handles, DXF dangling handle
references, source hash, and `dropped_on_save`. A clean in-memory document can
still be lossy for a different format, so target selection is part of the
audit:
```json
{
"ocs_session_id": "SESSION_FROM_OCS_SESSIONS",
"op": "audit",
"parameters": {"target_format": "dwg", "target_version": "2000"}
}
```
Use `save_verified` for the delivery write. It requires an explicit absolute
path, refuses overwrite and lossy conversion unless separately acknowledged,
writes the requested version, reopens it, checks the actual version, compares
the semantic entity manifest, audits raw DXF handle references, and returns a
SHA-256 hash. A verification failure preserves the output for diagnosis and
reports it as failed instead of silently declaring success.
```json
{
"ocs_session_id": "SESSION_FROM_OCS_SESSIONS",
"request": {
"op": "save_verified",
"request_id": "deliver-2026-09-22-1",
"path": "C:\\drawings\\issued\\plan.dwg",
"target_format": "dwg",
"target_version": "2000",
"overwrite": false
}
}
```
Call `ocs_read` with `op: "record_schema"` before editing an unfamiliar record. With no parameters it lists the complete generated type registry. A collection returns the record types accepted by that collection even when the current drawing has no instance of a type. Supplying both `collection` and `type` returns the type's complete dependency graph, flattened property paths, JSON types, optional and sequence markers, enum variants, integer bounds, unambiguous unit annotations, identity fields, and write rules:
```json
+152
View File
@@ -0,0 +1,152 @@
"""End-to-end GUI/MCP acceptance with versioned, verified CAD outputs."""
import base64
import json
from pathlib import Path
import struct
import sys
import time
import uuid
from mcp_eval import Client, META
def request_id(prefix: str) -> str:
return f"accept-{prefix}-{uuid.uuid4().hex}"
def main() -> None:
server = Path(sys.argv[1] if len(sys.argv) > 1 else "target/debug/OpenCADStudio.exe").resolve()
root = Path(sys.argv[2] if len(sys.argv) > 2 else "target/acceptance").resolve()
output = root / time.strftime("%Y%m%d-%H%M%S")
output.mkdir(parents=True, exist_ok=False)
client = Client(server)
session = None
report: dict = {"ok": False, "server": str(server), "output": str(output)}
try:
sessions = client.tool("ocs_sessions", {"launch_if_none": True})["result"]
session = sessions[0]["session_id"]
while sessions[0].get("modal"):
client.tool(
"ocs_execute",
{
"ocs_session_id": session,
"request": {
"op": "action",
"request_id": request_id("close-modal"),
"name": "close_modal",
},
},
)
sessions = client.tool("ocs_sessions", {"launch_if_none": False})["result"]
client.tool(
"ocs_execute",
{
"ocs_session_id": session,
"request": {"op": "new", "request_id": request_id("new")},
},
)
drawing = client.tool(
"ocs_execute",
{
"ocs_session_id": session,
"response_detail": "changed_entities",
"request": {
"op": "batch",
"request_id": request_id("draw"),
"steps": [
{"op": "run", "cmd": "PLINE 0,0 200,0 200,120 0,120 C"},
{"op": "run", "cmd": "CIRCLE 50,60 15"},
{"op": "run", "cmd": "CIRCLE 150,60 15"},
{"op": "run", "cmd": "LINE 0,60 200,60"},
{"op": "run", "cmd": "TEXT 100,135 8 0 OPEN CAD MCP ACCEPTANCE"},
{"op": "action", "name": "zoom_extents"},
],
},
},
)
assert drawing["completed_steps"] == 6, drawing
audits = {}
saves = {}
targets = [("dwg", "2000"), ("dwg", "2013"), ("dwg", "2018"), ("dxf", "2000")]
for target_format, target_version in targets:
key = f"{target_format}-{target_version}"
audit = client.tool(
"ocs_read",
{
"ocs_session_id": session,
"op": "audit",
"parameters": {
"target_format": target_format,
"target_version": target_version,
},
},
)
assert audit["ok"] is True, audit
audits[key] = audit
destination = output / f"mcp-acceptance-{target_version}.{target_format}"
saved = client.tool(
"ocs_execute",
{
"ocs_session_id": session,
"response_detail": "full",
"request": {
"op": "save_verified",
"request_id": request_id(f"save-{key}"),
"path": str(destination),
"target_format": target_format,
"target_version": target_version,
},
},
)
verified = saved.get("result", saved)
assert verified["verified"] is True, saved
assert destination.is_file(), destination
saves[key] = verified
capture = client.rpc(
"tools/call",
{
"name": "ocs_capture",
"arguments": {
"ocs_session_id": session,
"scope": "viewport",
"max_dimension": 1600,
},
"_meta": META,
},
)
image = capture["content"][0]
assert image["type"] == "image" and image["mimeType"] == "image/png", capture
png = base64.b64decode(image["data"], validate=True)
assert png[:8] == b"\x89PNG\r\n\x1a\n", "capture is not a PNG"
width, height = struct.unpack(">II", png[16:24])
capture_path = output / "viewport.png"
capture_path.write_bytes(png)
report.update(
{
"ok": True,
"session_id": session,
"drawn_entities": drawing["changed_entities"],
"audits": audits,
"saves": saves,
"capture": {"path": str(capture_path), "width": width, "height": height},
"metrics": {
"tool_calls": client.tool_calls,
"tasks": client.tasks,
"rpc_calls": client.rpc_calls,
},
}
)
finally:
client.close()
report_path = output / "report.json"
report_path.write_text(json.dumps(report, indent=2), encoding="utf-8")
print(json.dumps(report, indent=2))
if __name__ == "__main__":
main()
+15 -1
View File
@@ -5,6 +5,7 @@ from pathlib import Path
import subprocess
import sys
import time
import uuid
MODERN = "2026-07-28"
@@ -91,6 +92,19 @@ def main() -> None:
assert "io.modelcontextprotocol/tasks" in discovery["capabilities"]["extensions"]
sessions = client.tool("ocs_sessions", {"launch_if_none": True})["result"]
session = sessions[0]["session_id"]
while sessions[0].get("modal"):
client.tool(
"ocs_execute",
{
"ocs_session_id": session,
"request": {
"op": "action",
"request_id": f"eval-close-modal-{uuid.uuid4().hex}",
"name": "close_modal",
},
},
)
sessions = client.tool("ocs_sessions", {"launch_if_none": False})["result"]
active = next(
document for document in sessions[0]["documents"]
if document["id"] == sessions[0]["document_id"]
@@ -98,7 +112,7 @@ def main() -> None:
if active.get("start"):
client.tool(
"ocs_execute",
{"ocs_session_id": session, "request": {"op": "new", "request_id": "eval-new"}},
{"ocs_session_id": session, "request": {"op": "new", "request_id": f"eval-new-{uuid.uuid4().hex}"}},
)
base = 1_000_000 + int(time.time()) % 100_000
+5 -2
View File
@@ -52,7 +52,7 @@ def legacy(server: Path) -> None:
names = set(definitions)
assert names == TOOLS, names
execute_request = definitions["ocs_execute"]["inputSchema"]["properties"]["request"]
assert len(execute_request["oneOf"]) == 35
assert len(execute_request["oneOf"]) == 38
op_enum = execute_request["properties"]["op"]["enum"]
for shipped in ("entities_create", "entities_delete", "entities_transform",
"block_define", "block_delete", "file_identity", "xdata_set",
@@ -65,9 +65,12 @@ def legacy(server: Path) -> None:
assert execute_request["properties"]["steps"]["maxItems"] == 64
assert execute_request["properties"]["cmd"]["examples"][0] == "LINE 0,0 10,10"
assert "set_properties" in execute_request["properties"]["op"]["enum"]
assert "save_verified" in execute_request["properties"]["op"]["enum"]
assert "record_schema" in definitions["ocs_read"]["inputSchema"]["properties"]["op"]["enum"]
assert "audit" in definitions["ocs_read"]["inputSchema"]["properties"]["op"]["enum"]
read_parameters = definitions["ocs_read"]["inputSchema"]["properties"]["parameters"]["properties"]
assert {"collection", "where", "paths"} <= set(read_parameters)
assert {"collection", "where", "paths", "target_format", "target_version"} <= set(read_parameters)
assert execute_request["properties"]["target_version"]["enum"][0] == "R14"
assert execute_request["properties"]["kind"]["enum"] == [
"text", "token", "point", "entity", "structure", "selection", "enter"
]
+17 -4
View File
@@ -33,8 +33,8 @@ Two request styles share the channel, told apart by the `protocol` field:
| Style | Shape | Operations |
|---|---|---|
| protocol 1 | `{"protocol":1,"request_id":"…","document_id":N,"op":"…"}` — the same requests and replies as `ocs_execute` / `ocs_read`, described in the README | `new`, `open`, `save`, `activate`, `select`, `run`, `start`, `input`, `cancel`, `stop`, `action`, `property`, `set_properties`, `embed_image`, `undo`, `redo`; reads `state`, `capabilities`, `query`, `entities`, `records`, `record_schema`, `layers`, `header`, `properties`, `measure`, `history`, `commands`, `events`, `operation` |
| legacy | `{"op":"…"}` without `protocol` | `new`, `open`, `run`, `entities`, `query`, `records`, `record_schema`, `capabilities`, `layers`, `header`, `select`, `save`, `undo`, `redo` |
| protocol 1 | `{"protocol":1,"request_id":"…","document_id":N,"op":"…"}` — the same requests and replies as `ocs_execute` / `ocs_read`, described in the README | `new`, `open`, `save`, `save_verified`, `activate`, `select`, `run`, `start`, `input`, `cancel`, `stop`, `action`, `property`, `set_properties`, `embed_image`, `undo`, `redo`; reads `state`, `capabilities`, `audit`, `query`, `entities`, `records`, `record_schema`, `layers`, `header`, `properties`, `measure`, `history`, `commands`, `events`, `operation` |
| legacy | `{"op":"…"}` without `protocol` | `new`, `open`, `run`, `entities`, `audit`, `query`, `records`, `record_schema`, `capabilities`, `layers`, `header`, `select`, `save`, `undo`, `redo` |
The two styles differ in how a waiting command is treated. A protocol-1 `run`
or `start` while a command is still active is refused with `command_busy`:
@@ -104,8 +104,19 @@ README describes.
`OpenCADStudio --export IN OUT` loads `IN` and writes `OUT` without a window or
a session. The output format follows the extension: `.dxf` writes DXF, any other
extension writes DWG at the document's own version. The exit code is 0 on
success and 1 when the input cannot be read or the output cannot be written.
extension writes DWG at the document's own version. Add `--target-version R14`
(or `2000`, `2004`, `2007`, `2010`, `2013`, `2018`) to choose explicitly. The
exit code is 0 on success, 1 when the input/output fails, and 2 for an invalid
target version.
The verified interoperability lane uses version 2000 for a DWG -> DXF
round-trip. R14 DWG files can be written and reopened, but the current R14
DWG -> DXF path can emit object dictionaries that are orphaned from the root,
which leaves layout and placeholder ownership invalid. `audit` and
`save_verified` reject that ASCII DXF instead of reporting a false green. Do
not deliver that conversion unless `save_verified` passes. Raw
group-code handle validation is not applied to binary DXF and is reported as
skipped in the audit result.
## Interactive steps
@@ -129,3 +140,5 @@ For commands whose answers cannot be written on one line, use `start` and then
| measure | `{"protocol":1,"op":"measure","handles":["2A"]}` — kernel-computed length, area, bounds and mass properties; no window needed | `ocs_read` with `op: "measure"` and `parameters.handles` |
| capture | `{"protocol":1,"op":"capture","path":"out.png"}` — needs the editor window, so it answers `gui_required` here | `ocs_capture` with `scope` (`viewport` or `window`) and `max_dimension` |
| `set_properties` | requires `collection`; colours are the serialised enum — `{"Index": 1}` or `"ByLayer"`; a colour name such as `"Red"` is rejected with `invalid_value` | same request through `ocs_execute` |
| audit | `{"protocol":1,"op":"audit","target_format":"dwg","target_version":"2000"}` | `ocs_read` with `op: "audit"` and the same target parameters |
| save verified | protocol-1 `save_verified` with absolute `path`, unique `request_id`, and target fields | `ocs_execute` with `op: "save_verified"`; returns hash, reopened version and semantic manifest |
+67
View File
@@ -0,0 +1,67 @@
param(
[Parameter(Mandatory = $true)]
[string]$Directory,
[Parameter(Mandatory = $true)]
[string]$ConsolePath,
[int]$TimeoutSeconds = 25
)
$ErrorActionPreference = 'Stop'
$drawingDirectory = (Resolve-Path -LiteralPath $Directory).Path
$scriptPath = Join-Path $PSScriptRoot 'reference_audit.scr'
$targets = @(
@{ Version = '2000'; File = 'mcp-acceptance-2000.dwg' },
@{ Version = '2013'; File = 'mcp-acceptance-2013.dwg' },
@{ Version = '2018'; File = 'mcp-acceptance-2018.dwg' }
)
$results = foreach ($target in $targets) {
$drawing = Join-Path $drawingDirectory $target.File
if (-not (Test-Path -LiteralPath $drawing -PathType Leaf)) {
throw "Missing acceptance drawing: $drawing"
}
$isolation = Join-Path $env:TEMP "ocs-reference-acceptance-$($target.Version)"
New-Item -ItemType Directory -Force -Path $isolation | Out-Null
$start = [Diagnostics.ProcessStartInfo]::new()
$start.FileName = $ConsolePath
$start.UseShellExecute = $false
$start.CreateNoWindow = $true
$start.RedirectStandardOutput = $true
$start.RedirectStandardError = $true
$start.StandardOutputEncoding = [Text.Encoding]::Unicode
$start.StandardErrorEncoding = [Text.Encoding]::Unicode
@(
'/i', $drawing,
'/s', $scriptPath,
'/l', 'en-US',
'/isolate', "ocsaccept$($target.Version)", $isolation,
'/readonly'
) | ForEach-Object { [void]$start.ArgumentList.Add($_) }
$process = [Diagnostics.Process]::new()
$process.StartInfo = $start
[void]$process.Start()
$stdout = $process.StandardOutput.ReadToEndAsync()
$stderr = $process.StandardError.ReadToEndAsync()
$forcedTermination = -not $process.WaitForExit($TimeoutSeconds * 1000)
if ($forcedTermination) {
$process.Kill($true)
$process.WaitForExit()
}
$text = $stdout.Result + $stderr.Result
$log = Join-Path $drawingDirectory "reference-audit-$($target.Version).log"
[IO.File]::WriteAllText($log, $text, [Text.UTF8Encoding]::new($false))
$passed = [regex]::IsMatch($text, 'Total errors found\s+0\s+fixed\s+0')
[pscustomobject]@{
Version = $target.Version
Passed = $passed
ForcedTermination = $forcedTermination
ExitCode = $process.ExitCode
Log = $log
}
}
$results | Format-Table -AutoSize
if ($results.Passed -contains $false) {
exit 1
}
+3
View File
@@ -0,0 +1,3 @@
_.AUDIT
_N
_.QUIT