web: XREFs, data links & extraction, ARCHIVE transmittals, worker-side saves

The last four big web-vs-desktop gaps.

XREFs: XATTACH reads the picked drawing's bytes and registers them
name-keyed (the image/PDF-underlay pattern), pre-parses on the web
worker, and the commit merge produces the same xref block flow as
native. Opened drawings resolve xrefs from the registry (trailing-name
match) before cache build; unresolved references produce ONE warning
listing the names and the fix — drop the referenced files on the page
(drops and picker opens now register their bytes) and reopen or
XRELOAD. XRELOAD works off the registry on unsaved drawings too.

Data Link & Extraction: spreadsheet links read picked xlsx/csv bytes
(calamine from slices / CSV from string) through a session registry,
so preview, refresh, and table-insert work on web; extraction takes
external drawings via a multi-file picker parsed on the worker, and
outputs CSV/TXT/.dxex via download_bytes. The disabled 'Add folder'
button explains why.

ARCHIVE/ETRANSMIT: assembles the drawing plus registered images, PDF
underlays, and session plot tables into a ZIP delivered as
{stem}_transmit.zip; skipped dependencies are reported by name.

Save/autosave off the UI thread: the parse worker gains a serialize
action (protocol v5) — the document is prepared once on the main
thread (clone, styles, dimension bake), then the DWG/DXF writer pass,
the dominant cost, runs on the worker; any worker failure falls back
to inline serialization with the identical shared implementation.
The save path also clears the dirty flag only when the tab didn't
change mid-write, and the runtime-fields sidecar now carries
graphic_data, fixing a pre-existing loss of raw DWG round-trip bytes
on web-opened documents.

(cherry picked from commit 2e100634)

Ported onto main (#1717 review): the new ocs_save crate builds on the
pinned codec; the hand-written ZIP writer is replaced by the zip crate
already in the tree; ARCHIVE packs registered xref bytes too and
serializes document_for_save() like a web Save; the web save prepares
document_for_save() (prepare_document_for_save now takes the copy by
value, so no second clone); the wasm resolve_path consults the xref
registry, so every reference lookup — not just resolve — finds dropped
files; XRELOAD keeps main's undo, unload and stale-baseline handling;
native data extraction streams the source drawings one at a time.

crates/ocs_save/Cargo.toml

@@ -0,0 +1,15 @@
+[package]
+name = "ocs_save"
+version = "0.1.0"
+edition = "2021"
+publish = false
+
+# Pure-Rust SAVE pipeline pieces shared by the app crate (native host and the
+# main wasm instance) and the browser parse/serialize worker (`ocs_web_worker`),
+# so the DWG/DXF byte serialization and the worker wire format each have
+# exactly one implementation. No UI/app dependencies — only the codec + serde.
+
+[dependencies]
+codec = { package = "opencadcodec", git = "https://github.com/HakanSeven12/opencadcodec.git", rev = "8189f83", features = ["serde"] }
+bincode = "1.3"
+serde = { version = "1", features = ["derive"] }

crates/ocs_save/src/lib.rs

@@ -0,0 +1,290 @@
+//! Shared SAVE/serialize pipeline pieces.
+//!
+//! Both sides of the browser worker boundary use this crate so the DWG/DXF
+//! byte serialization and the worker wire format have exactly one
+//! implementation:
+//!
+//! - the app crate's inline `io::save_to_bytes` (native host fallback and the
+//!   web fallback path when the worker is unavailable), and
+//! - `ocs_web_worker`'s `serialize_document` action, which runs the same
+//!   serialization on the dedicated worker thread.
+//!
+//! The document-crossing helpers (bincode encode/decode plus the runtime-field
+//! sidecar) are also shared with the existing `parse_document` action, which
+//! returns documents over the same boundary in the opposite direction.
+
+use codec::{CadDocument, DwgWriter, DxfWriter};
+
+/// Worker protocol version. Bumped whenever a request/response payload shape
+/// changes; the `?v=` cache-buster on the worker URL must match. v5 adds the
+/// `serialize` action and extends the runtime-field sidecar with
+/// `graphic_data`.
+pub const PROTOCOL_VERSION: u16 = 5;
+
+/// Serialize a document to an in-memory byte buffer, format chosen by `ext`
+/// (`dxf` → DXF, anything else → DWG). The document is expected to be fully
+/// prepared for saving (version stamped, current styles synced, dimension
+/// blocks baked) — see the app crate's `io::prepare_document_for_save`, which
+/// owns that app-side half of the pipeline.
+pub fn serialize_document(doc: &CadDocument, ext: &str) -> Result<Vec<u8>, String> {
+    match ext.to_lowercase().as_str() {
+        "dxf" => DxfWriter::new(doc).write_to_vec().map_err(|e| e.to_string()),
+        _ => {
+            let mut buf = std::io::Cursor::new(Vec::new());
+            DwgWriter::write_to_writer(&mut buf, doc).map_err(|e| e.to_string())?;
+            Ok(buf.into_inner())
+        }
+    }
+}
+
+// ── Runtime-field sidecar ──────────────────────────────────────────────────
+
+/// Per-entity fields the codec marks `serde(skip)` on `EntityCommon` (raw
+/// DWG storage/round-trip data the logical model does not own). They do not
+/// survive a bincode crossing on their own, so every document that crosses
+/// the worker boundary carries them beside the payload and re-installs them
+/// after decode — otherwise the saved file would silently lose per-entity
+/// linetype/material/plotstyle handles and raw graphic bytes.
+#[derive(serde::Serialize, serde::Deserialize)]
+pub struct EntityRuntimeFields {
+    pub handle: u64,
+    pub linetype_handle: Option<u64>,
+    pub color_book_handle: Option<u64>,
+    pub face_visual_style_handle: Option<u64>,
+    pub edge_visual_style_handle: Option<u64>,
+    pub material_flags: u8,
+    pub material_handle: Option<u64>,
+    pub shadow_flags: u8,
+    pub plotstyle_flags: u8,
+    pub plotstyle_handle: Option<u64>,
+    pub entity_mode: Option<u8>,
+    pub has_ds_data: bool,
+    pub graphic_data: Option<Vec<u8>>,
+}
+
+/// Collect the serde-skipped runtime fields of every entity in `doc`.
+pub fn extract_entity_runtime_fields(doc: &CadDocument) -> Vec<EntityRuntimeFields> {
+    doc.entities()
+        .map(|entity| {
+            let common = entity.common();
+            let handle = |value: Option<codec::Handle>| value.map(|handle| handle.value());
+            EntityRuntimeFields {
+                handle: common.handle.value(),
+                linetype_handle: handle(common.linetype_handle),
+                color_book_handle: handle(common.color_book_handle),
+                face_visual_style_handle: handle(common.face_visual_style_handle),
+                edge_visual_style_handle: handle(common.edge_visual_style_handle),
+                material_flags: common.material_flags,
+                material_handle: handle(common.material_handle),
+                shadow_flags: common.shadow_flags,
+                plotstyle_flags: common.plotstyle_flags,
+                plotstyle_handle: handle(common.plotstyle_handle),
+                entity_mode: common.entity_mode,
+                has_ds_data: common.has_ds_data,
+                graphic_data: common.graphic_data.clone(),
+            }
+        })
+        .collect()
+}
+
+/// Re-install runtime fields collected by [`extract_entity_runtime_fields`]
+/// into a document that crossed a serde boundary. Entities whose handle no
+/// longer resolves are skipped silently (mirrors the parse direction).
+pub fn restore_entity_runtime_fields(document: &mut CadDocument, fields: Vec<EntityRuntimeFields>) {
+    let handle = |value: Option<u64>| value.map(codec::Handle::new);
+    for fields in fields {
+        let Some(entity) = document.get_entity_mut(codec::Handle::new(fields.handle)) else {
+            continue;
+        };
+        let common = entity.common_mut();
+        common.linetype_handle = handle(fields.linetype_handle);
+        common.color_book_handle = handle(fields.color_book_handle);
+        common.face_visual_style_handle = handle(fields.face_visual_style_handle);
+        common.edge_visual_style_handle = handle(fields.edge_visual_style_handle);
+        common.material_flags = fields.material_flags;
+        common.material_handle = handle(fields.material_handle);
+        common.shadow_flags = fields.shadow_flags;
+        common.plotstyle_flags = fields.plotstyle_flags;
+        common.plotstyle_handle = handle(fields.plotstyle_handle);
+        common.entity_mode = fields.entity_mode;
+        common.has_ds_data = fields.has_ds_data;
+        common.graphic_data = fields.graphic_data;
+    }
+}
+
+// ── Worker wire format (`serialize` action) ────────────────────────────────
+//
+// Request  (main → worker): bincode `(PROTOCOL_VERSION, ext, &CadDocument,
+//           Vec<EntityRuntimeFields>)` transferred as an ArrayBuffer.
+// Response (worker → main): bincode `(PROTOCOL_VERSION, Result<Vec<u8>,
+//           String>)` — the encoded DWG/DXF bytes on success, the writer's
+//           error text on failure.
+
+/// Encode a prepared document for the trip main → worker. Costs one bincode
+/// pass over the document plus the runtime-field sidecar; see the crate docs
+/// for why the sidecar exists.
+pub fn encode_request(doc: &CadDocument, ext: &str) -> Result<Vec<u8>, String> {
+    let runtime_fields = extract_entity_runtime_fields(doc);
+    bincode::serialize(&(PROTOCOL_VERSION, ext, doc, runtime_fields))
+        .map_err(|error| format!("encode save document for worker: {error}"))
+}
+
+/// Decode a `serialize` request on the worker: returns the document (with
+/// runtime fields restored) and the target extension.
+pub fn decode_request(bytes: &[u8]) -> Result<(CadDocument, String), String> {
+    let (version, ext, mut document, runtime_fields): (
+        u16,
+        String,
+        CadDocument,
+        Vec<EntityRuntimeFields>,
+    ) = bincode::deserialize(bytes).map_err(|error| format!("decode save document: {error}"))?;
+    if version != PROTOCOL_VERSION {
+        return Err(format!(
+            "serializer worker protocol mismatch: expected {PROTOCOL_VERSION}, received {version}"
+        ));
+    }
+    restore_entity_runtime_fields(&mut document, runtime_fields);
+    Ok((document, ext))
+}
+
+/// Encode the worker's reply: the serialized drawing bytes, or the error text.
+pub fn encode_response(result: Result<Vec<u8>, String>) -> Result<Vec<u8>, String> {
+    bincode::serialize(&(PROTOCOL_VERSION, result))
+        .map_err(|error| format!("encode save response: {error}"))
+}
+
+/// Decode the worker's reply on the main thread, rejecting stale-protocol
+/// payloads.
+pub fn decode_response(bytes: &[u8]) -> Result<Result<Vec<u8>, String>, String> {
+    let (version, result): (u16, Result<Vec<u8>, String>) =
+        bincode::deserialize(bytes).map_err(|error| format!("decode save response: {error}"))?;
+    if version != PROTOCOL_VERSION {
+        return Err(format!(
+            "serializer worker protocol mismatch: expected {PROTOCOL_VERSION}, received {version}"
+        ));
+    }
+    Ok(result)
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use codec::entities::{EntityType, Line, Point};
+    use codec::types::Vector3;
+
+    fn sample_document() -> CadDocument {
+        let mut doc = CadDocument::new();
+        let mut pt = Point::new();
+        pt.common.handle = doc.allocate_handle();
+        pt.common.layer = "RUNTIME-FIELDS".to_string();
+        // Fill every serde-skipped field with a recognizable value.
+        pt.common.linetype_handle = Some(codec::Handle::new(0xABCD));
+        pt.common.color_book_handle = Some(codec::Handle::new(0x101));
+        pt.common.material_flags = 0b11;
+        pt.common.material_handle = Some(codec::Handle::new(0x202));
+        pt.common.plotstyle_flags = 0b11;
+        pt.common.plotstyle_handle = Some(codec::Handle::new(0x303));
+        pt.common.shadow_flags = 7;
+        pt.common.entity_mode = Some(2);
+        pt.common.has_ds_data = true;
+        pt.common.graphic_data = Some(vec![0xDE, 0xAD, 0xBE, 0xEF]);
+        let _ = doc.add_entity(EntityType::Point(pt));
+        let mut line = Line::new();
+        line.start = Vector3::new(0.0, 0.0, 0.0);
+        line.end = Vector3::new(10.0, 0.0, 0.0);
+        line.common.handle = doc.allocate_handle();
+        let _ = doc.add_entity(EntityType::Line(line));
+        doc
+    }
+
+    // The sidecar is the whole reason documents don't lose raw DWG storage
+    // fields when they cross the worker boundary: serde skips them, so
+    // encode → decode must restore every one.
+    #[test]
+    fn request_round_trip_restores_runtime_fields() {
+        let doc = sample_document();
+        let encoded = encode_request(&doc, "dwg").expect("encode");
+        let (decoded, ext) = decode_request(&encoded).expect("decode");
+        assert_eq!(ext, "dwg");
+        assert_eq!(decoded.entities().count(), 2);
+        let point = decoded
+            .entities()
+            .find(|e| matches!(e, EntityType::Point(_)))
+            .expect("point survived");
+        let common = point.common();
+        assert_eq!(common.layer, "RUNTIME-FIELDS");
+        assert_eq!(common.linetype_handle.map(|h| h.value()), Some(0xABCD));
+        assert_eq!(
+            common.color_book_handle.map(|h| h.value()),
+            Some(0x101)
+        );
+        assert_eq!(common.material_flags, 0b11);
+        assert_eq!(common.material_handle.map(|h| h.value()), Some(0x202));
+        assert_eq!(common.plotstyle_flags, 0b11);
+        assert_eq!(
+            common.plotstyle_handle.map(|h| h.value()),
+            Some(0x303)
+        );
+        assert_eq!(common.shadow_flags, 7);
+        assert_eq!(common.entity_mode, Some(2));
+        assert!(common.has_ds_data);
+        assert_eq!(common.graphic_data.as_deref(), Some(&[0xDE, 0xAD, 0xBE, 0xEF][..]));
+    }
+
+    #[test]
+    fn response_round_trip_ok_and_err() {
+        let ok = decode_response(&encode_response(Ok(vec![1, 2, 3])).expect("encode"))
+            .expect("decode")
+            .expect("inner ok");
+        assert_eq!(ok, vec![1, 2, 3]);
+        let err = decode_response(&encode_response(Err("writer failed".to_string())).expect("encode"))
+            .expect("decode")
+            .expect_err("inner err");
+        assert_eq!(err, "writer failed");
+    }
+
+    #[test]
+    fn response_rejects_foreign_protocol_version() {
+        let stale =
+            bincode::serialize(&(PROTOCOL_VERSION - 1, Ok::<Vec<u8>, String>(vec![]))).unwrap();
+        assert!(decode_response(&stale).is_err());
+    }
+
+    #[test]
+    fn request_rejects_foreign_protocol_version() {
+        let stale = bincode::serialize(&(
+            PROTOCOL_VERSION - 1,
+            "dwg",
+            CadDocument::new(),
+            Vec::<EntityRuntimeFields>::new(),
+        ))
+        .unwrap();
+        assert!(decode_request(&stale).is_err());
+    }
+
+    // The ext dispatch both save paths (inline fallback and worker) share:
+    // `dxf` must produce an ASCII DXF, anything else the DWG sentinel.
+    #[test]
+    fn serialize_document_picks_format_by_extension() {
+        let mut doc = CadDocument::new();
+        doc.version = codec::DxfVersion::AC1032;
+        let dxf = serialize_document(&doc, "dxf").expect("dxf bytes");
+        assert!(!dxf.is_empty());
+        assert!(dxf.starts_with(b"  0") || dxf.starts_with(b"0\n"), "not a DXF header");
+        let dwg = serialize_document(&doc, "dwg").expect("dwg bytes");
+        assert!(dwg.starts_with(b"AC10"), "not a DWG sentinel");
+        // Case-insensitive: the tab state and save dialog both pass through.
+        let dwg_upper = serialize_document(&doc, "DWG").expect("dwg bytes");
+        assert!(dwg_upper.starts_with(b"AC10"));
+    }
+
+    // The worker path must serialize to the same bytes the inline fallback
+    // would — the whole point of sharing this implementation.
+    #[test]
+    fn serialize_document_is_deterministic_across_calls() {
+        let doc = sample_document();
+        let a = serialize_document(&doc, "dxf").expect("bytes a");
+        let b = serialize_document(&doc, "dxf").expect("bytes b");
+        assert_eq!(a, b);
+    }
+}

crates/ocs_web_worker/Cargo.toml

@@ -14,5 +14,8 @@ serde = { version = "1", features = ["derive"] }
 console_error_panic_hook = "0.1"
 getrandom = { version = "0.3", features = ["wasm_js"] }
 js-sys = "0.3"
+# Shared save serialization + worker wire format (one implementation for both
+# this worker and the app's inline fallback).
+ocs_save = { path = "../ocs_save" }
 sha2 = "0.10"
 wasm-bindgen = "0.2"

crates/ocs_web_worker/src/lib.rs

@@ -8,23 +8,11 @@ use sha2::{Digest, Sha256};
 use wasm_bindgen::prelude::*;
 
 const HASH_MARKER: &str = "\nreport-source-sha256:";
-const PROTOCOL_VERSION: u16 = 4;
 
-#[derive(serde::Serialize)]
-struct EntityRuntimeFields {
-    handle: u64,
-    linetype_handle: Option<u64>,
-    color_book_handle: Option<u64>,
-    face_visual_style_handle: Option<u64>,
-    edge_visual_style_handle: Option<u64>,
-    material_flags: u8,
-    material_handle: Option<u64>,
-    shadow_flags: u8,
-    plotstyle_flags: u8,
-    plotstyle_handle: Option<u64>,
-    entity_mode: Option<u8>,
-    has_ds_data: bool,
-}
+/// Version of the shared worker wire format (see `ocs_save`). Both sides of
+/// the boundary — this crate and the app's `io::web_worker` — check payloads
+/// against it.
+const PROTOCOL_VERSION: u16 = ocs_save::PROTOCOL_VERSION;
 
 /// Parse DWG/DXF on a dedicated browser worker and return a compact serialized
 /// document. The main wasm instance only deserializes and installs it, so the
@@ -143,6 +131,28 @@ pub fn sha256_document(bytes: Uint8Array) -> String {
     sha256_document_bytes(&bytes.to_vec())
 }
 
+/// Serialize an already-prepared document to DWG/DXF bytes on the dedicated
+/// browser worker, so the expensive writer pass (bit-level DWG encoding /
+/// ASCII DXF formatting) never occupies the UI thread. `encoded_document` is
+/// the shared `ocs_save` request payload (document + runtime-field sidecar +
+/// target extension); the reply is the shared response payload (protocol
+/// version + `Result<bytes, error text>`). The main wasm instance only decodes
+/// the result — the save-side counterpart of [`parse_document`].
+#[wasm_bindgen]
+pub fn serialize_document(
+    encoded_document: Uint8Array,
+    report_stage: &Function,
+) -> Result<Uint8Array, JsValue> {
+    console_error_panic_hook::set_once();
+    report_stage.call1(&JsValue::NULL, &JsValue::from_str("decode document"))?;
+    let (document, ext) = ocs_save::decode_request(&encoded_document.to_vec())?;
+    report_stage.call1(&JsValue::NULL, &JsValue::from_str("serialize drawing"))?;
+    let result = ocs_save::serialize_document(&document, &ext);
+    report_stage.call1(&JsValue::NULL, &JsValue::from_str("copy output"))?;
+    let encoded = ocs_save::encode_response(result)?;
+    Ok(Uint8Array::from(encoded.as_slice()))
+}
+
 fn report_fingerprint_needed(stats: &codec::ReadStats) -> bool {
     stats.recovered() || stats.skipped_source_records > 0 || !stats.stream_completed
 }
@@ -174,33 +184,7 @@ fn encode_result(
     let runtime_fields = result
         .as_ref()
         .ok()
-        .map(|outcome| {
-            outcome
-                .document
-                .entities()
-                .map(|entity| {
-                    let common = entity.common();
-                    EntityRuntimeFields {
-                        handle: common.handle.value(),
-                        linetype_handle: common.linetype_handle.map(|handle| handle.value()),
-                        color_book_handle: common.color_book_handle.map(|handle| handle.value()),
-                        face_visual_style_handle: common
-                            .face_visual_style_handle
-                            .map(|handle| handle.value()),
-                        edge_visual_style_handle: common
-                            .edge_visual_style_handle
-                            .map(|handle| handle.value()),
-                        material_flags: common.material_flags,
-                        material_handle: common.material_handle.map(|handle| handle.value()),
-                        shadow_flags: common.shadow_flags,
-                        plotstyle_flags: common.plotstyle_flags,
-                        plotstyle_handle: common.plotstyle_handle.map(|handle| handle.value()),
-                        entity_mode: common.entity_mode,
-                        has_ds_data: common.has_ds_data,
-                    }
-                })
-                .collect::<Vec<_>>()
-        })
+        .map(|outcome| ocs_save::extract_entity_runtime_fields(&outcome.document))
         .unwrap_or_default();
     let encoded = bincode::serialize(&(
         PROTOCOL_VERSION,