web: XREFs, data links & extraction, ARCHIVE transmittals, worker-side saves

The last four big web-vs-desktop gaps.

XREFs: XATTACH reads the picked drawing's bytes and registers them
name-keyed (the image/PDF-underlay pattern), pre-parses on the web
worker, and the commit merge produces the same xref block flow as
native. Opened drawings resolve xrefs from the registry (trailing-name
match) before cache build; unresolved references produce ONE warning
listing the names and the fix — drop the referenced files on the page
(drops and picker opens now register their bytes) and reopen or
XRELOAD. XRELOAD works off the registry on unsaved drawings too.

Data Link & Extraction: spreadsheet links read picked xlsx/csv bytes
(calamine from slices / CSV from string) through a session registry,
so preview, refresh, and table-insert work on web; extraction takes
external drawings via a multi-file picker parsed on the worker, and
outputs CSV/TXT/.dxex via download_bytes. The disabled 'Add folder'
button explains why.

ARCHIVE/ETRANSMIT: assembles the drawing plus registered images, PDF
underlays, and session plot tables into a ZIP delivered as
{stem}_transmit.zip; skipped dependencies are reported by name.

Save/autosave off the UI thread: the parse worker gains a serialize
action (protocol v5) — the document is prepared once on the main
thread (clone, styles, dimension bake), then the DWG/DXF writer pass,
the dominant cost, runs on the worker; any worker failure falls back
to inline serialization with the identical shared implementation.
The save path also clears the dirty flag only when the tab didn't
change mid-write, and the runtime-fields sidecar now carries
graphic_data, fixing a pre-existing loss of raw DWG round-trip bytes
on web-opened documents.

(cherry picked from commit 2e100634)

Ported onto main (#1717 review): the new ocs_save crate builds on the
pinned codec; the hand-written ZIP writer is replaced by the zip crate
already in the tree; ARCHIVE packs registered xref bytes too and
serializes document_for_save() like a web Save; the web save prepares
document_for_save() (prepare_document_for_save now takes the copy by
value, so no second clone); the wasm resolve_path consults the xref
registry, so every reference lookup — not just resolve — finds dropped
files; XRELOAD keeps main's undo, unload and stale-baseline handling;
native data extraction streams the source drawings one at a time.
This commit is contained in:
Afonsop authored and Hakan Seven committed 2026-10-12 12:02:04 +03:00
1 parent 9cd898a17a
commit 61c32c7f74
43 files changed
+2187 -336

No files matched your search

+288 -25
View File
@@ -42,8 +42,13 @@ pub mod xref_model;
use crate::scene::DerivedCaches;
use codec::entities::EntityType;
// The path-based readers are native-only; web parsing runs through the
// parse worker / the xref bytes registry instead.
#[cfg(not(target_arch = "wasm32"))]
use codec::io::dwg::DwgReader;
use codec::{CadDocument, DwgReadOptions, DwgWriter, DxfReader, DxfReaderConfiguration, DxfWriter};
#[cfg(not(target_arch = "wasm32"))]
use codec::{DwgReadOptions, DxfReader, DxfReaderConfiguration};
use codec::{CadDocument, DwgWriter, DxfWriter};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU16, AtomicU32, AtomicU8, Ordering};
use std::sync::Arc;
@@ -608,6 +613,10 @@ pub async fn open_bytes_web(
bytes: Arc<[u8]>,
progress: Arc<OpenProgressState>,
) -> WebOpenOutcome {
// The opened drawing's own bytes enter the xref registry too: another
// open drawing may reference it by name, and the registry — not the
// absent filesystem — is what re-resolves that reference in-session.
crate::io::xref::register_source(&name, Arc::new(bytes.to_vec()));
let size_bytes = bytes.len() as u64;
let result = load_web_bytes(&name, &bytes, progress.clone(), false, "", None).await;
let keep_for_recovery = result
@@ -758,9 +767,16 @@ async fn load_web_bytes(
recovery_available: true,
});
}
// XREFs inside the opened drawing resolve from the in-session bytes
// registry (see `resolve_web_xrefs`) before the caches snapshot the
// document, so merged xref geometry is part of the caches — the same
// phase order the native open uses.
let (xref_infos, xref_dropped) = resolve_web_xrefs(&mut doc, progress.clone()).await;
let mut caches = crate::scene::build_derived_caches(&doc);
caches.corrupt_dropped = dropped;
caches.read_stats = Some(outcome.stats);
caches.xrefs = xref_infos;
caches.xref_dropped = xref_dropped;
if source_sha256.is_none() && recovery_fingerprint_needed(&caches) {
source_sha256 = web_worker::sha256_document(bytes).await.ok();
}
@@ -814,6 +830,91 @@ pub async fn pick_and_parse_web(
Ok(Some((name, document)))
}
/// Web: parse drawing bytes already in hand on the shared worker — the
/// byte-level counterpart of `load_file` for callers that ran their own
/// picker (Data Extraction's multi-file source list). Same normalization as
/// every other web-opened drawing.
#[cfg(target_arch = "wasm32")]
pub async fn parse_drawing_bytes_web(name: &str, bytes: &[u8]) -> Result<CadDocument, String> {
parse_web_document(name, bytes).await
}
/// Web: resolve the XREF blocks inside a freshly opened drawing against the
/// in-session bytes registry — the browser counterpart of the native open's
/// xref phase. Every referenced file whose bytes are registered (picked or
/// dropped earlier this session) is parsed on the shared worker first, so
/// the UI thread only runs the (cheap) merge; unregistered references come
/// back as `NotFound` for the open handler to warn about.
#[cfg(target_arch = "wasm32")]
async fn resolve_web_xrefs(
doc: &mut CadDocument,
progress: Arc<OpenProgressState>,
) -> (Vec<crate::io::xref::XrefInfo>, usize) {
let pending: Vec<String> = doc
.block_records
.iter()
.filter(|br| (br.flags.is_xref || br.flags.is_xref_overlay) && !br.xref_path.is_empty())
.map(|br| br.xref_path.clone())
.collect();
let total = pending.len().max(1);
for (index, raw_path) in pending.iter().enumerate() {
progress.set(crate::app::OPEN_PHASE_XREF, 5900, index, total);
let Some(key) = crate::io::xref::find_registered_source(raw_path) else {
continue;
};
let Some(bytes) = crate::io::xref::registered_bytes(&key) else {
continue;
};
// The strict worker refuses recoverable parses; those fall through
// to a direct registry read inside the resolve so their stats (and
// the Failed status derived from them) match the native loader.
if let Ok((outcome, _)) = web_worker::parse_document(&key, &bytes, false, "").await {
crate::io::xref::register_parsed(&key, outcome);
}
}
progress.set(crate::app::OPEN_PHASE_XREF, 5900, total, total);
// `set_fraction` is native-only; the web phase reports completed/total
// directly, which the open overlay surfaces alongside the fixed weight.
let xref_progress: Arc<dyn Fn(usize, usize) + Send + Sync> = {
let progress = Arc::clone(&progress);
Arc::new(move |completed, total| {
progress.set(crate::app::OPEN_PHASE_XREF, 5900, completed, total);
})
};
crate::io::xref::resolve_xrefs_with_progress(doc, Path::new(""), Some(xref_progress))
}
/// Web XATTACH pick: read the chosen drawing's bytes, keep them in the xref
/// registry (so later reloads and re-opens of the host drawing re-resolve
/// the reference in-session), and pre-parse on the worker so the
/// commit-time merge stays cheap. A parse failure defers to commit time,
/// which re-reads the registered bytes and reports the same Failed status
/// the native attach would. `Err("Cancelled")` mirrors the native pick.
#[cfg(target_arch = "wasm32")]
pub async fn pick_and_register_xref_web() -> Result<PathBuf, String> {
let Some(handle) = crate::sys::file_dialog()
.set_title(crate::t!("Select External Reference File").as_ref())
.add_filter(
crate::t!("CAD Files").as_ref(),
&["dwg", "dxf", "bak", "DWG", "DXF", "BAK"],
)
.add_filter(crate::t!("DWG Files").as_ref(), &["dwg", "DWG"])
.add_filter(crate::t!("DXF Files").as_ref(), &["dxf", "DXF"])
.add_filter(crate::t!("Backup Files").as_ref(), &["bak", "BAK"])
.pick_file()
.await
else {
return Err("Cancelled".to_string());
};
let name = handle.file_name();
let bytes = handle.read().await;
crate::io::xref::register_source(&name, Arc::new(bytes.clone()));
if let Ok((outcome, _)) = web_worker::parse_document(&name, &bytes, false, "").await {
crate::io::xref::register_parsed(&name, outcome);
}
Ok(PathBuf::from(name))
}
/// Parse a drawing from bytes using its name or recovery-file signature.
#[cfg(not(target_arch = "wasm32"))]
pub fn load_bytes(name: &str, bytes: Vec<u8>) -> Result<CadDocument, String> {
@@ -870,6 +971,8 @@ pub fn load_bytes_finalized(path: &Path, bytes: Vec<u8>) -> Result<(CadDocument,
/// Load a DWG or DXF file directly from a path (auto-detect by extension).
/// Peek at a file's leading bytes to tell a DWG (version tag "AC10xx") from a
/// DXF. Used for `.bak` copies, whose extension hides the real format.
/// Native-only: the web reads registered bytes instead of the filesystem.
#[cfg(not(target_arch = "wasm32"))]
fn sniff_dwg_or_dxf(path: &Path) -> String {
use std::io::Read;
let mut buf = [0u8; 6];
@@ -905,6 +1008,10 @@ pub fn load_file(_path: &Path) -> Result<CadDocument, String> {
Err(crate::t!("Opening a drawing by path is not available in the browser.").into_owned())
}
/// Path-based reader with progress — native-only: on the web xrefs resolve
/// from the bytes registry (`io::xref::read_registered_outcome`), never a
/// filesystem path.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn load_file_with_progress(
path: &Path,
_progress: Option<Arc<dyn Fn(u16) + Send + Sync>>,
@@ -1001,12 +1108,14 @@ fn load_file_for_open(
finalize_loaded_outcome(path, outcome).map_err(OpenAttemptFailure::from)
}
#[cfg(not(target_arch = "wasm32"))]
struct ReaderFailure {
message: String,
#[cfg(not(target_arch = "wasm32"))]
recoverable: bool,
}
#[cfg(not(target_arch = "wasm32"))]
impl ReaderFailure {
fn from_reader(error: codec::DxfError) -> Self {
Self {
@@ -1067,6 +1176,7 @@ fn merge_read_stats(
}
}
#[cfg(not(target_arch = "wasm32"))]
fn read_file_attempt(
path: &Path,
progress: Option<Arc<dyn Fn(u16) + Send + Sync>>,
@@ -1094,6 +1204,7 @@ fn read_file_attempt(
}
}
#[cfg(not(target_arch = "wasm32"))]
fn finalize_loaded_outcome(
path: &Path,
mut outcome: codec::ReadOutcome,
@@ -1116,6 +1227,7 @@ fn finalize_loaded_outcome(
Ok(outcome)
}
#[cfg(not(target_arch = "wasm32"))]
fn read_dwg_path(
path: &Path,
progress: Option<Arc<dyn Fn(u16) + Send + Sync>>,
@@ -1236,6 +1348,7 @@ fn read_drawing_snapshot(path: &Path) -> std::io::Result<Vec<u8>> {
}
}
#[cfg(not(target_arch = "wasm32"))]
fn read_dxf_path(path: &Path, failsafe: bool) -> Result<codec::ReadOutcome, ReaderFailure> {
DxfReader::from_file(path)
.map_err(ReaderFailure::from_reader)?
@@ -1312,6 +1425,7 @@ fn normalize_block_origins(doc: &mut CadDocument) {
/// are untouched. File handles are reused when still free so a repair +
/// resave stays handle-stable; otherwise a fresh handle is minted and the
/// record is pointed at it.
#[cfg(not(target_arch = "wasm32"))]
fn materialize_missing_block_markers(doc: &mut CadDocument) {
use codec::entities::{Block, BlockEnd};
use codec::{EntityType, Handle};
@@ -1386,6 +1500,9 @@ fn materialize_missing_block_markers(doc: &mut CadDocument) {
/// that doesn't exist on this machine. Resolve each image to a usable path:
/// the stored path if it exists, otherwise the same file name next to the
/// drawing — and write the result onto the entity so the renderer finds it.
/// Native-only: its callers resolve against a real folder; web images bind
/// through the name-keyed byte registry instead.
#[cfg(not(target_arch = "wasm32"))]
fn resolve_raster_image_paths(doc: &mut CadDocument, base_dir: Option<&Path>) {
use codec::objects::ObjectType;
use codec::EntityType;
@@ -2133,6 +2250,13 @@ pub fn save(doc: &CadDocument, path: &Path) -> Result<(), String> {
/// `doc.version`). Used by the web build, which hands the bytes to a browser
/// download instead of writing a path — the byte-buffer counterpart to
/// [`save_as_version`].
///
/// This is the *inline* path (main thread on wasm). The web SAVE/AUTOSAVE
/// flows prefer [`prepare_document_for_save`] +
/// [`serialize_prepared_document`], which runs the writer half on the browser
/// worker; both routes share the same preparation and the same writer
/// dispatch (`ocs_save::serialize_document`), so the produced bytes are
/// identical.
#[allow(dead_code)] // web/wasm build only; unused in the native target
pub fn save_to_bytes(
doc: &CadDocument,
@@ -2141,35 +2265,14 @@ pub fn save_to_bytes(
) -> Result<Vec<u8>, String> {
let perf = crate::perf::enabled();
let total_started = iced::time::Instant::now();
let clone_started = iced::time::Instant::now();
let mut doc = doc.clone();
let clone_ms = clone_started.elapsed().as_secs_f64() * 1000.0;
doc.version = version;
let styles_started = iced::time::Instant::now();
sync_current_styles_on_save(&mut doc);
let styles_ms = styles_started.elapsed().as_secs_f64() * 1000.0;
let dimensions_started = iced::time::Instant::now();
crate::modules::draw::modify::explode::bake_dimension_blocks(&mut doc);
let dimensions_ms = dimensions_started.elapsed().as_secs_f64() * 1000.0;
let doc = prepare_document_for_save(doc.clone(), version);
let write_started = iced::time::Instant::now();
let result = match ext.to_lowercase().as_str() {
"dxf" => DxfWriter::new(&doc)
.write_to_vec()
.map_err(|e| e.to_string()),
_ => {
let mut buf = std::io::Cursor::new(Vec::new());
DwgWriter::write_to_writer(&mut buf, &doc).map_err(|e| e.to_string())?;
Ok(buf.into_inner())
}
};
let result = ocs_save::serialize_document(&doc, ext);
if perf {
let bytes = result.as_ref().map_or(0, Vec::len);
crate::perf_record!(
"[perf] save-bytes total={:.1}ms clone={:.1} styles={:.1} dimensions={:.1} write={:.1} bytes={} entities={} objects={}",
"[perf] save-bytes total={:.1}ms write={:.1} bytes={} entities={} objects={}",
total_started.elapsed().as_secs_f64() * 1000.0,
clone_ms,
styles_ms,
dimensions_ms,
write_started.elapsed().as_secs_f64() * 1000.0,
bytes,
doc.entities().count(),
@@ -2179,6 +2282,59 @@ pub fn save_to_bytes(
result
}
/// Prepare a save copy of a document: stamp the target DXF version, sync the
/// current-style markers into their header handles / dictionary variables, and
/// bake missing dimension geometry blocks. The app-side half of the save
/// pipeline — it must run where the live document lives (the worker crate
/// cannot reach the app's style/dimension modules). It takes the copy by
/// value, so callers that already built one (`document_for_save`) pay no
/// second clone and the live document is never touched. Both
/// [`save_to_bytes`] and the web worker offload call this exactly once per
/// save, so there is one implementation of the preparation for every target.
pub fn prepare_document_for_save(mut doc: CadDocument, version: codec::DxfVersion) -> CadDocument {
let perf = crate::perf::enabled();
doc.version = version;
let styles_started = perf.then(iced::time::Instant::now);
sync_current_styles_on_save(&mut doc);
let styles_ms = crate::perf::elapsed_ms(styles_started);
let dimensions_started = perf.then(iced::time::Instant::now);
crate::modules::draw::modify::explode::bake_dimension_blocks(&mut doc);
let dimensions_ms = crate::perf::elapsed_ms(dimensions_started);
if perf {
crate::perf_record!(
"[perf] save-prepare styles={:.1} dimensions={:.1} entities={} objects={}",
styles_ms,
dimensions_ms,
doc.entities().count(),
doc.objects.len(),
);
}
doc
}
/// Web SAVE/AUTOSAVE: offload the DWG/DXF writer pass to the browser worker.
/// `prepared` comes from [`prepare_document_for_save`] — the serialization
/// itself (the expensive half) runs off the UI thread and the encoded bytes
/// come back here. Falls back to inline serialization when the worker cannot
/// spawn or errors, so a save never hard-fails just because the worker is
/// unavailable.
#[cfg(target_arch = "wasm32")]
pub async fn serialize_prepared_document(
prepared: &CadDocument,
ext: &str,
) -> Result<Vec<u8>, String> {
let encoded = match ocs_save::encode_request(prepared, ext) {
Ok(encoded) => encoded,
// The worker cannot help if the document cannot even be encoded —
// serialize inline from the document we still hold.
Err(error) => {
log::warn!("{error}; serializing inline");
return ocs_save::serialize_document(prepared, ext);
}
};
web_worker::serialize_document(encoded).await
}
// ── Post-load fixups ──────────────────────────────────────────────────────
// Resolve the current text / dimension / multiline style from the handle the
@@ -3259,3 +3415,110 @@ mod pre_r2000_layer_plot_tests {
);
}
}
// ── Save-pipeline prepare tests ────────────────────────────────────────────
//
// `prepare_document_for_save` is the extracted app-side half of the save
// pipeline, shared by the inline `save_to_bytes` and the web worker offload —
// these pin the behaviours both routes depend on.
#[cfg(test)]
mod prepare_save_tests {
use super::*;
use codec::entities::{Dimension, DimensionLinear};
use codec::tables::TextStyle;
use codec::types::Vector3;
// The live document must never observe the save preparation: version
// stamp, style sync and dimension baking all happen on the clone only.
#[test]
fn prepare_stamps_version_and_leaves_live_document_untouched() {
let mut doc = CadDocument::new();
doc.version = codec::DxfVersion::AC1027;
let prepared = prepare_document_for_save(doc.clone(), codec::DxfVersion::AC1032);
assert_eq!(prepared.version, codec::DxfVersion::AC1032);
assert_eq!(doc.version, codec::DxfVersion::AC1027);
}
// The current text style marker must resolve from its name to the table
// entry's handle on the prepared clone (the DWG header stores a handle),
// while the live document keeps its previous marker.
#[test]
fn prepare_syncs_current_style_markers() {
let mut doc = CadDocument::new();
crate::io::linetypes::populate_document(&mut doc);
let mut style = TextStyle::new("ANNOT");
style.handle = doc.allocate_handle();
let style_handle = style.handle;
doc.text_styles.add(style).unwrap();
doc.header.current_text_style_name = "ANNOT".to_string();
let live_handle = doc.header.current_text_style_handle;
let prepared = prepare_document_for_save(doc.clone(), doc.version);
assert_eq!(prepared.header.current_text_style_handle, style_handle);
assert_eq!(doc.header.current_text_style_handle, live_handle);
}
// A dimension without its anonymous geometry block must get one baked on
// the clone (the writer emits the *D block), with the group-10 definition
// point mirrored into `base` — while the live document keeps neither.
#[test]
fn prepare_bakes_missing_dimension_blocks() {
let mut doc = CadDocument::new();
crate::io::linetypes::populate_document(&mut doc);
let mut dim = DimensionLinear::new(
Vector3::new(0.0, 0.0, 0.0),
Vector3::new(10.0, 0.0, 0.0),
);
dim.definition_point = Vector3::new(5.0, 7.0, 0.0);
dim.base.definition_point = Vector3::ZERO;
dim.base.common.handle = doc.allocate_handle();
let handle = dim.base.common.handle;
doc.add_entity(EntityType::Dimension(Dimension::Linear(dim)))
.unwrap();
let prepared = prepare_document_for_save(doc.clone(), doc.version);
let EntityType::Dimension(Dimension::Linear(linear)) =
prepared.get_entity(handle).unwrap()
else {
panic!("dimension missing from prepared clone");
};
assert!(
linear.base.block_name.starts_with("*D"),
"anonymous dimension block not baked: {}",
linear.base.block_name
);
assert!(prepared.block_records.get(&linear.base.block_name).is_some());
assert_eq!(linear.base.definition_point, linear.definition_point);
// The live document is untouched: no block name, stale group-10.
let EntityType::Dimension(Dimension::Linear(live)) =
doc.get_entity(handle).unwrap()
else {
panic!("dimension missing from live document");
};
assert!(live.base.block_name.is_empty());
assert_eq!(live.base.definition_point, Vector3::ZERO);
}
// The whole point of the split: the inline fallback (`save_to_bytes`) and
// the worker route (`prepare_document_for_save` → `ocs_save` on the
// worker) must produce identical bytes for the same input.
#[test]
fn save_to_bytes_equals_prepare_then_shared_serialize() {
let mut doc = CadDocument::new();
crate::io::linetypes::populate_document(&mut doc);
let mut dim = DimensionLinear::new(
Vector3::new(0.0, 0.0, 0.0),
Vector3::new(10.0, 0.0, 0.0),
);
dim.base.common.handle = doc.allocate_handle();
doc.add_entity(EntityType::Dimension(Dimension::Linear(dim)))
.unwrap();
for ext in ["dwg", "dxf"] {
let inline = save_to_bytes(&doc, ext, codec::DxfVersion::AC1032).expect("inline");
let prepared = prepare_document_for_save(doc.clone(), codec::DxfVersion::AC1032);
let shared = ocs_save::serialize_document(&prepared, ext).expect("shared");
assert_eq!(inline, shared, "inline and shared serialize diverge for {ext}");
}
}
}
+5 -2
View File
@@ -132,8 +132,11 @@ pub fn register_session_table(name: &str, bytes: Arc<Vec<u8>>) {
}
/// The bytes registered under `name` this session, matched
/// case-insensitively the way the folder scan matches file names.
fn session_table(name: &str) -> Option<Arc<Vec<u8>>> {
/// case-insensitively the way the folder scan matches file names. The web
/// ARCHIVE/ETRANSMIT packages a drawing's registered plot table into the
/// transmittal; `None` means the table was not picked this session (the
/// built-ins ship with the app and need no transmittal copy).
pub fn session_table(name: &str) -> Option<Arc<Vec<u8>>> {
session_tables()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
+5
View File
@@ -130,6 +130,11 @@ pub async fn open_dropped(
bytes: Vec<u8>,
progress: std::sync::Arc<crate::io::OpenProgressState>,
) -> crate::io::WebOpenOutcome {
// The dropped drawing's bytes also enter the xref registry (name-keyed,
// like dropped raster images): another drawing may reference it by
// name, and the registry — not the absent filesystem — is what
// re-resolves that reference when its host is reopened or reloaded.
super::xref::register_source(&name, std::sync::Arc::new(bytes.clone()));
progress.set(crate::app::OPEN_PHASE_READING, 500, 1, 2);
let bytes: std::sync::Arc<[u8]> = std::sync::Arc::from(bytes);
let size_bytes = bytes.len() as u64;
+95 -44
View File
@@ -7,24 +7,10 @@ use wasm_bindgen::{JsCast, JsValue};
use web_sys::{ErrorEvent, MessageEvent, Worker, WorkerOptions, WorkerType};
const HASH_MARKER: &str = "\nreport-source-sha256:";
const PROTOCOL_VERSION: u16 = 4;
const WORKER_URL: &str = "ocs-parse-worker.js?v=4";
#[derive(serde::Deserialize)]
struct EntityRuntimeFields {
handle: u64,
linetype_handle: Option<u64>,
color_book_handle: Option<u64>,
face_visual_style_handle: Option<u64>,
edge_visual_style_handle: Option<u64>,
material_flags: u8,
material_handle: Option<u64>,
shadow_flags: u8,
plotstyle_flags: u8,
plotstyle_handle: Option<u64>,
entity_mode: Option<u8>,
has_ds_data: bool,
}
/// Version of the shared worker wire format (`ocs_save`); the `?v=` on the
/// worker URL cache-busts to match. v5 adds the `serialize` action.
const PROTOCOL_VERSION: u16 = ocs_save::PROTOCOL_VERSION;
const WORKER_URL: &str = "ocs-parse-worker.js?v=5";
pub(super) async fn parse_document(
name: &str,
@@ -58,7 +44,7 @@ pub(super) async fn parse_document(
Result<codec::ReadOutcome, (String, Option<codec::ReadStats>)>,
Option<String>,
bool,
Vec<EntityRuntimeFields>,
Vec<ocs_save::EntityRuntimeFields>,
) = bincode::deserialize(&bytes)
.map_err(|error| super::OpenLoadError::from(error.to_string()))?;
if payload.0 != PROTOCOL_VERSION {
@@ -69,7 +55,10 @@ pub(super) async fn parse_document(
}
match payload.1 {
Ok(mut outcome) => {
restore_entity_runtime_fields(&mut outcome.document, payload.4);
ocs_save::restore_entity_runtime_fields(
&mut outcome.document,
payload.4,
);
Ok((outcome, payload.2))
}
Err((message, read_stats)) => Err(super::OpenLoadError {
@@ -136,30 +125,6 @@ pub(super) async fn parse_document(
result
}
fn restore_entity_runtime_fields(
document: &mut codec::CadDocument,
fields: Vec<EntityRuntimeFields>,
) {
let handle = |value: Option<u64>| value.map(codec::Handle::new);
for fields in fields {
let Some(entity) = document.get_entity_mut(codec::Handle::new(fields.handle)) else {
continue;
};
let common = entity.common_mut();
common.linetype_handle = handle(fields.linetype_handle);
common.color_book_handle = handle(fields.color_book_handle);
common.face_visual_style_handle = handle(fields.face_visual_style_handle);
common.edge_visual_style_handle = handle(fields.edge_visual_style_handle);
common.material_flags = fields.material_flags;
common.material_handle = handle(fields.material_handle);
common.shadow_flags = fields.shadow_flags;
common.plotstyle_flags = fields.plotstyle_flags;
common.plotstyle_handle = handle(fields.plotstyle_handle);
common.entity_mode = fields.entity_mode;
common.has_ds_data = fields.has_ds_data;
}
}
pub(super) async fn sha256_document(bytes: &[u8]) -> Result<String, super::OpenLoadError> {
let options = WorkerOptions::new();
options.set_type(WorkerType::Module);
@@ -226,6 +191,92 @@ fn decode_worker_error(message: String) -> super::OpenLoadError {
super::OpenLoadError::new(message.to_string(), source_sha256)
}
/// Run the DWG/DXF writer pass for SAVE/AUTOSAVE on the browser worker.
/// `encoded_request` is the shared `ocs_save` payload (prepared document +
/// runtime-field sidecar + target extension); the worker replies with the
/// encoded drawing bytes. Any worker failure — cannot spawn, script error,
/// stale protocol — falls back to decoding the document here and serializing
/// inline on this thread, so a save never hard-fails because the worker is
/// unavailable (strict CSP, blocked worker script, out-of-memory spawn...).
pub(super) async fn serialize_document(encoded_request: Vec<u8>) -> Result<Vec<u8>, String> {
match serialize_document_via_worker(&encoded_request).await {
Ok(bytes) => Ok(bytes),
Err(worker_error) => {
log::warn!("save worker unavailable, serializing inline: {worker_error}");
let (document, ext) = ocs_save::decode_request(&encoded_request)?;
ocs_save::serialize_document(&document, &ext)
}
}
}
async fn serialize_document_via_worker(encoded_request: &[u8]) -> Result<Vec<u8>, String> {
let options = WorkerOptions::new();
options.set_type(WorkerType::Module);
let worker =
Worker::new_with_options(WORKER_URL, &options).map_err(|error| js_error(error))?;
let (sender, receiver) =
iced::futures::channel::oneshot::channel::<Result<Vec<u8>, String>>();
let sender = Rc::new(RefCell::new(Some(sender)));
let message_sender = sender.clone();
let on_message = Closure::<dyn FnMut(MessageEvent)>::new(move |event: MessageEvent| {
let data = event.data();
let ok = Reflect::get(&data, &JsValue::from_str("ok"))
.ok()
.and_then(|value| value.as_bool())
.unwrap_or(false);
let result = if ok {
Reflect::get(&data, &JsValue::from_str("data"))
.map_err(|error| js_error(error))
.and_then(|value| {
let bytes = Uint8Array::new(&value).to_vec();
ocs_save::decode_response(&bytes)
})
.and_then(|result| result)
} else {
let message = Reflect::get(&data, &JsValue::from_str("error"))
.ok()
.and_then(|value| value.as_string())
.unwrap_or_else(|| "CAD serializer worker failed".to_string());
Err(message)
};
if let Some(sender) = message_sender.borrow_mut().take() {
let _ = sender.send(result);
}
});
worker.set_onmessage(Some(on_message.as_ref().unchecked_ref()));
let error_sender = sender;
let on_error = Closure::<dyn FnMut(ErrorEvent)>::new(move |event: ErrorEvent| {
if let Some(sender) = error_sender.borrow_mut().take() {
let _ = sender.send(Err(event.message()));
}
});
worker.set_onerror(Some(on_error.as_ref().unchecked_ref()));
let payload = Object::new();
Reflect::set(
&payload,
&JsValue::from_str("action"),
&JsValue::from_str("serialize"),
)
.map_err(|error| js_error(error))?;
let input = Uint8Array::from(encoded_request);
Reflect::set(&payload, &JsValue::from_str("document"), &input.buffer())
.map_err(|error| js_error(error))?;
let transfer = Array::new();
transfer.push(&input.buffer());
worker
.post_message_with_transfer(&payload, &transfer)
.map_err(|error| js_error(error))?;
let result = receiver
.await
.map_err(|_| "CAD serializer worker closed without a result".to_string())?;
worker.terminate();
result
}
fn js_error(value: JsValue) -> String {
value
.as_string()
+317
View File
@@ -7,6 +7,7 @@ use codec::types::{Handle, Vector3};
use codec::{CadDocument, EntityType};
use rustc_hash::{FxHashMap as HashMap, FxHashSet as HashSet};
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
#[cfg(not(target_arch = "wasm32"))]
type SourceFingerprint = crate::io::edit_lock::FileFingerprint;
@@ -44,6 +45,197 @@ pub struct XrefInfo {
pub source_sha256: Option<String>,
}
/// Names of the xrefs that resolved to nothing (`NotFound`) — the list the
/// one-line "unresolved references" warning after a web open shows.
pub fn unresolved_xref_names(infos: &[XrefInfo]) -> Vec<String> {
infos
.iter()
.filter(|info| info.status == XrefStatus::NotFound)
.map(|info| info.name.clone())
.collect()
}
// ── In-session xref byte registry (web) ────────────────────────────────
//
// The browser exposes no filesystem behind the path a drawing's XREF block
// carries, so the web build keeps the bytes of every DWG/DXF the user picked
// or dropped, keyed by file name — the xref counterpart of
// `image_model::register_source`. Resolution matches a block-record's
// `xref_path` against the registry by trailing file name (case-insensitive),
// the only component two machines can be expected to agree on. Native builds
// never consult the registry: `resolve_path` reads the real filesystem, so
// their behaviour is byte-identical to before. The registry itself compiles
// on every target so its pure matching logic stays unit-testable on the
// native host.
/// Lower-cased trailing component of a path-style xref reference. Drawings
/// store xref paths as the author wrote them (often an absolute foreign
/// path), so the file name is the only part a picked or dropped file on
/// another machine can be expected to match.
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))] // unit-tested on native
pub(crate) fn file_name_key(path: &str) -> String {
path.rsplit(['/', '\\'])
.next()
.unwrap_or_default()
.to_ascii_lowercase()
}
/// Do two xref references name the same file? Compares trailing components
/// case-insensitively (the web has no path beyond the name at all, and the
/// Windows paths most drawings were authored on are case-insensitive).
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))] // unit-tested on native
pub(crate) fn same_file_name(a: &str, b: &str) -> bool {
let (a, b) = (file_name_key(a), file_name_key(b));
!a.is_empty() && a == b
}
fn source_cache() -> &'static Mutex<HashMap<String, Arc<Vec<u8>>>> {
static SOURCES: OnceLock<Mutex<HashMap<String, Arc<Vec<u8>>>>> = OnceLock::new();
SOURCES.get_or_init(|| Mutex::new(HashMap::default()))
}
fn parsed_cache() -> &'static Mutex<HashMap<String, codec::ReadOutcome>> {
static PARSED: OnceLock<Mutex<HashMap<String, codec::ReadOutcome>>> = OnceLock::new();
PARSED.get_or_init(|| Mutex::new(HashMap::default()))
}
/// Keep a picked or dropped drawing's bytes, keyed by the file name the
/// drawing's xref references will be matched against. Registering supersedes
/// any eager parse cached for the same file, so a re-pick (or a drop that
/// finally supplies a missing reference) reloads from the new bytes.
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))] // mirrors image_model::register_source
pub fn register_source(name: &str, bytes: Arc<Vec<u8>>) {
source_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.insert(name.to_string(), bytes);
parsed_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.retain(|parsed, _| !same_file_name(parsed, name));
}
/// Cache an off-thread parse (web parse worker) for the next resolve to
/// merge. The entry is one-shot: a resolve consumes it, so a later reload
/// re-reads the registered bytes instead of replaying a stale document.
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))]
pub(crate) fn register_parsed(name: &str, outcome: codec::ReadOutcome) {
parsed_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.insert(name.to_string(), outcome);
}
/// Registry key whose trailing file name matches `raw` (case-insensitive) —
/// the web stand-in for `resolve_path`, which has no filesystem to consult.
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))] // unit-tested on native
pub(crate) fn find_registered_source(raw: &str) -> Option<String> {
source_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.keys()
.find(|key| same_file_name(key, raw))
.cloned()
}
/// Bytes registered under exactly `key`.
#[cfg_attr(not(target_arch = "wasm32"), allow(dead_code))]
pub(crate) fn registered_bytes(key: &str) -> Option<Arc<Vec<u8>>> {
source_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.get(key)
.cloned()
}
/// Take the cached parse for `key`, if one is pending.
#[cfg(any(target_arch = "wasm32", test))]
fn take_parsed(key: &str) -> Option<codec::ReadOutcome> {
parsed_cache()
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
.remove(key)
}
/// Read a registered xref on the web: the cached off-thread parse when the
/// pick/open pre-pass produced one, otherwise a direct parse of the
/// registered bytes (the eager parse never caches recoverable outcomes —
/// those must keep their stats so phase 2 reports them as Failed, like the
/// native reader does).
#[cfg(target_arch = "wasm32")]
fn read_registered_outcome(
key: &str,
progress: Option<Arc<dyn Fn(u16) + Send + Sync>>,
) -> Result<codec::ReadOutcome, String> {
if let Some(mut outcome) = take_parsed(key) {
finalize_xref_outcome(&mut outcome);
return Ok(outcome);
}
let Some(bytes) = registered_bytes(key) else {
return Err(format!("no registered bytes for xref \"{key}\""));
};
let mut outcome = read_outcome_from_bytes(key, bytes, progress)?;
finalize_xref_outcome(&mut outcome);
Ok(outcome)
}
/// Apply the same post-parse fixups the file-open tail applies, so merged
/// xref content behaves like natively loaded content.
#[cfg(target_arch = "wasm32")]
fn finalize_xref_outcome(outcome: &mut codec::ReadOutcome) {
let doc = &mut outcome.document;
super::normalize_block_origins(doc);
if outcome.stats.source_format == Some(codec::SourceFormat::Dxf) {
super::fix_dxf_dimension_rotations(doc);
super::fix_dxf_layout_plot_settings(doc);
}
super::fix_viewport_status_flags(doc);
super::fix_current_style_names(doc);
}
/// Parse DWG/DXF bytes into a reader outcome — `load_file_with_progress`
/// for the browser, where the "file" is the registry entry. Recoverable
/// parse issues stay in the stats instead of failing the read, matching the
/// native xref loader's behaviour (phase 2 turns them into Failed).
#[cfg(target_arch = "wasm32")]
fn read_outcome_from_bytes(
name: &str,
bytes: Arc<Vec<u8>>,
progress: Option<Arc<dyn Fn(u16) + Send + Sync>>,
) -> Result<codec::ReadOutcome, String> {
use codec::io::dwg::DwgReader;
use codec::DxfReader;
// The readers keep their stream and want an owned, 'static buffer —
// take the registry `Arc`'s contents by value (one copy; the worker
// pre-parse makes this fallback path the exception, not the rule).
let bytes = bytes.to_vec();
let ext = name.rsplit('.').next().unwrap_or_default().to_lowercase();
// A `.bak`/`.sv$` copy hides the real format behind its extension —
// sniff the leading bytes, exactly as the path-based reader does.
let effective = if matches!(ext.as_str(), "bak" | "sv$") {
super::sniff_dwg_or_dxf_bytes(&bytes).to_string()
} else {
ext.clone()
};
let outcome = match effective.as_str() {
"dwg" => {
let mut reader = DwgReader::from_stream(std::io::Cursor::new(bytes));
if let Some(progress) = progress {
reader.set_progress_callback(progress);
}
reader.read_with_stats()
}
"dxf" => DxfReader::from_reader(std::io::Cursor::new(bytes))
.and_then(|reader| reader.read_with_stats()),
_ => return Err(format!("Unsupported file format: .{ext}")),
};
let outcome = outcome.map_err(|error| error.to_string())?;
if !outcome.stats.has_usable_drawing_data() {
return Err("initial read returned no source drawing records".to_string());
}
Ok(outcome)
}
/// Scan `doc` for XREF block-records, resolve their paths relative to
/// `base_dir`, and populate each xref block with entities from the
/// referenced file.
@@ -148,9 +340,16 @@ fn resolve_xrefs_with_filter(
});
callback
});
#[cfg(not(target_arch = "wasm32"))]
let xref_outcome = resolved
.as_ref()
.map(|path| super::load_file_with_progress(path, nested_progress));
// Web: read the registry — the cached off-thread parse when the
// pick/open pre-pass made one, otherwise the registered bytes.
#[cfg(target_arch = "wasm32")]
let xref_outcome = resolved
.as_ref()
.map(|path| read_registered_outcome(&path.to_string_lossy(), nested_progress));
let source_sha256 = resolved.as_ref().and_then(|path| {
let needs_fingerprint = match &xref_outcome {
Some(Ok(outcome)) => {
@@ -414,8 +613,17 @@ pub fn layer_overrides(doc: &CadDocument, name: &str, raw_path: &str, base_dir:
})
}
/// Web: there is no filesystem behind a stored path, so a reference resolves
/// to the name-keyed bytes registry instead (trailing-name match); the
/// returned path is the registry key.
#[cfg(target_arch = "wasm32")]
fn resolve_path(raw: &str, _base_dir: &Path) -> Option<PathBuf> {
find_registered_source(raw).map(PathBuf::from)
}
/// Try to build an absolute path from a raw xref path string.
/// Handles absolute paths, relative paths, and Windows-style separators.
#[cfg(not(target_arch = "wasm32"))]
fn resolve_path(raw: &str, base_dir: &Path) -> Option<PathBuf> {
let normalised = raw.replace('\\', "/");
let p = PathBuf::from(&normalised);
@@ -3279,3 +3487,112 @@ fn relative_point_cloud_path(doc: &mut CadDocument, handle: Handle, host: &Path)
def.source_filename = relative;
}
}
#[cfg(test)]
mod web_registry_tests {
use super::*;
fn outcome() -> codec::ReadOutcome {
codec::ReadOutcome::new(CadDocument::new(), codec::ReadStats::default())
}
#[test]
fn file_name_key_takes_the_trailing_component_case_folded() {
assert_eq!(file_name_key("Site Plan.dwg"), "site plan.dwg");
assert_eq!(
file_name_key(r"D:\Projects\Arch\XREFS\Elevation.DWG"),
"elevation.dwg"
);
assert_eq!(file_name_key("../consultant/model.dxf"), "model.dxf");
assert_eq!(file_name_key(""), "");
}
#[test]
fn same_file_name_ignores_path_and_case_but_not_empty() {
assert!(same_file_name(
r"C:\foreign\abs\PATH\Elevation.DWG",
"elevation.dwg"
));
assert!(same_file_name("a/x.dwg", r"b\X.Dwg"));
assert!(!same_file_name("elevation.dwg", "elevation-2.dwg"));
// An empty trailing component names nothing — never a match, or a
// drawing with a blank xref path would resolve to any empty key.
assert!(!same_file_name("", ""));
assert!(!same_file_name("", "x.dwg"));
}
#[test]
fn registry_finds_a_registered_source_by_trailing_name() {
let registered = "Registry-Find-Site.dwg";
register_source(registered, Arc::new(vec![1, 2, 3]));
// The drawing's stored path is the author's absolute foreign path;
// only the trailing name can match, case-insensitively.
assert_eq!(
find_registered_source(r"E:\2009\Consultant\REGISTRY-FIND-SITE.DWG"),
Some(registered.to_string())
);
assert_eq!(find_registered_source("registry-find-site.dxf"), None);
assert_eq!(
registered_bytes(registered).map(|bytes| bytes.as_ref().clone()),
Some(vec![1, 2, 3])
);
}
#[test]
fn register_source_supersedes_a_cached_parse_of_the_same_file() {
let name = "Registry-Supersede-Plan.dwg";
register_parsed(name, outcome());
// Re-registering the same file under a differently-cased path name
// must drop the stale parse so a reload re-reads the new bytes.
register_source("registry-supersede-plan.DWG", Arc::new(vec![9]));
assert!(take_parsed(name).is_none());
// And the fresh bytes are what the registry hands back.
assert_eq!(
find_registered_source(name).as_deref(),
Some("registry-supersede-plan.DWG")
);
}
#[test]
fn unresolved_xref_names_lists_not_found_entries_only() {
let infos = vec![
XrefInfo {
name: "Loaded.dwg".into(),
path: "Loaded.dwg".into(),
status: XrefStatus::Loaded,
diagnostics: Vec::new(),
read_stats: None,
source_sha256: None,
},
XrefInfo {
name: "Missing A.dwg".into(),
path: r"C:\gone\Missing A.dwg".into(),
status: XrefStatus::NotFound,
diagnostics: Vec::new(),
read_stats: None,
source_sha256: None,
},
XrefInfo {
name: "Broken.dwg".into(),
path: "Broken.dwg".into(),
status: XrefStatus::Failed,
diagnostics: Vec::new(),
read_stats: None,
source_sha256: None,
},
XrefInfo {
name: "Missing B.dxf".into(),
path: "Missing B.dxf".into(),
status: XrefStatus::NotFound,
diagnostics: Vec::new(),
read_stats: None,
source_sha256: None,
},
];
assert_eq!(
unresolved_xref_names(&infos),
vec!["Missing A.dwg".to_string(), "Missing B.dxf".to_string()]
);
assert!(unresolved_xref_names(&[]).is_empty());
}
}