docs: verification and validation; published benchmarks; fixtures (#31)

* docs: verification and validation against theory, experiments and published benchmarks; fixtures

Owner decision 2026-10-13: test against experimental and theoretical data; results published by
vendors (e.g. Abaqus benchmark and verification manuals) may be used as cited reference values for
our own models of the same problems; once a case matches, its result becomes a fixture so fidelity
cannot regress. Commercial products are still never run and their files and text never copied.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NmAttHhUZTokgPeEJia8sm

* docs: V&V policy in AGENTS.md, ADR 0001 and the roadmap

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NmAttHhUZTokgPeEJia8sm

---------

Co-authored-by: jyviko <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>

AGENTS.md

@@ -48,12 +48,16 @@ People trust ModelCraft with their models; a crash loses their work. **This outr
   HyperMesh, ANSA, Femap, Patran, ANSYS, …): no test data, measurements, decks, results or ids
   produced with them. Deck formats come from the vendors' publicly available reference manuals;
   we write our own descriptions and our own examples, never copy manual text or example decks.
-- **Expected values are analytic.** Reference tests compare with closed-form solutions (patch
-  tests, beams, plates, Lamé cylinders, Kirsch plates, natural frequencies, wave and shock
-  solutions) and with mesh invariants (positive Jacobians, conserved volume, watertight
-  boundaries). A separate **benchmark** suite may compare with reference values published in
-  papers (lid-driven cavity, Schäfer–Turek, Turek–Hron, Taylor bar), each tagged with its source;
-  the analytic suite is the gate. Never values produced with a commercial product.
+- **Verification and validation (docs/verification.md).** Every solver capability is checked
+  three ways, each case in `tests/oracle/` tagged with its source: **theory** (closed-form
+  solutions, patch tests, mesh invariants), **experiment** (measurements published in papers and
+  reports) and **published benchmarks** (reference results published by benchmark bodies and by
+  solver vendors' benchmark and verification manuals, e.g. NAFEMS, Abaqus). Owner decision
+  2026-10-13: published vendor results may be used as reference values, cited per case. We never
+  run a commercial product, never copy its input files, text or figures: each case is our own
+  model built from the published description, and only the published numbers are used. Once our
+  result matches its reference within the stated tolerance, it is frozen as a **fixture**
+  (`fixture.json`), so any later change that moves it fails CI (fidelity never drifts silently).
 - **Algorithms come from papers.** Meshing and solver methods are implemented from published
   papers and theses, cited in the module docs and in `docs/meshing/literature.md`.
 - **Never read, copy or link GPL/LGPL/AGPL code**: TetGen, CGAL, Gmsh, Netgen, CalculiX,

ROADMAP.md

@@ -33,4 +33,5 @@ hex meshes today and it gives the best elements where stresses are read.
 | M12 CFD: lattice Boltzmann | D3Q19/D3Q27 on grids voxelised from the face-tagged surface; TRT, regularised and cumulant collision; interpolated walls, inlets and outlets; LES; octree refinement shared with the hex mesher; wgpu backend (deterministic per device, checked against the CPU) | planned |
 | M13 Fluid–structure interaction | Immersed-boundary coupling of the lattice Boltzmann fluid with the explicit or implicit structure; interface force transfer; sub-cycling; Aitken then interface quasi-Newton coupling; free-surface flow | planned |
 | M14 Blast and impact fluids | EOS-driven fluids in the explicit solver: ALE multi-material or SPH, with fluid–structure contact | planned |
+| Continuous: verification and validation | Every capability checked against theory (closed forms), experiments (published measurements) and published benchmarks (NAFEMS, vendor benchmark and verification manuals such as Abaqus's), each case our own model in `tests/oracle/` with its cited reference and tolerance; `cargo xtask oracle` runs them and writes docs/oracle.md; once a case matches, our result is frozen as a fixture so fidelity cannot regress (docs/verification.md) | started |
 | Continuous: desktop and agents | Desktop app (viewport, model tree, commands, control channel: done; toolbar of every capability from the catalogue, ready or coming soon with its milestone, icons drawn with VectorCraft, docs/features.md: done), MCP server, browser build, command schema as the public API | started |

docs/adr/0001-libraries.md

@@ -89,8 +89,11 @@ dependency of `modelcraft-geometry` only behind its non-default `scripts` featur
 tooling (`modelcraft-testkit`) enables; the shipped library opens designs without it.
 
 **Commercial products** (Abaqus, Nastran, LS-DYNA, LS-PrePost, HyperMesh, ANSA, Femap, Patran,
-ANSYS) are never run, scripted or used as a source of data. Deck formats come from the vendors'
-public reference manuals, described in our own words with our own examples.
+ANSYS) are never run or scripted, and their input files, text and figures are never copied. Deck
+formats come from the vendors' public reference manuals, described in our own words with our own
+examples. Published numerical results (benchmark and verification manuals) may serve as cited
+reference values for our own models of the same problems (owner decision 2026-10-13;
+docs/verification.md).
 
 ## Consequences
 

docs/verification.md

@@ -0,0 +1,29 @@
+# Verification and validation
+
+Every solver capability is checked against three kinds of reference, and every accepted result is
+frozen so it cannot drift.
+
+| Kind | What it checks | Sources |
+|---|---|---|
+| Theory | The code solves the equations right | Closed-form solutions (beams, plates, Lamé, Kirsch, waves, shocks, Poiseuille…), patch tests, convergence rates, mesh invariants |
+| Experiment | The model predicts reality | Measurements published in papers and reports (Taylor bar, cavity and cylinder flows, FSI experiments…) |
+| Published benchmarks | Agreement with established codes on standard problems | NAFEMS benchmarks, vendor benchmark and verification manuals (e.g. Abaqus Benchmarks and Verification manuals), other published reference solutions |
+
+Rules:
+- Each case is **our own model** of the problem, built from the published description (geometry,
+  material, loads, boundary conditions). We never run a commercial product and never copy its
+  input files, manual text or figures; only the published numbers are used, with a citation.
+- Each case lives in `tests/oracle/<case>/`:
+  - `recipe.json` — the model as ModelCraft commands (or a SolidCraft recipe for geometry);
+  - `expected.json` — the reference values with `kind` (theory | experiment | benchmark),
+    `source` (citation: document, table or figure, page), `quantity`, `value`, `tolerance` and
+    the reason for that tolerance (mesh, measurement scatter, model differences);
+  - `fixture.json` — ModelCraft's accepted result, written by `cargo xtask oracle --bless <case>`
+    only after the case passes against `expected.json`.
+- `cargo xtask oracle` runs every case, compares with `expected.json` (pass/fail with error) and
+  with `fixture.json` (any change beyond the fixture tolerance fails CI, even if the reference
+  still passes), and writes `docs/oracle.md` with the table of cases, sources, errors and status.
+- A fixture is updated only deliberately (`--bless`), in a PR that says why the result moved
+  (a better element, a finer default mesh), never to hide a regression.
+- The theory suite is the gate for every PR; experiment and benchmark cases run in CI too, with
+  their cited tolerances.