Public reference tests: analytic and FreeCAD expected values (#310)

The oracle no longer uses Fusion measurements. tests/oracle/<case>/ now holds the 76 recipes
(our own format, with recorded measurements and tool-specific fields removed) and an
expected.json per case:

- 55 analytic cases: closed-form values from tools/oracle-analytic/ (stdlib Python, formulas
  in each case's "method").
- 21 FreeCAD cases: the same recipe built with FreeCAD's public scripting API by
  tools/freecad-oracle/ (an external program, not a dependency), with a reference.stl for 20.

solvecraft-cli oracle compares body count, volume, area, centre of mass, bbox, face counts
where the geometry fixes them, and, against reference.stl, the sampled Hausdorff distance and
the principal moments (spatial.rs from #301). cargo xtask oracle reads tests/oracle, writes
docs/oracle.md with the source of each value, and supports known_failure marks for tracked
bugs. CI runs it on every PR; a manual workflow regenerates the FreeCAD values. step-corpus,
seam normalisation and the through-all emulation, which only existed to compare with Fusion,
are removed.

75/76 pass; 75-sm-bracket-holes is a known failure (#296).


Claude-Session: https://claude.ai/code/session_01RjGw4gA5ZvKrU8MQP6sBgT

Co-authored-by: bherbruck <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
This commit is contained in:
authored and GitHub committed 2026-10-12 10:43:43 -04:00
1 parent f0dda667bb
commit e12758ff91
204 files changed
+27106 -932

No files matched your search

+10 -6
View File
@@ -119,7 +119,7 @@ the most complete description of the house style.
|---|---|---|
| Unit and engine tests | next to the code, or `*_tests.rs` files | `cargo xtask test` (or `cargo test --workspace --profile ci`) |
| UI scenarios | `crates/ui-egui/tests/scenarios/*.json` | part of `cargo test` |
| Fusion oracle | `cargo xtask oracle` | needs the local oracle data (see below) |
| Reference tests | `cargo xtask oracle` | `tests/oracle/` (see below) |
**UI scenarios** drive the real app headless: start a tool, click a plane or a sketch point,
press keys, then assert on the result.
@@ -150,11 +150,15 @@ in the PR. `cargo xtask test` builds the tests with it and then runs every test
`cargo xtask test -- shell`. Tests run beside each other on a busy machine, so a test must not
depend on wall-clock timing.
**The Fusion oracle** is a set of 76 reference parts, each built in Fusion from a recipe.
SolidCraft rebuilds the same recipe and compares volume, area and topology counts with Fusion's
measurements; [docs/oracle.md](docs/oracle.md) has the current results. The recipes and
measurements live outside the repo (`plan/`, gitignored), because they come from a Fusion
install. If you work on geometry and want to check against it, ask in an issue.
**The reference tests** are 76 parts in [`tests/oracle/`](tests/oracle/), each a `recipe.json`
in our own format with an `expected.json` beside it. SolidCraft rebuilds the recipe and compares
body count, volume, area, centre of mass and bounding box (and, where there is a `reference.stl`,
the surface itself). Expected values are either worked out in closed form
(`tools/oracle-analytic/`) or computed by building the same recipe in FreeCAD, run as a separate
program (`tools/freecad-oracle/`); each file says which. [docs/oracle.md](docs/oracle.md) has
the current results, and CI runs `cargo xtask oracle` on every pull request. A case that fails
because of a known bug names its issue in `expected.json` (`"known_failure": "#123"`); never
change an expected value to match SolidCraft's output.
## Sending a change