Robustness goldens: one test per seed, so CI spreads them (full runs ~33 → ~15-20 min) (#1851)
Thanks @jyviko.
This commit is contained in:
1 parent
0292fa35e6
commit
7ba2d5db47
6 files changed
+134
-52
No files matched your search
@@ -77,7 +77,7 @@ Pull requests skip a few heavy tests (each takes one to four minutes); they run
|
||||
`main`, nightly, and in `cargo xtask test`:
|
||||
|
||||
- `solidcraft-engine`: `tests::hostile_params_never_panic`, everything in `tests::robustness` and
|
||||
`tests::robustness_goldens` (pull requests that touch `crates/kernel` run its `sample_*` part),
|
||||
the seeds of `tests::robustness_goldens` (`seed_00` … `seed_95`; pull requests that touch `crates/kernel` run every 8th),
|
||||
`pattern_suppress_tests::enclosure_arc_extinction_keeps_a_real_sharp_inner_lip_after_resize_and_reopen`,
|
||||
`pattern_suppress_tests::enclosure_datums_follow_wall_height_and_board_depth_with_material_proofs`,
|
||||
`cmd::plastic::tests::enclosure_snap_fit_follows_resized_front_wall`, the `atomic_commands` contract tests
|
||||
|
||||
+3
-3
@@ -43,13 +43,13 @@ its steps, how it ended (ok, or the kind and the start of the error it was refus
|
||||
for the design it left the bodies' total volume and area (within a relative 1e-6) and their
|
||||
face, edge and shell counts (exact). `tests::robustness_goldens` fails on any change, better or
|
||||
worse, so a kernel change shows every seed it moves. Pull requests that touch `crates/kernel`
|
||||
run every 8th seed (`sample_0` … `sample_3`, about two minutes); pushes to `main` and the
|
||||
nightly run them all (`all_0` … `all_3`).
|
||||
run every 8th seed; pushes to `main` and the nightly run them all. Each seed is its own test
|
||||
(`seed_00` … `seed_95`), so the CI shards and their cores share them out.
|
||||
|
||||
To record an intended change, give the reason:
|
||||
|
||||
```
|
||||
ROBUSTNESS_GOLDENS_BLESS="why it changed" cargo test --profile ci -p solidcraft-engine --lib robustness_goldens::all
|
||||
ROBUSTNESS_GOLDENS_BLESS="why it changed" cargo test --profile ci -p solidcraft-engine --lib robustness_goldens::seed_
|
||||
```
|
||||
|
||||
The changed seeds and the reason are added to the file's `notes`; say why in the pull request too.
|
||||
|
||||
Reference in new issue
Block a user