fix(gpu): a crash the device caused still costs the backend, however late

#1519 stopped the sentinel blaming a backend for a run that had already
drawn for SENTINEL_PROOF_DELAY, on the reasoning that a backend which
kept the application alive that long had proved itself. That reasoning
does not hold for the failure this repository sees most on integrated
hardware.

Measured here on an Intel Iris Xe: with `--backend dx12` a large drawing
aborts the process 110 s in, long past the proof delay, so the sentinel
carried `survived=1` and the next launch went straight back to dx12 and
crashed again. The same binary on `--backend vulkan` opens the same
drawings with no out-of-memory report and no uncaptured errors at all,
so the backend really is the difference and moving off it really is the
cure. D3D12 is fine while the interface draws and only fails once the
whole scene uploads, which is why the clock is the wrong question.

The crash report from #1521 answers the right one. It records the
backend that was live and what the panic was, and it is named for the
process that wrote it, so it joins exactly to the `pid` the sentinel
already stores — no guessing from whichever report is newest, and no
new state to keep. A sentinel marked survived is now excused only when
that run left no report, or left one that has nothing to do with the
graphics device.

The test that matters is the pair, and both were run end to end:

  killed from Task Manager after 40 s, no report  -> dx12 kept
  device crash after 110 s, report names wgpu     -> dx12 skipped, Vulkan

`is_device_failure` is deliberately narrow: an indexing bug in drawing
code that happens to run on a frame must not cost the user a working
backend.

Co-Authored-By: Claude Opus 5 <[email protected]>

src/sys.rs

@@ -429,6 +429,62 @@ pub mod crash_log {
         crate::config::config_dir().map(|path| path.join("crash_logs"))
     }
 
+    /// The report a given process left behind, if it left one.
+    ///
+    /// Reports are named for the process that wrote them, so a caller holding
+    /// a process id from somewhere else — the GPU crash sentinel does — can
+    /// ask what killed that exact run instead of guessing from the newest
+    /// file, which on a busy machine may belong to a different one.
+    pub fn report_for_pid(pid: u32) -> Option<String> {
+        let directory = directory()?;
+        let suffix = format!("-{pid}.log");
+        let entry = std::fs::read_dir(directory)
+            .ok()?
+            .flatten()
+            .find(|entry| {
+                entry
+                    .file_name()
+                    .to_string_lossy()
+                    .ends_with(suffix.as_str())
+            })?;
+        std::fs::read_to_string(entry.path()).ok()
+    }
+
+    /// Whether a report describes the graphics device giving up.
+    ///
+    /// Deliberately narrow. Anything that reaches wgpu — an exhausted device,
+    /// a lost one, a buffer that was never created — is the backend's
+    /// business; a panic in the drawing code that happens to run on a frame
+    /// is not, and must not cost the user a working backend.
+    pub fn is_device_failure(report: &str) -> bool {
+        let mut panic_line = "";
+        let mut at_line = "";
+        for line in report.lines() {
+            if let Some(rest) = line.strip_prefix("panic: ") {
+                panic_line = rest;
+            } else if let Some(rest) = line.strip_prefix("at: ") {
+                at_line = rest;
+            }
+        }
+        // The message wgpu panics with names the call, and the location names
+        // wgpu itself; either alone is enough, since a stripped build can
+        // leave the message terse.
+        let haystack = format!("{panic_line} {at_line}").to_ascii_lowercase();
+        ["wgpu", "out of memory", "buffer", "surface", "device lost"]
+            .iter()
+            .any(|needle| haystack.contains(needle))
+    }
+
+    /// The backend a report says was live, as written by [`report_from`].
+    pub fn backend_in_report(report: &str) -> Option<String> {
+        report
+            .lines()
+            .find_map(|line| line.strip_prefix("gpu backend: "))
+            .map(str::trim)
+            .filter(|backend| !backend.is_empty() && *backend != "(default)")
+            .map(str::to_string)
+    }
+
     /// Chain a report writer onto the current panic hook.
     ///
     /// Call once, early: a panic before this runs is still silent.
@@ -562,6 +618,46 @@ pub mod crash_log {
             );
         }
 
+        /// Only the device's own failures may cost a backend. A panic in
+        /// drawing code that merely happened to run on a frame must not.
+        #[test]
+        fn a_device_failure_is_told_apart_from_an_ordinary_panic() {
+            let device = super::report_from(
+                "Error in Buffer::get_mapped_range: Validation Error",
+                "wgpu-29.0.4/src/backend/wgpu_core.rs:2253:18",
+                1,
+            );
+            assert!(super::is_device_failure(&device), "{device}");
+
+            let oom = super::report_from("Out of Memory", "src/scene/pipeline/mod.rs:10:1", 1);
+            assert!(super::is_device_failure(&oom), "{oom}");
+
+            let ordinary = super::report_from(
+                "index out of bounds: the len is 3 but the index is 7",
+                "src/app/commands/draw.rs:120:5",
+                1,
+            );
+            assert!(
+                !super::is_device_failure(&ordinary),
+                "an indexing bug is not the graphics device's fault:
+{ordinary}"
+            );
+        }
+
+        /// The backend line is what the sentinel joins on, so it has to come
+        /// back out — and "(default)" names nothing to blame.
+        #[test]
+        fn the_backend_is_read_back_out_of_a_report() {
+            // SAFETY: single-threaded test mutating a process-local variable.
+            unsafe { std::env::set_var("WGPU_BACKEND", "dx12") };
+            let named = super::report_from("boom", "x.rs:1:1", 0);
+            assert_eq!(super::backend_in_report(&named).as_deref(), Some("dx12"));
+
+            unsafe { std::env::remove_var("WGPU_BACKEND") };
+            let unnamed = super::report_from("boom", "x.rs:1:1", 0);
+            assert_eq!(super::backend_in_report(&unnamed), None);
+        }
+
         /// A panic with no message still has to produce a filed report rather
         /// than nothing at all.
         #[test]