Bryan Falkowski Release check · Engineer view · Paper

Can I trust this eval run?

A dashboard over the 285 runs of the support-agent eval in Is This Run Valid?. Every run gets two answers: did it pass the validity checks, and did its score move outside the normal range? A failed check comes with evidence. A passed check means no known problem was found, not that the run is proven good.

Start here

All runs at a glance

Normal range: 78.6% to 87.4%, the mean of ten clean runs plus or minus two standard deviations. One ticket is 1.67 points, so a swing of two or three tickets between clean runs is normal. For runs that passed or passed with caveats, the score is the one that would be published (excluded tickets left out). For failed runs, it is the plain score a dashboard without checks would have shown.

Over time

Runs in the order they started, all on October 2, 2026, between 13:15 and 15:02 UTC. Many ran in parallel, so the axis is run order, not clock time. Faults were grouped by batch for the study, so this is a replay of the experiment, not a product's history. Hover a point for details; click it to open the run.

Can I use it?RunScorevs normalMain issueStarted