[Fix] (Trace, Bench, CI): compute p50 by the device's own median rule, fail the profile guard closed, and give the new control step its sibling's environment

- format_benchmark printed a p50 taken with the nearest-rank rule beside a medianFrameCpuMs the
  device computes as the average of the two middle frames, and documented the two as one rule; on
  an even window they differ (the pre-flight printed p50=8.261ms next to medianCpuMs=271.766).
  p50 now goes through series_median, which is SummarizeSeries' rule transcribed; p95 and p99 stay
  nearest rank, which is the device's rule for p95 and the honest extension of it for the p99 the
  device does not compute at all
- require_verified_profile treated a profile that simply omits PROFILE_VERIFIED as verified, which
  is the fail-open default a profile written by copying another one inherits - exactly the case the
  guard exists for. It defaults to unverified now, odinlite.env carries PROFILE_VERIFIED=1
  explicitly (it is the one profile that earned it), and the refusal says "says 0, or says nothing"
- the two new profiles claimed profile.sh refuses an unverified profile; it has no such check and
  needs none - it records a simpleperf profile and pins nothing. The claim is corrected in both
  profiles and in the README rather than a guard added where there is nothing to guard
- the handle-ABA / CSO control step in test.yml set only MOBILEGL_ITEST_REQUIRE_GPU while its
  sibling verify step sets the three MOBILEGL_MAGMA_* fixes and arms core dumps. It runs the same
  DirectVulkan binary on the same runner, so a crash there left no core; it now carries both
This commit is contained in:
2026-09-07 23:18:09 -04:00
parent a5d1136c02
commit af20dba6db
8 changed files with 73 additions and 20 deletions
@@ -226,12 +226,32 @@ def read_benchmark(case, backend, run_index):
return report
def series_median(values):
"""The median, by the rule SummarizeSeries uses on the device.
trace_replay_core.cpp's SeriesSummary takes the middle element of an odd window and the
AVERAGE of the two middle elements of an even one, so p50 has to be computed the same way or
the line would print a p50 next to a medianFrameCpuMs that disagreed with it for a reason
nobody could see. (It is the only one of the three that is not a nearest rank: the device's
p95 is.)
"""
if not values:
return -1.0
ordered = sorted(values)
middle = len(ordered) // 2
if len(ordered) % 2 == 1:
return ordered[middle]
return 0.5 * (ordered[middle - 1] + ordered[middle])
def nearest_rank_percentile(values, fraction):
"""Nearest-rank percentile, the same rule SummarizeSeries uses on the device.
"""Nearest-rank percentile, the rule SummarizeSeries uses on the device for p95.
Nearest rank rather than an interpolating percentile so that every number printed here is a
frame that was actually observed, and so that a p95 computed on this side agrees exactly with
the p95 the device reported for the same window.
the p95 the device reported for the same window. The device computes no p99 at all - that is
the whole reason benchmark.json carries the full series - so p99 is this rule extended, and
p50 is NOT computed here (see series_median).
"""
if not values:
return -1.0
@@ -245,11 +265,12 @@ def nearest_rank_percentile(values, fraction):
def cpu_tail(report):
"""The trailing window of the per-frame CPU series, or [] when the run collected none.
benchmark.json carries the WHOLE frameCpuTimesMs[] array precisely so that percentiles the
device does not compute - p50 and p99, which are what the paired A/B publishes - are a
host-side reduction over an artefact that already exists. The window is the same trailing
tailFrames the device summarised, so the numbers below sit beside the device's own without
being about a different set of frames.
benchmark.json carries the WHOLE frameCpuTimesMs[] array precisely so that p99 - which the
device does not compute, and which the paired A/B publishes beside p50 - is a host-side
reduction over an artefact that already exists. The window is the same trailing tailFrames the
device summarised, so the numbers below sit beside the device's own without being about a
different set of frames; p50 is recomputed here by the device's own median rule, so it agrees
with medianFrameCpuMs on the same run rather than merely sitting next to it.
"""
series = report.get("frameCpuTimesMs") or []
if not series:
@@ -277,7 +298,7 @@ def format_benchmark(report):
if window:
line += (
f" | cpu mean={report.get('meanFrameCpuMs', -1):.3f}ms"
f" p50={nearest_rank_percentile(window, 0.50):.3f}ms"
f" p50={series_median(window):.3f}ms"
f" p95={report.get('p95FrameCpuMs', -1):.3f}ms"
f" p99={nearest_rank_percentile(window, 0.99):.3f}ms"
)