[Feat] (Bench, Pipe): run the blend-toggle case in CI, give G7 a negative control, and record the two campaign devices

- DriverBenchStateToggle runs mc_state_toggle as its own ctest entry. The case has been in
  kBenchCases since P0 and nothing executed it, so nothing would have noticed it rotting - and it
  is the exact enable/draw/disable/draw shape the microbenchmark P2 owes the GO/NO-GO measures.
  About 1.2 s inside an existing three-minute job.
- scripts/g7_negative_control.sh breaks the pipeline/dynamic split on purpose: it inserts two
  boundaries so ColorMasks becomes a dynamic chunk of its own, which keeps the partition sorted,
  non-overlapping and complete - so it still COMPILES - while making glColorMask bump
  m_pipelineStateVersion without moving the pipeline-subset hash. A non-zero ctest is the pass.
- Everything that could make that control lie is refused rather than reported: a missing
  SetterConsistency test exits 2 instead of reading "no tests matched" as a failure; a tree that
  is already red or already broken exits 2; a patched table that does not compile exits 2, since
  a build break would prove the static_asserts work rather than that the test still checks; and
  the restore is from byte-for-byte copies (never from git, so a dirty tree is given back
  intact), followed by a rebuild and a re-run that must be green. --verify-patch-only exercises
  the mechanism where the test does not exist yet and says explicitly that it is not a pass.
- Profiles for the two campaign devices, and the guard that stops them being trusted early. Both
  carry PROFILE_VERIFIED=0 and every device-specific field is TODO_VERIFY_ON_DEVICE rather than a
  guess: the harness pins through MediaTek nodes and 35d0befa is a Qualcomm part, where
  `su -c 'echo ... > /proc/ppm/...'` fails with a zero exit and the run would report numbers it
  believes were pinned. bench.sh and session.sh now refuse an unverified profile unless
  --allow-unverified-profile is passed, which warns that the run is not comparable with a pinned
  one. The README records what earns PROFILE_VERIFIED=1.
This commit is contained in:
2026-09-07 23:18:09 -04:00
parent e9499d38bd
commit b9c137e146
7 changed files with 440 additions and 1 deletions
+19
View File
@@ -30,6 +30,25 @@ frequency-pin integrity.
5. Root required (frequency pinning, GPU busy sampling).
6. Write a device profile under `devices/` (see `devices/odinlite.env`).
A profile carries `PROFILE_VERIFIED=1` only once its sysfs nodes and OPPs have been read
off *that* device and one pinned window has been checked against them
(`big_cur`/`little_cur`/`gpu_cur_khz` in the result JSON must match the pins). Until then
it says `PROFILE_VERIFIED=0` and `bench.sh` / `session.sh` refuse to run against it unless
`--allow-unverified-profile` is passed, which labels the run unpinned in the warning.
That refusal exists because the pin path is silent when it is wrong: the harness writes
through `/proc/ppm/policy/hard_userlimit_*` and `/proc/gpufreq/gpufreq_opp_freq`, which are
MediaTek nodes, and `su -c 'echo ... > /proc/...'` against a device that has neither fails
without a non-zero exit. The run then reports numbers it believes were taken under a pin.
## Devices
| profile | device | verified |
|---|---|---|
| `devices/odinlite.env` | AYN Odin Lite, MT6877 / Mali-G68 | yes |
| `devices/xiaomi-adreno830.env` | Xiaomi, Snapdragon 8 Elite / Adreno 830 (`35d0befa`) | **no** - Qualcomm pin path not yet taught to `bench.sh` |
| `devices/oppo-mali.env` | Oppo / ColorOS, MediaTek + Mali (`3B159D009VZ00000`) | **no** - OPPs and thermal zone not yet read off the device |
## Usage
```
+29
View File
@@ -14,6 +14,7 @@
# Usage:
# bench.sh --device devices/odinlite.env --backend magma [--samples 30]
# [--warmup 180] [--label mylabel] [--no-pin]
# [--allow-unverified-profile]
# backend: magma | espryt | mobileglues (reference)
#
# Output: one JSON line on stdout (also appended to results/results.jsonl) with
@@ -37,6 +38,7 @@ SAMPLES=30
WARMUP=180
LABEL=""
DO_PIN=1
ALLOW_UNVERIFIED_PROFILE=0
WORLD_LOAD_TIMEOUT=420
while [ $# -gt 0 ]; do
@@ -47,6 +49,7 @@ while [ $# -gt 0 ]; do
--warmup) WARMUP=$2; shift 2 ;;
--label) LABEL=$2; shift 2 ;;
--no-pin) DO_PIN=0; shift ;;
--allow-unverified-profile) ALLOW_UNVERIFIED_PROFILE=1; shift ;;
*) echo "unknown arg: $1" >&2; exit 2 ;;
esac
done
@@ -55,6 +58,32 @@ done
# shellcheck disable=SC1090
. "$DEVICE_ENV"
# A device profile that has not been read off its device yet is refused here rather than acted
# on. The failure it prevents is silent and expensive: the pin path below is MediaTek-specific
# (/proc/ppm, /proc/gpufreq), `su -c 'echo ... > /proc/...'` fails without a non-zero exit, and a
# run against a profile whose nodes do not exist reports numbers it believes were taken under a
# frequency pin. The pin-integrity fields sampled at window end are the only clue, and they are
# read after the run rather than before it.
#
# PROFILE_VERIFIED=1 means: somebody read the cpufreq policies, the GPU OPP and the thermal zone
# TYPE off THIS device, ran one pinned window, and checked big_cur/little_cur/gpu_cur_khz in the
# result JSON against the pins. Nothing else earns it.
require_verified_profile() {
if [ "${PROFILE_VERIFIED:-1}" = "1" ]; then return 0; fi
if [ "$ALLOW_UNVERIFIED_PROFILE" = "1" ]; then
echo "[warn] $DEVICE_ENV declares PROFILE_VERIFIED=0 and --allow-unverified-profile was passed:" >&2
echo "[warn] the frequency pins and the thermal gate in it are UNCONFIRMED, so any number this" >&2
echo "[warn] run produces is not comparable with a pinned one." >&2
return 0
fi
echo "$DEVICE_ENV declares PROFILE_VERIFIED=0: its sysfs nodes and OPPs have not been read off" >&2
echo "the device, so pinning would fail silently and the run would look pinned but not be." >&2
echo "Fill in the TODO_VERIFY_ON_DEVICE fields, confirm one pinned window, set PROFILE_VERIFIED=1 -" >&2
echo "or pass --allow-unverified-profile to measure anyway and label the result unpinned." >&2
exit 2
}
require_verified_profile
case "$BACKEND" in
espryt) RENDERER=$RENDERER_ESPRYT ;;
magma) RENDERER=$RENDERER_MAGMA ;;
+48
View File
@@ -0,0 +1,48 @@
# Device profile: Oppo / ColorOS, Mali GPU, adb serial 3B159D009VZ00000.
#
# The second of the two devices the disaggregation campaign is measured on (the other is
# devices/xiaomi-adreno830.env). Same purpose: keep the pinning and thermal protocol in the
# repository rather than in one operator's shell history.
#
# ============================ NOT YET DEVICE-VERIFIED ============================
# PROFILE_VERIFIED=0, and bench.sh / session.sh / profile.sh refuse to run against it unless
# --allow-unverified-profile is passed. This part is a MediaTek SoC, so unlike the Adreno
# profile the harness's existing /proc/ppm + /proc/gpufreq pin path is probably the right one -
# but "probably" is exactly the state a measurement profile must not ship in. The cluster
# indices, the available OPPs, the top GPU OPP and the thermal zone TYPE all differ between
# MediaTek generations, and odinlite.env's values are for an MT6877, not for this device.
#
# To promote it: read the four TODO fields off the device
# (`cat /sys/devices/system/cpu/cpufreq/policy*/scaling_available_frequencies`,
# `cat /proc/gpufreq/gpufreq_opp_dump`, `for tz in /sys/class/thermal/thermal_zone*; do
# echo "$tz $(cat $tz/type)"; done`), run one pinned window, check big_cur/little_cur/gpu_cur_khz
# in the result JSON against the pins, then set PROFILE_VERIFIED=1.
#
# ColorOS traps that belong with this device, and cost a run each when forgotten:
# * the first install of a not-yet-installed package blocks on
# com.oplus.appdetail InstallGuideActivity until "continue install" is tapped
# (`input tap 353 2349` on the 1272x2772 panel);
# * a foreign-signed APK has to be uninstalled before a rebuild will install;
# * pass MSYS_NO_PATHCONV=1 on every adb invocation from Git Bash, or a /data/... argument is
# rewritten into a Windows path.
# =================================================================================
PROFILE_VERIFIED=0
PIN_STYLE=ppm
DEVICE_SERIAL=3B159D009VZ00000
# Campaign protocol constants (perf-test-protocol): big 1.96 GHz, little 1.55 GHz, GPU at its
# top OPP, 40 C start gate. As above, the kHz values are the protocol's targets and the nearest
# actual OPP has to be confirmed on the device.
CPU_BIG_POLICY=TODO_VERIFY_ON_DEVICE
CPU_BIG_FREQ=1958000
CPU_LITTLE_POLICY=TODO_VERIFY_ON_DEVICE
CPU_LITTLE_FREQ=1550000
# MediaTek legacy gpufreq, same node family as odinlite. The top OPP is device-specific.
GPU_PIN_KHZ=
GPU_UTIL_NODE=/sys/kernel/ged/hal/gpu_utilization
GPU_CURFREQ_NODE=/sys/kernel/ged/hal/current_freqency
THERMAL_ZONE_TYPE=TODO_VERIFY_ON_DEVICE
THERMAL_START_MAX_MC=40000
@@ -0,0 +1,54 @@
# Device profile: Xiaomi, Snapdragon 8 Elite (Adreno 830), adb serial 35d0befa.
#
# One of the two devices the disaggregation campaign is measured on (the other is
# devices/oppo-mali.env). It exists so that the pinning and thermal protocol the campaign
# actually runs is written down in the repository instead of living in one operator's shell
# history, and so that a `--device` argument names something reviewable.
#
# ============================ NOT YET DEVICE-VERIFIED ============================
# PROFILE_VERIFIED=0 below, and bench.sh / session.sh / profile.sh REFUSE to run against a
# profile that says so unless --allow-unverified-profile is passed. Two of the values here are
# protocol constants that are known (the campaign pins big 1.96 GHz / little 1.55 GHz and gates
# at 40 C), but the sysfs node names and the exact available OPPs are NOT: this is a Qualcomm
# part and the harness was written against MediaTek, where the pin goes through
# /proc/ppm/policy/hard_userlimit_* and the GPU through /proc/gpufreq/gpufreq_opp_freq. Neither
# path exists on this SoC - Adreno pins through /sys/class/kgsl/kgsl-3d0/devfreq/{min,max}_freq
# and its cpufreq policies are not policy6/policy0.
#
# A profile that quietly wrote MediaTek paths on this device would be the worst outcome
# available: `su -c 'echo ... > /proc/ppm/...'` fails silently, bench.sh would report a run it
# believes was pinned, and the pin-integrity fields it samples at window end would be the only
# clue. So the unknown fields are left EMPTY and marked, rather than guessed, and the refusal is
# the mechanism that keeps them from being used before somebody has read them off the device.
#
# To promote this profile: fill in the four TODO fields from the device
# (`cat /sys/devices/system/cpu/cpufreq/policy*/scaling_available_frequencies`,
# `ls /sys/class/kgsl/kgsl-3d0/devfreq/`, `for tz in /sys/class/thermal/thermal_zone*; do
# echo "$tz $(cat $tz/type)"; done`), teach bench.sh the Qualcomm pin path, run one pinned
# window, check big_cur/little_cur/gpu_cur_khz in the result JSON against the pins, and only
# then set PROFILE_VERIFIED=1 in the same commit as the bench.sh change.
# =================================================================================
PROFILE_VERIFIED=0
PIN_STYLE=qualcomm-kgsl
DEVICE_SERIAL=35d0befa
# Campaign protocol constants (perf-test-protocol): big 1.96 GHz, little 1.55 GHz, GPU at its
# top OPP, and a 40 C start gate. The kHz values are the protocol's targets; the nearest actual
# OPP has to be read off the device before they are used, because a cpufreq write that names a
# frequency the policy does not offer is rounded silently.
CPU_BIG_POLICY=TODO_VERIFY_ON_DEVICE
CPU_BIG_FREQ=1958400
CPU_LITTLE_POLICY=TODO_VERIFY_ON_DEVICE
CPU_LITTLE_FREQ=1555200
# Adreno pins through the kgsl devfreq knobs, not /proc/gpufreq. Left empty deliberately: see
# the block above.
GPU_PIN_KHZ=
GPU_UTIL_NODE=/sys/class/kgsl/kgsl-3d0/gpubusy
GPU_CURFREQ_NODE=/sys/class/kgsl/kgsl-3d0/gpuclk
# Thermal gate: 40 C, the campaign's threshold. The zone TYPE differs per SoC and bench.sh
# matches on it by name, so it has to be read off the device.
THERMAL_ZONE_TYPE=TODO_VERIFY_ON_DEVICE
THERMAL_START_MAX_MC=40000
+29 -1
View File
@@ -7,6 +7,7 @@
#
# Usage: session.sh --device devices/odinlite.env [--backend magma|espryt|mobileglues]
# [--settle 150] [--retries 3] [--no-pin]
# [--allow-unverified-profile]
# Exits 0 with the game in-world (after settle seconds), 1 otherwise.
# NOTE: leaves the game running AND the frequency pins active (that is the
# point of a session). When done: am force-stop the game and unpin via
@@ -24,7 +25,7 @@ RENDERER_ESPRYT=5e273ee2-baca-4c81-8e48-b63feefb9ba8
RENDERER_MAGMA=2be0dc10-1eef-4ce2-b512-b266dd33fd9e
RENDERER_MOBILEGLUES=com.fcl.plugin.mobileglues
DEVICE_ENV="" BACKEND="" SETTLE=150 RETRIES=3 DO_PIN=1
DEVICE_ENV="" BACKEND="" SETTLE=150 RETRIES=3 DO_PIN=1 ALLOW_UNVERIFIED_PROFILE=0
while [ $# -gt 0 ]; do
case "$1" in
--device) DEVICE_ENV=$2; shift 2 ;;
@@ -32,12 +33,39 @@ while [ $# -gt 0 ]; do
--settle) SETTLE=$2; shift 2 ;;
--retries) RETRIES=$2; shift 2 ;;
--no-pin) DO_PIN=0; shift ;;
--allow-unverified-profile) ALLOW_UNVERIFIED_PROFILE=1; shift ;;
*) echo "unknown arg: $1" >&2; exit 2 ;;
esac
done
[ -n "$DEVICE_ENV" ] || { echo "need --device" >&2; exit 2; }
# shellcheck disable=SC1090
. "$DEVICE_ENV"
# A device profile that has not been read off its device yet is refused here rather than acted
# on. The failure it prevents is silent and expensive: the pin path below is MediaTek-specific
# (/proc/ppm, /proc/gpufreq), `su -c 'echo ... > /proc/...'` fails without a non-zero exit, and a
# run against a profile whose nodes do not exist reports numbers it believes were taken under a
# frequency pin. The pin-integrity fields sampled at window end are the only clue, and they are
# read after the run rather than before it.
#
# PROFILE_VERIFIED=1 means: somebody read the cpufreq policies, the GPU OPP and the thermal zone
# TYPE off THIS device, ran one pinned window, and checked big_cur/little_cur/gpu_cur_khz in the
# result JSON against the pins. Nothing else earns it.
require_verified_profile() {
if [ "${PROFILE_VERIFIED:-1}" = "1" ]; then return 0; fi
if [ "$ALLOW_UNVERIFIED_PROFILE" = "1" ]; then
echo "[warn] $DEVICE_ENV declares PROFILE_VERIFIED=0 and --allow-unverified-profile was passed:" >&2
echo "[warn] the frequency pins and the thermal gate in it are UNCONFIRMED, so any number this" >&2
echo "[warn] run produces is not comparable with a pinned one." >&2
return 0
fi
echo "$DEVICE_ENV declares PROFILE_VERIFIED=0: its sysfs nodes and OPPs have not been read off" >&2
echo "the device, so pinning would fail silently and the run would look pinned but not be." >&2
echo "Fill in the TODO_VERIFY_ON_DEVICE fields, confirm one pinned window, set PROFILE_VERIFIED=1 -" >&2
echo "or pass --allow-unverified-profile to measure anyway and label the result unpinned." >&2
exit 2
}
require_verified_profile
ADB="adb -s $DEVICE_SERIAL"
log() { echo "[session] $*" >&2; }