swung0x48
af20dba6db
- format_benchmark printed a p50 taken with the nearest-rank rule beside a medianFrameCpuMs the device computes as the average of the two middle frames, and documented the two as one rule; on an even window they differ (the pre-flight printed p50=8.261ms next to medianCpuMs=271.766). p50 now goes through series_median, which is SummarizeSeries' rule transcribed; p95 and p99 stay nearest rank, which is the device's rule for p95 and the honest extension of it for the p99 the device does not compute at all - require_verified_profile treated a profile that simply omits PROFILE_VERIFIED as verified, which is the fail-open default a profile written by copying another one inherits - exactly the case the guard exists for. It defaults to unverified now, odinlite.env carries PROFILE_VERIFIED=1 explicitly (it is the one profile that earned it), and the refusal says "says 0, or says nothing" - the two new profiles claimed profile.sh refuses an unverified profile; it has no such check and needs none - it records a simpleperf profile and pins nothing. The claim is corrected in both profiles and in the README rather than a guard added where there is nothing to guard - the handle-ABA / CSO control step in test.yml set only MOBILEGL_ITEST_REQUIRE_GPU while its sibling verify step sets the three MOBILEGL_MAGMA_* fixes and arms core dumps. It runs the same DirectVulkan binary on the same runner, so a crash there left no core; it now carries both
device_bench — in-game FPS benchmark harness
Scripted, repeatable in-game FPS measurement for MobileGL's two Android backends (Espryt/DirectGLES and Magma/DirectVulkan) plus a MobileGlues reference run, driven through the FCL fordebug flavor. Intended for A/B performance work and release regression gates on real devices.
How it measures
FCL's in-game FPS overlay counts eglSwapBuffers calls natively (renderer-
agnostic, not vsync-capped when the game runs with vsync off). When the overlay
is enabled, FCL's FPS thread logs one FCLFPS: <n> logcat line per second;
bench.sh collects those lines during the measurement window and reports
mean / median / min / max / stdev, alongside GPU busy%, SoC temperature, and
frequency-pin integrity.
One-time setup (per device / world)
-
Install the FCL fordebug flavor (
com.tungsten.fcl.mgdebug.debug). Its splash auto-launches the selected profile into the prepared world after a 5 s countdown. -
In-game menu: enable show FPS (persists in
files/menu_setting.json). -
Prepare the benchmark world: fixed camera position, gamerules
doMobSpawning/doDaylightCycle/doWeatherCycle=false, then save & quit once.bench.shalwaysam force-stops the game (never saves), so every run replays the same state. -
options.txt: desiredrenderDistance,enableVsync:false, highmaxFps,inactivityFpsLimit:"minimized"(the "afk" default locks 30 fps after 60 s without input and ruins the window). -
Root required (frequency pinning, GPU busy sampling).
-
Write a device profile under
devices/(seedevices/odinlite.env).A profile carries
PROFILE_VERIFIED=1only once its sysfs nodes and OPPs have been read off that device and one pinned window has been checked against them (big_cur/little_cur/gpu_cur_khzin the result JSON must match the pins). Until then it saysPROFILE_VERIFIED=0andbench.sh/session.shrefuse to run against it unless--allow-unverified-profileis passed, which labels the run unpinned in the warning. A profile that omits the key entirely is refused the same way - the guard defaults to unverified, so copying a verified profile and editing the serial cannot inherit its verdict. (profile.shpins nothing - it records a simpleperf profile - so it carries no such guard.)That refusal exists because the pin path is silent when it is wrong: the harness writes through
/proc/ppm/policy/hard_userlimit_*and/proc/gpufreq/gpufreq_opp_freq, which are MediaTek nodes, andsu -c 'echo ... > /proc/...'against a device that has neither fails without a non-zero exit. The run then reports numbers it believes were taken under a pin.
Devices
| profile | device | verified |
|---|---|---|
devices/odinlite.env |
AYN Odin Lite, MT6877 / Mali-G68 | yes |
devices/xiaomi-adreno830.env |
Xiaomi, Snapdragon 8 Elite / Adreno 830 (35d0befa) |
no - Qualcomm pin path not yet taught to bench.sh |
devices/oppo-mali.env |
Oppo / ColorOS, MediaTek + Mali (3B159D009VZ00000) |
no - OPPs and thermal zone not yet read off the device |
Usage
./bench.sh --device devices/odinlite.env --backend magma # 30 samples, 180 s warmup
./bench.sh --device devices/odinlite.env --backend espryt --label after-fix-X
./bench.sh --device devices/odinlite.env --backend mobileglues # reference
Results append to results/results.jsonl; per-run screenshots (pre.png,
post.png) land in results/<timestamp>-<backend>[-label]/ — always eyeball
them: the pre/post pair must show the same scene, or the run is invalid.
Protocol discipline (hard-won, do not skip)
- Thermal gate: the script waits for the profile's start-temperature threshold. Runs started hot are not comparable to runs started cool.
- Warmup 180 s: ART JIT takes ~3 min to plateau (62→67→84 fps ramp was measured); short warmups underestimate by 10-20%.
- Pins can be overridden by the thermal engine. The result JSON records
big_cur/little_cur/gpu_cur_khzsampled at window end — discard the run if they do not match the profile pins. - Paired runs: absolute FPS drifts across sessions (camera angle, world state). A/B comparisons must be back-to-back runs in the same session.
- F3 off for standard numbers (the F3 debug overlay multiplies per-draw overhead and skews backends differently).
- The FPS overlay itself must be ON (it is what produces the FCLFPS lines).