mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-09-09 04:38:30 +09:00
swung0x48
b1c37699b1
- The verified-profile guard read the process environment as well as the profile: the test ran after the source, so PROFILE_VERIFIED=1 exported in an operator's shell re-opened the fail-open hole for every profile that says nothing. Both scripts now set PROFILE_VERIFIED=0 immediately before sourcing, so the file is the only thing that can answer. - A --device path that cannot be sourced was diagnosed as an unverified profile, because both scripts cd to their own directory first and neither checked readability. The path is now also tried relative to the directory the script was invoked from (which is what a repo-root-relative --device means), and an unreadable one is reported as unreadable, naming both places tried. - Verified: exported PROFILE_VERIFIED=1 + an unverified profile -> rc 2; exported 1 + a profile with no key -> rc 2; a repo-root-relative path -> resolved, then refused for its own reason; a missing file -> "cannot read the device profile"; odinlite.env -> past the guard; --allow-unverified-profile -> the three warnings, then proceeds. - test.yml's new step described a mechanism the tree does not have. G6's and G10's entries are registered in the pull build too - they must be, for G2's name-for-name comparison - and skip inside their bodies. The step's value is unchanged and its comment now says the true thing: the `test` job runs those names as a column of skips, and this is the first CI job that unpacks a build which compiled the assertions. - trace_benchmark takes the wall baseline before the CPU baseline, the order OnFrameBoundary already reads them in, so frame 0 stops reporting a CPU delta biased upward against its own wall delta; and it includes <time.h> rather than <ctime> for the POSIX names it uses.
85 lines
4.8 KiB
Markdown
85 lines
4.8 KiB
Markdown
# device_bench — in-game FPS benchmark harness
|
|
|
|
Scripted, repeatable in-game FPS measurement for MobileGL's two Android backends
|
|
(Espryt/DirectGLES and Magma/DirectVulkan) plus a MobileGlues reference run,
|
|
driven through the FCL fordebug flavor. Intended for A/B performance work and
|
|
release regression gates on real devices.
|
|
|
|
## How it measures
|
|
|
|
FCL's in-game FPS overlay counts `eglSwapBuffers` calls natively (renderer-
|
|
agnostic, not vsync-capped when the game runs with vsync off). When the overlay
|
|
is enabled, FCL's FPS thread logs one `FCLFPS: <n>` logcat line per second;
|
|
`bench.sh` collects those lines during the measurement window and reports
|
|
mean / median / min / max / stdev, alongside GPU busy%, SoC temperature, and
|
|
frequency-pin integrity.
|
|
|
|
## One-time setup (per device / world)
|
|
|
|
1. Install the FCL **fordebug** flavor (`com.tungsten.fcl.mgdebug.debug`). Its
|
|
splash auto-launches the selected profile into the prepared world after a 5 s
|
|
countdown.
|
|
2. In-game menu: enable **show FPS** (persists in `files/menu_setting.json`).
|
|
3. Prepare the benchmark world: fixed camera position, gamerules
|
|
`doMobSpawning/doDaylightCycle/doWeatherCycle=false`, then save & quit once.
|
|
`bench.sh` always `am force-stop`s the game (never saves), so every run
|
|
replays the same state.
|
|
4. `options.txt`: desired `renderDistance`, `enableVsync:false`, high `maxFps`,
|
|
`inactivityFpsLimit:"minimized"` (the "afk" default locks 30 fps after 60 s
|
|
without input and ruins the window).
|
|
5. Root required (frequency pinning, GPU busy sampling).
|
|
6. Write a device profile under `devices/` (see `devices/odinlite.env`).
|
|
|
|
A profile carries `PROFILE_VERIFIED=1` only once its sysfs nodes and OPPs have been read
|
|
off *that* device and one pinned window has been checked against them
|
|
(`big_cur`/`little_cur`/`gpu_cur_khz` in the result JSON must match the pins). Until then
|
|
it says `PROFILE_VERIFIED=0` and `bench.sh` / `session.sh` refuse to run against it unless
|
|
`--allow-unverified-profile` is passed, which labels the run unpinned in the warning.
|
|
**A profile that omits the key entirely is refused the same way** - the guard defaults to
|
|
unverified, so copying a verified profile and editing the serial cannot inherit its verdict.
|
|
Only the file can answer: both scripts reset `PROFILE_VERIFIED=0` immediately before sourcing
|
|
it, so `PROFILE_VERIFIED=1` exported in your shell does not re-open the hole. Nor does a
|
|
profile path that cannot be read get mistaken for an unverified one - it is reported as
|
|
unreadable, and a path relative to the directory you ran the script from is resolved.
|
|
(`profile.sh` pins nothing - it records a simpleperf profile - so it carries no such guard.)
|
|
|
|
That refusal exists because the pin path is silent when it is wrong: the harness writes
|
|
through `/proc/ppm/policy/hard_userlimit_*` and `/proc/gpufreq/gpufreq_opp_freq`, which are
|
|
MediaTek nodes, and `su -c 'echo ... > /proc/...'` against a device that has neither fails
|
|
without a non-zero exit. The run then reports numbers it believes were taken under a pin.
|
|
|
|
## Devices
|
|
|
|
| profile | device | verified |
|
|
|---|---|---|
|
|
| `devices/odinlite.env` | AYN Odin Lite, MT6877 / Mali-G68 | yes |
|
|
| `devices/xiaomi-adreno830.env` | Xiaomi, Snapdragon 8 Elite / Adreno 830 (`35d0befa`) | **no** - Qualcomm pin path not yet taught to `bench.sh` |
|
|
| `devices/oppo-mali.env` | Oppo / ColorOS, MediaTek + Mali (`3B159D009VZ00000`) | **no** - OPPs and thermal zone not yet read off the device |
|
|
|
|
## Usage
|
|
|
|
```
|
|
./bench.sh --device devices/odinlite.env --backend magma # 30 samples, 180 s warmup
|
|
./bench.sh --device devices/odinlite.env --backend espryt --label after-fix-X
|
|
./bench.sh --device devices/odinlite.env --backend mobileglues # reference
|
|
```
|
|
|
|
Results append to `results/results.jsonl`; per-run screenshots (`pre.png`,
|
|
`post.png`) land in `results/<timestamp>-<backend>[-label]/` — always eyeball
|
|
them: the pre/post pair must show the same scene, or the run is invalid.
|
|
|
|
## Protocol discipline (hard-won, do not skip)
|
|
|
|
- **Thermal gate**: the script waits for the profile's start-temperature
|
|
threshold. Runs started hot are not comparable to runs started cool.
|
|
- **Warmup 180 s**: ART JIT takes ~3 min to plateau (62→67→84 fps ramp was
|
|
measured); short warmups underestimate by 10-20%.
|
|
- **Pins can be overridden by the thermal engine.** The result JSON records
|
|
`big_cur/little_cur/gpu_cur_khz` sampled at window end — discard the run if
|
|
they do not match the profile pins.
|
|
- **Paired runs**: absolute FPS drifts across sessions (camera angle, world
|
|
state). A/B comparisons must be back-to-back runs in the same session.
|
|
- **F3 off** for standard numbers (the F3 debug overlay multiplies per-draw
|
|
overhead and skews backends differently).
|
|
- The FPS overlay itself must be ON (it is what produces the FCLFPS lines).
|