[Feat] (DeviceBench): scripted in-game FPS benchmark harness (FCLFPS logcat sampling, MTK ppm frequency pinning, thermal gate, GPU busy telemetry)

This commit is contained in:
2026-08-26 01:51:04 -04:00
parent 12df061e0b
commit 386bd7e461
5 changed files with 364 additions and 0 deletions
+2
View File
@@ -0,0 +1,2 @@
results/
leveldat_*
+58
View File
@@ -0,0 +1,58 @@
# device_bench — in-game FPS benchmark harness
Scripted, repeatable in-game FPS measurement for MobileGL's two Android backends
(Espryt/DirectGLES and Magma/DirectVulkan) plus a MobileGlues reference run,
driven through the FCL fordebug flavor. Intended for A/B performance work and
release regression gates on real devices.
## How it measures
FCL's in-game FPS overlay counts `eglSwapBuffers` calls natively (renderer-
agnostic, not vsync-capped when the game runs with vsync off). When the overlay
is enabled, FCL's FPS thread logs one `FCLFPS: <n>` logcat line per second;
`bench.sh` collects those lines during the measurement window and reports
mean / median / min / max / stdev, alongside GPU busy%, SoC temperature, and
frequency-pin integrity.
## One-time setup (per device / world)
1. Install the FCL **fordebug** flavor (`com.tungsten.fcl.mgdebug.debug`). Its
splash auto-launches the selected profile into the prepared world after a 5 s
countdown.
2. In-game menu: enable **show FPS** (persists in `files/menu_setting.json`).
3. Prepare the benchmark world: fixed camera position, gamerules
`doMobSpawning/doDaylightCycle/doWeatherCycle=false`, then save & quit once.
`bench.sh` always `am force-stop`s the game (never saves), so every run
replays the same state.
4. `options.txt`: desired `renderDistance`, `enableVsync:false`, high `maxFps`,
`inactivityFpsLimit:"minimized"` (the "afk" default locks 30 fps after 60 s
without input and ruins the window).
5. Root required (frequency pinning, GPU busy sampling).
6. Write a device profile under `devices/` (see `devices/odinlite.env`).
## Usage
```
./bench.sh --device devices/odinlite.env --backend magma # 30 samples, 180 s warmup
./bench.sh --device devices/odinlite.env --backend espryt --label after-fix-X
./bench.sh --device devices/odinlite.env --backend mobileglues # reference
```
Results append to `results/results.jsonl`; per-run screenshots (`pre.png`,
`post.png`) land in `results/<timestamp>-<backend>[-label]/` — always eyeball
them: the pre/post pair must show the same scene, or the run is invalid.
## Protocol discipline (hard-won, do not skip)
- **Thermal gate**: the script waits for the profile's start-temperature
threshold. Runs started hot are not comparable to runs started cool.
- **Warmup 180 s**: ART JIT takes ~3 min to plateau (62→67→84 fps ramp was
measured); short warmups underestimate by 10-20%.
- **Pins can be overridden by the thermal engine.** The result JSON records
`big_cur/little_cur/gpu_cur_khz` sampled at window end — discard the run if
they do not match the profile pins.
- **Paired runs**: absolute FPS drifts across sessions (camera angle, world
state). A/B comparisons must be back-to-back runs in the same session.
- **F3 off** for standard numbers (the F3 debug overlay multiplies per-draw
overhead and skews backends differently).
- The FPS overlay itself must be ON (it is what produces the FCLFPS lines).
+207
View File
@@ -0,0 +1,207 @@
#!/usr/bin/env bash
# In-game FPS benchmark for MobileGL on a real device, driven through FCL.
#
# Prerequisites (one-time, manual):
# - FCL fordebug flavor installed (com.tungsten.fcl.mgdebug.debug); its splash
# screen auto-launches the selected profile/version into the prepared world.
# - FCL in-game menu "show FPS" toggle enabled (menu_setting.json showFps=true):
# the FPS overlay thread logs one "FCLFPS: <n>" logcat line per second.
# - The benchmark world saved with a deterministic state (mob spawning /
# daylight cycle / weather gamerules off) and the desired options.txt
# (renderDistance, vsync off, maxFps high).
# - Rooted device (frequency pinning + GPU utilization sampling).
#
# Usage:
# bench.sh --device devices/odinlite.env --backend magma [--samples 30]
# [--warmup 180] [--label mylabel] [--no-pin]
# backend: magma | espryt | mobileglues (reference)
#
# Output: one JSON line on stdout (also appended to results/results.jsonl) with
# mean/median/min/max FPS, GPU busy%, temperatures, and pin-integrity flags.
# Screenshots (pre/post measurement) land in results/<timestamp>-<label>/.
set -u -o pipefail
cd "$(dirname "$0")"
# Git Bash: stop MSYS from rewriting /sys/... arguments into C:/Program Files/...
export MSYS_NO_PATHCONV=1 MSYS2_ARG_CONV_EXCL='*'
PKG=com.tungsten.fcl.mgdebug.debug
ACTIVITY=$PKG/com.tungsten.fcl.activity.SplashActivity
RENDERER_ESPRYT=5e273ee2-baca-4c81-8e48-b63feefb9ba8
RENDERER_MAGMA=2be0dc10-1eef-4ce2-b512-b266dd33fd9e
RENDERER_MOBILEGLUES=com.fcl.plugin.mobileglues
DEVICE_ENV=""
BACKEND=""
SAMPLES=30
WARMUP=180
LABEL=""
DO_PIN=1
WORLD_LOAD_TIMEOUT=420
while [ $# -gt 0 ]; do
case "$1" in
--device) DEVICE_ENV=$2; shift 2 ;;
--backend) BACKEND=$2; shift 2 ;;
--samples) SAMPLES=$2; shift 2 ;;
--warmup) WARMUP=$2; shift 2 ;;
--label) LABEL=$2; shift 2 ;;
--no-pin) DO_PIN=0; shift ;;
*) echo "unknown arg: $1" >&2; exit 2 ;;
esac
done
[ -n "$DEVICE_ENV" ] && [ -n "$BACKEND" ] || { echo "need --device and --backend" >&2; exit 2; }
# shellcheck disable=SC1090
. "$DEVICE_ENV"
case "$BACKEND" in
espryt) RENDERER=$RENDERER_ESPRYT ;;
magma) RENDERER=$RENDERER_MAGMA ;;
mobileglues) RENDERER=$RENDERER_MOBILEGLUES ;;
*) echo "unknown backend: $BACKEND" >&2; exit 2 ;;
esac
ADB="adb -s $DEVICE_SERIAL"
STAMP=$(date +%Y%m%d-%H%M%S)
RUNLABEL="${STAMP}-${BACKEND}${LABEL:+-$LABEL}"
OUTDIR="results/$RUNLABEL"
mkdir -p "$OUTDIR"
log() { echo "[bench] $*" >&2; }
# Quote the whole su invocation for the DEVICE shell, or redirects run unprivileged.
sushell() { $ADB shell "su -c '$*'"; }
read_temp() {
$ADB shell "for tz in /sys/class/thermal/thermal_zone*; do
if [ \"\$(cat \$tz/type)\" = \"$THERMAL_ZONE_TYPE\" ]; then cat \$tz/temp; break; fi; done" | tr -d '\r'
}
# MTK: plain cpufreq sysfs writes are reverted by the vendor boost/PowerHAL within
# seconds — pin through ppm hard_userlimit instead (cluster indices: 0=little, 1=big).
pin_freqs() {
log "pinning CPU big=$CPU_BIG_FREQ little=$CPU_LITTLE_FREQ gpu=${GPU_PIN_KHZ}kHz (ppm)"
sushell "echo 1 $CPU_BIG_FREQ > /proc/ppm/policy/hard_userlimit_max_cpu_freq;
echo 1 $CPU_BIG_FREQ > /proc/ppm/policy/hard_userlimit_min_cpu_freq;
echo 0 $CPU_LITTLE_FREQ > /proc/ppm/policy/hard_userlimit_max_cpu_freq;
echo 0 $CPU_LITTLE_FREQ > /proc/ppm/policy/hard_userlimit_min_cpu_freq" >/dev/null
sushell "echo $GPU_PIN_KHZ > /proc/gpufreq/gpufreq_opp_freq" >/dev/null
}
unpin_freqs() {
log "unpinning frequencies (restore DVFS)"
sushell "echo 1 -1 > /proc/ppm/policy/hard_userlimit_max_cpu_freq;
echo 1 -1 > /proc/ppm/policy/hard_userlimit_min_cpu_freq;
echo 0 -1 > /proc/ppm/policy/hard_userlimit_max_cpu_freq;
echo 0 -1 > /proc/ppm/policy/hard_userlimit_min_cpu_freq" >/dev/null
sushell "echo 0 > /proc/gpufreq/gpufreq_opp_freq" >/dev/null
}
cleanup() {
$ADB shell am force-stop $PKG >/dev/null 2>&1
[ "$DO_PIN" = 1 ] && unpin_freqs
$ADB shell svc power stayon false >/dev/null 2>&1
}
trap cleanup EXIT
# --- 1. Wake, unlock, keep screen on, fan to sport ---------------------------
$ADB shell input keyevent KEYCODE_WAKEUP >/dev/null
$ADB shell input keyevent 82 >/dev/null
$ADB shell svc power stayon true >/dev/null
$ADB shell settings put global fan_mode 3 2>/dev/null
wakefulness=$($ADB shell dumpsys power | grep -o 'mWakefulness=[A-Za-z]*' | head -1)
log "screen: $wakefulness"
# --- 2. Thermal gate ----------------------------------------------------------
log "thermal gate: waiting for $THERMAL_ZONE_TYPE <= $THERMAL_START_MAX_MC"
for i in $(seq 1 60); do
T=$(read_temp)
[ "$T" -le "$THERMAL_START_MAX_MC" ] && break
log " temp=$T, cooling... ($i)"
sleep 10
done
TEMP_START=$(read_temp)
log "start temp: $TEMP_START"
# --- 3. Select renderer (device-side sed, proven under run-as) ---------------
for known in $RENDERER_ESPRYT $RENDERER_MAGMA $RENDERER_MOBILEGLUES; do
[ "$known" = "$RENDERER" ] && continue
$ADB shell run-as $PKG sed -i "s/$known/$RENDERER/g" files/config.json
done
log "renderer now: $($ADB shell run-as $PKG grep renderer files/config.json | tr -d '\r' | tr -s ' ' | sort -u | tr '\n' ' ')"
# --- 4. Pin frequencies -------------------------------------------------------
[ "$DO_PIN" = 1 ] && pin_freqs
# --- 5. Launch, wait for world ------------------------------------------------
$ADB shell am force-stop $PKG
MCLOG="/storage/emulated/0/FCL/.minecraft/versions/*/logs/latest.log"
$ADB shell "rm -f $MCLOG /sdcard/MG/latest.log" 2>/dev/null
$ADB logcat -c 2>/dev/null
log "launching $ACTIVITY"
$ADB shell am start -n $ACTIVITY >/dev/null
WORLD_UP=0
for i in $(seq 1 $((WORLD_LOAD_TIMEOUT / 5))); do
sleep 5
if ! $ADB shell pidof $PKG >/dev/null; then
# process may legitimately restart once (launcher -> game process)
:
fi
if $ADB shell "grep -l -e 'logged in with entity id' -e 'Preparing spawn area: 100' $MCLOG" >/dev/null 2>&1; then
WORLD_UP=1; break
fi
done
if [ "$WORLD_UP" != 1 ]; then
log "world did not load within ${WORLD_LOAD_TIMEOUT}s"
$ADB exec-out screencap -p > "$OUTDIR/failed-load.png" 2>/dev/null
echo "{\"label\":\"$RUNLABEL\",\"error\":\"world-load-timeout\"}"
exit 1
fi
log "world is up; warmup ${WARMUP}s"
sleep "$WARMUP"
# --- 6. Measure ---------------------------------------------------------------
$ADB exec-out screencap -p > "$OUTDIR/pre.png" 2>/dev/null
TEMP_MID=$(read_temp)
GPU_BUSY_SAMPLES=""
FPS_FILE="$OUTDIR/fps.txt"
: > "$FPS_FILE"
log "sampling $SAMPLES FPS values (1/s) + GPU busy"
$ADB logcat -v raw -s FCLFPS:I > "$OUTDIR/fclfps.log" &
LOGCAT_PID=$!
for i in $(seq 1 "$SAMPLES"); do
sleep 1
B=$(sushell "cat $GPU_UTIL_NODE" | tr -d '\r' | awk '{print $1}')
GPU_BUSY_SAMPLES="$GPU_BUSY_SAMPLES $B"
done
kill $LOGCAT_PID 2>/dev/null
wait $LOGCAT_PID 2>/dev/null
grep -E '^[0-9]+$' "$OUTDIR/fclfps.log" | tail -n "$SAMPLES" > "$FPS_FILE"
$ADB exec-out screencap -p > "$OUTDIR/post.png" 2>/dev/null
TEMP_END=$(read_temp)
# Pin integrity: sample live freqs right at the end of the window (game still hot).
BIG_CUR=$($ADB shell cat /sys/devices/system/cpu/cpufreq/$CPU_BIG_POLICY/scaling_cur_freq | tr -d '\r')
LITTLE_CUR=$($ADB shell cat /sys/devices/system/cpu/cpufreq/$CPU_LITTLE_POLICY/scaling_cur_freq | tr -d '\r')
GPU_CUR=$(sushell "cat $GPU_CURFREQ_NODE" | tr -d '\r' | awk '{print $NF}')
# --- 7. Stats -----------------------------------------------------------------
STATS=$(sort -n "$FPS_FILE" | awk '
{ v[NR]=$1; s+=$1 }
END {
if (NR==0) { print "0 0 0 0 0 0"; exit }
mean=s/NR; med=v[int((NR+1)/2)];
for(i=1;i<=NR;i++) ss+=(v[i]-mean)^2;
sd=(NR>1)?sqrt(ss/(NR-1)):0;
printf "%d %.1f %d %d %d %.1f", NR, mean, med, v[1], v[NR], sd
}')
set -- $STATS
N=$1 MEAN=$2 MED=$3 MIN=$4 MAX=$5 SD=$6
GPU_BUSY_MEAN=$(echo "$GPU_BUSY_SAMPLES" | tr ' ' '\n' | grep -E '^[0-9]+$' | awk '{s+=$1;n++} END{if(n) printf "%.0f", s/n; else print 0}')
RESULT=$(printf '{"label":"%s","backend":"%s","samples":%s,"fps_mean":%s,"fps_median":%s,"fps_min":%s,"fps_max":%s,"fps_sd":%s,"gpu_busy_mean":%s,"temp_start_mc":%s,"temp_mid_mc":%s,"temp_end_mc":%s,"big_cur":%s,"little_cur":%s,"gpu_cur_khz":%s,"pinned":%s,"warmup_s":%s}' \
"$RUNLABEL" "$BACKEND" "$N" "$MEAN" "$MED" "$MIN" "$MAX" "$SD" "$GPU_BUSY_MEAN" \
"$TEMP_START" "$TEMP_MID" "$TEMP_END" "$BIG_CUR" "$LITTLE_CUR" "${GPU_CUR:-0}" "$DO_PIN" "$WARMUP")
mkdir -p results
echo "$RESULT" >> results/results.jsonl
echo "$RESULT"
+29
View File
@@ -0,0 +1,29 @@
# Device profile: AYN Odin Lite (MT6877 Dimensity 900, Mali-G68 MC4, Android 11)
# 2x A78 (policy6, max 2400000) + 6x A55 (policy0, max 2000000), GPU top OPP 902MHz.
# Panel: 1080x1920 @ 60Hz (presented FPS caps at 60 - render-side FPS comes from the
# FCLFPS logcat tag, which counts eglSwapBuffers; it is NOT vsync-capped when the
# game runs with vsync off).
DEVICE_SERIAL=MTK0002207301023500
# Frequency pins, enforced via /proc/ppm/policy/hard_userlimit_* (plain cpufreq
# sysfs writes are reverted by the vendor game boost within seconds). The stock
# performance_mode=2 boost pins big=2400000/little=2000000 anyway, so we pin at
# the same values; bench.sh records live freqs at window end to catch thermal
# clamping. Fan must be in sport mode (settings put global fan_mode 3) or the
# SoC runs away past 80C.
CPU_BIG_POLICY=policy6
CPU_BIG_FREQ=2400000
CPU_LITTLE_POLICY=policy0
CPU_LITTLE_FREQ=2000000
# GPU pin via legacy MTK gpufreq: echo <khz> > /proc/gpufreq/gpufreq_opp_freq
# (0 restores DVFS). Top OPP on this device is 902000.
GPU_PIN_KHZ=902000
# Thermal gate: wait until this zone is at or below the threshold before starting.
THERMAL_ZONE_TYPE=mtktscpu
THERMAL_START_MAX_MC=50000
# MTK ged GPU utilization node; first field of the triple is busy %.
GPU_UTIL_NODE=/sys/kernel/ged/hal/gpu_utilization
GPU_CURFREQ_NODE=/sys/kernel/ged/hal/current_freqency
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# CPU-profile the running game with simpleperf (DWARF call graphs) and produce
# a symbolized report on the host. Run while the game is in-world (e.g. during
# a bench.sh warmup, or standalone after launching the game manually).
#
# Usage:
# profile.sh --device devices/odinlite.env [--duration 30] [--label hot1]
# [--freq 800]
#
# Requires: debuggable app (fordebug flavor), host NDK simpleperf, and the
# unstripped libMobileGL.so from the same build as the installed APK
# (MobileGL/build/intermediates/merged_native_libs/fordebug/mergeFordebugNativeLibs/out/lib/arm64-v8a).
#
# Notes carried over from earlier campaigns:
# - DWARF unwinding (-g): frame-pointer call graphs are broken on these builds.
# - The MC render thread is a JVM thread with a generic name (Thread-NN, varies
# per run) — find it in the report with --sort comm first.
# - Use Git Bash, not PowerShell (binary pull corruption via > redirect).
set -u -o pipefail
cd "$(dirname "$0")"
PKG=com.tungsten.fcl.mgdebug.debug
DEVICE_ENV=""
DURATION=30
FREQ=800
LABEL=prof
while [ $# -gt 0 ]; do
case "$1" in
--device) DEVICE_ENV=$2; shift 2 ;;
--duration) DURATION=$2; shift 2 ;;
--freq) FREQ=$2; shift 2 ;;
--label) LABEL=$2; shift 2 ;;
*) echo "unknown arg: $1" >&2; exit 2 ;;
esac
done
[ -n "$DEVICE_ENV" ] || { echo "need --device" >&2; exit 2; }
# shellcheck disable=SC1090
. "$DEVICE_ENV"
ADB="adb -s $DEVICE_SERIAL"
STAMP=$(date +%Y%m%d-%H%M%S)
OUTDIR="results/${STAMP}-${LABEL}"
mkdir -p "$OUTDIR"
$ADB shell pidof $PKG >/dev/null || { echo "game not running" >&2; exit 1; }
echo "[profile] recording ${DURATION}s @ ${FREQ}Hz (DWARF)..." >&2
$ADB shell simpleperf record --app $PKG -e cpu-clock -f "$FREQ" -g \
--duration "$DURATION" -o /data/local/tmp/mgprof.data || exit 1
(cd "$OUTDIR" && MSYS_NO_PATHCONV=1 adb -s "$DEVICE_SERIAL" pull /data/local/tmp/mgprof.data perf.data >/dev/null)
echo "[profile] pulled to $OUTDIR/perf.data" >&2
# Host-side symbolization if the NDK simpleperf scripts are available.
SIMPLEPERF_DIR="${SIMPLEPERF_DIR:-$LOCALAPPDATA/Android/Sdk/ndk/28.2.13676358/simpleperf}"
SYMDIR="../../build/intermediates/merged_native_libs/fordebug/mergeFordebugNativeLibs/out/lib/arm64-v8a"
if [ -d "$SIMPLEPERF_DIR" ] && [ -d "$SYMDIR" ]; then
echo "[profile] building binary cache (symbolized)..." >&2
(cd "$OUTDIR" && python "$SIMPLEPERF_DIR/binary_cache_builder.py" -i perf.data -lib "../../$SYMDIR" >/dev/null 2>&1)
echo "[profile] per-thread summary:" >&2
"$SIMPLEPERF_DIR/bin/windows/x86_64/simpleperf.exe" report -i "$OUTDIR/perf.data" \
--symfs "$OUTDIR/binary_cache" --sort comm -n 2>/dev/null | head -25
echo "[profile] done; drill down with:" >&2
echo " $SIMPLEPERF_DIR/bin/windows/x86_64/simpleperf.exe report -i $OUTDIR/perf.data --symfs $OUTDIR/binary_cache --comms <renderthread> --sort symbol -n | head -40" >&2
else
echo "[profile] NDK simpleperf or symbol dir missing; raw perf.data kept at $OUTDIR" >&2
fi