MobileGL/tools/device_bench/disagg/README.md

Disaggregated-mode device perf harness

A/B/C/D performance matrix and profiling harness for MobileGL's transport arms (monolith / inproc / spawn+shm / spawn+tcp), built for the adb-attached Android bench device. Written from Git Bash on a Windows host; requires MSYS_NO_PATHCONV=1 (the scripts set it themselves).

Prerequisites: the disaggregated trace APK installed (./gradlew -p android-plugin :app:assembleTraceRelease -Pmobilegl.buildDisaggregated=ON -Pmobilegl.buildDisaggregatedInproc=ON -Pmobilegl.pipePush=ON -Pmobilegl.debuggableRelease=true, then sign with the debug keystore and adb install -r), and trace fixtures under tools/trace_replay/fixtures/.

All scripts honor SER=<adb serial> (default 2f7cbe2e) and PERF_WORK=<results root> (default <repo>/.trace-work/perf-compare, git-ignored).

run_matrix.sh — perf matrix driver

bash run_matrix.sh [smoke|prep|openra|rd12|all]

Interleaved A/B/C/D rounds (monolith -> inproc -> shm -> tcp), N reps per arm (OPENRA_REPS / RD12_REPS, default 3). Interleaving is the substitute for frequency pinning on unrooted devices. Per run it force-stops the app, clears stale results, starts/stops the tcp server for the tcp arm, verifies the arm markers in logcat (Config: MOBILEGL_TRANSPORT=..., spawn ARMED, run-ahead ARMED — a run without them is invalid), and pulls result.json / benchmark.json / PipeStats dumps into $PERF_WORK/<workload>/<arm>-r<n>/.

benchmark mode never snapshots, so ssim stays -1; the matrix adds one non-benchmark ssim round per workload/arm for the golden gate.

make_report.py — aggregate into REPORT.md

python make_report.py [results_dir]

Renders raw per-rep tables, best-of-N aggregates with spread, the ssim gate, per-run arm verification, and merged client+server PipeStats rollups. Note the spawn-arm traps: client-side frames=0 is structural (use the server dump), and logcat MGPipe stats: mixes non-counter lines — trust the JSON dumps.

profile_matrix.sh + run_profile_batch.sh — profiling lane

Same runner with PROF=1: adds simpleperf (perf_event_paranoid=-1 device) for the client and server processes, ftrace sched capture for exact wakeup latencies (shell-writable on this device), and the /proc sampler below.

bash run_profile_batch.sh          # the canned batch; edit the run list inside
PROF=1 ONE=tcp WL=rd12 REP=1 bash profile_matrix.sh one   # single arm

ONE / WL / REP / EXTRA_ENV / EXTRA_SERVER_ENV select and parameterize a single run; e.g. EXTRA_SERVER_ENV=MOBILEGL_IPC_SPIN_US=2000 for the deep-idle-exit A/B.

sampler.c — device-side /proc sampler

sampler <pid> <duration_s> <interval_ms> <out.csv>. Per-tid blocked-syscall histogram (/proc/<tid>/syscall) + CPU/switch counters. Exists because strace attach is SELinux-blocked for shell uid on the bench device, while same-uid /proc reads via run-as work. Build for the device with the NDK (aarch64-linux-android*-clang), push, run under run-as.

tcp_path_check.sh — run before any cross-host measurement

Connects from WSL to a closed port on the phone. End to end, the phone refuses it; if the connect is accepted, something on the host (here v2rayN's xray_tun, WSL's default route) is terminating TCP and relaying it. That fakes sub-ms RTTs, and on 2026-09-28 it stalled the MobileGL control stream. The stall read as a server hang at eglMakeCurrent. Exit 0 = end to end, 1 = middlebox, 2 = filtered. Fix: ip route replace <phone>/32 via <LAN gateway> dev eth0 as root in WSL (temporary; delete it afterwards).

fcl_accept.sh — P12 exit gate (a)

FCL's Minecraft renders into the Render Server's on-screen window on the same phone, then fcl_accept.sh <backend> kill kills the server and checks the client latches DEVICE LOST and FCL survives. Needs the FCL fork's version setting "keepRunningInBackground" ON in config.json (FCL pauses the game the moment its Activity leaves the foreground). The server is started through the Render Server screen's documented extras. Results: the phase notes.

crosshost_accept.sh + crosshost_report.py — P12 exit gate (b)

A WSL client replays the openra and rd12 traces against the phone's render server over TCP + stream (ssim gate, then PRESENT_CREDIT 1/2/3 benchmarks). Adds a temporary /32 route around the host's TUN proxy, checks the path with tcp_path_check.sh, and removes the route and stops the server on exit. crosshost_report.py reduces the output to a table. Results and the P6.5 link numbers: the phase notes.

netbench.sh — Wi-Fi TCP throughput baseline

Understates the link: the toybox nc sink capped at 18-22 MiB/s while the render server read 52 MB/s on the same Wi-Fi. Use the server's P65ServerFrame read rate as the throughput number (enable it with MOBILEGL_SERVER_FRAME_STATS=1 or setprop debug.mobilegl.server_frame_stats 1; it is off by default).

WSL -> phone bulk transfer over toybox nc sink, 3 reps; refuses to run unless tcp_path_check.sh passes. The nc listener must keep stdin open or it exits on EOF and RSTs the connection, and pkill must be anchored ('^toybox nc') or it kills the adb shell that is about to start the listener.

phone_strace.sh — session-child strace

Attaches the device /system/bin/strace to the server's session child as the app uid via run-as (shell-uid ptrace is blocked).

P15 FCL probes: fcl_probe.sh, fcl_count.sh and their reducers

fcl_probe.sh <name> <lib> <backend> <transport> <outdir> [simpleperf=1] swaps a library into FCL (rooted device), launches it, waits for steady in-world frames (MC's Time elapsed: line, then four fps-log windows of more than 150 frames each, relaunching on the known start-up death), then records:

• 4 s of ftrace: sched, kgsl cmdbatch/waittimestamp, and atrace gfx for the app;
• schedstat and gpu_clock_stats bracketing that window;
• with the last argument at 1, 6 s of simpleperf --trace-offcpu -p <game pid>.

The caller restores FCL with fcl_p14.sh restore. fcl_count.sh does the same launch for a p15count.patch build (per-draw (VAO, program) pair and between-draw entry statistics, armed by debug.mobilegl.p15count=1 or MOBILEGL_P15_COUNT=1, window MOBILEGL_P15_COUNT_WINDOW).

Reducers, all reading a probe directory:

• snapdiff.py: per-thread CPU and runqueue wait;
• offwait.py <trace> <tid> <present-slice-regex>: sleep and runnable time per frame, keyed by the atrace slices open at switch-out and the waker, plus GPU busy;
• sfalign.py: SurfaceFlinger setTransactionState duration and phase against flushTransactions;
• slices.py: atrace slice totals per thread;
• fcl_split.py: layer split, on- vs off-CPU;
• fcl_incl.py: inclusive libMobileGL functions;
• fcl_drvcaller.py: driver time by entry and MobileGL caller;
• fcl_stacks.py: top stacks.

The simpleperf reducers need SIMPLEPERF_DIR and a binary_cache from binary_cache_builder.py. Findings: the phase notes §2.

Clocks, bench sessions and run validity (P15)

Every performance number carries the device state it was measured under.

• pin_clocks.sh pins CPU and GPU to a named profile (PROFILE=sustained|gpumax|gpulow|cpulow|cpuhunt|gpuhunt, see the script header; sustained is the default) and records it on the device.
• bench_session.sh start|stop:
  • start records, then stops, the services that move frequency caps (thermal-engine, the QTI perf HAL, vendor.perfservice, Lenovo's performance and hyperschedule_hal_service). It releases their leftover requests in /sys/kernel/msm_performance/parameters/cpu_{max,min}_freq, which otherwise clamp scaling_max_freq under any pin, and then pins.
  • stop restarts every service that was running and verifies it is running again.
  • Wrap every device job so stop always runs (trap on EXIT/INT/TERM).
• freq_sampler.sh runs on the device for each measurement window and samples every 2 s: CPU cur/max per policy, GPU cur, kgsl thermal level, busy and temperatures.
• freqcheck.py decides VALID/INVALID against the intended pin (one OPP of tolerance, GPU checked while busy). Invalid runs are discarded and re-run after cool-down.
• devstate.sh <outdir> [trace] writes the per-run state.txt line: profile and clocks, validity with the observed min/max frequencies and peak temperatures, daemons, MC version, swap interval and anland on/off. fcl_probe.sh and matrix_p14.sh (benchmark mode) call it.