From 710beefd02487c71a5cc358ca4c29a30134b4e79 Mon Sep 17 00:00:00 2001 From: Swung0x48 Date: Sun, 4 Oct 2026 06:44:45 -0400 Subject: [PATCH] [Docs] (Performance): glmark2 optimization handoff with the landed fixes, current numbers, remaining work and the bench scripts --- docs/Performance/glmark2-baseline.md | 2 +- docs/Performance/glmark2-handoff.md | 176 ++++++++++++++++++ tools/device_bench/anland_glmark2/ab.sh | 6 + tools/device_bench/anland_glmark2/callers.py | 30 +++ tools/device_bench/anland_glmark2/dep.sh | 11 ++ tools/device_bench/anland_glmark2/gprof.sh | 12 ++ tools/device_bench/anland_glmark2/incl.py | 46 +++++ tools/device_bench/anland_glmark2/lm.sh | 4 + tools/device_bench/anland_glmark2/mktables.py | 66 +++++++ tools/device_bench/anland_glmark2/offcpu.py | 71 +++++++ tools/device_bench/anland_glmark2/pbbatch.sh | 28 +++ tools/device_bench/anland_glmark2/pbclk.sh | 26 +++ tools/device_bench/anland_glmark2/pbcool.sh | 5 + tools/device_bench/anland_glmark2/pbkill.sh | 4 + tools/device_bench/anland_glmark2/pbparse.py | 33 ++++ tools/device_bench/anland_glmark2/pbprobe.sh | 10 + tools/device_bench/anland_glmark2/pbprof.sh | 20 ++ tools/device_bench/anland_glmark2/pbrun.sh | 13 ++ tools/device_bench/anland_glmark2/pbsamp.sh | 8 + tools/device_bench/anland_glmark2/pbset.sh | 5 + tools/device_bench/anland_glmark2/pbshown.sh | 4 + tools/device_bench/anland_glmark2/profrun.sh | 12 ++ tools/device_bench/anland_glmark2/smoke.sh | 14 ++ tools/device_bench/anland_glmark2/stfix.sh | 10 + tools/device_bench/anland_glmark2/symagg.py | 27 +++ tools/device_bench/anland_glmark2/timeline.py | 27 +++ tools/device_bench/anland_glmark2/visset.sh | 9 + tools/device_bench/anland_glmark2/wtest.sh | 17 ++ 28 files changed, 695 insertions(+), 1 deletion(-) create mode 100644 docs/Performance/glmark2-handoff.md create mode 100644 tools/device_bench/anland_glmark2/ab.sh create mode 100644 tools/device_bench/anland_glmark2/callers.py create mode 100644 tools/device_bench/anland_glmark2/dep.sh create mode 100644 tools/device_bench/anland_glmark2/gprof.sh create mode 100644 tools/device_bench/anland_glmark2/incl.py create mode 100644 tools/device_bench/anland_glmark2/lm.sh create mode 100644 tools/device_bench/anland_glmark2/mktables.py create mode 100644 tools/device_bench/anland_glmark2/offcpu.py create mode 100644 tools/device_bench/anland_glmark2/pbbatch.sh create mode 100644 tools/device_bench/anland_glmark2/pbclk.sh create mode 100644 tools/device_bench/anland_glmark2/pbcool.sh create mode 100644 tools/device_bench/anland_glmark2/pbkill.sh create mode 100644 tools/device_bench/anland_glmark2/pbparse.py create mode 100644 tools/device_bench/anland_glmark2/pbprobe.sh create mode 100644 tools/device_bench/anland_glmark2/pbprof.sh create mode 100644 tools/device_bench/anland_glmark2/pbrun.sh create mode 100644 tools/device_bench/anland_glmark2/pbsamp.sh create mode 100644 tools/device_bench/anland_glmark2/pbset.sh create mode 100644 tools/device_bench/anland_glmark2/pbshown.sh create mode 100644 tools/device_bench/anland_glmark2/profrun.sh create mode 100644 tools/device_bench/anland_glmark2/smoke.sh create mode 100644 tools/device_bench/anland_glmark2/stfix.sh create mode 100644 tools/device_bench/anland_glmark2/symagg.py create mode 100644 tools/device_bench/anland_glmark2/timeline.py create mode 100644 tools/device_bench/anland_glmark2/visset.sh create mode 100644 tools/device_bench/anland_glmark2/wtest.sh diff --git a/docs/Performance/glmark2-baseline.md b/docs/Performance/glmark2-baseline.md index 62e174f7..dd8aaf1c 100644 --- a/docs/Performance/glmark2-baseline.md +++ b/docs/Performance/glmark2-baseline.md @@ -1,6 +1,6 @@ # glmark2 baseline: MobileGL (Espryt, Magma) vs the stock kgsl driver stack on anland -Measured 2026-10-04 on the anland Plasma desktop. This is the phase-1 baseline for the +Measured 2026-10-04 on the anland Plasma desktop. **What was optimized since, the current numbers and the remaining work: [glmark2-handoff.md](glmark2-handoff.md).** This is the phase-1 baseline for the performance work: approximate numbers, how they were taken, where MobileGL loses, and a ranked list of general fixes. The goal was "where does MobileGL fall short", not lab-grade accuracy, so the scores are **approximate** (1-3 valid runs per configuration, see n in the tables); the profiling diff --git a/docs/Performance/glmark2-handoff.md b/docs/Performance/glmark2-handoff.md new file mode 100644 index 00000000..f8c68098 --- /dev/null +++ b/docs/Performance/glmark2-handoff.md @@ -0,0 +1,176 @@ +# glmark2 performance work: handoff + +Status on 2026-10-04 after the first optimization pass on top of the +[baseline](glmark2-baseline.md). Everything here was measured on the anland Plasma desktop of the +Lenovo TB321FU (Adreno 750) at **pinned clocks** (CPU 1.80 / 2.25 / 2.25 / 2.25 GHz, GPU 680 MHz; +see the baseline). Single runs or two at most per point, so the numbers are **approximate** +(light scenes vary about +-15% run to run). The tablet was in portrait for the later runs, so the +"after" fullscreen numbers are 1600x2560 and the "before" ones 2560x1600 (same pixel count). + +## Headline + +glmark2 score, `-b :duration=5`, default scene list: + +| | stock kgsl stack | Espryt before | Espryt now | Magma before | Magma now | +|---|---|---|---|---|---| +| es2, 800x600 window | ~1076-1180 | ~1319 | **~1887** | ~778 | **~1213** | +| es2, fullscreen | ~1177 | ~572 | ~561 | ~354 | ~404 | +| desktop GL, 800x600 window | ~1161 | ~1283 | **~1809** | ~743 | **~1186** | + +The "now" es2 window scores include the two `buffer update-method=map` scenes, which MobileGL now +runs (they were Unsupported before), so they are a fair comparison with stock's. + +Per scene (es2 window unless marked F = fullscreen; FPS): + +| scene | stock | Espryt before | Espryt now | Magma before | Magma now | +|---|---|---|---|---|---| +| texture | 1269 | 1672 | ~2210 | 915 | ~1580 | +| build use-vbo=false | 1095 | 467 | ~1215 | 472 | ~1040-1190 | +| build use-vbo=true | 1287 | 1675 | ~2450 | 1062 | ~1730 | +| desktop blur | 976 | 275 | **~1000** | 157 | ~680 | +| desktop shadow | 1044 | 234 | **~1000** | 125 | ~830 | +| ideas | 688 | 138 | ~550 | 66 | ~354 | +| terrain | 443 | 149 | ~220 | 115 | ~157 | +| shadow | 975 | 1139 | ~1840 | 644 | ~1170 | +| refract | 838 | 981 | ~1280 | 331 | ~520 | +| jellyfish | 1185 | 1438 | ~2170 | 841 | ~1155 | +| buffer map (2 scenes) | 330 / 408 | Unsupported | ~465 | Unsupported | ~431 | +| texture F | ~1430 | ~726 | ~597 | ~445 | ~398 | +| terrain F | - | - | ~142 | - | ~112 | + +**Remaining gap:** Espryt is at or above stock in every windowed scene except **ideas (0.8x)** and +**terrain (0.5x)**. Magma is behind on most draw-heavy scenes (ideas 0.5x, terrain 0.35x, desktop +0.7-0.8x, refract 0.6x). **Fullscreen** is the biggest gap on both backends (0.4-0.5x of stock), +and nothing landed here addresses it. + +## Done (all on origin/feat/disaggregated) + +| commit | change | why / evidence | effect (approx) | +|---|---|---|---| +| `4722fa38` | the baseline report | - | - | +| `66b47a10` | **glFinish waits for the GPU** on the remote path (a server fence created, waited with `GL_SYNC_FLUSH_COMMANDS_BIT`, destroyed); **Magma submits** once 1024 retired per-clear objects pile up with nothing submitted; **ES contexts advertise** `GL_OES_rgb8_rgba8`, `GL_OES_depth24`, `GL_OES_mapbuffer` (with `glMapBufferOES` / `glUnmapBufferOES` / `glGetBufferPointervOES` exported as the core calls) | glFinish only waited for the server's apply, so off-screen rendering ran unbounded (glmark2 `--off-screen` "6846" / "12117" fps) and drove Magma's server to `VK_ERROR_OUT_OF_HOST_MEMORY` in `ClearWireFramebuffer`; es2 `--off-screen` could not start (glmark2 picked `GL_RGBA4` and no depth) | off-screen now real numbers (Espryt texture ~866, Magma ~800) and no crash; es2 `--off-screen` runs; the two buffer-map scenes run | +| `2de0d6dd` | **client-array buffers are reused** from a per-share-group pool (respecify, a non-blocking record, instead of a blocking ResourceCreate + destroy per array per draw); contiguous element ranges copied in one memcpy; the fetch plan skips sorting an already ascending range | desktop blur/shadow paid 28/52 blocking creates per frame, build use-vbo=false 2; `std::sort` was 27% of the client in build use-vbo=false | build use-vbo=false Espryt 467 -> ~1100, Magma 472 -> ~1040; blocking replies per frame in desktop 29/53 -> 1 (fps of desktop barely moved at this step: the next item was the limit) | +| `d627fca3` | **targeted wake-ups:** the client publishes the applied sequence it waits for (`RingControl::producerWakeSeq`, its own cache line); the server rings a parked client only once that sequence is applied (or the event ring filled); the parked flag is stored with release so the notifier sees the target | while the client waited for its present reply the server rang it after every record it applied: 18% of Espryt's apply thread was `SocketDoorbell::Notify`, and every ring was a futile wake-up and re-park on the client | Espryt desktop ~250 -> ~800, ideas 138 -> ~380, texture 1672 -> ~2190; Magma desktop ~150 -> ~550-650, ideas 45 -> ~137, refract 372 -> ~540 | +| `963f4d81` | Magma **reuses completed command buffers** (pool resets them at the next begin) instead of `vkFreeCommandBuffers` + `vkAllocateCommandBuffers` per submission | `FreeRetiredCommandBuffersCompletedUpTo` was 15% of Magma's apply thread in ideas | small: Magma texture / desktop +~10% | +| `915882b2` | the server **times records only when** `MOBILEGL_SERVER_FRAME_STATS` is on | two `steady_clock::now()` per record were ~5% of Espryt's apply thread | with the next row | +| `e5ee31b0` | the anland server build uses **`android-30`** (the APK's minSdk) in `build-android.sh` / `reproduce.md`: `thread_local` is native ELF TLS instead of emulated TLS | `__emutls_get_address` + `pthread_getspecific` were 7-12% of Espryt's apply thread. **An existing build dir keeps its API level: reconfigure (delete it) to get android-30** | Espryt (with the row above) desktop ~800 -> ~1000, ideas ~380 -> ~550, terrain 168 -> ~220; Magma unchanged | +| `bb31f374` | Magma **keeps the wire draw's render pass open** across draws to the same attachments. Every command that may not sit inside a pass (barriers, copies, clears, blits, dispatches, other passes' begins) in the resource managers and wire helpers first calls `EndActiveRenderPassOn(cb)` (`RenderPassGuard.h`); the per-draw "make image writes visible" barrier for sampled images is only recorded when an image may have been written since the last one (`WireImageWriteEpoch`, bumped by every pass end, out-of-pass work and storage-image draw); same-layout attachment barriers are skipped when the pass continues | `SetupWireDraw` ended and began a VkRenderPass for every draw (ideas: 305 passes per frame; End/Begin ~21% of the apply thread, plus `vkEndCommandBuffer` cost), and on a tiler each pass loads and stores its attachments | Magma ideas ~146 -> ~354, desktop shadow ~640 -> ~830; visually checked on ideas, terrain, shadow, refract, jellyfish, desktop, bump, conditionals, effect2d, Plasma, Chrome and the lock screen | + +Each landed with the host unit tests green (WSL clang, `ctest -L unit -E DirectVulkan`, 2851 +tests) and a device check on both backends (frames shown, Plasma, Chrome page + scroll input, +`kscreenlocker_greet --testing`). + +## Findings + +- **The common shape before the fixes: nothing was saturated.** GPU 7-40% busy, client and server + threads each ~30-50% busy, both waiting for each other with a wake-up per handoff. Removing + futile wake-ups (`d627fca3`) was worth more than any per-draw CPU saving. +- **Espryt after the fixes:** the remaining windowed gaps are ideas and terrain. Its present reply + is still a synchronous round trip per frame, and the server's per-draw state sync + (`DG::PrepareForDraw`, 28% of the apply thread in ideas before) is now the larger share. +- **Magma after the fixes:** CPU-bound on the server. Remaining per-draw costs in + `SetupWireDraw` (descriptor sets, `GetOrCreateProgram` / pipeline hash, `BuildWireVertexInput`, + vectors allocated per draw, scudo lock contention ~20% before), and per frame + `BlitDefaultFramebufferToSharedImage` (~30% of the apply thread: frame flush + a second submit + for the copy + `vkEndCommandBuffer`) plus `Present()` of the client's offscreen default surface + (a swapchain acquire + submit that nobody looks at) and GPU progress markers per submit. +- **Fullscreen:** every frame is rendered into the server's default framebuffer and then copied + into the window's shared image (Espryt `glBlitFramebuffer`, Magma `vkCmdCopyImage`); at 4 MP the + GPU is ~98% busy on texture at ~600 fps while stock does ~1430 and is even faster fullscreen than + windowed (it presumably skips composition). + +**Dead end: an adaptive client spin.** Growing the doorbell spin up to 1 ms towards the measured +reply latency (and shrinking it on long waits) gave +30-80% on light scenes (texture, jellyfish, +shadow) on Magma but cost refract -35% (370 -> 236) and fullscreen terrain -35% (90 -> 58), +reproducibly; the present reply simply got slower while the client spun. Polling the shared line +less often (backoff up to 64 iterations) did not change that, so it is not cache-line ping-pong. A +static `MOBILEGL_IPC_SPIN_US=1000` on the client showed the same mixed picture on Espryt. Not +landed; the targeted wake-up removed most of what the spin bought without the regressions. + +**Traps:** +- **Pin the clocks.** Unpinned, the stock stack's light scenes ran at GPU 310 MHz / big cores 672 + MHz and scored 425 instead of ~1100. +- **glFinish and `--off-screen`.** Before `66b47a10` off-screen MobileGL numbers were meaningless. +- **The stock desktop turns black mid-run** (a pure-black 21707-byte screenshot) while + SurfaceFlinger keeps latching ~125 buffers/s, and its score then jumps to ~1420-1500. Probe the + screen every ~15 s during stock runs and discard runs with a black probe; restart the stock + container + app (`stfix.sh`) to get the picture back. A powerdevil inhibit did not prevent it. +- **The MobileGL app's extra-keys bar** shrinks its desktop to ~2560x1410; set + `extra_keys_mode=with_keyboard` for fullscreen comparisons (it was restored afterwards). +- **The MobileGL session restores a System Settings window** at login; close it before measuring. +- **Rotation:** the stock app follows auto-rotate; check glmark2's "Surface Size" line. +- **Symbolizing:** the APK's server lib is stripped; a RelWithDebInfo build of the same tree was at + +0x80 for every function (check with `llvm-nm -D` on both). Container clients: simpleperf prints + offsets from the start of the executable mapping (add the r-x segment's page-aligned vaddr, + 0x4fb000 for the lib of 2026-10-04). Off-CPU (`sched_switch`) samples of container processes + get their user frames attributed to Android's libs, so use the client's own `P65LinkMetrics` for + its waits. +- `pkill -f