swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-11-07 18:13:23 +09:00
Code Issues Packages Projects Releases Wiki Activity
Files
6cecfcd22f362ada4129ab834f4eb5d963e09560
MobileGL/docs/Disaggregated/notes/perf-p15/HANDOFF.md
T

17 KiB
Raw Blame History

P15 handoff (living document)

The plan and its measurements are in PLAN-P15.md; the audit is in PARITY-AUDIT.md.

Target (PLAN §0.1, revised by the user 2026-10-08):

  • CPU scenes: FCL presenting-thread CPU at or below dev monolith (Magma <= 1.79, Espryt <= 2.14 ms/frame), dev measured in the same session, ~5 % noise.
  • GPU scene (Iris + BSL): fps within ~5 % of MobileGlues on both backends.

Latest (2026-10-08, after S0)

Every number below carries its profile and passed the validity check (state.txt). cpuhunt = CPU about 1.5/1.25 GHz, GPU 903 MHz (cleanly CPU-bound); interval 0, anland off, daemons stopped.

  • The S0 milestone's absolute numbers are superseded. The perf stack's msm_performance caps (787 / 1286 / 1075 / 1248 MHz) were clamping the CPU under the pin for some of that session. Only same-session deltas from it stand. Measurement now uses bench_session.sh and the sampler (PLAN §1), and the probe reports only clean fps windows taken before any tracing (P15PROBE trace-start marker, 25 s settle).

  • FCL vanilla 1.21.5, cpuhunt, presenting-thread CPU ms/frame:

    build Espryt Magma
    dev 1.76 1.60
    p15k (S0 + VAO shadow) 2.10 2.03
    p15l (+ vertex-elements dedup) 2.10 2.08

    MobileGlues: 1.58 in an earlier cpuhunt session (needs a same-session re-run with clean windows). Floor gap: Espryt +0.34, Magma +0.43 ms.

  • Driver-call redundancy:

    • Measured with a diagnostic build that counts every host GLES/EGL call and redundant re-sets (p15diag worktree, never committed; HostCount.inl and VkCount.cpp).
    • Espryt sent 3717 host calls/frame against MC's ~874: 2118 glDisableVertexAttribArray, 378 enable, 378 divisor, all re-sets.
    • Two fixes, each independently removes the walk churn:
      • b4e209e1: a shadow of each driver VAO (850 calls/frame);
      • c8a5abe5: the client no longer re-creates an identical vertex-elements record on every glBindVertexBuffer. With the shadow off it takes 3724 down to 1228.
    • CPU gain is small (about 0.1 ms): the calls were cheap in the driver.
    • Magma issues about 380 Vulkan calls/frame with essentially none redundant.

  • Where Espryt's gap to dev is now: the driver time equals dev's (0.50 vs 0.48). The excess is libMobileGL itself, 0.75 vs 0.32: client half 0.51, backend 0.24, applier 0.02. The client half is spread thin (validate, tracker, emitters, residual copy): S1's draw-delta.

  • Inproc (cpuhunt):

    • Espryt reaches 306 fps vs MobileGlues about 319; GL thread 1.68 ms, apply thread 1.83 ms of a 3.27 ms frame.
    • Neither thread is saturated. The apply thread waits in queueBuffer on a GPU fence (present pacing), and the GL thread waits on the apply thread.
    • MOBILEGL_IPC_PRESENT_CREDIT=2 changes nothing (300 fps).
    • Beating MobileGlues on inproc needs a present-path change (stage P territory, dropped by the user), not CPU work.

  • S0b: the fixture minecraft-1.21.5-vanilla-fcl-in-world is landed (4e27342e) and is in the SSIM gate as fcl (32/32 on p15j).

  • Knobs on FCL: adb shell setprop debug.mobilegl.env "MOBILEGL_A=1;MOBILEGL_B=0" (49c6394d); fclab.sh takes a 6th field K=V,K=V.

  • MC's AFK frame limit: inactivityFpsLimit:"afk" (MC 1.21.2+) made later probe windows and profiles spin in System.nanoTime. fcl_scene.sh now sets "minimized" for bench scenes and restores the original options files (/data/local/tmp/p15-opts-121{5,4}.txt). Profiles taken before this are partly AFK-polluted.

  • BSL at gpuhunt (CPU 2380/1689 MHz, GPU 578 MHz, all 100 % GPU busy):

    • MobileGlues 53.4 fps;
    • dev Magma 53.2;
    • p15l Magma 51.8 (3 reps, -3.0 % vs MobileGlues: target met, -2.6 % vs dev to bisect);
    • Espryt dev and p15l 48.3 (-9.6 %): the GPU gap, inherited from dev.

  • Same-session cpuhunt, apply thread:

    build fps apply ms/frame
    MobileGlues 329 CPU 1.55
    Magma inproc p15m 300 1.6
    Espryt inproc 296 1.65-1.8
    • Inproc beats MobileGlues only when the apply thread's CPU is below MobileGlues' (both pay the same queueBuffer wait). About 0.3 ms per frame beyond the apply CPU is still unexplained.
    • 500b63ea: per-store reuse of renamed-away wire buffers (Magma apply -0.13 ms).
    • ONE_TIME_SUBMIT: no effect (A/B'd, knob dropped).
    • Magma's driver work is about 2x dev's: vkResetCommandBuffer 0.09 + vkEndCommandBuffer 0.11 ms/frame.

  • Inproc serialization found (2026-10-08, p15l cpuhunt profiles): every blit reached the client as BlitNamedFramebuffer, a verb with no MGP_VERB_OP_LIST row, so ClientVerbIsBarriered answered barriered and the GL thread quiesced the applier once per frame (1.65 ms/frame parked in WaitForApplyToCatchUp, plus 0.11 from the following glBindTexture). The apply thread then idled 0.24 ms parked plus 0.2 ms spinning in Doorbell::Wait. e00db8e6 replaces the inverse and its barriered fallback with a verb -> record-op table (MG_Pipe/VerbRecordOps.h, one row per verb by static_assert; no knob, user decision).

    • Same-session blab (cpuhunt): Magma inproc 319.6 -> 334.9 fps (+4.8 %), Espryt inproc 324.4 -> 338.1 (+4.2 %).
    • Wait census (MOBILEGL_IPC_WAIT_STATS=1, 7acf3ea8): FCL vanilla inproc waits only on the present credit (1.00/frame). BSL waits 7 times per frame on three kWaitApplied rows plus 22 follow-on quiesces; see WAIT-AUDIT.md. User decision applied in 2fbcc224: fire-and-forget rows async, inproc creates and PBO readbacks no longer wait, and the dual block (MOBILEGL_IPC_ROLE_SPLIT_STATE, now default on) lets a barriered verb cost one wait. BSL census after it still owed (gpuhunt only; FCL's 1.21.4-Fabric profile exited with signal 34 at JVM start on both backends in the last attempt).
    • db915f76: a client parked on the present credit is rung only by the credit (apply-thread futex wakes 0.056 ms/frame before).
    • Frames in flight (0b96a3ef): MOBILEGL_FRAMES_IN_FLIGHT (default 3) drives Magma's frame contexts, Espryt's post-present fence wait and the split present credit max(1, N-2); the Magma name is an alias. Default latency unchanged; the FIF A/B is the user's input for the default.
    • Present credit 2 (cens, cpuhunt, all VALID): Magma 309.0 -> 319.9, Espryt 299.1 -> 319.1 fps vs MobileGlues 321.9 in the same session; apply CPU 1.40-1.43 vs MobileGlues' GL thread 1.57. Credit 2 costs a frame of latency: user decision. The remaining inproc gap is the queueBuffer fence wait (GPU-side), not CPU.
    • BSL at cpuhunt (GPU 903 MHz) reaches GPU 95 C and kgsl thermal_pwrlevel 1 (834 MHz cap): the census run was INVALID twice and the job was stopped. Run BSL only at gpuhunt.
    • Apply-thread frame (Magma inproc): on-CPU 1.75 (0.2 of it spin), queueBuffer fence wait 1.45, parked 0.24 ms.
    • Magma apply on-CPU by verb: draws 0.55 (SetupWireDraw 0.40, of it descriptor binding 0.13), present 0.37 (End CB 0.10, queueBuffer CPU 0.08, submit 0.07, wire-store sweep 0.035), clear 0.19 (the frame's recording begin 0.09 + pass begin 0.05), buffer writes 0.09, blit 0.06 (two image views and a VkFramebuffer created and retired per blit).

  • Device contention, root cause (2026-10-08): the anland app was relaunched by fclab.sh's own end-of-run monkey (launchedFromUid=0); its activity takes the foreground back whenever paused, so FCL launched over it never reaches the world (RenderDoc job: 0 of 4 worlds). c41dfdda: bench_session.sh start force-stops it and stop relaunches it only if it ran before.

  • Open user decision: Espryt's apply thread spends 0.06 ms/frame in LatchIfGuiltyBeforeApply (glGetGraphicsResetStatus is a kgsl ioctl, queried on every wake). Rate-limiting it to 2 ms breaks ServerLoopEglLatchTest.AGuiltyResetEndsTheSessionBeforeItsNextRecord..., the designed promptness guarantee, so it was reverted. Trading that guarantee for 0.06 ms is the user's call.

  • CI: 4a8ecf9a green. 49c6394d: one job failed on a hosted runner's apt-get update timeout (45 min, infrastructure); the failed job is re-run.

  • Next:

    • RenderDoc (rdc GLES layer, remote server stopped): Espryt vs MobileGlues BSL pass structure;
    • S1 (client validate 0.25 ms) and S3 (backend by-handle syncs, descriptor resolve) for the CPU floor and the inproc apply thread.

Status (2026-10-08)

Branch: p15impl (worktree .claude/worktrees/p15, short path W:/p15), pushed to feat/disaggregated.

S0.1 parity audit: written.

S0.2, Magma wire non-draw ports:

  • M1a, clears become part of the draw pass (ClearWireFramebufferInPass; PrepareWireDrawPass split out of SetupWireDraw):
    • A whole-area clear that begins the pass begins it with a LOAD_OP_CLEAR variant of the pass's render pass (RenderPassEntry::clearVariants, compatible with the framebuffer).
    • Any other clear (into the open pass, or scissored) is vkCmdClearAttachments in the pass. A forced pass restart for depth clears (an LRZ hypothesis) was tried and reverted: it cost CPU and fixed nothing.
    • Kill switches: MOBILEGL_MAGMA_INPASS_CLEAR=0; MOBILEGL_MAGMA_INPASS_CLEAR_DEFAULT=1 puts default-framebuffer clears in the pass too (off by default, see the open item).
  • Open item: default-framebuffer clears stay standalone. With them in the pass, the openra retrace on Adreno 750 loses its palette-sampled sprites (SSIM 0.956378, deterministic). What was measured:
    • Both vkCmdClearAttachments and LOAD_OP_CLEAR fail; so does splitting the clear calls.
    • Colour-only or depth-only in the pass, with the other part standalone, passes.
    • Event traces: the first draw flushes the recording (the palette glTexImage2D's upload flush, FlushPendingCommands) right after the clear. Command order otherwise matches the standalone path.
    • Not explained yet. Lavapipe and the InPassClear scenarios on FBOs are clean.
    • Ruled out by reading: a missing ordering edge. Default-framebuffer images get the same pass-begin barrier as FBO images (a full ALL_COMMANDS memory barrier, or their transition), with their layout tracked per swapchain index through the swapchain object.
    • Still open: the in-pass path commits the freshly acquired swapchain image (DefaultFramebufferWriteIndex(true) in PrepareWireDrawPass) at the clear; check whether the standalone path's clear and the in-pass clear address the same index on the trace route.
    • Cost of leaving it: small for FCL (MC renders into its own FBO and blits to the default one).
  • M2, draw-pass cache across frames: one cache, entries pruned when their image epochs move, destroyed after their last submit. Kill switch: MOBILEGL_MAGMA_PASS_CACHE=0.
  • M3, blit descriptor sets: one pool per frame slot.
  • M9, converted vertex streams (wrongly marked "have" in the first audit). The wire arm re-converted the whole remaining stream on every draw: Sodium's multi-draws over one arena cost 49 ms/frame of CPU on BSL (19.5 fps against dev's 64.7).
    • Now one conversion per (store content serial, layout), kept across frames in VkBufferManager (LookupConvertedStream / StoreConvertedStream).
    • Each entry owns a vertex buffer. Bounded at 192 MB (least recently used first); entries idle for 256 frames go. A replaced buffer is released when its frame slot comes round.
    • The content serial is process-wide monotonic and moves on every host write, respecify, GPU-write mark, shader-writable bind and donation.
    • Kill switch: MOBILEGL_MAGMA_VERTEX_CONVERSION_CACHE=0.
  • M4, write renaming. A busy glBufferSubData used to take a staged copy, which ended the render pass with two full barriers (FCL: ~5.6 per frame, all on one 288-byte store). Now:
    • A store of 64 KB or less keeps a host shadow (64 MB budget), from respecify on.
    • A busy write into a shadowed store gives it a fresh VkBuffer filled from the shadow and releases the old one like a glBufferData orphan. No copy is recorded and the pass stays open.
    • GPU-written marks, shader-writable binds, donation and import drop the shadow.
    • ReadWireBuffer serves shadowed stores from the shadow, so no host wait.
    • Kill switch: MOBILEGL_MAGMA_WRITE_RENAME=0.
  • Tests:
    • InPassClearScenario: 4 cases, monolith and split, both backends.
    • ConvertedVertexStreamScenario: 3 cases (reuse, same-frame sub-data, cross-frame sub-data). Red-once: a lookup that ignores the content serial turns both sub-data cases red.
    • WriteRenameScenario: 4 cases (order and carry, one-byte writes, uniform and index buffers, multi-frame). Red-once: a rename that skips the shadow fill turns all 4 red, which also proves the rename engages.
    • Red-once checks: the colour index and the scissor (2 cases red); the depth clear-load bit (depth/stencil case red).
    • Syncval gates green (cache 7/7, run-ahead 14/14), plus the InPassClear cases under syncval.
  • Gates:
    • SSIM 24/24 (openra, rd12, bsl × 4 arms × 2 backends) on p15e.
    • Host unit and integration-gpu green, apart from the known environmental ServerSpawn / FuzzArm2 / PeerLatch / spawn ColdStart failures.
    • CompositorRecovery flakes under -j16 and passes alone 3/3.

S0 milestone (one device session, FCL, interval 0, 2 reps; runs/ms-s0i):

scene build Magma CPU ms/frame Magma fps Espryt CPU ms/frame Espryt fps GPU busy
vanilla 1.21.5 (CPU) fin 3.27 170 2.81 190 12-15 %
p15h (S0 without M4) 2.59 191 - - 15 %
p15i (S0) 2.56 190 2.77 193 13-15 %
dev 1.79 (n=1) 234 2.14 218 14 %
MobileGlues 1.59 256 15 %
BSL (GPU) p15i 7.26 77.2 7.61 70.6 99 %
dev 13.85 66.1 7.17 70.5 85-100 %
MobileGlues 6.25 78.2 98-100 %

(BSL before the converted-stream cache: Magma 19.5 fps at 26 % GPU busy, 49 ms CPU/frame.)

  • Espryt's base lib (p15plan, unchanged) read 2.84 in the same session: the earlier "p15e Espryt regression" was session variance. S0 does not touch Espryt.
  • Target status. Magma BSL is met (within 1.3 % of MobileGlues, GPU-bound). Espryt BSL is GPU-bound at 99 % and 10 % below MobileGlues, same as dev: a GPU-side gap still to explain (the load/store and clear audit of PLAN §0.1). CPU scene: Magma 2.56 vs MobileGlues 1.59, Espryt 2.77.
  • Where Magma's CPU goes now (p15i profile, runs/prof-hi, ms/frame inclusive): VerbChannel::Port 0.85, OnDrawVbo 0.62, SetupWireDraw 0.46, Present 0.32, MGPipeValidateForVerb 0.27, in-pass clears 0.18. The draw path is S1-S3's; S0's non-draw ports are done. Proof that M4 engages: StagedWireRangeCopy 0.067 is gone and RenameBusyWireStore costs 0.048 (a VMA allocation per rename; a recycle pool could take most of it).

Stage P: dropped for the target.

Harness

purpose where
FCL probes and reducers tools/device_bench/disagg/ (README "P15 FCL probes")
FCL milestone (CPU + BSL scenes, fin / new / dev / MobileGlues) scratchpad p15/fcl_milestone.sh, fcl_scene.sh, fclab.sh, fclsum.py
SSIM gate (openra, rd12, bsl × 4 arms × 2 backends) scratchpad p15/ssim_gate.sh; matrix_p14.sh now takes PREP_WLS and a bsl workload
host CI-shaped build (WSL archlinux) ~/mgl-p15-ci31, scratchpad p15/wsl_ci.sh, w_suite.sh, w_syncval.sh, w_sv_one.sh, w_one.sh
trace APK builds scratchpad build_trace.sh W:/p15 .<tag> (rerun once on the IncrementalSplitterRunnable failure)

Host-build trap: editing a header while ninja is compiling leaves objects newer than the header but built from its old text. This happened once and gave a bad_alloc in every scenario. Never edit sources during a build; touch the edited headers if in doubt.

Device facts:

  • Clocks: CPU 2035.2 MHz on policies 2/5/7 and 1574.4 MHz on policy0, GPU 680 MHz (new default since 2026-10-08, pending the soak). Pinned per session by bench_session.sh, which every devjob.sh job runs.
  • Frequency daemons are stopped only inside a bench session: thermal-engine, perf2-hal-1-0, vendor.perfservice, performance and hyperschedule_hal_service. Their prior state is in /data/local/tmp/p15-bench-session while a session is open, and they are restarted and verified on exit. If that file exists with no job running, a session was left open: run bench_session.sh stop.
  • The perf stack's game-mode caps in /sys/kernel/msm_performance/parameters/cpu_max_freq survive its services and silently override the pin. The session releases them.
  • FCL's original lib md5 is 00c09f0a….
  • MC 1.21.5 options md5 is 93b3f708…; it is backed up at /data/local/tmp/p15-options.txt.
  • Iris properties are backed up at /data/local/tmp/p15-iris.properties.
  • fcl_scene.sh restore restores Iris; fcl_p14.sh restore restores the lib and config.json.
  • The anland desktop is stopped during runs and restarted afterwards.
Reference in New Issue View Git Blame Copy Permalink