[Docs] (P15): swap-interval stage, user decisions, and the FCL probe tools.

This commit is contained in:
2026-10-08 00:22:22 -04:00
parent 408b9fcade
commit 84ec06f3fc
13 changed files with 808 additions and 29 deletions
+175 -29
View File
@@ -35,18 +35,46 @@ This is the plan, plus the measurements it rests on (section 2). Nothing here is
epochs (a client one and a server one);
- per-VAO and per-program slices that persist across frames.
It does not build a pair-keyed packet table.
4. **Swap interval 0 does not unlimit the frame rate today, and interval ≥1 vsyncs correctly.**
- MC 1.21.5 in FCL runs with vsync off (interval 0) and maxFps 260, which is MC's "unlimited".
- At interval 1, every build locks to 165.0 fps.
- At interval 0, every build is still partly display-coupled: the GL thread blocks ~2 ms/frame
in `queueBuffer`. That includes dev and the MobileGlues plugin.
- The cause is on the platform side. The app's own buffer-queue binder thread holds the queue
mutex across a SurfaceFlinger call, and SurfaceFlinger in turn waits synchronously on the
composer HAL.
- Stage P moves the present off the GL thread to a present worker, so interval 0 is CPU-bound
and interval ≥1 is still paced to the display (section 2.4).
Stage order: 0 (parity audit + wire non-draw fast path), 0b (an FCL 1.21.5 trace fixture), 1
(client draw-delta prototype), 2 (per-VAO / per-program slices), 3 (applier + backend
draw-delta), 4 (split arms, verify, default-on).
Stage order:
- 0: parity audit + wire non-draw fast path;
- 0b: an FCL 1.21.5 trace fixture;
- P: swap-interval-correct presentation. It is independent of 1-4 and can run in parallel after 0;
- 1: client draw-delta prototype;
- 2: per-VAO / per-program slices;
- 3: applier + backend draw-delta;
- 4: split arms, verify, default-on.
Decisions taken with the user (2026-10-08):
- Presenting-thread CPU ms/frame is the primary FCL metric, with fps beside it.
- Swap interval must be honoured properly. This is stage P.
- minSdk / native TLS is the user's own change; P15 does not touch it.
- The probe tools are committed under `tools/device_bench/disagg/`.
- No Espryt deferred-clear port: its clear and blit costs match dev's.
## 1. Measurement rules for P15
- **FCL:**
- The primary metric is the presenting-thread CPU in ms/frame, which MobileGL's fps log already
prints. Report fps beside it, never alone.
- Above ~165 fps, frame time ≈ CPU + ~2.4 ms of display-coupled wait (section 2.1). So fps
ratios understate CPU ratios and are not linear in CPU saved.
- Until stage P lands, at interval 0 and above ~165 fps, frame time ≈ CPU + ~2.4 ms of
display-coupled wait (section 2.1). So fps ratios understate CPU ratios and are not linear in
CPU saved.
- Record the swap interval of every FCL reading (MC's `enableVsync` in
`/sdcard/FCL/.minecraft/versions/1.21.5/options.txt`; the default bench is `false`). Back the
file up before editing and restore it md5-checked.
- At interval ≥1 the per-frame CPU of every build rises by 0.3-0.7 ms (section 2.4), because the
thread idles between frames. Compare CPU only within one interval setting.
- Take readings only from in-world steady windows. The probe waits for MC's `Time elapsed:`
line and then four fps-log windows of more than 150 frames each.
- **Trace (rd12, openra, and the FCL 1.21.5 fixture from stage 0b):**
@@ -72,15 +100,14 @@ draw-delta), 4 (split arms, verify, default-on).
## 2. Measurements behind the plan (2026-10-07)
Tools are in the session scratchpad `p15/`, to be landed under `tools/device_bench/disagg/`
(open question 4):
Tools are in `tools/device_bench/disagg/` (README section "P15 FCL probes"):
- `fcl_probe.sh`: waits for in-world frames, then runs 4 s of ftrace (sched, kgsl cmdbatch, atrace
gfx) and 6 s of `simpleperf --trace-offcpu -p`;
- `offwait.py`: off-CPU time by open atrace slice and waker;
- `sfalign.py`: SurfaceFlinger transaction phase;
- `fclsplit.py` / `incl.py` / `drvcaller.py`: layer split, inclusive functions, and driver time
by caller;
- `P15Count.inl` (measurement-only patch): per-draw pair and gap statistics.
- `fcl_split.py` / `fcl_incl.py` / `fcl_drvcaller.py` / `fcl_stacks.py`: layer split, inclusive
functions, driver time by caller, and stacks;
- `fcl_count.sh` + `p15count.patch` (measurement-only): per-draw pair and gap statistics.
### 2.1 Item 1: where FCL's wait is
@@ -108,9 +135,12 @@ FCL MC 1.21.5, in world, view distance 2, monolith. The figures for the presenti
- **Other checks:**
- GPU busy is ~14 % (union of kgsl `adreno_cmdbatch_retired`).
- The swapchain is MAILBOX with 6 images, so present mode and image count are not the cause.
- **Consequence:** this is not a MobileGL defect and not a feat regression. Making the present
asynchronous would raise every build's FCL fps above 165, but only by producing frames SF drops.
It is not proposed as a perf item; open question 2 covers it.
- **SurfaceFlinger's side:** its binder thread sleeps inside `setTransactionState` until it is
woken by `binder:1555_x`, a thread of `vendor.qti.hardware.display.composer-service`. That
thread is busy with its `HWCDisplay::CommitOrPrepare` cycle. So the composer HAL's display
cycle sets the phase.
- **Consequence:** this is not a MobileGL defect and not a feat regression. The fix for interval 0
is stage P (section 2.4).
### 2.2 FCL CPU by function (render thread, on-CPU ms/frame, inclusive)
@@ -180,6 +210,52 @@ What sits between draws:
- **openra:** every draw changes blend, texture, attrib pointers and sub-data. No per-draw
shortcut applies, and none is needed: openra is not where the gap is.
### 2.4 Swap interval: what is requested, what each backend does, what happens
**What MC requests:**
- The bench world runs with `enableVsync:false` and `maxFps:260` (MC's "unlimited"). So the app
asks for interval 0 through FCL's GLFW → `eglSwapInterval`.
- That interval 0 reaches Espryt's host EGL is shown by behaviour: the host default of 1 would cap
at 165 fps, but Espryt runs at 194-225.
- For Magma, `enableVsync:true` logs `Picked present mode: VK_PRESENT_MODE_FIFO_KHR` and locks to
165 fps, so the interval is plumbed.
**How each backend honours it today:**
- **Espryt:**
- `EGLImpl::SwapInterval` → `BackendObject::SetEGLSwapInterval` → `DirectGLES::SetSwapInterval`
→ host `eglSwapInterval`. On Android that is `ANativeWindow` `setSwapInterval`; 0 is the
async queue mode.
- The interval is re-applied when the window surface is recreated.
- **Magma:**
- `DirectVulkan::SetSwapInterval` → `VulkanRenderer::SetSwapInterval`.
- A changed interval recreates the swapchain between frames.
- `ChooseSwapchainPresentMode`: 0 → IMMEDIATE, then MAILBOX, then FIFO; ≥1 → FIFO; negative →
FIFO_RELAXED, then FIFO; no request → MAILBOX first.
- Android offers no IMMEDIATE here, so interval 0 is MAILBOX with 6 images.
- Magma creates one queue. `m_presentQueue` is the same queue as `m_graphicsQueue` when the
families match.
**Measured, FCL, monolith, same session, mean of 4 fps-log windows** (presenting-thread CPU in
ms/frame):
| build | interval 1: fps | interval 1: CPU | interval 0: fps | interval 0: CPU | interval 0: GL-thread sleep | interval 0: runnable |
|---|---|---|---|---|---|---|
| dev Magma | 164.9 | 2.43 | 235.5 | 1.76 | 2.02 | 0.35 |
| feat Magma | 163.2 | 2.92 | 172.8 | 3.12 | 2.02 | 0.44 |
| dev Espryt | 164.9 | 2.37 | 216.5 | 2.14 | 1.94 | 0.36 |
| feat Espryt | 163.4 | 2.72 | 194.3 | 2.73 | 1.87 | 0.41 |
| MobileGlues plugin (control) | 164.9 | ~1.9 | 248.4 | ~1.7 | 1.98 | 0.34 |
- The MobileGlues CPU figures come from schedstat (the plugin has no fps log).
- **Interval 1 is correct everywhere:** FIFO, or host interval 1. The waits sit in
`dequeueBuffer` / `AcquireNextImageKHR` and in the render thread's own `setTransactionState`.
- **Interval 0 is wrong everywhere,** including dev and the control. The GL thread still spends
~2 ms/frame in `queueBuffer` → `BLASTBufferQueue::onFrameAvailable`, waiting for the mutex that
the app's binder thread holds across `setTransactionState`, and that call is phase-locked to the
composer HAL (section 2.1).
- **The async queue mode does not help.** It changes what happens to the buffers, not who holds the
mutex.
## 3. Stages
### Stage 0: parity audit, then the wire non-draw fast path (Magma first)
@@ -274,6 +350,81 @@ Espryt's deleted mechanisms. Port only those that show in profiles.
not, the fixture does not represent FCL.
- **Order:** best done before stage 0.2 lands, so stage 0 can be A/B'd on a trace as well as on FCL.
### Stage P: swap-interval-correct presentation (both backends)
**Goal:**
- With interval 0, presentation never paces the app: FCL fps becomes 1 / presenting-thread CPU.
- With interval ≥1 the app is still paced to the display: 165 fps ±1 % on this device, with no
extra latency beyond one queued frame.
**Design: a present worker that owns the blocking present.**
- The GL thread finishes a frame and hands it to the worker through a bounded queue (depth 1 at
interval ≥1, depth 2 at 0). It returns at once.
- The worker performs the present call that may block.
- At interval ≥1, backpressure comes from the full queue: the GL thread waits for the worker,
which waits for the display. At interval 0 the queue never fills, because the worker's blocked
windows (≤3.4 ms, about once per refresh) are absorbed by the swapchain's spare images.
- The worker exists only for window surfaces. Pbuffer and offscreen surfaces keep the inline path.
**Magma:**
- The GL thread keeps `vkAcquireNextImageKHR` and the submit, which signals the image's
render-done semaphore.
- The worker calls `vkQueuePresentKHR`.
- Queue access must be externally synchronised, and the WSI's `queueBuffer` runs inside
`vkQueuePresentKHR`. So the worker needs **its own VkQueue**: a second queue of the graphics
family, or a present-capable family.
- Check the Adreno queue count first.
- If only one queue exists, the fallback is a queue mutex released before the WSI blocks. That
is not possible from outside the loader, so a single-queue device keeps the inline present
and logs it once.
- Swapchain recreation (resize, interval change, `OUT_OF_DATE`) drains the worker first.
- The frame-in-flight fences already gate the reuse of images.
**Espryt:** `eglSwapBuffers` is bound to the context's thread, so the prototype compares two routes:
- **(a) Own the native window.**
- Dequeue `ANativeWindowBuffer`s on the GL thread (or worker) and wrap each as an EGLImage
(`EGL_NATIVE_BUFFER_ANDROID`, cached by buffer identity: generation + id, never a pointer).
- Render the default framebuffer into it.
- The worker queues it with an `EGL_ANDROID_native_fence_sync` fd.
- This route also needs buffer count, transform / pre-rotation, damage, buffer age and
`setSwapInterval` on the window itself.
- No extra blit. The anland shared-image route already does part of this.
- **(b) A present thread with a shared context.**
- The default framebuffer becomes an offscreen FBO.
- The present thread waits on a fence, blits to its window surface and calls `eglSwapBuffers`.
- Costs one full-screen blit per frame (800×1280; small on Adreno 750), a second context, and
redirecting default-framebuffer reads.
- Prototype (a) first.
**Split arms:**
- The server presents on the apply thread (inproc) or in the server process (spawn/tcp). The
worker sits there.
- The present credit is returned when the frame is handed to the worker, not when `queueBuffer`
returns.
**Tests:**
- interval 0 → fps bounded by CPU, and the GL thread never sleeps in `queueBuffer`
(`offwait.py` shows no `queueBuffer` sleep on the GL thread);
- interval 1 → 165 fps ±1 %, and so is interval 2 → 82.5;
- interval changes at runtime (MC's vsync toggle);
- surface resize / rotation; eglMakeCurrent away and back with frames queued;
- teardown and device loss with a frame in the worker;
- the anland desktop (a split-arm present on the same device);
- negative control: a knob that makes the worker present inline must bring the ~2 ms
`queueBuffer` sleep back.
- Kill switch: `MOBILEGL_PRESENT_WORKER=0`.
**Expected at FCL interval 0** (presenting-thread CPU is unchanged; fps approaches 1/CPU):
| build | today | after stage P | after stage 0 as well |
|---|---|---|---|
| dev Magma | ~236 | ~450-550 | |
| feat Magma | ~173 | ~300-330 | ~450 |
| feat Espryt | ~194 | ~350 | |
At interval ≥1 nothing changes. Confidence: medium. It depends on the second queue (Magma) and on
route (a) working (Espryt).
### Stage 1: client draw-delta prototype (Espryt monolith)
This is the client half of the old sketch, re-keyed on what section 2.3 shows.
@@ -388,8 +539,7 @@ This is the client half of the old sketch, re-keyed on what section 2.3 shows.
- a (VAO, program) pair table: 0 % hit on rd12, and FCL's hits are covered by the draw delta;
- record batching (measured and dropped in P14);
- protocol changes;
- asynchronous present (open question 2);
- native TLS (open question 3);
- native TLS / minSdk (the user's own change);
- openra, whose gaps change state on every draw.
### 4.2 Per-layer state for one draw on the fast path
@@ -526,18 +676,14 @@ What the totals mean:
- **Gains below estimate:** the stage 1 prototype is the go/no-go for stages 2-3.
- **FCL fixture fidelity:** stage 0b's P15Count check.
## 7. Open questions for the user
## 7. Questions resolved with the user (2026-10-08)
1. **The FCL metric.** Do you agree to make presenting-thread CPU ms/frame the primary FCL metric?
fps above refresh rate is display-coupled (section 2.1).
2. **Asynchronous present.** Presenting from a separate thread would cut the ~2 ms/frame BBQ wait
from the render thread on every build, dev included. It would raise FCL fps above refresh, but
the extra frames are ones SF drops; at or below refresh nothing changes. Leave it out (my
recommendation), or plan it as a separate item?
3. **Native TLS** (minSdk 29, +1.7 % Espryt): still your call from P14.
4. **The P15 measurement tools.** May I commit `fcl_probe.sh`, `offwait.py`, `sfalign.py`,
`fclsplit.py`, `drvcaller.py` and `incl.py` under `tools/device_bench/disagg/` with the plan?
They are tooling, not product code. This assignment was docs-only, so they are not in this
commit.
5. **Magma deferred clears, Espryt side:** the wire arm's Espryt clears cost the same as dev's, so
no port is planned there. Confirm.
1. FCL metric: presenting-thread CPU ms/frame is primary, with fps beside it.
2. Swap interval: interval 0 must unlimit the frame rate, and interval ≥1 must vsync. This is
stage P.
3. minSdk 29 / native TLS: the user decides and changes it. P15 does not touch it.
4. The probe tools are committed under `tools/device_bench/disagg/`.
5. No Espryt deferred-clear port: Espryt's clear and blit costs match dev's.
Still open (for stage P): does Adreno 750 expose a second graphics-family queue for the Magma
present worker? Which Espryt route, (a) or (b)?