diff --git a/docs/Disaggregated/notes/perf-p15/HANDOFF.md b/docs/Disaggregated/notes/perf-p15/HANDOFF.md index 6e94da17..35099e7b 100644 --- a/docs/Disaggregated/notes/perf-p15/HANDOFF.md +++ b/docs/Disaggregated/notes/perf-p15/HANDOFF.md @@ -81,8 +81,22 @@ CPU about 1.5/1.25 GHz, GPU 903 MHz (cleanly CPU-bound); interval 0, anland off, client as `BlitNamedFramebuffer`, a verb with no `MGP_VERB_OP_LIST` row, so `ClientVerbIsBarriered` answered barriered and the GL thread quiesced the applier once per frame (1.65 ms/frame parked in `WaitForApplyToCatchUp`, plus 0.11 from the following `glBindTexture`). - The apply thread then idled 0.24 ms parked plus 0.2 ms spinning in `Doorbell::Wait`. c07b8c40 - maps the verb to its `blit` row (kill switch `MOBILEGL_PIPE_BLIT_RUN_AHEAD=0`). + The apply thread then idled 0.24 ms parked plus 0.2 ms spinning in `Doorbell::Wait`. e00db8e6 + replaces the inverse and its barriered fallback with a verb -> record-op table + (`MG_Pipe/VerbRecordOps.h`, one row per verb by static_assert; no knob, user decision). + - Same-session blab (cpuhunt): Magma inproc 319.6 -> 334.9 fps (+4.8 %), Espryt inproc + 324.4 -> 338.1 (+4.2 %). + - Wait census (`MOBILEGL_IPC_WAIT_STATS=1`, 7acf3ea8): FCL vanilla inproc waits only on the + present credit (1.00/frame). BSL waits 7 times per frame on three kWaitApplied rows plus 22 + follow-on quiesces; see [WAIT-AUDIT.md](WAIT-AUDIT.md) (awaiting the user's review). + - db915f76: a client parked on the present credit is rung only by the credit (apply-thread + futex wakes 0.056 ms/frame before). + - Present credit 2 (cens, cpuhunt, all VALID): Magma 309.0 -> 319.9, Espryt 299.1 -> 319.1 fps + vs MobileGlues 321.9 in the same session; apply CPU 1.40-1.43 vs MobileGlues' GL thread 1.57. + Credit 2 costs a frame of latency: user decision. The remaining inproc gap is the queueBuffer + fence wait (GPU-side), not CPU. + - BSL at cpuhunt (GPU 903 MHz) reaches GPU 95 C and kgsl thermal_pwrlevel 1 (834 MHz cap): the + census run was INVALID twice and the job was stopped. Run BSL only at gpuhunt. - Apply-thread frame (Magma inproc): on-CPU 1.75 (0.2 of it spin), queueBuffer fence wait 1.45, parked 0.24 ms. - Magma apply on-CPU by verb: draws 0.55 (SetupWireDraw 0.40, of it descriptor binding 0.13), diff --git a/docs/Disaggregated/notes/perf-p15/WAIT-AUDIT.md b/docs/Disaggregated/notes/perf-p15/WAIT-AUDIT.md new file mode 100644 index 00000000..b47a9050 --- /dev/null +++ b/docs/Disaggregated/notes/perf-p15/WAIT-AUDIT.md @@ -0,0 +1,73 @@ +# P15 wait-class audit (draft for the user's review) + +Commands are recorded and replayed asynchronously by default. A record makes the client wait only +because its row in `MG_Pipe/PipeCalls.def` has a wait class other than `kWaitNone`. Since e00db8e6 +the client's per-verb barrier answer is exactly the wait class of the record its emitter publishes +(`MG_Pipe/VerbRecordOps.h`). This file lists every row that still waits, says why, and flags the +candidates for async. **No row is converted until the user has seen this table.** Each conversion +needs a red-once test. + +The rule for keeping a wait: +- **true reply**: the client acts on an answer it cannot derive itself; or +- **present pacing**. + +Anything else (a `kWaitApplied` row whose apply still reads a BARRIER_PULLED or residual field, or a +reply the client could derive) is a candidate. + +## Measured hit rates (FCL, inproc, cpuhunt, `MOBILEGL_IPC_WAIT_STATS=1`) + +Waits per frame in steady state, 600-frame windows: + +| wait | vanilla 1.21.5 (Magma, Espryt) | BSL (Magma) | +|---|---|---| +| present credit | 1.00 | 1.00 | +| record `ResourceCreate` (blocking create) | 0.00-0.01 | 2-3 | +| record `ResourceCopyRegion` (glCopyImageSubData) | 0 | 3 | +| record `GenerateMipmap` | 0 | 3 | +| record `CopyFramebufferToTexture` (glCopyTexSubImage2D) | 0 | 1 | +| barriered verbs quiescing the applier at validate | 0 | 8 (CopyImageSubData 3, GenerateMipmap 3, CopyTexSubImage2D 1, ClientWaitSync 1) | +| follow-on quiesce `MGPipeNoteFrontendMutation` | 0 | 14-16 | +| follow-on quiesce `MGPipeValidateForVerb/residual` | 0 | 8 | +| ring space | 0 | 0 | + +- Vanilla meets the target: the present credit is the only per-frame wait. +- On BSL, every barriered verb costs about three waits, not one: the record's own applied wait, the + quiesce before the fill, and then a re-quiesce at each frontend mutation until the next + unbarriered verb (`g_lastFillWasBarriered`). Making the three `kWaitApplied` verbs async removes + all of these at once. + +## Rows that wait + +| row | class | why it waits today | FCL hits/frame | verdict | +|---|---|---|---|---| +| `Present` | kWaitPresent | present credit, taken before the encode | 1 | keep (pacing) | +| `GetCaps` | kWaitReply | the caps snapshot | 0 (startup) | keep (true reply) | +| `ResourceCreate` | kWaitReply | accept/decline feeds the client's publish latch; a create window defers it for some objects | BSL 2-3 | **candidate**: find which creates take the blocking path every frame on BSL and why the window refuses them | +| `ResourceRespecify` | kWaitReply | texture half until the object's create is accepted; already fire-and-forget once published (`MGPipeResourceRespecifyWantsItsReply`) | 0 | keep (relaxed per call already) | +| `MapPersistent` | kWaitReply | the client needs the mapping | 0 (startup) | keep (true reply) | +| `FenceStatus`, `FenceWait` | kWaitReply | GPU state the app asked for | BSL 1 (ClientWaitSync) | keep (true reply) | +| `QueryAvailable`, `QueryResult`, `QueryTimestamp` | kWaitReply | GPU results | 0 | keep (true reply) | +| `SetTextureParams` | kWaitReply | waits only before the texture's create is accepted; fire-and-forget after | 0 | keep (relaxed per call already) | +| `ResourceSubData` | kWaitReply | buffer half and wire texture half never wait (`MGPipeSubDataWantsItsReply`) | 0 | keep (relaxed per call already) | +| `ResourceReadback`, `GetTextureImage`, `ReadPixels` | kWaitReply | the bytes | 0 | keep (true reply); **note**: the verb table names the reply op for ReadPixels / GetTexImage, so a pack-buffer read (which publishes the kWaitNone `*ToBuffer` record) is still barriered at validate. A per-call answer would make PBO reads async | +| `SharedImage` | kWaitReply | AHB/dma-buf handle exchange (anland) | 0 | keep (true reply) | +| `GenerateMipmap` | kWaitApplied | its apply read `GetActiveTextureUnit` and the unit's binding slot from the residual block (PipeCalls.def / ClientSession.cpp); P13 W4b gave the record arm the texture by handle (`VerbMipRes`) | BSL 3 | **candidate**: check whether any backend path still reads the residual unit; if not, kWaitNone | +| `ResourceCopyRegion` | kWaitApplied | its apply read `GetTextureObject`, a BARRIER_PULLED row whose retiring phase was P7 (ID-118); P13 W4b resolves both endpoints by handle | BSL 3 | **candidate**: same check | +| `CopyFramebufferToTexture` | kWaitApplied | the read framebuffer and the destination were pulled; P13 W4b resolves the destination by handle (`VerbCopyTexDst`) and P13 W4c names framebuffers by handle | BSL 1 | **candidate**: same check | +| `SetStorageBlockBinding` | kWaitApplied | probes a registry | 0 | keep for now (not on any FCL path) | +| `BeginStreamOutput`, `EndStreamOutput`, `PauseStreamOutput`, `ResumeStreamOutput`, `BindStreamOutput` | kWaitApplied | XFB span accounting and the capture-binding shadow | 0 | keep for now (not on any FCL path) | +| `ApplierReset` | kWaitApplied | orders a make-current reset against the verbs after it | 0 (make-current) | keep | + +Count note: the X rows in PipeCalls.def parse as 59 kWaitNone, 15 kWaitReply, 10 kWaitApplied and +1 kWaitPresent (85 rows). + +## Proposed order (after the user's go-ahead) + +1. `GenerateMipmap`, `ResourceCopyRegion`, `CopyFramebufferToTexture` to kWaitNone, each with a + red-once test (a poison or verify build that aborts on a residual read under an unbarriered + apply). BSL then loses 7 record waits, 7 validate quiesces and the 14-16 follow-on quiesces per + frame. +2. `ResourceCreate` on BSL: find the 2-3 blocking creates per frame and why the create window + declines them. +3. Pack-buffer `ReadPixels` / `GetTexImage`: a per-call barrier answer instead of the verb-level + reply op.