From 98b2c37384c7d3e31f4139e444dc0609527fb87b Mon Sep 17 00:00:00 2001 From: Swung0x48 Date: Fri, 9 Oct 2026 01:14:52 -0400 Subject: [PATCH] Docs (P15): monolith diff vs dev and H2 rows; Magma records 12.6 passes and 27.5 barriers/frame against dev's 4.1 and 3.4. --- .../notes/perf-p15/MONOLITH-DIFF.md | 288 ++++++++++++++++++ .../notes/perf-p15/SCOREBOARD.md | 23 ++ 2 files changed, 311 insertions(+) create mode 100644 docs/Disaggregated/notes/perf-p15/MONOLITH-DIFF.md diff --git a/docs/Disaggregated/notes/perf-p15/MONOLITH-DIFF.md b/docs/Disaggregated/notes/perf-p15/MONOLITH-DIFF.md new file mode 100644 index 00000000..9525527c --- /dev/null +++ b/docs/Disaggregated/notes/perf-p15/MONOLITH-DIFF.md @@ -0,0 +1,288 @@ +# Monolith render-thread diff: dev vs p15 (FCL MC 1.21.5 vanilla, H2, cpuhunt) + +The fixed function-level baseline for the monolith CPU cuts (SCOREBOARD.md has the fps +reference). Every cut starts from a symbolized profile of the p15 side and ends with a +before/after of the same function. + +**Sources:** +- **Profiles:** simpleperf `cpu-clock --trace-offcpu --call-graph fp -f 4000 --duration 6`, taken + after the measurement windows of an H2 run (FCL pinned to cpu2-6). +- **dev:** 08124c99 in `runs/safter/dev-Direct{Vulkan,GLES}-r1`. +- **p15:** p15t = HEAD 3def5cb2 (p15s 0b96a3ef plus the Magma blit view cache) in + `runs/pt/t-Direct{Vulkan,GLES}-r1`. +- **Scope and units:** render thread only (the GL thread, which in monolith also applies), off-CPU + samples dropped, in ms/frame. Frames are the run's measured fps × 6 s; profiling overhead + inflates both sides alike. +- **Reducer:** `layerdiff.py` in the scratchpad. + +**Attribution:** each sample goes to its innermost libMobileGL function, then to one layer. +- The leaf DSO decides first: driver (Adreno), libc+TLS (libc, libc++, `__emutls_get_address`, + TLS init), or kernel. +- Otherwise the function's namespace decides: + - client frontend: `MG_Impl`, `MG_State`; + - pipe record: `MG_Pipe` validate, tracker and emitters, plus the record channel; + - applier: `MG_Record` sinks and record emitters, plus `MG_Pipe` applier state; + - backend: `MG_Backend`. +- Samples with no libMobileGL frame are app-side (JIT/JVM). + +**Caveat on function rows:** some functions were renamed between dev and p15, so a "+" row and a +"-" row can be the same work. Read such pairs together: +- Espryt `FlushPendingRangesNow` -> `FlushPendingRangesFrom` (0.105 -> 0.116); +- `BackendVertexArrayObject::SyncToBackend` -> `SyncToBackendFromApplier` (0.062 -> 0.058); +- `BackendFramebufferObject::SyncToBackend` -> `SyncToBackendByHandle`; +- Magma `SetupDraw` -> `SetupWireDraw`. + +## Where the excess lives (summary) + +| layer | Magma diff | Espryt diff | what it is | +|---|---|---|---| +| pipe record | +0.36 | +0.35 | `MGPipeValidateForVerb` (0.34 inclusive on Magma). Its pieces: framebuffer-state build 0.06, vertex-buffer emit 0.04, global constants 0.03, slot lookups (`FindByLifetimeId` + `Acquire`) 0.05, residual `CopyField` 0.02, tracker 0.02 | +| driver | +0.23 | +0.06 | Magma: per-frame fixed Vulkan work (below). Espryt: about even once renames are paired | +| libc+TLS | +0.22 | +0.14 | `__emutls_get_address` 0.055-0.075 plus `pGLContext` TLS init 0.016 (dev: ~0). Magma: libc under the extra command-buffer and pass begins. Espryt: `QueryCurrentSurfaceSize` | +| client frontend | +0.12 | +0.17 | Spread thin. Atomics (`__aarch64_ldadd8_*`) +0.02, state lookups 0.005 each | +| backend | -0.10 | +0.11 | Magma's wire backend is cheaper than dev's `SetupDraw` (no XXH64 vertex-input hash). Espryt pays per-handle registry lookups | +| JIT/JVM (app) | +0.04 | +0.08 | Not ours; profiling noise between runs | + +**Magma per-frame fixed costs** (inclusive ms/frame, dev -> p15): +- BeginRenderPass 0.045 -> 0.117. +- EndCommandRecording 0.054 -> 0.113. +- BeginCommandRecording 0.052 -> 0.089. +- EndRenderPass 0.023 -> 0.033. +- Present 0.303 -> 0.352. +- p15 also has `SyncTextureResourceByHandle` 0.068, of which `UploadPendingWireLevels` is 0.038 + and `FlushPendingCommands` 0.025. + +Cause: every wire texture upload first submits the frame's recorded work +(`FlushWirePendingCommandsForTextureUpload`) and then uploads in a separate batch. MC updates +textures every frame, so the frame pays an extra submit, an extra command-buffer end/begin and a +render-pass restart. `GpuProgressMarkers::SubmitBracketed` is NOT new work: it wraps Present's +queue submit, which dev pays inside `Present` (0.121 vs 0.115). + +**Espryt-only:** `QueryCurrentSurfaceSize` costs 0.065 ms/frame and dev pays nothing for it. It +is two `eglQuerySurface` calls on every dirty render-state sync (draws 0.048, clears 0.022). It +substitutes the surface for a scissor box the app never wrote (and for an empty viewport). + +## Correction after cut 1 (2026-10-09): what the Magma fixed costs really are + +Cut 1 (wire texture uploads stop flushing the frame recording) engaged on device: +`FlushPendingCommands` and `SubmitPendingCommandBuffer` left the profile, and +`UploadPendingWireLevels` went from 0.038 to 0.016. But Begin/EndCommandRecording and +BeginRenderPass did not move. H2 c1: 254.1 -> 249.8 fps, inside the noise. + +Their cost is in the driver and scales with what is recorded: + +| driver call | dev | p15 | +|---|---|---| +| vkResetCommandBuffer | 0.044 | 0.077 | +| vkEndCommandBuffer | 0.053 | 0.111 | +| vkCmdBeginRenderPass | 0.037 | 0.105 | + +Host call counts per frame (diagnostic counting builds; FCL vanilla; 600-frame windows): +- **Magma, Vulkan calls: dev 331.6 vs p15 379.** + - The draw-level stream is identical and almost free of repeats: DrawIndexed 86.1, + BindVertexBuffers 87.1, BindDescriptorSets 87.1, BindPipeline 15.1, UpdateDescriptorSets 14, + dynamic state 2 each. + - Render passes: 4.1 -> 12.6. + - Pipeline barriers: 3.4 -> 27.5. + - CopyBufferToImage: 0.1 -> 2.2. + - By call site (unwinding counter): + - Pass begins: 6.5 at draws, 5.2 at clears, 1 at the blit. + - Pass ends: 5.3 from `ResolveWireImageDescriptor` (sampling an image written earlier), 4.2 + from `TransitionWireImage` in the in-pass clear, 1.2 from `TransitionWireImage` in + `SetupWireDraw`, 2 at the blit. + - Barriers: 11.8 from `TransitionWireImage` at draw-pass begin, 6.3 at clear-pass begin, 5.6 + from `ResolveWireImageDescriptor`, 2 at the blit, 1 at present. + - Cause: each new wire draw pass runs `TransitionWireImage(GENERAL -> GENERAL)` per + attachment. That ends the open pass and records a full ALL_COMMANDS memory barrier whether or + not a hazard exists. +- **Espryt, host GLES calls: dev 850.7 (479 repeat their previous arguments) vs p15 835.6 (446).** + - p15 does not issue more GLES calls than dev. + - In both, most of the repeats are glVertexAttribPointer 302 + glVertexAttribIPointer 75: the + same layout re-pointed at a new VBO on every draw. + +## Cuts, biggest first + +**Revised order:** +1. **1b (Magma):** explicit external subpass dependencies on the wire draw passes. Barrier only + images last written by transfer or storage, once and coalesced before the pass. + Estimate: passes 12.6 -> ~5, barriers 27 -> ~5, and the driver's command-buffer and + pass-begin cost with them. +2. **Cut 1 stays local** (no measurable gain). + +The original list follows. + + +Estimates are ms/frame on the H2 render thread. Each cut gets a before/after profile of its +function and an H2 A/B. + +1. **Magma: record wire texture uploads into the frame's command buffer** instead of + flush + separate batch. Estimate 0.10-0.15. It removes the extra submit, the command-buffer + end/begin and the pass restart. Correctness-sensitive: draw / TexSubImage / draw ordering, and + staging lifetime per frame. Red-once with the existing between-draws upload scenarios, plus + syncval. +2. **Shared: the pipe-record layer**, about 0.35 on both backends. Parts: + - (a) framebuffer-state build per verb: memo on the framebuffer's version; + - (b) slot lookups by lifetime id; + - (c) residual `CopyField` on monolith, where `fillOwed` is always true; + - (d) vertex-buffer and global-constant emit. + Each part is 0.02-0.06. Profile-driven, one at a time. +3. **Shared: TLS.** `__emutls_get_address` plus TLS init is ~0.07-0.09. The structural fix is + native TLS (minSdk, the user's call). The in-code mitigation is to read + `pGLContext` / `gPipeInputs` once per verb and pass the pointer down. +4. **Espryt: memoize the surface size** per draw surface, invalidated at swap and make-current. + Estimate 0.06. +5. **Magma: the clear path** (`glClear` 0.285 vs dev ~0.13): first re-profile after cut 1, since + much of it is the command-buffer and pass begins cut 1 removes. + +## Magma (DirectVulkan): +0.88 ms/frame on the render thread (schedstat: 2.84 vs 2.00) + +| layer | dev | p15 | diff | +|---|---|---|---| +| client frontend | 0.257 | 0.376 | +0.119 | +| pipe record | 0.000 | 0.363 | +0.363 | +| applier | 0.000 | 0.026 | +0.026 | +| backend | 0.516 | 0.416 | -0.100 | +| driver | 0.394 | 0.622 | +0.228 | +| libc+TLS | 0.213 | 0.430 | +0.218 | +| kernel (from MobileGL) | 0.112 | 0.115 | +0.003 | +| mgl other | 0.066 | 0.049 | -0.017 | +| JIT/JVM (app) | 0.659 | 0.697 | +0.038 | +| kernel (app) | 0.049 | 0.047 | -0.001 | +| **total** | 2.265 | 3.142 | +0.877 | + +Top 15 p15 excess rows: + +| layer | function (attributed) | dev | p15 | diff | +|---|---|---|---|---| +| libc+TLS | `__emutls_get_address` | 0.000 | 0.075 | +0.075 | +| driver | `VkRenderPassManager::BeginRenderPass` | 0.029 | 0.081 | +0.052 | +| driver | `FrameContext::EndCommandRecording` | 0.045 | 0.090 | +0.045 | +| kernel (from MobileGL) | `GpuProgressMarkers::SubmitBracketed` | 0.000 | 0.045 | +0.045 | +| driver | `VulkanRenderer::SetupWireDraw` | 0.000 | 0.040 | +0.040 | +| pipe record | `MG_Pipe::MGPipeValidateForVerb` | 0.000 | 0.035 | +0.035 | +| backend | `VulkanRenderer::SetupWireDraw` | 0.000 | 0.034 | +0.034 | +| driver | `VulkanRenderer::DrawElements` | 0.101 | 0.129 | +0.028 | +| pipe record | `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId` | 0.000 | 0.026 | +0.026 | +| driver | `GpuProgressMarkers::SubmitBracketed` | 0.000 | 0.024 | +0.024 | +| driver | `UniformManager::ResolveWireImageDescriptor` | 0.000 | 0.023 | +0.023 | +| pipe record | `MG_Pipe::MGPipeTracker::Update` | 0.000 | 0.022 | +0.022 | +| libc+TLS | `VkRenderPassManager::BeginRenderPass` | 0.010 | 0.032 | +0.022 | +| libc+TLS | `FrameContext::BeginCommandRecording` | 0.018 | 0.039 | +0.021 | + +Per layer (top 6 each): + +| layer | function (attributed) | dev | p15 | diff | +|---|---|---|---|---| +| client frontend | `__aarch64_ldadd8_acq_rel` | 0.013 | 0.023 | +0.010 | +| client frontend | `MG_State::GLState::FramebufferAttachmentObject::IsEmpty` | 0.000 | 0.008 | +0.007 | +| client frontend | `MG_State::GLState::BufferState::GetBindingSlot` | 0.000 | 0.006 | +0.006 | +| client frontend | `__aarch64_ldadd8_relax` | 0.004 | 0.010 | +0.006 | +| client frontend | `MG_State::GLState::ShareGroupState::GetBufferObject` | 0.000 | 0.005 | +0.005 | +| client frontend | `MG_State::GLState::FramebufferObject::CheckCompleteness` | 0.000 | 0.004 | +0.004 | +| pipe record | `MG_Pipe::MGPipeValidateForVerb` | 0.000 | 0.035 | +0.035 | +| pipe record | `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId` | 0.000 | 0.026 | +0.026 | +| pipe record | `MG_Pipe::MGPipeTracker::Update` | 0.000 | 0.022 | +0.022 | +| pipe record | `MG_Pipe::MGPipeFillAccess::CopyField` | 0.000 | 0.021 | +0.021 | +| pipe record | `MG_Pipe::MGPipeVertexInputEmitter::EmitVertexBuffers` | 0.000 | 0.018 | +0.018 | +| pipe record | `MG_Pipe::MGPipeProgramEmitter::EmitGlobalConstants` | 0.000 | 0.011 | +0.011 | +| applier | `MG_Record::RecordVerbSink::OnDrawVbo` | 0.000 | 0.005 | +0.005 | +| applier | `MG_Record::{anon}::EmitDrawRecord` | 0.000 | 0.004 | +0.004 | +| applier | `MG_Record::{anon}::EmitClear` | 0.000 | 0.003 | +0.003 | +| applier | `MG_Record::{anon}::EmitIndexedDraw` | 0.000 | 0.003 | +0.003 | +| applier | `MG_Record::{anon}::EmitDrawElements` | 0.000 | 0.003 | +0.003 | +| applier | `MG_Pipe::MGPipeApplierState::DrawFramebuffer` | 0.000 | 0.003 | +0.003 | +| backend | `VulkanRenderer::SetupWireDraw` | 0.000 | 0.034 | +0.034 | +| backend | `VulkanRenderer::GetOrCreatePipelineWithInput` | 0.000 | 0.021 | +0.021 | +| backend | `UniformManager::PrepareWireTextureResources` | 0.000 | 0.018 | +0.018 | +| backend | `_t const&, std::__ndk1::tuple, std::__ndk1::tuple<> >` | 0.000 | 0.016 | +0.016 | +| backend | `UniformManager::ResolveWireImageDescriptor` | 0.000 | 0.014 | +0.014 | +| backend | `VkTextureManager::ResolveWireTextureStorage` | 0.000 | 0.012 | +0.012 | +| driver | `VkRenderPassManager::BeginRenderPass` | 0.029 | 0.081 | +0.052 | +| driver | `FrameContext::EndCommandRecording` | 0.045 | 0.090 | +0.045 | +| driver | `VulkanRenderer::SetupWireDraw` | 0.000 | 0.040 | +0.040 | +| driver | `VulkanRenderer::DrawElements` | 0.101 | 0.129 | +0.028 | +| driver | `GpuProgressMarkers::SubmitBracketed` | 0.000 | 0.024 | +0.024 | +| driver | `UniformManager::ResolveWireImageDescriptor` | 0.000 | 0.023 | +0.023 | +| libc+TLS | `__emutls_get_address` | 0.000 | 0.075 | +0.075 | +| libc+TLS | `VkRenderPassManager::BeginRenderPass` | 0.010 | 0.032 | +0.022 | +| libc+TLS | `FrameContext::BeginCommandRecording` | 0.018 | 0.039 | +0.021 | +| libc+TLS | `thread-local initialization routine for MG_State::pGLContext` | 0.000 | 0.017 | +0.017 | +| libc+TLS | `EndActiveRenderPassOn` | 0.000 | 0.014 | +0.014 | +| libc+TLS | `FrameContext::EndCommandRecording` | 0.009 | 0.023 | +0.014 | + +## Espryt (DirectGLES): +0.93 ms/frame on the render thread (schedstat: 3.13 vs 2.34) + +| layer | dev | p15 | diff | +|---|---|---|---| +| client frontend | 0.238 | 0.404 | +0.166 | +| pipe record | 0.000 | 0.349 | +0.349 | +| applier | 0.000 | 0.041 | +0.041 | +| backend | 0.245 | 0.356 | +0.111 | +| driver | 0.800 | 0.861 | +0.062 | +| libc+TLS | 0.209 | 0.350 | +0.141 | +| kernel (from MobileGL) | 0.209 | 0.221 | +0.011 | +| mgl other | 0.134 | 0.113 | -0.021 | +| JIT/JVM (app) | 0.693 | 0.774 | +0.081 | +| kernel (app) | 0.055 | 0.050 | -0.005 | +| **total** | 2.584 | 3.518 | +0.935 | + +Top 15 p15 excess rows: + +| layer | function (attributed) | dev | p15 | diff | +|---|---|---|---|---| +| driver | `BufferImpl::{anon}::FlushPendingRangesFrom` | 0.000 | 0.091 | +0.091 | +| libc+TLS | `__emutls_get_address` | 0.000 | 0.055 | +0.055 | +| driver | `VertexArrayImpl::BackendVertexArrayObject::SyncToBackendFromApplier` | 0.000 | 0.040 | +0.040 | +| pipe record | `MG_Pipe::MGPipeValidateForVerb` | 0.000 | 0.040 | +0.040 | +| driver | `DrawElements` | 0.276 | 0.312 | +0.036 | +| pipe record | `MG_Pipe::MGPipeTracker::Update` | 0.000 | 0.029 | +0.029 | +| libc+TLS | `QueryCurrentSurfaceSize` | 0.000 | 0.026 | +0.026 | +| libc+TLS | `BufferImpl::{anon}::FlushPendingRangesFrom` | 0.000 | 0.025 | +0.025 | +| driver | `FramebufferImpl::BackendFramebufferObject::SyncToBackendByHandle` | 0.000 | 0.025 | +0.025 | +| mgl other | `QueryCurrentSurfaceSize` | 0.000 | 0.023 | +0.023 | +| pipe record | `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId` | 0.000 | 0.022 | +0.022 | +| pipe record | `MG_Pipe::MGPipeVertexInputEmitter::EmitVertexBuffers` | 0.000 | 0.021 | +0.021 | +| pipe record | `MG_Pipe::MGPipeFillAccess::CopyField` | 0.000 | 0.018 | +0.018 | +| backend | `VertexArrayImpl::BackendVertexArrayObject::SyncToBackendFromApplier` | 0.000 | 0.018 | +0.018 | + +Per layer (top 6 each): + +| layer | function (attributed) | dev | p15 | diff | +|---|---|---|---|---| +| client frontend | `__aarch64_ldadd8_acq_rel` | 0.012 | 0.026 | +0.014 | +| client frontend | `__aarch64_ldadd8_relax` | 0.003 | 0.013 | +0.011 | +| client frontend | `MG_State::GLState::FramebufferAttachmentObject::IsEmpty` | 0.000 | 0.006 | +0.006 | +| client frontend | `MG_State::GLState::ShareGroupState::ValidateBufferObject` | 0.000 | 0.006 | +0.006 | +| client frontend | `MG_State::GLState::BufferState::GetBindingSlot` | 0.001 | 0.006 | +0.005 | +| client frontend | `MG_State::GLState::GLContext::GetProgramForDraw` | 0.002 | 0.007 | +0.005 | +| pipe record | `MG_Pipe::MGPipeValidateForVerb` | 0.000 | 0.040 | +0.040 | +| pipe record | `MG_Pipe::MGPipeTracker::Update` | 0.000 | 0.029 | +0.029 | +| pipe record | `MG_Pipe::MGPipeSlotAllocator::FindByLifetimeId` | 0.000 | 0.022 | +0.022 | +| pipe record | `MG_Pipe::MGPipeVertexInputEmitter::EmitVertexBuffers` | 0.000 | 0.021 | +0.021 | +| pipe record | `MG_Pipe::MGPipeFillAccess::CopyField` | 0.000 | 0.018 | +0.018 | +| pipe record | `MG_Pipe::MGPipeProgramEmitter::EmitGlobalConstants` | 0.000 | 0.009 | +0.009 | +| applier | `MG_Record::RecordVerbSink::OnDrawVbo` | 0.000 | 0.009 | +0.009 | +| applier | `MG_Record::{anon}::EmitDrawRecord` | 0.000 | 0.009 | +0.009 | +| applier | `MG_Pipe::MGPipeApplierState::DrawFramebuffer` | 0.000 | 0.005 | +0.005 | +| applier | `MG_Record::{anon}::EmitIndexedDraw` | 0.000 | 0.004 | +0.004 | +| applier | `MG_Record::{anon}::EmitDrawElements` | 0.000 | 0.002 | +0.002 | +| applier | `MG_Pipe::MGPipeApplierState::ReadFramebuffer` | 0.000 | 0.002 | +0.002 | +| backend | `VertexArrayImpl::BackendVertexArrayObject::SyncToBackendFromApplier` | 0.000 | 0.018 | +0.018 | +| backend | `TextureImpl::BackendTextureObject::IsDrawSyncCleanByRecord` | 0.000 | 0.013 | +0.013 | +| backend | `BufferImpl::EnsureBufferResourceForHandle` | 0.000 | 0.012 | +0.012 | +| backend | `eRegistry