Compare commits

...
Author SHA1 Message Date
RISC-1145 ee124018a2 [Fix] (git) Added the ignored item .gradle 2026-08-20 21:46:01 +08:00
RISC-1145 d7ce0c48ef [Fix] (Cmake) Fixed the issue of including the header files of the ASIO library 2026-08-20 21:44:12 +08:00
swung0x48 eadf7bc474 [Fix] (TraceReplay): import iterationRP repair flags in the desktop CLI
Initialize the desktop replay Request from the three iterationRP environment flags before loading MobileGL. Without this, the Request defaults caused the replay core to unset CI's exported flags, leaving all repairs disabled despite the workflow configuration.
2026-08-20 00:42:34 -04:00
swung0x48 00a326ef78 [CI] (MG_IntegrationTest): enable iterationRP repairs in the integration gate
Run the lavapipe Program 203 golden test with the same subgroup scratch, derived topology, and missing-barrier repairs as the iterationRP retrace matrix. This keeps the prerequisite integration job from failing before retrace jobs can start.
2026-08-20 00:03:52 -04:00
swung0x48 c09045fe59 [Fix, Test] (DirectVulkan, ShaderTranspiler, TraceReplay): repair iterationRP's missing reduction barrier
Program 203 reuses prefixSumCache for a second subgroup reduction before every workgroup invocation has consumed the first result. Add a fingerprint-gated SPIR-V pass that inserts the missing Workgroup acquire-release barrier while preserving native subgroup operations.

Keep the repair opt-in behind MOBILEGL_ITERATIONRP_FIX_BARRIER, cover insertion, pass-through, and idempotence, and enable it together with the existing iterationRP subgroup repairs for the matching Linux and Android CI retraces.
2026-08-19 23:42:17 -04:00
swung0x48 5bd8ef01e5 [Test] (MG_IntegrationTest): pin Program 203's complete golden output across desktop and Android
Add a deterministic iterationRP Program 203 fixture that dispatches the original shader and compares every RG16F texel against fixed half-float golden bits. This catches both a wrong exposure result and collateral writes without retaining a serial reference shader.

Make MobileGLIntegrationTest runnable as a standalone Android executable by linking the shared MobileGL library and backing EGL with an AImageReader window; desktop keeps its static-library pbuffer path.

Validation: Adreno 830 passes with 0/262656 mismatches; lavapipe reproduces the current reduction defect with 1/262656 mismatches at the exposure texel.
2026-08-19 22:30:30 -04:00
swung0x48 3181ed2c5a [Fix] (DirectGLES): repair the emulation-guard mask restore and scissor the resolve fallback's staging blit correctly
Final audit round over the DirectGLES scratch/shadow mechanisms; three verified
defects fixed:

- ~ScopedEmulationDrawState restored the APPLICATION's per-buffer colour masks,
  not what SyncRenderState actually pushed: a widened attachment's alpha-off
  doctoring (g_syncedColorMaskAlphaWidenMask) was dropped while the memo still
  claimed it applied, so the next sync early-outed and draws wrote fragment
  alpha into the widened buffer - breaking the stored-alpha==1.0 invariant the
  widen discipline exists to protect. The restore now re-applies the doctoring.
- The same restore loop gated on the core glColorMaski name only, while the sync
  push falls back to glColorMaskiEXT/OES: EXT/OES-only devices were left holding
  buffer 0's mask broadcast across every draw buffer with the shadow recording
  the divergent set (never repaired). The restore now uses the same three-way
  pointer fallback.
- ResolveThenBlit ran its resolve-into-scratch staging blit under the
  application's scissor: a box not covering the scratch-origin rect clipped the
  resolve silently (no GL error), and the second blit then copied stale scratch
  renderbuffer texels into the destination. The staging blit now runs scissor-off
  (shadow-tracked, like ScopedScissorDisable); the caller-visible blit keeps its
  native scissor semantics.
2026-08-19 16:41:31 -04:00
swung0x48 6aed3b08f3 [Fix] (DirectVulkan, GLImpl, MG_State, ShaderTranspiler): second audit round over the remaining memo sites
Six more verified defects from the residual memo/cache mechanisms:

- Program resource cache (DirectVulkan reflection): glShaderStorageBlockBinding
  deliberately does not bump the backend state version, and the SSO pipeline
  composite is unnamed so the by-name in-place patch can never reach its slot -
  the composite kept serving pre-rebind SSBO bindings. The cache now keys on the
  program's block-binding version; a binding-only change re-applies the overrides
  by name instead of re-running spirv-reflect. SetShaderStorageBlockBinding also
  gains the equality bail-out its uniform-block sibling has, so the composite
  mirror's replay stops churning the version every draw.
- LinkProgram's allowVSOnlyPrograms function-static latch never set its own
  initialized flag (dead memo, re-read every call) - and completing it would have
  frozen a per-backend capability across re-initialization. Replaced with a fresh
  per-link read from the null-checked active backend.
- Query object registry: drained at full library teardown (DestroyAllQueryObjects,
  mirroring DestroyAllSyncObjects) - undeleted queries and their backend wrappers
  leaked across Destroy/Initialize cycles, stale ids stayed IsQuery == GL_TRUE in
  the re-initialized library, and a later delete could hand the old backend's
  wrapper to a different backend's DeleteBackendQuery.
- Converted vertex streams and the host-side EBO max-index scan now SyncGpuWrites
  before reading the coherent mapping: XFB/SSBO/image writes are merely recorded
  at that point, so the conversion read pre-write bytes (the restart-index
  rewrite already synced; these two host reads did not).
- Zero-stride converted bindings: both converters rejected stride 0, making the
  factory's documented single-element conversion unreachable and silently
  dropping every draw using such a binding; the stride is substituted with the
  element size for the one-element case.
- DemoteFloat64Pass block relayout: measurement queued into the module eagerly,
  so a mid-struct failure left a half-relaid-out block (compacted offsets before
  the failing member, 64-bit offsets after) while claiming the block was left
  alone. Decoration writes are now collected and committed only when the whole
  block measures successfully.
2026-08-19 16:41:28 -04:00
swung0x48 281467a345 [Fix] (DirectGLES, DirectVulkan, MG_State): close stale-cache, A-B-A and state-leak holes across the memo layers
Audit of every memoization implementation; sixteen verified defects fixed:

DirectGLES backend:
- Broadcast draw-buffer memo: cleared at MakeCurrent/DestroyEGLContext like its
  sibling shadows; its identity+version key is only monotonic within one GLContext,
  so a library teardown + re-init could false-hit on a recycled FBO address.
- Backend texture id re-mint (RecreateBackendTexture) now bumps an attachment
  generation that the SyncCurrentFBO gate and every FBO twin compare, so driver
  FBOs re-attach instead of keeping the deleted texture name; the attachment walk
  re-enters until the generation is quiescent (a walk itself can re-mint).
- Buffer id re-mint (persistent-map adoption, immutable-store retire) now bumps a
  generation the VAO twin sync compares, forcing a full re-emit of the baked
  glVertexAttribPointer / element-array bindings that frontend versions cannot see.
- VAO element-array sync memo: bound-object identity joins the wrapping Uint16
  slot version (same pairing the ResolvedDrawBuffers IBO memo already uses).

DirectVulkan backend:
- EBO slice memo gains the mapped-buffer guard its vertex-binding sibling has: a
  shadow-backed persistent map mutates with no epoch bump, so a hit must decline.
- VkClearManager::MergeClearPayload keeps colorEncoding/colorInt/colorUint with
  the color, so deferred glClearBufferiv/uiv no longer degrade to all-zero float.
- GetOrCreateComputePipeline no longer memoizes a failed creation (same contract
  as PipelineFactory): a transient driver failure was permanently disabling every
  dispatch of that program.
- Explicit-LOD-0 verdict memo keys on the sampling-resolution generation; sampler
  filter/aniso/LOD setters bump only that counter, so the old key served a stale
  verdict (wrong SPIR-V variant) after glTexParameter/glSamplerParameter changes.
- SetupDraw fast path declines instead of re-arming on a moved sampling-resolution
  generation (the snapshot bakes the LOD verdict into its pipeline), and
  recomputes the XfbCapture bit so the first draw after glBeginTransformFeedback
  cannot bind the undecorated variant and silently capture nothing.
- VertexInputStateFactory eviction epoch is drawn from a process-wide source: VAO
  state-pointer memos outlive the factory across renderer recreation, and a fresh
  factory restarting at epoch 1 would dereference a dead factory's entry.
- Cached render passes re-read the live renderbuffer clear payload at begin (the
  clear VALUE is not in the pass hash; the entry's inline snapshot replayed the
  creation-time color and dropped the newly queued one).
- FramebufferObject gains a never-reused lifetime id, keyed into the render-pass
  fast-path memo and the SetupDraw snapshot beside the raw pointer + Uint16
  version pair, which address reuse plus fresh version counts could equal.
- SyncTextureResource's preserved-content image goes through the deferred-release
  ring on both failure paths instead of a synchronous destructor under the GPU.

MG_State frontend:
- Layer-1 compile memo is env-disciplined like layers 2/3: a node computed against
  a dead CompileEnv (e.g. pre-capability fallback limits) no longer answers
  glCompileShader forever once the environment's content changes.
- Pipeline composite cache rebuilds from each stage program's last-link shader
  snapshot (new LinkedShaderRef list + pinned link inputs) instead of the live
  attach list and current compile nodes: post-link glAttachShader/glCompileShader
  must not leak into the composite while the (lifetimeId, linkVersion) signature
  still hits - GL's "as last linked" rule.
2026-08-19 16:41:26 -04:00
swung0x48 c7e36986e7 [Fix] (ShaderTranspiler, DirectVulkan, MG_IntegrationTest): patch both of iterationRP's under-declared subgroup scratch arrays
The previous commit's fingerprint was pinned to one array's incidental
dimensions - workgroup exactly 32x16x1, element exactly vec2, length
exactly 32 - which is the auto-exposure reduction and nothing else. The
pack ships the same idiom twice:

  - auto-exposure:  32x16 (512 invocations), shared vec2 prefixSumCache[32]
  - RTW warp:       1024 invocations,        shared float prefixSumCache[64]

so the warp kept writing 128 subgroups into 64 entries on an 8-lane
device and the retrace stayed bit-identically wrong (ssim 0.027902).

Key the fingerprint on the pack's idiom instead of one array's shape: a
workgroup array of 32-bit floats indexed by gl_SubgroupID, fed by a
subgroup scan, whose declared length is below ceil(invocations / native
width). Three properties keep that a targeted repair rather than a
general array resizer:

  - the index must BE gl_SubgroupID (through OpCopyObject, a signedness
    OpBitcast, or a spill whose every store is that id), so an index
    masked or clamped into range is left alone;
  - the >= 16-lane early-out is retained, so every module on the devices
    the pack was written for passes through byte-identical;
  - growth is certified against maxComputeSharedMemorySize using a
    natural-alignment layout model, and declined outright when a
    declaration cannot be sized, so a patched module can never fail
    pipeline creation where the original would not have.

Verified against the shaders the CI trace actually contains: of the 14
compute modules in the fixture exactly these two change, the other
twelve are byte-identical, and all fourteen pass spirv-val. The
integration scenario grows a second case for the 1024-invocation shape;
both abort with heap corruption when the patch is disabled.
2026-08-19 16:36:06 -04:00
53 changed files with 2215 additions and 245 deletions
+3
View File
@@ -420,6 +420,9 @@ jobs:
MOBILEGL_USE_ANGLE: ${{ matrix.backend.name == 'DirectGLES' && '1' || '0' }}
MOBILEGL_TRACE_ANGLE_VARIANT: ${{ matrix.case.name == 'minecraft-1.21.4-fabric-iris-bliss-in-world' && '90a62123d794' || 'ec889e6ea831' }}
MOBILEGL_MAGMA_R11G11B10F_FALLBACK: ${{ matrix.backend.name == 'DirectVulkan' && '1' || '0' }}
MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH: ${{ matrix.backend.name == 'DirectVulkan' && matrix.case.name == 'minecraft-1.21.4-fabric-iris-iterationrp-in-world' && '1' || '0' }}
MOBILEGL_DERIVE_NUM_SUBGROUPS: ${{ matrix.backend.name == 'DirectVulkan' && matrix.case.name == 'minecraft-1.21.4-fabric-iris-iterationrp-in-world' && '1' || '0' }}
MOBILEGL_ITERATIONRP_FIX_BARRIER: ${{ matrix.backend.name == 'DirectVulkan' && matrix.case.name == 'minecraft-1.21.4-fabric-iris-iterationrp-in-world' && '1' || '0' }}
run: |
apk_file="android-retrace-apks/MobileGL-plugin-trace-release-${GITHUB_SHA}.apk"
test -f "${apk_file}"
+9
View File
@@ -265,6 +265,9 @@ jobs:
# crash stack without burning a CI round on an in-workflow debugger.
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH: "1"
MOBILEGL_DERIVE_NUM_SUBGROUPS: "1"
MOBILEGL_ITERATIONRP_FIX_BARRIER: "1"
run: |
ulimit -c unlimited
sudo sysctl -w kernel.core_pattern='/tmp/core.%e.%p'
@@ -639,6 +642,12 @@ jobs:
if [ '${{ matrix.backend }}' = 'DirectVulkan' ]; then
export MOBILEGL_MAGMA_R11G11B10F_FALLBACK=1
fi
if [ '${{ matrix.backend }}' = 'DirectVulkan' ] \
&& [ '${{ matrix.case }}' = 'minecraft-1.21.4-fabric-iris-iterationrp-in-world' ]; then
export MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH=1
export MOBILEGL_DERIVE_NUM_SUBGROUPS=1
export MOBILEGL_ITERATIONRP_FIX_BARRIER=1
fi
# The blended depth-write quirk auto-enables only on Qualcomm, which no CI
# runner has, so force it on for the OIT case it exists to fix. ForceOn
# bypasses only the vendor gate, so this exercises the real strip on
+1
View File
@@ -27,3 +27,4 @@ MobileGL/MG*/cmake-build*
tools/trace_replay/work/
__pycache__/
*.py[cod]
/.gradle
+9 -1
View File
@@ -285,6 +285,7 @@ set(SOURCE_FILES
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RebaseInstanceIndexPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/ZeroBaseVertexPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DeriveNumSubgroupsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPBarrierPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPSubgroupScratchPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EmulateSubgroupsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/NormalizeRectCoordinatesPass.cpp
@@ -462,7 +463,7 @@ set(MOBILEGL_INCLUDE_DIR
# Header-only submodule: no add_subdirectory, no link target. Only
# MG_Util/Async/ShaderCompilePool.cpp includes it, and it stays behind that file's
# pimpl so no consumer target needs this path.
${CMAKE_SOURCE_DIR}/3rdparty/asio/asio/include
${CMAKE_SOURCE_DIR}/3rdparty/asio/include
)
add_library(${CMAKE_PROJECT_NAME} SHARED
@@ -674,3 +675,10 @@ if (NOT ANDROID)
add_subdirectory(tools/trace_replay)
endif()
endif()
# The integration binary is also useful as a standalone adb-shell executable.
# Android cannot use the desktop-only MobileGL_s target, so its CMake module
# links libMobileGL.so and creates an AImageReader-backed window instead.
if (ANDROID AND MOBILEGL_BUILD_INTEGRATION_TEST)
add_subdirectory(MobileGL/MG_IntegrationTest)
endif()
+4
View File
@@ -100,6 +100,10 @@ namespace MobileGL::MG_Config {
// itself on >= 16-lane devices. Auto is ON; ForceOff replays the pack's bug
// verbatim.
QuirkOverride FixIterationRPSubgroupScratch = QuirkOverride::Auto;
// MOBILEGL_ITERATIONRP_FIX_BARRIER: repair Program 203's missing workgroup
// rendezvous between its two reductions over prefixSumCache. Off by default and
// fingerprint-gated by FixIterationRPBarrierPass when enabled.
Bool IterationRPFixBarrier = false;
// MOBILEGL_DERIVE_NUM_SUBGROUPS: replace compute gl_NumSubgroups loads with
// ceil(workgroup invocations / gl_SubgroupSize) on the NATIVE subgroup path
// (ShaderTranspiler::DeriveNumSubgroupsPass). Auto is ON: GL requires
+1
View File
@@ -171,6 +171,7 @@ namespace MobileGL::MG_ConfigLoader {
features.MagmaEmulateSubgroup = QueryEnvFlag("MOBILEGL_MAGMA_EMULATE_SUBGROUP");
features.FixIterationRPSubgroupScratch =
QueryEnvQuirkOverride("MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH");
features.IterationRPFixBarrier = QueryEnvFlag("MOBILEGL_ITERATIONRP_FIX_BARRIER");
features.DeriveNumSubgroups = QueryEnvQuirkOverride("MOBILEGL_DERIVE_NUM_SUBGROUPS");
features.AdvertiseFp64 = QueryEnvFlag("MOBILEGL_ADVERTISE_FP64");
features.MagmaR11G11B10FFallback = QueryEnvFlag("MOBILEGL_MAGMA_R11G11B10F_FALLBACK");
+6
View File
@@ -15,6 +15,7 @@
#include <MG_Impl/GLImpl/Texture/ProxyTexture.h>
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <MG_Impl/GLImpl/Query/GL_Query.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
@@ -51,6 +52,11 @@ namespace MobileGL {
// before a re-initialized library could pair them with the wrong
// backend's DeleteSync).
MG_Impl::GLImpl::DestroyAllSyncObjects();
// Queries die with their contexts for the same reason, and their registry
// is the same shape of process-global map: drain it here too, while the
// function table can still pair each backend handle with the backend that
// minted it.
MG_Impl::GLImpl::DestroyAllQueryObjects();
MG_Backend::pActiveBackendObject.reset();
MG_State::pGLContext.reset();
MG_State::pEGLContext.reset();
+51 -8
View File
@@ -1431,6 +1431,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_fboSyncedSlotVersions[SizeT(target)] = slotVersion;
g_fboSyncedObjectVersions[SizeT(target)] = objectVersion;
g_fboSyncedObjects[SizeT(target)] = fbo;
g_fboSyncedBackendIdGenerations[SizeT(target)] = g_attachmentBackendIdGeneration;
}
void SyncCurrentFBO() {
@@ -1461,9 +1462,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
const Uint16 slotVersion = slot.GetVersion();
const Uint16 objectVersion = currentFBO ? currentFBO->GetObjectVersion() : 0;
auto* currentPtr = currentFBO.get();
// The backend-id generation joins the triple: a backend texture re-mint
// (RecreateBackendTexture) moves no frontend version, so without it the
// early-out would keep the driver FBO on the deleted texture name.
if (slotVersion == g_fboSyncedSlotVersions[SizeT(target)] &&
objectVersion == g_fboSyncedObjectVersions[SizeT(target)] &&
currentPtr == g_fboSyncedObjects[SizeT(target)]) {
currentPtr == g_fboSyncedObjects[SizeT(target)] &&
g_fboSyncedBackendIdGenerations[SizeT(target)] == g_attachmentBackendIdGeneration) {
lastUpdatedFBO = currentPtr;
continue;
}
@@ -2129,6 +2134,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
static Bool g_broadcastMemoValid = false;
static Uint g_broadcastMemoCount = 1;
// The identity+version key above is only monotonic WITHIN one GLContext: a
// library teardown + re-init frees every FramebufferObject and restarts the
// draw slot's counter at zero, so a recycled FBO address with coinciding
// fresh versions would false-hit. Cleared at the same boundaries as the
// structurally identical SyncCurrentFBO trio (InvalidateFramebufferBindingCache).
void InvalidateBroadcastMemo() {
g_broadcastMemoValid = false;
}
void SyncCurrentProgram(const SharedPtr<MG_State::GLState::ProgramObject>& currentProgram) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
@@ -2312,6 +2326,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
FramebufferImpl::g_fboSyncedSlotVersions[(SizeT)target] = slot.GetVersion();
FramebufferImpl::g_fboSyncedObjectVersions[(SizeT)target] = fbo ? fbo->GetObjectVersion() : 0;
FramebufferImpl::g_fboSyncedObjects[(SizeT)target] = fbo.get();
FramebufferImpl::g_fboSyncedBackendIdGenerations[(SizeT)target] =
FramebufferImpl::g_attachmentBackendIdGeneration;
}
static void BindCurrentProgramWithResources(
@@ -3950,8 +3966,19 @@ namespace MobileGL::MG_Backend::DirectGLES {
Bool resolved = g_GLESFuncs.glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE;
if (resolved) {
DrainBlitErrors();
// A blit is scissored like a draw (the replicate path's guard documents the
// same rule): the application's box would clip this resolve into the
// scratch, and the second blit would then copy never-written scratch texels
// into the destination - silently, since scissor clipping raises no GL
// error. Disable for the staging blit only; the caller-visible blit below
// keeps the blit's native scissor semantics. Tracked via the render-state
// shadow, exactly like ScopedScissorDisable.
const Bool scissorWasEnabled =
(RenderStateImpl::g_syncedRenderStateParameters.ScissorTestEnabledMask & 1u) != 0;
if (scissorWasEnabled) g_GLESFuncs.glDisable(GL_SCISSOR_TEST);
g_GLESFuncs.glBlitFramebuffer(left, bottom, right, top, 0, 0, width, height, GL_COLOR_BUFFER_BIT,
GL_NEAREST);
if (scissorWasEnabled) g_GLESFuncs.glEnable(GL_SCISSOR_TEST);
resolved = g_GLESFuncs.glGetError() == GL_NO_ERROR;
}
if (resolved) {
@@ -4097,13 +4124,25 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
// The per-draw-buffer colour masks are not covered by the non-indexed
// glColorMask above.
for (Uint index = 0; index < MG_State::GLState::FramebufferObject::MAX_DRAW_BUFFERS; ++index) {
const BoolVec4& colorMask = RenderStateImpl::g_syncedRenderStateParameters.ColorMasks[index];
if (g_GLESFuncs.glColorMaski) {
g_GLESFuncs.glColorMaski(index, colorMask.x() ? GL_TRUE : GL_FALSE,
colorMask.y() ? GL_TRUE : GL_FALSE, colorMask.z() ? GL_TRUE : GL_FALSE,
colorMask.w() ? GL_TRUE : GL_FALSE);
// glColorMask above. Restore what the SYNC actually pushed, not the raw
// application masks: a widened attachment's alpha write is forced off by
// SyncRenderState and memoized in g_syncedColorMaskAlphaWidenMask, and the
// next sync early-outs on an unchanged version - restoring the undoctored
// mask here would leave alpha writes enabled on the widened buffer with
// nothing left to repair it. Same three-way pointer fallback as
// SyncRenderState's push: gating on the core name alone left EXT/OES-only
// devices holding buffer 0's mask broadcast across every buffer.
const auto colorMaskiFn = g_GLESFuncs.glColorMaski ? g_GLESFuncs.glColorMaski
: g_GLESFuncs.glColorMaskiEXT ? g_GLESFuncs.glColorMaskiEXT
: g_GLESFuncs.glColorMaskiOES;
if (colorMaskiFn) {
for (Uint index = 0; index < MG_State::GLState::FramebufferObject::MAX_DRAW_BUFFERS; ++index) {
BoolVec4 colorMask = RenderStateImpl::g_syncedRenderStateParameters.ColorMasks[index];
if (index < 32 && (RenderStateImpl::g_syncedColorMaskAlphaWidenMask & (1u << index)) != 0) {
colorMask.w() = false;
}
colorMaskiFn(index, colorMask.x() ? GL_TRUE : GL_FALSE, colorMask.y() ? GL_TRUE : GL_FALSE,
colorMask.z() ? GL_TRUE : GL_FALSE, colorMask.w() ? GL_TRUE : GL_FALSE);
}
}
if (m_pausedTransformFeedback && g_GLESFuncs.glResumeTransformFeedback) {
@@ -8304,6 +8343,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Conservatively drop the redundant-glUseProgram guard: re-issuing one bind
// after a MakeCurrent is cheaper than trusting a possibly-reset context.
PrgramImpl::g_lastUsedBackendProgramId = 0;
// The GLContext becoming current may be a fresh one whose slot versions
// restarted at zero; the broadcast memo's key is only monotonic within one.
PrgramImpl::InvalidateBroadcastMemo();
BufferImpl::InvalidateIndexedBufferBindingCache();
BufferImpl::InvalidatePixelBufferBindingCaches();
FramebufferImpl::InvalidateFramebufferBindingCache();
@@ -8832,6 +8874,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
FramebufferImpl::InvalidateFramebufferBindingCache();
VertexArrayImpl::InvalidateVAOBindingCache();
PixelStoreImpl::InvalidatePackStateCache();
PrgramImpl::InvalidateBroadcastMemo();
// Texture ids belong to the dying context; wrappers destroyed later must
// not glDeleteTextures a recycled name in a successor context.
++g_backendContextGeneration;
+65 -6
View File
@@ -577,6 +577,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// immutable storage, and any prior mutable store is replaced anyway.
if (resource->id != 0) {
NoteBufferIdDeleted(resource->id);
// Driver VAOs may have this id baked into attribute/element bindings
// keyed on frontend versions this re-mint does not move.
++g_bufferBackendIdGeneration;
g_GLESFuncs.glDeleteBuffers(1, &resource->id);
resource->id = 0;
resource->immutableStorage = false;
@@ -833,6 +836,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_bufferMutationEpoch.fetch_add(1, std::memory_order_release);
}
// See the declaration: re-mints of a live resource's driver id. Written only on
// the context thread (both re-mint sites run there), read only by the VAO sync.
Uint64 g_bufferBackendIdGeneration = 0;
void RegisterBufferBackendOps() {
MG_State::GLState::SetBufferBackendOps(&g_glesBufferBackendOps);
// Frontend writes issued while ops were unregistered advanced change
@@ -960,6 +967,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
// here, on the thread that can, and the id is re-minted below.
if (resource->immutableStorage && !resource->persistentMapped && resource->id != 0) {
NoteBufferIdDeleted(resource->id);
// Same as the persistent-map re-mint: the dying id may be baked into
// driver VAO bindings whose frontend versions do not move for this.
++g_bufferBackendIdGeneration;
g_GLESFuncs.glDeleteBuffers(1, &resource->id);
resource->id = 0;
resource->immutableStorage = false;
@@ -1640,8 +1650,24 @@ namespace MobileGL::MG_Backend::DirectGLES {
// PrepareForDraw's BindCurrentVAO establishes the draw binding regardless.
const Uint32 currentConfigVersion = stateVAOObject->GetConfigVersion();
const Uint16 currentIndexBufferVersion = stateVAOObject->GetIndexBufferBindingSlot().GetVersion();
const Bool attributesDirty = !m_hasSyncedConfigVersion || m_syncedConfigVersion != currentConfigVersion;
const Bool indexBufferDirty = currentIndexBufferVersion != m_syncedIndexBufferVersion;
// A live buffer's driver id was re-minted since this twin's last emit
// (persistent-map adoption / immutable-store retire): every baked binding may
// hold the dead id while every frontend version still matches, so force a
// full re-emit. Read once; each buffer re-mints at most once per walk (its
// first EnsureBufferResource this draw), before its id is baked, so stamping
// the entry value at the end is exact - and a stale stamp only costs one
// extra full emit.
const Uint64 currentBufferIdGeneration = BufferImpl::g_bufferBackendIdGeneration;
const Bool bufferIdsRemitted = m_syncedBufferIdGeneration != currentBufferIdGeneration;
const Bool attributesDirty =
bufferIdsRemitted || !m_hasSyncedConfigVersion || m_syncedConfigVersion != currentConfigVersion;
// Identity joins the version compare: the slot version is a wrapping Uint16,
// so a wrapped-back count with a different buffer bound must still read dirty.
const MG_State::GLState::BufferObject* currentIndexBufferObject =
stateVAOObject->GetIndexBufferBindingSlot().GetBoundObject().get();
const Bool indexBufferDirty = bufferIdsRemitted ||
currentIndexBufferVersion != m_syncedIndexBufferVersion ||
currentIndexBufferObject != m_syncedIndexBufferObject;
// The baseInstance shift lives in the attribute offsets the driver already holds, so
// a change of baseInstance has to re-emit the divisor'd arrays even when the frontend
@@ -1675,10 +1701,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
Bool needsSyncFormat = allAttributeVersions[attribIndex].FormatVersion !=
m_syncedAttributeVersions[attribIndex].FormatVersion;
Bool needsSyncBuffer = allAttributeVersions[attribIndex].BufferVersion !=
m_syncedAttributeVersions[attribIndex].BufferVersion;
Bool needsSyncFormat = bufferIdsRemitted || allAttributeVersions[attribIndex].FormatVersion !=
m_syncedAttributeVersions[attribIndex].FormatVersion;
Bool needsSyncBuffer = bufferIdsRemitted || allAttributeVersions[attribIndex].BufferVersion !=
m_syncedAttributeVersions[attribIndex].BufferVersion;
if (!needsSyncFormat && !needsSyncBuffer && !needsSyncBaseInstance) continue;
// Defence in depth. The frontend already declines glVertexAttribLFormat on this
@@ -1798,6 +1824,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (indexBufferSynced) {
m_syncedIndexBufferVersion = currentIndexBufferVersion;
m_syncedIndexBufferObject = currentIndexBufferObject;
}
}
@@ -1809,6 +1836,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (emitAttributes) {
m_syncedFetchBaseInstance = fetchBaseInstance;
}
m_syncedBufferIdGeneration = currentBufferIdGeneration;
}
void BackendVertexArrayObject::SyncClientSideAttributesForDrawArrays(
@@ -1946,6 +1974,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
void BackendTextureObject::RecreateBackendTexture() {
if (m_backendTextureId != 0) {
ScratchFBOImpl::NoteTextureIdDeleted(m_backendTextureId);
// Application FBO twins that attached the dying id memoize on FRONTEND
// attachment versions, which this backend-side re-mint does not move;
// without this bump their driver FBOs would keep the deleted name
// attached forever (see g_attachmentBackendIdGeneration).
++FramebufferImpl::g_attachmentBackendIdGeneration;
if (m_contextGeneration == g_backendContextGeneration) {
g_GLESFuncs.glDeleteTextures(1, &m_backendTextureId);
}
@@ -3517,6 +3550,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_backendReadBuffer = GL_NONE;
std::fill(m_syncedFrontendAttachmentVersions.begin(), m_syncedFrontendAttachmentVersions.end(),
static_cast<Uint16>(~0u));
// Every attachment version is invalidated above, so the next walk re-attaches
// everything regardless; stamp the generation so it does not re-arm twice.
m_syncedBackendIdGeneration = g_attachmentBackendIdGeneration;
}
static Bool SyncAttachmentObject(GLenum glFBOTarget,
@@ -4005,6 +4041,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
// -------------------- Attach texture to backend FBO -----------------------
// A backend texture id was re-minted since this twin's last walk
// (RecreateBackendTexture): any point here may still hold the dead id while
// its frontend attachment version is unchanged, so the memo below would skip
// exactly the attachment that needs repair. Re-arm every point first.
if (m_syncedBackendIdGeneration != g_attachmentBackendIdGeneration) {
std::fill(m_syncedFrontendAttachmentVersions.begin(), m_syncedFrontendAttachmentVersions.end(),
static_cast<Uint16>(~0u));
m_syncedBackendIdGeneration = g_attachmentBackendIdGeneration;
}
const auto& attachments = stateFBOObject->GetAllAttachmentObjects();
const auto& attachmentVersions = stateFBOObject->GetAllFramebufferAttachmentVersions();
for (SizeT i = 0; i < attachments.size(); ++i) {
@@ -4093,6 +4138,18 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
#endif
}
// The walk itself can re-mint an id (SyncAttachmentObject ->
// SyncMipmapsToBackend -> RecreateBackendTexture), invalidating points this
// walk already attached or version-skipped - e.g. one texture attached at two
// points. Re-enter until the generation is quiescent: every pass syncs each
// dirty texture clean, so each repeat finds strictly fewer re-mints and the
// common case (no re-mint) never takes a second pass. The head's draw/read-
// buffer syncs are memoized against their own shadows, so a repeat re-walks
// only the attachments.
if (m_syncedBackendIdGeneration != g_attachmentBackendIdGeneration) {
SyncToBackend(stateFBOObject, asTarget);
}
}
GLenum BackendFramebufferObject::GetBackendAttachmentType(FramebufferAttachmentType frontendAtt) const {
@@ -4119,6 +4176,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedObjectVersions = {0};
Array<MG_State::GLState::FramebufferObject*, SizeT(FramebufferTarget::FramebufferTargetCount)>
g_fboSyncedObjects = {};
Uint64 g_attachmentBackendIdGeneration = 0;
Array<Uint64, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedBackendIdGenerations = {0};
} // namespace FramebufferImpl
namespace ScratchFBOImpl {
+36
View File
@@ -346,6 +346,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
// client-attribute staging buffers): scrub every buffer-binding shadow that
// could false-skip when the name is recycled.
void NoteBufferIdDeleted(Uint id);
// Bumped whenever a live GLESBufferResource's driver id is retired and re-minted
// while its frontend buffer stays alive (persistent-map adoption, immutable-store
// retire). The VAO twins' baked glVertexAttribPointer / element-array bindings
// key on FRONTEND versions, which a backend-side re-mint does not move - without
// this generation the driver VAO would keep fetching through the deleted id (or
// its retained store) forever. Compared and stamped by
// BackendVertexArrayObject::SyncToBackend.
extern Uint64 g_bufferBackendIdGeneration;
// Redundant-bind cache for INDEXED buffer bindings (glBindBufferBase/Range on
// GL_UNIFORM_BUFFER / GL_SHADER_STORAGE_BUFFER): skips the GL call when the
// (id, range) already at that index matches, like the array-buffer/texture/
@@ -465,6 +473,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
Array<Uint, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> m_clientAttributeBufferIds;
Bool m_isInitialized = false;
Uint16 m_syncedIndexBufferVersion = 0;
// Identity of the buffer the version above was stamped against. Raw and never
// dereferenced: the slot version is a wrapping Uint16 (see the ResolvedDrawBuffers
// IBO memo and the packed_pixels postmortem at BindCurrentFBO), so the version
// alone would read a wrapped-back count with a different buffer bound as clean.
const MG_State::GLState::BufferObject* m_syncedIndexBufferObject = nullptr;
// Aggregate gate over the per-attribute walk below: the frontend bumps its config
// version on every per-attribute version bump (the three Bump*Version functions are
// its only writers), so an unchanged config version proves every per-attribute
@@ -480,6 +493,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Kept here because it describes what was last EMITTED, which is what the next sync
// has to correct.
Uint32 m_syncedFetchBaseInstance = 0;
// BufferImpl::g_bufferBackendIdGeneration as of this twin's last emit. A
// mismatch means some live buffer's driver id was re-minted since; the ids
// baked into the driver VAO's attribute/element bindings may be dead even
// though every frontend version matches, so the next sync re-emits them all.
Uint64 m_syncedBufferIdGeneration = 0;
};
extern StateBackendObjectRegistry<MG_State::GLState::VertexArrayObject, BackendVertexArrayObject>
@@ -799,6 +817,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
using FramebufferObject = MG_State::GLState::FramebufferObject;
FramebufferObject::FramebufferAttachmentVersionArray m_syncedFrontendAttachmentVersions = {0};
// g_attachmentBackendIdGeneration as of this twin's last attachment walk. A
// mismatch means some backend texture id was re-minted since, and any of this
// twin's attachment points may still hold the dead id even though the frontend
// attachment versions match - so the walk re-attaches everything first.
Uint64 m_syncedBackendIdGeneration = 0;
};
extern StateBackendObjectRegistry<MG_State::GLState::FramebufferObject, BackendFramebufferObject>
@@ -888,6 +911,19 @@ namespace MobileGL::MG_Backend::DirectGLES {
extern Array<MG_State::GLState::FramebufferObject*, SizeT(FramebufferTarget::FramebufferTargetCount)>
g_fboSyncedObjects;
// Bumped whenever a live backend texture's driver id is re-minted while its
// frontend texture may still be attached to application FBOs
// (BackendTextureObject::RecreateBackendTexture - e.g. a respecify of a texture
// whose backend storage went immutable). The FBO twins' attachment memos key on
// FRONTEND attachment versions, which a backend-side re-mint does not move, so
// the driver FBO would keep the deleted texture name attached forever. The
// SyncCurrentFBO gate compares this generation (below) to re-enter the sync,
// and each twin re-arms its per-attachment memo on a mismatch (SyncToBackend).
extern Uint64 g_attachmentBackendIdGeneration;
// What g_attachmentBackendIdGeneration was when SyncCurrentFBO last stamped each
// target; part of the synced tuple above.
extern Array<Uint64, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedBackendIdGenerations;
// Driver-level READ/DRAW framebuffer-binding shadow. Every backend
// glBindFramebuffer routes through BindFramebufferId so scoped helpers can
// save/restore the current binding without a glGetIntegerv round-trip (that
@@ -69,6 +69,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// slot's ownership unambiguous.
Uint64 programLifetimeId = 0;
Uint32 backendStateVersion = 0;
// glShaderStorageBlockBinding deliberately does NOT bump the backend state
// version, and the pipeline composite is unnamed so the in-place patch in
// DirectVulkan::ShaderStorageBlockBinding can never reach its slot - the
// mirror replay bumps only the program's block-binding version. Without this
// key the composite's slot kept serving the pre-rebind block.binding.
Uint32 blockBindingVersion = 0;
Vector<StorageBlockResource> storageBlocks;
Vector<BufferVariableResource> bufferVariables;
GLint computeWorkGroupSize[3] = {1, 1, 1};
@@ -156,18 +162,33 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto& cache = g_programResourceCaches[program.GetExternalIndex()];
const Uint64 programLifetimeId = program.GetLifetimeId();
const Uint32 backendStateVersion = program.GetBackendStateVersion();
const Uint32 blockBindingVersion = program.GetBlockBindingVersion();
// The lifetime id must match too: a new program that reuses a deleted
// program's name and happens to land on the same backendStateVersion (both
// count from zero) would otherwise be served the dead program's reflection.
if (cache.programLifetimeId == programLifetimeId &&
cache.backendStateVersion == backendStateVersion &&
(!cache.storageBlocks.empty() || !cache.bufferVariables.empty())) {
if (cache.blockBindingVersion != blockBindingVersion) {
// Only the block bindings moved (glShaderStorageBlockBinding, or the
// pipeline composite's mirror replay - neither touches the backend
// state version): the reflection itself is unchanged, so re-apply the
// overrides by name instead of re-running spirv-reflect. Overrides
// only ever accumulate, so a block without one still holds its
// declared binding.
for (auto& block : cache.storageBlocks) {
const Int rebound = program.GetShaderStorageBlockBindingOverride(block.name);
if (rebound >= 0) block.binding = static_cast<Uint32>(rebound);
}
cache.blockBindingVersion = blockBindingVersion;
}
return cache;
}
cache = {};
cache.programLifetimeId = programLifetimeId;
cache.backendStateVersion = backendStateVersion;
cache.blockBindingVersion = blockBindingVersion;
Vector<SpvReflectShaderModule> modules;
Vector<Bool> validModules;
@@ -3190,10 +3190,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
// GL_KHR_shader_subgroup handling (SubgroupSupportPolicy.h). Native subgroup
// operations execute natively; two module repairs keep the GL contract intact
// operations execute natively; module repairs keep the GL contract intact
// around them. The opt-in emulation path replaces them only on devices with no
// subgroup support at all (MOBILEGL_MAGMA_EMULATE_SUBGROUP).
if (shaders[i] && shaders[i]->GetShaderStage() == ShaderStage::Compute) {
// Program 203 broadcasts the first reduction through
// prefixSumCache[0], then lets the second reduction overwrite that
// scratch without first rendezvousing all readers. Patch that exact
// fingerprint before either native or emulated subgroup lowering.
if (m_subgroupPolicy.fixIterationRPBarrier) {
Vector<Uint> patchedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::FixIterationRPBarrierForVulkan(
moduleSpirvs[i], patchedSpirv, enableSpirvValidation)) {
moduleSpirvs[i] = std::move(patchedSpirv);
} else {
MGLOG_E("ProgramFactory: iterationRP barrier patch failed for program %u; "
"Program 203 keeps its shared-scratch race",
program.GetExternalIndex());
}
}
if (m_subgroupPolicy.emulateSubgroups) {
Vector<Uint> emulatedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::EmulateSubgroupsForVulkan(
@@ -3213,6 +3228,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<Uint> patchedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
moduleSpirvs[i], patchedSpirv, m_subgroupPolicy.nativeSubgroupSize,
m_subgroupPolicy.maxComputeSharedMemoryBytes,
enableSpirvValidation)) {
moduleSpirvs[i] = std::move(patchedSpirv);
} else {
@@ -381,6 +381,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct SubgroupLoweringPolicy {
Bool emulateSubgroups = false; // MOBILEGL_MAGMA_EMULATE_SUBGROUP, no-native-support devices
Bool fixIterationRPSubgroupScratch = false; // patch iterationRP's under-declared scratch
Bool fixIterationRPBarrier = false; // repair Program 203's shared-scratch race
Bool deriveNumSubgroups = false; // repair the NumSubgroups builtin
Bool requireFullSubgroups = false; // computeFullSubgroups enabled on the device
Uint32 nativeSubgroupSize = 0;
@@ -287,8 +287,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (m_frameBoundaryCounter - it->second->lastUsedFrameBoundary > kRetireAgeBoundaries) {
it = m_cache.erase(it);
// Invalidate every VAO's state-pointer memo: the erased node's
// address may be reused by a future insert.
++m_evictionEpoch;
// address may be reused by a future insert. Advance through the
// process-wide source so the value stays unique across factory
// instances (see the member comment).
m_evictionEpoch = ++s_evictionEpochSource;
} else {
++it;
}
@@ -125,7 +125,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// construction); a memo is honored only while its recorded epoch
// matches, so an evicted entry can never be dereferenced through a
// stale memo.
Uint64 m_evictionEpoch = 1;
//
// Drawn from a process-wide source, never a per-instance counter: the VAO
// memos outlive this factory (they live on pGLContext's VAOs, the renderer
// is destroyed and recreated on EGL surface release/re-create), so a fresh
// factory restarting at a dead factory's epoch value would honor its
// dangling entry pointers. The constructor takes a value strictly greater
// than anything a predecessor ever stamped, so a dead factory's memo can
// never compare equal here - the same never-reused idiom as the lifetime ids.
// Single-threaded like the rest of the factory (renderer-thread only).
static inline Uint64 s_evictionEpochSource = 0;
Uint64 m_evictionEpoch = ++s_evictionEpochSource;
static inline XXH64_state_t* m_hashState = XXH64_createState();
};
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -166,7 +166,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void VkClearManager::MergeClearPayload(ClearAttachmentPayload& dst, const ClearAttachmentPayload& src) {
dst.mask |= src.mask;
if ((src.mask & GL_COLOR_BUFFER_BIT) != 0) {
// The whole colour story travels together (same rule as
// VkRenderPassManager::QueueRenderbufferClear): a glClearBufferiv/uiv
// payload carries its value in colorInt/colorUint and its branch selector
// in colorEncoding - dropping them here would leave the pending clear
// reading as an all-zero float one.
dst.color = src.color;
dst.colorEncoding = src.colorEncoding;
dst.colorInt = src.colorInt;
dst.colorUint = src.colorUint;
}
if ((src.mask & GL_DEPTH_BUFFER_BIT) != 0) {
dst.depth = src.depth;
@@ -831,6 +831,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// recreated since (texture + renderbuffer image epochs), and no pending clear (which alters
// load ops). Any of these differing forces the full recompute below. Portable to VK 1.1.
if (activeRenderPass != nullptr && m_rpFastValid && m_rpFastFbo == &fbo &&
m_rpFastFboLifetimeId == fbo.GetLifetimeId() &&
m_rpFastFboVersion == fbo.GetObjectVersion() && m_rpFastSwapchainIndex == swapchainImageIndex &&
m_rpFastTexEpoch == m_textureManager.GetTextureImageEpoch() &&
m_rpFastRbEpoch == m_renderbufferImageEpoch &&
@@ -855,6 +856,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// epochs AFTER ComputeHash: its attachment SyncTexture can create an image (bump the epoch).
m_rpFastValid = true;
m_rpFastFbo = &fbo;
m_rpFastFboLifetimeId = fbo.GetLifetimeId();
m_rpFastFboVersion = fbo.GetObjectVersion();
m_rpFastSwapchainIndex = swapchainImageIndex;
m_rpFastTexEpoch = m_textureManager.GetTextureImageEpoch();
@@ -1507,7 +1509,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ClearAttachmentPayload clearPayload{};
SharedPtr<MG_State::GLState::ITextureObject> liveTexture;
if (pending.hasInlinePayload) {
clearPayload = pending.inlinePayload;
// The inline payload was snapshotted when the entry was CREATED, but the
// clear VALUE is not part of the entry's hash - a cache hit with a newer
// glClear would replay the creation-time value and drop the new one (the
// texture path below is immune because it re-reads the live payload).
// Same defense as ClearAttachmentsOnActiveRenderPass: prefer the live
// pending clear, fall back to the snapshot only when none is queued.
if (s_renderPassManager != nullptr &&
s_renderPassManager->GetPendingRenderbufferClear(pending.renderbuffer, clearPayload)) {
if ((clearPayload.mask & GL_COLOR_BUFFER_BIT) != 0 && pending.renderbuffer != nullptr &&
MG_Util::GetBaseInternalFormatComponentCount(pending.renderbuffer->GetInternalFormat()) ==
3) {
// RGB renderbuffers are backed by an RGBA image; the missing alpha reads as 1.
ForceOpaqueClearAlpha(clearPayload);
}
} else {
clearPayload = pending.inlinePayload;
}
} else {
if (pending.key.texture == nullptr ||
!s_clearManager->GetPendingClear(pending.key, clearPayload, liveTexture)) {
@@ -289,6 +289,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// or a pending clear. Portable to Vulkan 1.1 (no dynamic_rendering / imageless FB needed).
Bool m_rpFastValid = false;
const MG_State::GLState::FramebufferObject* m_rpFastFbo = nullptr;
// The FBO's never-reused lifetime id joins the raw pointer + Uint16 version:
// a deleted FBO reallocated at the same address whose fresh setup performed
// the same number of version bumps would otherwise compare equal (both count
// from 0), serving the dead framebuffer's pass to the new object.
Uint64 m_rpFastFboLifetimeId = 0;
Uint16 m_rpFastFboVersion = 0;
Uint32 m_rpFastSwapchainIndex = 0;
Uint64 m_rpFastTexEpoch = 0;
@@ -1993,6 +1993,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
texture.GetExternalIndex(),
MG_Util::ConvertTextureUploadTargetToString(uploadTarget).c_str(),
static_cast<Int>(format), static_cast<Uint32>(imageInfo.usage));
// The preserved image was written by GPU work that may still be in flight
// (preserve requires layout != UNDEFINED); park it on the deferred ring
// like every other destruction path instead of letting the unique_ptr
// destroy it synchronously under the GPU.
if (preservedResource) {
DeferResourceRelease(Move(*preservedResource));
}
return false;
}
}
@@ -2015,6 +2022,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static_cast<Int>(imageInfo.samples), static_cast<Int>(imageInfo.format));
resource.image = VK_NULL_HANDLE;
resource.allocation = nullptr;
// Same as the probe failure above: the preserved live image must go through
// the deferred ring, never a synchronous destructor while frames that
// reference it are still in flight.
if (preservedResource) {
DeferResourceRelease(Move(*preservedResource));
}
return false;
}
++m_textureImageEpoch; // a new attachment image invalidates cached render passes
@@ -3063,6 +3063,7 @@ void main() {
subgroupPolicy.emulateSubgroups = ShouldEmulateSubgroups(m_nativeSubgroupSupported);
subgroupPolicy.fixIterationRPSubgroupScratch =
m_nativeSubgroupSupported && ShouldFixIterationRPSubgroupScratch();
subgroupPolicy.fixIterationRPBarrier = ShouldFixIterationRPBarrier();
subgroupPolicy.deriveNumSubgroups =
m_nativeSubgroupSupported && ShouldDeriveNumSubgroups();
subgroupPolicy.requireFullSubgroups = m_computeFullSubgroupsFeatureEnabled;
@@ -3323,6 +3324,11 @@ void main() {
indexView.indexByteSize > bufferSize - indexView.indexByteOffset) {
return false;
}
// Recorded-but-unexecuted GPU writes (XFB capture, SSBO, storage texel
// buffer) land in the coherent mapping this scan is about to read;
// submit-and-wait first, exactly like the restart-index rewrite does.
// A no-op unless the gpu-write flag is set.
indexBufferShared->SyncGpuWrites();
indexBufferShared->SyncPersistentMappedRange();
indexBytes = indexBufferShared->MappedData() + indexView.indexByteOffset;
} else {
@@ -3560,6 +3566,16 @@ void main() {
const Uint8* sourceData, SizeT sourceStride,
SizeT elementSize, SizeT elementCount,
BufferSlice& outSlice) -> Bool {
// A resolved stride of 0 is the binding model's "never advance" (see the
// factory's layout notes): exactly one element is converted and every vertex
// reads it. That single element is read at offset 0, so the stride is never
// actually used - but both converters reject 0 as a degenerate input, which
// made the documented single-element conversion unreachable and silently
// dropped every draw using such a binding. Substitute the element's own
// size; the caller's cache key still carries the distinct stride 0.
if (sourceStride == 0 && elementCount == 1) {
sourceStride = elementSize;
}
const void* uploadData = nullptr;
VkDeviceSize uploadSize = 0;
switch (conversion) {
@@ -3693,6 +3709,12 @@ void main() {
return false;
}
// A GPU-written source (XFB capture, SSBO, storage texel buffer) has its
// bytes produced by commands that are merely RECORDED at this point, and
// MappedData() aliases the coherent GPU memory they will write into -
// converting now would read pre-write garbage. Submit-and-wait first,
// mirroring the restart-index rewrite; a flag-test no-op otherwise.
sourceBufferShared->SyncGpuWrites();
sourceBufferShared->SyncPersistentMappedRange();
const SizeT availableElementCount =
sourceStride == 0 ? 1 : 1 + (sourceSize - baseOffset - elementSize) / sourceStride;
@@ -3986,7 +4008,7 @@ void main() {
// Skips the per-draw GetBackendResource chase into a cold resource object.
Bool sliceStillValid = false;
const Uint64 frameSerial = m_bufferManager.GetFrameSerial();
if (indexMemo->indexFrameSerial == frameSerial &&
if (indexMemo->indexFrameSerial == frameSerial && !indexMemo->indexBufferMapped &&
indexMemo->indexSliceEpochCounter == m_bufferManager.GetSliceEpochCounter()) {
sliceStillValid = true;
}
@@ -4049,6 +4071,9 @@ void main() {
indexMemo->indexVkBuffer = slice.buffer;
indexMemo->indexSliceOffset = slice.offset;
indexMemo->indexFrameSerial = m_bufferManager.GetFrameSerial();
// A host-mapped EBO can mutate its shadow with no epoch bump; the hit
// path declines on this flag (mirror of anyBufferMapped).
indexMemo->indexBufferMapped = indexBufferShared->IsMapped();
}
}
const VkDeviceSize indexBindOffset =
@@ -5718,6 +5743,22 @@ void main() {
if (program.GetBackendStateVersion() != snap.programVersion) {
return false;
}
// glBegin/EndTransformFeedback moves no key this fast path otherwise observes
// (the design makes capture a compile-option FLAG precisely because no version
// bumps, VulkanRenderer.h's pipeline-memo note) - but the snapshot bakes that
// flag into resolvedTransformFlags and the pipeline. Recompute the one dynamic
// bit (the full path's exact predicate) and decline on a mismatch, or the first
// captured draw after glBeginTransformFeedback would bind the undecorated
// variant and silently capture nothing while the CPU bookkeeping advances.
const Bool wantsXfbCapture = m_transformFeedbackFeatureEnabled &&
MG_State::pGLContext->IsTransformFeedbackActive() &&
program.GetTransformFeedbackVaryingCount() > 0;
const Bool snapHasXfbCapture =
static_cast<Bool>(ProgramFactory::CompileOptionFlags(snap.resolvedTransformFlags) &
ProgramFactory::CompileOptionBit::XfbCapture);
if (wantsXfbCapture != snapHasXfbCapture) {
return false;
}
// A changed VAO does NOT decline: the VAO only feeds the pipeline's vertex
// input state (re-resolved below through the layout-keyed memo, so N VAOs
// sharing one attribute layout share one pipeline) and the vertex/index
@@ -5731,6 +5772,7 @@ void main() {
const auto& drawFbo =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Draw).GetBoundObject();
if (static_cast<const void*>(drawFbo.get()) != snap.drawFbo ||
drawFbo->GetLifetimeId() != snap.drawFboLifetimeId ||
drawFbo->GetObjectVersion() != snap.fboVersion) {
return false;
}
@@ -5914,8 +5956,14 @@ void main() {
}
const Uint64 samplingResolutionGeneration = MG_State::pGLContext->GetSamplingResolutionGeneration();
if (samplingResolutionGeneration != snap.samplingResolutionGeneration) {
snap.samplingResolutionGeneration = samplingResolutionGeneration;
samplerDescriptorsUnchanged = false;
// Decline, not re-arm: snap.resolvedTransformFlags bakes the
// ExplicitLod0Sampling verdict, which reads the effective sampler's
// filters/aniso/LOD range - exactly the state this counter tracks.
// Re-arming the stamp here would rebuild the descriptors but keep the
// stale SPIR-V variant forever (every later draw compares equal again).
// Same shape as the erase-epoch declines above; costs one full-path draw
// per sampler/shape change, and the full path's LOD memo re-probes.
return false;
}
// Everything the full path would re-resolve is provably unchanged - or, for
@@ -6095,11 +6143,18 @@ void main() {
const Uint64 lodProgramLifetimeId = program.GetLifetimeId();
const Uint32 lodProgramVersion = program.GetBackendStateVersion();
const Uint64 lodBindGeneration = MG_State::pGLContext->GetTextureBindGeneration();
// The probe also reads the EFFECTIVE sampler's filters/aniso/LOD range
// (ProgramSamplesOnlySingleLevelTextures), and those setters bump ONLY the
// sampling-resolution generation - not the texture params version the sum
// below covers. Without this key a filter/aniso change would keep serving
// the stale verdict.
const Uint64 lodSamplingGeneration = MG_State::pGLContext->GetSamplingResolutionGeneration();
Bool lodMemoHit = false;
if (m_lastLodDecisionValid && m_lastSampledSetValid &&
m_lastLodProgramLifetimeId == lodProgramLifetimeId &&
m_lastLodProgramVersion == lodProgramVersion &&
m_lastLodBindGeneration == lodBindGeneration && m_lastLodBaseFlags == transformFlags &&
m_lastLodBindGeneration == lodBindGeneration &&
m_lastLodSamplingGeneration == lodSamplingGeneration && m_lastLodBaseFlags == transformFlags &&
m_lastSampledSetProgramLifetimeId == lodProgramLifetimeId &&
m_lastSampledSetProgramVersion == lodProgramVersion &&
m_lastSampledSetBindGeneration == lodBindGeneration) {
@@ -6124,6 +6179,7 @@ void main() {
m_lastLodProgramLifetimeId = lodProgramLifetimeId;
m_lastLodProgramVersion = lodProgramVersion;
m_lastLodBindGeneration = lodBindGeneration;
m_lastLodSamplingGeneration = lodSamplingGeneration;
m_lastLodBaseFlags = baseFlags;
m_lastLodResultFlags = transformFlags;
m_lastLodParamsSum = 0; // filled below once the sampled set is known
@@ -6467,6 +6523,7 @@ void main() {
snap.vaoLifetimeId = vao.GetLifetimeId();
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.drawFbo = drawFbo.get();
snap.drawFboLifetimeId = drawFbo->GetLifetimeId();
snap.fboVersion = drawFbo->GetObjectVersion();
snap.drawFboIsDefault = drawFboIsDefault;
snap.viewportCount = ResolveDrawViewportCount(programObj.writesViewportIndexBuiltin);
@@ -13825,6 +13882,15 @@ void main() {
VkPipeline pipeline = VK_NULL_HANDLE;
VK_VERIFY(vkCreateComputePipelines(m_device, VK_NULL_HANDLE, 1, &pipelineInfo, nullptr, &pipeline),
"GetOrCreateComputePipeline, vkCreateComputePipelines");
// A failed creation must never be memoized - same contract as
// PipelineFactory::GetOrCreatePipeline: caching the null would serve it back
// for the rest of the process and every dispatch of this program would be
// silently skipped. Retrying costs one failed vkCreateComputePipelines per
// dispatch, which is the correct price.
if (pipeline == VK_NULL_HANDLE) {
MGLOG_E("GetOrCreateComputePipeline: vkCreateComputePipelines failed; not caching the failure");
return VK_NULL_HANDLE;
}
m_computePipelines.emplace(programObj.hash, pipeline);
return pipeline;
}
@@ -807,6 +807,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 m_lastLodProgramVersion = 0;
Uint64 m_lastLodBindGeneration = 0;
Uint64 m_lastLodParamsSum = 0;
// Sampling-resolution generation at probe time. The probe reads the effective
// sampler's filters/aniso/LOD range, whose setters bump only this counter -
// the params-version sum above never moves for them.
Uint64 m_lastLodSamplingGeneration = 0;
ProgramFactory::CompileOptionFlags m_lastLodBaseFlags = {};
ProgramFactory::CompileOptionFlags m_lastLodResultFlags = {};
@@ -844,6 +848,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 vaoLifetimeId = 0;
Uint32 vaoConfigVersion = 0;
const void* drawFbo = nullptr;
// Never-reused lifetime id beside the raw pointer + Uint16 version: a
// deleted FBO recycled at the same address with the same fresh version
// count would otherwise compare equal (same ABA as the render-pass
// manager's fast-path memo).
Uint64 drawFboLifetimeId = 0;
Uint16 fboVersion = 0;
Bool drawFboIsDefault = false;
Uint renderStateVersion = 0;
@@ -1067,6 +1076,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBuffer indexVkBuffer = VK_NULL_HANDLE;
VkDeviceSize indexSliceOffset = 0;
Uint64 indexFrameSerial = 0;
// The EBO carried a host map when the slice was recorded - the mirror of
// anyBufferMapped on the vertex half. A shadow-backed (non-adopted)
// persistent map mutates its shadow with no API call and no epoch bump, so
// the one-compare rescue must decline and re-run the acquire, whose
// SyncPersistentMappedRange is the push-down. A map taken AFTER the record
// is already covered: AcquirePersistentMap bumps the slice epoch for the
// request itself, adopted or declined.
Bool indexBufferMapped = false;
// Bound per draw (first bindingCount elements).
VkBuffer vkBuffers[kMaxBindings] = {};
@@ -18,9 +18,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
//
// Native subgroups are the implementation whenever the device has them, whatever
// their width - subgroup operations execute on the hardware paths they were made
// for. Two module-level repairs keep the GL contract intact around them:
// for. Module-level repairs keep the GL contract intact around them:
// - FixIterationRPSubgroupScratchPass patches the one known pack bug: iterationRP's
// prefixSumCache[32], under-declared for sub-16-lane devices (8-lane lavapipe);
// - FixIterationRPBarrierPass repairs Program 203's race between two reductions
// reusing that scratch, when explicitly enabled;
// - DeriveNumSubgroupsPass replaces the one builtin drivers get wrong
// (gl_NumSubgroups) with the value the rest of the topology implies.
// The 32-lane shared-memory emulation (EmulateSubgroupsPass) is a LAST RESORT for
@@ -47,6 +49,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MG_Config::QuirkOverride::ForceOff;
}
inline Bool ShouldFixIterationRPBarrier() {
return MG_Config::Features.IterationRPFixBarrier;
}
inline Bool ShouldDeriveNumSubgroups() {
// Auto is ON: gl_NumSubgroups must agree with the gl_SubgroupID range for the GL
// contract to hold, and the derived ceil() value is the one the renderer can pin
+12 -13
View File
@@ -1057,21 +1057,20 @@ namespace MobileGL::MG_Impl::GLImpl {
return;
}
static Bool allowVSOnlyPrograms;
static Bool initialized = false;
if (!initialized) {
const auto& activeBackendObject = MG_Backend::pActiveBackendObject;
if (!activeBackendObject) {
MGLOG_E_ONCE("activeBackendObject is not initialized!");
return;
}
const auto& rendererInfo = activeBackendObject->GetRendererInfo();
allowVSOnlyPrograms = (Int)rendererInfo.StaticBackendCapability.AllowVSOnlyPrograms;
}
// Read fresh every link, never latched in a static: the capability is
// per-backend, and a latch would freeze it across a backend teardown +
// re-initialization (the previous function-static memo here never even set
// its own initialized flag, so it re-read every call anyway - this makes
// the always-fresh behavior the stated one). A struct-field read per
// glLinkProgram costs nothing.
const auto& activeBackendObject = MG_Backend::pActiveBackendObject;
if (activeBackendObject) {
programObject->SetMaxFragmentOutputColorNumber(activeBackendObject->GetDynamicParameters().MaxDrawBuffers);
if (!activeBackendObject) {
MGLOG_E_ONCE("activeBackendObject is not initialized!");
return;
}
const Bool allowVSOnlyPrograms =
activeBackendObject->GetRendererInfo().StaticBackendCapability.AllowVSOnlyPrograms;
programObject->SetMaxFragmentOutputColorNumber(activeBackendObject->GetDynamicParameters().MaxDrawBuffers);
programObject->Link(!allowVSOnlyPrograms);
}
@@ -648,4 +648,39 @@ namespace MobileGL::MG_Impl::GLImpl {
if (!ValidateQueryStreamIndex(__FUNCTION__, target, index)) return;
GetQueryiv(target, pname, params);
}
void DestroyAllQueryObjects() {
// Detach the registry under the lock, release outside it - same discipline
// (and the same accepted teardown race) as DestroyAllSyncObjects. Without
// this drain, every query the app left undeleted survived full library
// teardown in the process-global registry: the objects and their backend
// wrappers leaked across Destroy/Initialize cycles, stale ids kept
// answering IsQuery == GL_TRUE in the re-initialized library, and a later
// glDeleteQueries could hand the OLD backend's handle to a DIFFERENT
// backend's DeleteBackendQuery, which casts it to the wrong wrapper type.
UnorderedMap<GLuint, QueryObject*> orphans;
{
const std::lock_guard<std::mutex> lock(g_queryObjectsMutex);
orphans.swap(g_liveQueryObjects);
g_activeTimeElapsedQueryId = 0;
g_activePrimitivesWrittenQueryId = 0;
g_activePrimitivesGeneratedQueryId = 0;
g_activeSamplesPassedQueryId = 0;
}
if (orphans.empty()) {
return;
}
// Backend handles must be released by the backend that created them, so
// this runs while the function table is still populated. Both backends'
// DeleteBackendQuery are generation-guarded, so a handle whose renderer
// or ES context is already gone frees only the wrapper.
const auto deleteBackendQuery = MG_Backend::gBackendFunctionsTable.GL.DeleteBackendQuery;
for (const auto& [_, queryObject] : orphans) {
if (deleteBackendQuery && queryObject->backendHandle) {
deleteBackendQuery(queryObject->backendHandle);
}
delete queryObject;
}
MGLOG_D("DestroyAllQueryObjects: reclaimed %zu query object(s) the app left undeleted", orphans.size());
}
} // namespace MobileGL::MG_Impl::GLImpl
+9
View File
@@ -29,4 +29,13 @@ namespace MobileGL::MG_Impl::GLImpl {
void GetQueryBufferObjecti64v(GLuint id, GLuint buffer, GLenum pname, GLintptr offset);
void GetQueryBufferObjectui64v(GLuint id, GLuint buffer, GLenum pname, GLintptr offset);
void QueryCounter(GLuint id, GLenum target);
// Destroys every still-registered query object exactly as DeleteQueries would.
// GL requires queries to die with their context; called only from full library
// teardown (DestroyImpl), where no context survives on any thread, so the
// process-global registry can be drained wholesale. Must run while the backend
// function table is still populated: each backend handle has to be released by
// the backend that created it, never by a later re-initialized one (whose
// DeleteBackendQuery would cast the wrapper to the wrong backend's type).
// Same contract as DestroyAllSyncObjects.
void DestroyAllQueryObjects();
} // namespace MobileGL::MG_Impl::GLImpl
+25 -4
View File
@@ -24,9 +24,14 @@ set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(MGL_ITEST_ROOT ${CMAKE_CURRENT_LIST_DIR}/../..)
# Only meaningful where MobileGL_s exists (i.e. not Android).
if (NOT TARGET MobileGL_s)
message(STATUS "MobileGL_s is not available; skipping the integration test module")
# Desktop links the static implementation directly. Android runs the same
# executable from adb shell and links the shipping shared library instead.
if (ANDROID)
set(MGL_ITEST_MOBILEGL_TARGET MobileGL)
elseif (TARGET MobileGL_s)
set(MGL_ITEST_MOBILEGL_TARGET MobileGL_s)
else()
message(STATUS "No MobileGL library target is available; skipping the integration test module")
return()
endif()
@@ -69,6 +74,7 @@ add_executable(MobileGLIntegrationTest
Scenarios/UniformInitializerScenario.cpp
Scenarios/SwizzleAccessRoutineScenario.cpp
Scenarios/IterationRPFirstReductionScenario.cpp
Scenarios/IterationRPProgram203Scenario.cpp
Scenarios/IterationRPScratchFixScenario.cpp
Scenarios/ProgramPipelineScenario.cpp
Scenarios/ImageLoadStoreSsoScenario.cpp
@@ -94,9 +100,20 @@ target_include_directories(MobileGLIntegrationTest PRIVATE
# gtest, not gtest_main: Main.cpp installs the harness banner itself.
target_link_libraries(MobileGLIntegrationTest PRIVATE
GTest::gtest
MobileGL_s
${MGL_ITEST_MOBILEGL_TARGET}
)
if (ANDROID)
find_library(MGL_ITEST_ANDROID_LIBRARY android REQUIRED)
find_library(MGL_ITEST_LOG_LIBRARY log REQUIRED)
find_library(MGL_ITEST_MEDIANDK_LIBRARY mediandk REQUIRED)
target_link_libraries(MobileGLIntegrationTest PRIVATE
${MGL_ITEST_ANDROID_LIBRARY}
${MGL_ITEST_LOG_LIBRARY}
${MGL_ITEST_MEDIANDK_LIBRARY}
)
endif()
if (MSVC)
# Same reason as MG_Test/Backend/DirectVulkan: the GLES headers declare gl*
# as dllimport on Windows, so the in-library GL entry-point definitions only
@@ -105,6 +122,10 @@ if (MSVC)
endif()
target_compile_definitions(MobileGLIntegrationTest PRIVATE -DNOMINMAX)
if (ANDROID)
return()
endif()
# --- ctest wiring --------------------------------------------------------
# A bare libEGL on a glvnd box resolves to whatever vendor comes first, which is
# usually Mesa/llvmpipe - a software rasteriser silently replacing the GPU under
@@ -18,6 +18,11 @@
#if defined(_WIN32)
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#elif defined(__ANDROID__)
#include <android/hardware_buffer.h>
#include <android/native_window.h>
#include <media/NdkImage.h>
#include <media/NdkImageReader.h>
#endif
// MobileGL's own headers, in the order MobileGL/Includes.h uses them: GL/gl.h
@@ -37,7 +42,7 @@
// the only construction that is actually predictive here: MobileGL ABORTS
// (MOBILEGL_ASSERT -> SIGTRAP) rather than returning an error on an unusable
// platform, so nothing the parent can call in-process is allowed to be wrong.
#if !defined(_WIN32) && !defined(__APPLE__) && __has_include(<sys/wait.h>)
#if !defined(_WIN32) && !defined(__APPLE__) && !defined(__ANDROID__) && __has_include(<sys/wait.h>)
#define MGITEST_HAVE_FORK_PREFLIGHT 1
#include <csignal>
#include <ctime>
@@ -78,12 +83,58 @@ namespace MGITest {
CW_USEDEFAULT, CW_USEDEFAULT, kSurfaceWidth, kSurfaceHeight, nullptr, nullptr,
GetModuleHandleW(nullptr), nullptr);
}
#elif defined(__ANDROID__)
AImageReader* g_imageReader = nullptr;
ANativeWindow* g_imageReaderWindow = nullptr;
void DrainImageReader(void*, AImageReader* reader) {
AImage* image = nullptr;
if (AImageReader_acquireNextImage(reader, &image) == AMEDIA_OK && image != nullptr) {
AImage_delete(image);
}
}
bool CreateImageReaderWindow() {
if (g_imageReaderWindow != nullptr) return true;
constexpr int kMaxImages = 4;
const media_status_t status = AImageReader_newWithUsage(
kSurfaceWidth, kSurfaceHeight, AIMAGE_FORMAT_RGBA_8888,
AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE | AHARDWAREBUFFER_USAGE_GPU_COLOR_OUTPUT,
kMaxImages, &g_imageReader);
if (status != AMEDIA_OK || g_imageReader == nullptr) return false;
AImageReader_ImageListener listener = {nullptr, DrainImageReader};
AImageReader_setImageListener(g_imageReader, &listener);
if (AImageReader_getWindow(g_imageReader, &g_imageReaderWindow) != AMEDIA_OK ||
g_imageReaderWindow == nullptr) {
AImageReader_setImageListener(g_imageReader, nullptr);
AImageReader_delete(g_imageReader);
g_imageReader = nullptr;
return false;
}
ANativeWindow_acquire(g_imageReaderWindow);
return true;
}
void DestroyImageReaderWindow() {
if (g_imageReaderWindow != nullptr) {
ANativeWindow_release(g_imageReaderWindow);
g_imageReaderWindow = nullptr;
}
if (g_imageReader != nullptr) {
AImageReader_setImageListener(g_imageReader, nullptr);
AImageReader_delete(g_imageReader);
g_imageReader = nullptr;
}
}
#endif
bool UseWindowSurface() {
#if defined(_WIN32)
const char* value = std::getenv("MOBILEGL_ITEST_WINDOW_SURFACE");
return value != nullptr && value[0] != '\0' && std::strcmp(value, "0") != 0;
#elif defined(__ANDROID__)
return true;
#else
return false;
#endif
@@ -123,10 +174,10 @@ namespace MGITest {
// callers). surfaceless is the platform with no window-system dependency at
// all; the surface this file then creates is still a pbuffer, which every
// platform supports and which the amendment to this rule requires as the
// fallback shape. DISPLAY/WAYLAND_DISPLAY are cleared as well so that a
// fallback shape on desktop. Android instead supplies an AImageReader
// ANativeWindow. DISPLAY/WAYLAND_DISPLAY are cleared as well so that a
// driver that consults them directly cannot reintroduce the dependency
// behind EGL's back. Desktop-only file: MG_IntegrationTest never builds
// for Android, so no device path is affected.
// behind EGL's back.
void EnsureHeadlessPlatform() {
#if defined(__linux__) && !defined(__ANDROID__)
static bool done = false;
@@ -214,12 +265,21 @@ namespace MGITest {
return 6;
}
surface = eglCreateWindowSurface(display, config, g_testWindow, nullptr);
#elif defined(__ANDROID__)
if (!CreateImageReaderWindow()) {
outReason = "failed to create the Android AImageReader integration-test window";
return 6;
}
surface = eglCreateWindowSurface(display, config, g_imageReaderWindow, nullptr);
#endif
} else {
const EGLint pbufferAttribs[] = {EGL_WIDTH, kSurfaceWidth, EGL_HEIGHT, kSurfaceHeight, EGL_NONE};
surface = eglCreatePbufferSurface(display, config, pbufferAttribs);
}
if (surface == EGL_NO_SURFACE) {
#if defined(__ANDROID__)
DestroyImageReaderWindow();
#endif
outReason = WithEglError(useWindowSurface ? "eglCreateWindowSurface failed"
: "eglCreatePbufferSurface failed");
return 6;
@@ -548,6 +608,8 @@ namespace MGITest {
DestroyWindow(g_testWindow);
g_testWindow = nullptr;
}
#elif defined(__ANDROID__)
DestroyImageReaderWindow();
#endif
m_context = nullptr;
m_surface = nullptr;
@@ -14,11 +14,11 @@
// inspects backend state - both bugs this module pins were invisible to
// state-level assertions and visible only in pixels.
//
// Headless by construction, following MG_Benchmark/Driver/DriverBench.c: an EGL
// context on a PBUFFER surface. No window, no window manager, no human. Unlike
// DriverBench the scenarios do draw to the DEFAULT framebuffer (that is where
// the Y-flip lives) and do call eglSwapBuffers (that is the frame boundary the
// cross-frame scenarios need to be real).
// Headless by construction: desktop uses an EGL pbuffer and Android uses an
// AImageReader-backed ANativeWindow that needs no Activity. No window manager,
// no human. Unlike DriverBench the scenarios do draw to the DEFAULT framebuffer
// (that is where the Y-flip lives) and do call eglSwapBuffers (that is the frame
// boundary the cross-frame scenarios need to be real).
//
// One process is one backend: MOBILEGL_BACKEND_TYPE is latched at
// initialization, so the CMake wiring runs this binary once per backend rather
+7 -1
View File
@@ -31,8 +31,14 @@ namespace {
// silently bound to a workstation's window system is a different
// run from CI's and must be visible as one in the log.
const char* eglPlatform = std::getenv("EGL_PLATFORM");
std::fprintf(stderr, " renderer: %s\n surface: %dx%d pbuffer (headless, EGL_PLATFORM=%s)\n",
#if defined(__ANDROID__)
constexpr const char* surfaceKind = "AImageReader window";
#else
constexpr const char* surfaceKind = "pbuffer";
#endif
std::fprintf(stderr, " renderer: %s\n surface: %dx%d %s (headless, EGL_PLATFORM=%s)\n",
gl.RendererString().c_str(), gl.Width(), gl.Height(),
surfaceKind,
eglPlatform != nullptr ? eglPlatform : "<unset>");
} else if (MGITest::RequireGpu()) {
std::fprintf(stderr,
@@ -0,0 +1,379 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/IterationRPProgram203Scenario.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Full iterationRP Program 203 golden input/output fixture. The original shader
// consumes deterministic complete textures and uniforms, then its complete
// 512x513 RG16F output image is compared against fixed half-float golden bits.
// This catches both a wrong exposure slot and collateral scratch corruption.
#include <array>
#include <bit>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
namespace MGITest {
namespace {
constexpr int kSceneWidth = 854;
constexpr int kSceneHeight = 480;
constexpr int kPixelDataWidth = 512;
constexpr int kPixelDataHeight = 513;
constexpr std::size_t kSceneTexelCount =
static_cast<std::size_t>(kSceneWidth) * kSceneHeight;
constexpr std::size_t kPixelDataTexelCount =
static_cast<std::size_t>(kPixelDataWidth) * kPixelDataHeight;
struct Rgba32f {
float r, g, b, a;
};
struct Rg16 {
std::uint16_t r, g;
};
static_assert(sizeof(Rgba32f) == 16);
static_assert(sizeof(Rg16) == 4);
// Captured from the fixed fixture on Adreno 830. These are the exact
// RG16F storage bits for (0.806640625, 8.2578125), not rounded decimal
// comparisons performed by the test.
constexpr Rg16 kGoldenExposure = {0x3a74u, 0x4821u};
constexpr const char* kCommonSource = R"glsl(
#version 430 core
#extension GL_KHR_shader_subgroup_arithmetic : require
uniform int frameCounter;
uniform float frameTime;
uniform float aspectRatio;
uniform vec2 pixelSize;
uniform float nightVision;
uniform float darknessLightFactor;
uniform sampler2D colortex2;
uniform sampler2D pixelData2D;
layout(rg16f) uniform image2D img_pixelData2D;
float remapSaturate(float x, float e0, float e1) {
return clamp((x - e0) / (e1 - e0), 0.0f, 1.0f);
}
float GetExposureValue(float luminance) {
float aeCurve = 0.65f;
aeCurve = mix(aeCurve, clamp(aeCurve * 1.2f, 0.0f, 1.0f), nightVision);
aeCurve *= remapSaturate(luminance, 2.0f, 1.0f) * 0.6f + 0.4f;
float ae = pow(luminance, -aeCurve);
ae *= 1.0f - min(darknessLightFactor * 2.0f, 0.9f);
ae *= 8.5f;
return ae;
}
)glsl";
constexpr const char* kOriginalMain = R"glsl(
layout(local_size_x = 32, local_size_y = 16) in;
shared vec2 prefixSumCache[32];
void main() {
vec2 texCoord = (vec2(gl_GlobalInvocationID.xy) + 0.5f) * vec2(1.0f / 32.0f, 1.0f / 16.0f);
vec2 sampleCoord = texCoord * (1.0f / 64.0f);
sampleCoord.x += (15.0f / 32.0f) + pixelSize.x * 12.0f;
float tileExposure = dot(textureLod(colortex2, sampleCoord, 0.0f).rgb,
vec3(0.2125f, 0.7154f, 0.0721f));
vec2 sampleLuminance = vec2(tileExposure, 0.0f);
sampleLuminance = subgroupInclusiveAdd(sampleLuminance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint i = 0u; i < loopLength; ++i) {
if ((gl_SubgroupID & (1u << i)) > 0u) {
sampleLuminance += prefixSumCache[(gl_SubgroupID >> i << i) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
}
barrier();
}
if (gl_LocalInvocationIndex == 511u)
prefixSumCache[0] = sampleLuminance / 512.0f;
barrier();
float avg = prefixSumCache[0].x;
vec2 tileDistance = texCoord * 2.0f - 1.0f;
tileDistance.y /= aspectRatio;
float centerDistance = length(tileDistance);
float tileWeight = remapSaturate(centerDistance, 0.6f, 0.4f);
tileExposure = max(7.0E-7f, tileExposure);
float lumaWeight = avg / tileExposure;
lumaWeight = pow(lumaWeight, remapSaturate(avg, 0.02f, 0.001f) * 0.4f + 0.2f);
tileWeight *= lumaWeight;
vec2 sampleExposure = vec2(tileExposure * tileWeight, tileWeight);
sampleExposure = subgroupInclusiveAdd(sampleExposure);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleExposure;
barrier();
for (uint i = 0u; i < loopLength; ++i) {
if ((gl_SubgroupID & (1u << i)) > 0u) {
sampleExposure += prefixSumCache[(gl_SubgroupID >> i << i) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleExposure;
}
barrier();
}
if (gl_LocalInvocationIndex == 511u) {
float avgExposure = max(sampleExposure.x / sampleExposure.y * 29.3f, 1.0E-10f);
avgExposure = log2(avgExposure);
float prevAvgExposure = log2(texelFetch(pixelData2D, ivec2(0, 0), 0).x);
float frameTimeFixed = frameTime + step(frameCounter, 20) * 100.0f;
float exposureTime = clamp(frameTimeFixed * 2.0f, 0.0f, 1.0f);
avgExposure = mix(prevAvgExposure, avgExposure, exposureTime);
avgExposure = max(exp2(avgExposure), 1.0E-5f);
float exposure = GetExposureValue(avgExposure);
imageStore(img_pixelData2D, ivec2(0, 0), vec4(avgExposure, exposure, 0.0f, 0.0f));
}
}
)glsl";
GLuint CompileCompute(const char* mainSource, std::string* error) {
const std::array<const GLchar*, 2> sources = {kCommonSource, mainSource};
const GLuint shader = glCreateShader(GL_COMPUTE_SHADER);
glShaderSource(shader, static_cast<GLsizei>(sources.size()), sources.data(), nullptr);
glCompileShader(shader);
GLint compiled = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
if (compiled != GL_TRUE) {
std::array<char, 8192> log{};
glGetShaderInfoLog(shader, static_cast<GLsizei>(log.size() - 1), nullptr, log.data());
*error = log.data();
glDeleteShader(shader);
return 0;
}
const GLuint program = glCreateProgram();
glAttachShader(program, shader);
glLinkProgram(program);
glDeleteShader(shader);
GLint linked = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &linked);
if (linked != GL_TRUE) {
std::array<char, 8192> log{};
glGetProgramInfoLog(program, static_cast<GLsizei>(log.size() - 1), nullptr, log.data());
*error = log.data();
glDeleteProgram(program);
return 0;
}
return program;
}
std::vector<Rgba32f> MakeSceneInput() {
std::vector<Rgba32f> texels(kSceneTexelCount);
for (int y = 0; y < kSceneHeight; ++y) {
for (int x = 0; x < kSceneWidth; ++x) {
std::uint32_t h = static_cast<std::uint32_t>(x) * 0x9e3779b9u;
h ^= static_cast<std::uint32_t>(y) * 0x85ebca6bu;
h ^= h >> 16u;
h *= 0x7feb352du;
h ^= h >> 15u;
const float noise = static_cast<float>(h & 0xffffu) / 65535.0f;
float base = 0.0002f + noise * 0.075f;
const float dx = static_cast<float>(x - 420);
const float dy = static_cast<float>(y - 4);
base += 0.65f * std::exp(-(dx * dx + dy * dy) / 18.0f);
if (((x + y * 17) % 113) == 0) base += 1.75f;
texels[static_cast<std::size_t>(y) * kSceneWidth + x] =
{base * 0.83f, base * 1.07f, base * 1.31f, 1.0f};
}
}
return texels;
}
std::uint16_t FloatToHalf(float value) {
const std::uint32_t bits = std::bit_cast<std::uint32_t>(value);
const std::uint32_t sign = (bits >> 16u) & 0x8000u;
const std::uint32_t exponent = (bits >> 23u) & 0xffu;
std::uint32_t mantissa = bits & 0x7fffffu;
if (exponent == 0xffu) {
return static_cast<std::uint16_t>(sign | (mantissa == 0 ? 0x7c00u : 0x7e00u));
}
int halfExponent = static_cast<int>(exponent) - 127 + 15;
if (halfExponent >= 31) return static_cast<std::uint16_t>(sign | 0x7c00u);
if (halfExponent <= 0) {
if (halfExponent < -10) return static_cast<std::uint16_t>(sign);
mantissa |= 0x800000u;
const unsigned shift = static_cast<unsigned>(14 - halfExponent);
const std::uint32_t rounded = mantissa + ((1u << (shift - 1u)) - 1u) +
((mantissa >> shift) & 1u);
return static_cast<std::uint16_t>(sign | (rounded >> shift));
}
mantissa += 0xfffu + ((mantissa >> 13u) & 1u);
if ((mantissa & 0x800000u) != 0) {
mantissa = 0;
if (++halfExponent >= 31) return static_cast<std::uint16_t>(sign | 0x7c00u);
}
return static_cast<std::uint16_t>(sign | (static_cast<std::uint32_t>(halfExponent) << 10u) |
(mantissa >> 13u));
}
std::vector<Rg16> MakePixelDataInput() {
std::vector<Rg16> texels(kPixelDataTexelCount);
for (std::size_t i = 0; i < texels.size(); ++i) {
texels[i] = {FloatToHalf(0.35f + static_cast<float>(i % 97u) * 0.0025f),
FloatToHalf(-0.45f + static_cast<float>(i % 89u) * 0.01f)};
}
texels[0] = {FloatToHalf(0.73f), FloatToHalf(1.25f)};
return texels;
}
std::vector<Rg16> MakeGoldenOutput() {
std::vector<Rg16> golden = MakePixelDataInput();
golden[0] = kGoldenExposure;
return golden;
}
GLuint MakeTexture(GLenum internalFormat, GLenum format, GLenum type, int width, int height,
const void* data) {
GLuint texture = 0;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, static_cast<GLint>(internalFormat), width, height, 0, format,
type, data);
return texture;
}
void BindAndDispatch(GLuint program, GLuint scene, GLuint pixelData) {
glUseProgram(program);
glActiveTexture(GL_TEXTURE3);
glBindTexture(GL_TEXTURE_2D, scene);
glUniform1i(glGetUniformLocation(program, "colortex2"), 3);
glActiveTexture(GL_TEXTURE4);
glBindTexture(GL_TEXTURE_2D, pixelData);
glUniform1i(glGetUniformLocation(program, "pixelData2D"), 4);
glBindImageTexture(0, pixelData, 0, GL_FALSE, 0, GL_READ_WRITE, GL_RG16F);
glUniform1i(glGetUniformLocation(program, "img_pixelData2D"), 0);
glUniform1i(glGetUniformLocation(program, "frameCounter"), 100);
glUniform1f(glGetUniformLocation(program, "frameTime"), 1.0f / 60.0f);
glUniform1f(glGetUniformLocation(program, "aspectRatio"),
static_cast<float>(kSceneWidth) / kSceneHeight);
glUniform2f(glGetUniformLocation(program, "pixelSize"), 1.0f / kSceneWidth, 1.0f / kSceneHeight);
glUniform1f(glGetUniformLocation(program, "nightVision"), 0.23f);
glUniform1f(glGetUniformLocation(program, "darknessLightFactor"), 0.08f);
glDispatchCompute(1, 1, 1);
glMemoryBarrier(GL_TEXTURE_UPDATE_BARRIER_BIT | GL_SHADER_IMAGE_ACCESS_BARRIER_BIT);
}
std::vector<Rg16> ReadWholeRgTexture(GLuint texture) {
std::vector<Rg16> texels(kPixelDataTexelCount);
glBindTexture(GL_TEXTURE_2D, texture);
glGetTexImage(GL_TEXTURE_2D, 0, GL_RG, GL_HALF_FLOAT, texels.data());
return texels;
}
class IterationRPProgram203Scenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
GLint stages = 0;
GLint features = 0;
GLint invocations = 0;
glGetIntegerv(GL_SUBGROUP_SUPPORTED_STAGES_KHR, &stages);
glGetIntegerv(GL_SUBGROUP_SUPPORTED_FEATURES_KHR, &features);
glGetIntegerv(GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, &invocations);
const GLbitfield required =
GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR;
if ((static_cast<GLbitfield>(stages) & GL_COMPUTE_SHADER_BIT) == 0 ||
(static_cast<GLbitfield>(features) & required) != required || invocations < 512) {
GTEST_SKIP() << "requires 512-invocation basic+arithmetic compute subgroups";
}
std::string error;
m_original = CompileCompute(kOriginalMain, &error);
ASSERT_NE(m_original, 0u) << "original Program 203: " << error;
const std::vector<Rgba32f> scene = MakeSceneInput();
const std::vector<Rg16> pixelData = MakePixelDataInput();
m_scene = MakeTexture(GL_RGBA16F, GL_RGBA, GL_FLOAT, kSceneWidth, kSceneHeight, scene.data());
m_originalOutput =
MakeTexture(GL_RG16F, GL_RG, GL_HALF_FLOAT, kPixelDataWidth, kPixelDataHeight,
pixelData.data());
ASSERT_EQ(FirstGLError(), static_cast<GLenum>(GL_NO_ERROR));
}
void TearDown() override {
if (!Ready()) return;
const std::array<GLuint, 2> textures = {m_scene, m_originalOutput};
glDeleteTextures(static_cast<GLsizei>(textures.size()), textures.data());
if (m_original != 0) glDeleteProgram(m_original);
}
GLuint m_original = 0;
GLuint m_scene = 0;
GLuint m_originalOutput = 0;
};
} // namespace
TEST_F(IterationRPProgram203Scenario, FixedCompleteInputProducesFixedCompleteGoldenOutput) {
if (!Ready()) return;
BindAndDispatch(m_original, m_scene, m_originalOutput);
glFinish();
const std::vector<Rg16> actual = ReadWholeRgTexture(m_originalOutput);
const std::vector<Rg16> expected = MakeGoldenOutput();
ASSERT_EQ(FirstGLError(), static_cast<GLenum>(GL_NO_ERROR));
std::size_t mismatchTexels = 0;
std::size_t firstMismatch = actual.size();
for (std::size_t i = 0; i < actual.size(); ++i) {
if (actual[i].r != expected[i].r || actual[i].g != expected[i].g) {
if (firstMismatch == actual.size()) firstMismatch = i;
++mismatchTexels;
}
}
RecordProperty("program203_output_width", kPixelDataWidth);
RecordProperty("program203_output_height", kPixelDataHeight);
RecordProperty("program203_compared_texels", static_cast<long long>(actual.size()));
RecordProperty("program203_mismatch_texels", static_cast<long long>(mismatchTexels));
std::cout << "IterationRPProgram203Scenario complete-output actualExposureBits=(0x" << std::hex
<< actual[0].r << ", 0x" << actual[0].g << ") goldenExposureBits=(0x" << expected[0].r
<< ", 0x" << expected[0].g << std::dec << ") mismatches=" << mismatchTexels << '/'
<< actual.size() << '\n';
if (firstMismatch != actual.size()) {
const std::size_t x = firstMismatch % kPixelDataWidth;
const std::size_t y = firstMismatch / kPixelDataWidth;
ADD_FAILURE() << "complete Program 203 output differs at " << x << ',' << y
<< ": actual half bits=(0x" << std::hex << actual[firstMismatch].r << ", 0x"
<< actual[firstMismatch].g << ") golden half bits=(0x" << expected[firstMismatch].r
<< ", 0x" << expected[firstMismatch].g << std::dec << "); mismatched "
<< mismatchTexels << " of " << actual.size() << " texels";
}
EXPECT_EQ(mismatchTexels, 0u);
}
} // namespace MGITest
@@ -8,11 +8,13 @@
//
// Scenario - THE FIXTURE-SHAPED SUBGROUP REDUCTION, ON WHATEVER WIDTH THE DEVICE HAS.
//
// iterationRP's auto-exposure pass declares `shared vec2 prefixSumCache[32]` for a
// 512-invocation workgroup and combines per-subgroup subtotals through
// prefixSumCache[gl_SubgroupID]. The algorithm is width-agnostic; only the static 32
// bakes in "at most 32 subgroups", which every desktop capture satisfies and an 8-lane
// device (lavapipe: 64 subgroups) does not. DirectVulkan patches exactly that with
// iterationRP hard-sizes the scratch its subgroup prefix scans write through
// prefixSumCache[gl_SubgroupID], and ships that idiom twice: the auto-exposure pass
// declares `shared vec2 prefixSumCache[32]` for a 512-invocation workgroup, and the
// RTW importance warp declares `shared float prefixSumCache[64]` for a 1024-invocation
// one. Both algorithms are width-agnostic; only the static lengths bake in "at most 32
// (respectively 64) subgroups", which every desktop capture satisfies and an 8-lane
// device (lavapipe: 64 and 128 subgroups) does not. DirectVulkan patches exactly that with
// FixIterationRPSubgroupScratchPass, growing the array to ceil(invocations / native
// width) on the modules that match the pack's reduction fingerprint.
//
@@ -47,6 +49,10 @@ namespace MGITest {
constexpr std::uint32_t kInvocationCount = 512u;
// sum of 0..511, exactly representable and associativity-proof in fp32.
constexpr float kExpectedTotal = 130816.0f;
// The RTW warp's shape: 1024 invocations into a 64-entry float scratch.
constexpr std::uint32_t kWideInvocationCount = 1024u;
// sum of 0..1023, likewise exact in fp32.
constexpr float kWideExpectedTotal = 523776.0f;
constexpr const char* kComputeSource = R"(#version 430 core
#extension GL_KHR_shader_subgroup_basic : require
@@ -87,6 +93,50 @@ void main() {
}
atomicMax(outputData.maxSubgroupId, gl_SubgroupID);
}
)";
// The RTW importance warp's shape: a plain float scan over 1024 invocations
// into a 64-entry scratch. Same idiom, different dimensions - which is exactly
// what a fingerprint pinned to the exposure pass's shape walks past.
constexpr const char* kWideComputeSource = R"(#version 430 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 1024) in;
layout(std430, binding = 0) buffer Output {
float total;
uint numSubgroups;
uint maxSubgroupId;
} outputData;
shared float prefixSumCache[64];
void main() {
float importance = float(gl_LocalInvocationID.x);
float prefixSum = subgroupInclusiveAdd(importance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
prefixSum += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
}
barrier();
}
if (gl_LocalInvocationID.x == 1023u) {
outputData.total = prefixSum;
outputData.numSubgroups = gl_NumSubgroups;
}
atomicMax(outputData.maxSubgroupId, gl_SubgroupID);
}
)";
struct OutputBlock {
@@ -130,8 +180,7 @@ void main() {
"512-invocation workgroup";
}
m_program = CompileComputeProgram(kComputeSource);
ASSERT_NE(m_program, 0u) << m_buildLog;
m_maxInvocations = static_cast<std::uint32_t>(invocations);
glGenBuffers(1, &m_output);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_output);
@@ -181,7 +230,14 @@ void main() {
return program;
}
OutputBlock Dispatch() {
// Re-poisons the block, compiles the shape under test and runs it once.
OutputBlock Dispatch(const char* source) {
const OutputBlock poison{-1.0f, 0xa5a5a5a5u, 0u};
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_output);
glBufferSubData(GL_SHADER_STORAGE_BUFFER, 0, sizeof(OutputBlock), &poison);
m_program = CompileComputeProgram(source);
EXPECT_NE(m_program, 0u) << m_buildLog;
if (m_program == 0u) return OutputBlock{};
glUseProgram(m_program);
glDispatchCompute(1, 1, 1);
glMemoryBarrier(GL_BUFFER_UPDATE_BARRIER_BIT);
@@ -193,12 +249,13 @@ void main() {
GLuint m_program = 0;
GLuint m_output = 0;
std::uint32_t m_maxInvocations = 0;
std::string m_buildLog;
};
} // namespace
TEST_F(IterationRPScratchFixScenario, FixtureShapedReductionSumsEveryInvocation) {
const OutputBlock block = Dispatch();
const OutputBlock block = Dispatch(kComputeSource);
EXPECT_EQ(glGetError(), static_cast<GLenum>(GL_NO_ERROR));
// The topology diagnostics catch the failure modes by name before the sum does:
@@ -219,4 +276,27 @@ void main() {
<< "workgroup reduction produced " << block.total << " with gl_NumSubgroups="
<< block.numSubgroups;
}
// The pack's second instance of the same bug, and the one that kept the CI
// retrace red after the exposure pass alone was patched.
TEST_F(IterationRPScratchFixScenario, WideFixtureShapedReductionSumsEveryInvocation) {
if (m_maxInvocations < kWideInvocationCount) {
GTEST_SKIP() << "needs a " << kWideInvocationCount << "-invocation workgroup";
}
const OutputBlock block = Dispatch(kWideComputeSource);
EXPECT_EQ(glGetError(), static_cast<GLenum>(GL_NO_ERROR));
ASSERT_NE(block.numSubgroups, 0xa5a5a5a5u) << "invocation 1023 never reached its store";
EXPECT_GE(block.numSubgroups, 1u);
EXPECT_LE(block.numSubgroups, kWideInvocationCount);
EXPECT_LT(block.maxSubgroupId, block.numSubgroups)
<< "gl_SubgroupID exceeds gl_NumSubgroups - the inconsistency "
"DeriveNumSubgroupsPass exists to repair";
// Without the patch an 8-lane device writes prefixSumCache[64..127] out of
// bounds and this comparison fails.
EXPECT_EQ(block.total, kWideExpectedTotal)
<< "workgroup reduction produced " << block.total << " with gl_NumSubgroups="
<< block.numSubgroups;
}
} // namespace MGITest
+11 -3
View File
@@ -646,9 +646,17 @@ namespace MobileGL::MG_State {
for (SizeT stage = 0; stage < ProgramPipelineObject::kGraphicsStageCount; ++stage) {
const auto& stageProgram = pipeline->GetStageProgram(static_cast<ShaderStage>(stage));
if (!stageProgram) continue;
for (const auto& shader : stageProgram->GetAttachedShaders()) {
if (!shader || static_cast<SizeT>(shader->GetShaderStage()) != stage) continue;
composite->AttachShader(shader);
// The stage program contributes the shaders its LAST LINK consumed, never
// its live attach list: per GL 4.6 7.3/7.4 a pipeline stage executes the
// stage program as last linked - glAttachShader and glCompileShader take
// effect only at the program's next link - and neither of those moves the
// link version this cache keys on, so reading live state here would let a
// post-link attach or recompile leak into the composite while the signature
// still hits. The pinned (source, node) makes the composite's Link()
// consume the very inputs that link consumed.
for (const auto& ref : stageProgram->GetLinkedShaderSnapshot()) {
if (!ref.shader || static_cast<SizeT>(ref.shader->GetShaderStage()) != stage) continue;
composite->AttachShaderWithPinnedLinkInput(ref);
anyStage = true;
}
}
@@ -9,7 +9,18 @@
#include "FramebufferObject.h"
#include "MG_Util/Types.h"
#include <atomic>
namespace MobileGL::MG_State::GLState {
// Starts at 1 so a zero-initialized memo slot can never carry a live object's id.
// Atomic for the same reason as the VAO counter: it costs nothing, and a duplicate
// id would resurrect exactly the ABA this id exists to kill.
static std::atomic<Uint64> s_nextFramebufferLifetimeId{1};
Uint64 FramebufferObject::AllocateLifetimeId() {
return s_nextFramebufferLifetimeId.fetch_add(1, std::memory_order_relaxed);
}
// FramebufferAttachmentObject
FramebufferAttachmentObject::FramebufferAttachmentObject(
const SharedPtr<MG_State::GLState::ITextureObject>& texture, TextureUploadTarget textureUploadTarget, Int level,
@@ -148,13 +148,25 @@ namespace MobileGL {
Uint16 GetObjectVersion() const { return m_objectVersion; }
// Globally-unique, never-reused id for THIS object's lifetime - the same
// contract as VertexArrayObject::GetLifetimeId(), and needed for the same
// reason: neither the GL name nor the heap address can tell a
// deleted-and-recreated framebuffer from the original, and m_objectVersion
// starts at 0 for every new object, so a backend memo keyed on
// (pointer, version) alone would silently inherit the dead object's entry
// (see VkRenderPassManager's per-draw fast-path memo).
Uint64 GetLifetimeId() const { return m_lifetimeId; }
Uint GetExternalIndex() const;
Bool IsDefaultFramebuffer() const { return m_externalIndex == 0; }
private:
static Uint64 AllocateLifetimeId();
void BumpAttachmentVersion(FramebufferAttachmentType type);
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = AllocateLifetimeId();
FramebufferAttachmentObjectArray m_attachmentObjects;
FramebufferAttachmentVersionArray m_attachmentVersions;
@@ -393,6 +393,14 @@ namespace MobileGL::MG_State::GLState {
return true;
}
bool ProgramObject::AttachShaderWithPinnedLinkInput(const LinkedShaderRef& ref) {
if (!AttachShader(ref.shader)) {
return false;
}
m_pinnedLinkInputs[ref.shader.get()] = ref;
return true;
}
SizeT ProgramObject::DetachShader(const SharedPtr<ShaderObject>& shader) {
MGLOG_D("DetachShader called for shader %p from ProgramObject %u", shader.get(), m_externalIndex);
if (!ShaderIsAttached(shader)) {
@@ -475,6 +483,8 @@ namespace MobileGL::MG_State::GLState {
AddDefaultFragmentShaderIfMissing();
}
if (m_shaders.empty()) {
// This IS the last link now, and it consumed nothing.
m_linkedShaderSnapshot.clear();
m_artifacts.infoLog = "No shader objects are attached to program.";
MGLOG_E("ProgramObject %u: Link failed - no shader objects attached.", m_externalIndex);
return;
@@ -505,8 +515,19 @@ namespace MobileGL::MG_State::GLState {
Vector<SharedPtr<ShaderCompileTask>> deps;
deps.reserve(m_shaders.size());
task->in.shaders.reserve(m_shaders.size());
m_linkedShaderSnapshot.clear();
m_linkedShaderSnapshot.reserve(m_shaders.size());
for (const auto& shader : m_shaders) {
const SharedPtr<ShaderCompileTask>& node = shader->CompiledNodeForLink();
// A pipeline composite pins the (source, node) each stage program's LAST link
// consumed (AttachShaderWithPinnedLinkInput); an ordinary program takes the
// shader's current ones. Without the pin a post-link recompile would leak a
// shader the stage program never linked into the composite.
SharedPtr<const String> sourcePtr = shader->GetShaderSourcePtr();
SharedPtr<ShaderCompileTask> node = shader->CompiledNodeForLink();
if (const auto pinned = m_pinnedLinkInputs.find(shader.get()); pinned != m_pinnedLinkInputs.end()) {
sourcePtr = pinned->second.source;
node = pinned->second.node;
}
if (node) {
// This link is now an observer of that node's result, and the ShaderObject is
// no longer the only route to it: without the marker, the ordinary
@@ -515,7 +536,10 @@ namespace MobileGL::MG_State::GLState {
node->MarkLinkReferenced();
if (!node->IsTerminal()) deps.push_back(node);
}
task->in.shaders.push_back({shader->GetShaderStage(), shader->GetShaderSourcePtr(), node});
task->in.shaders.push_back({shader->GetShaderStage(), sourcePtr, node});
// What "as last linked" will mean for this program from now on - the pipeline
// composite cache rebuilds from exactly this set (GetProgramForDraw).
m_linkedShaderSnapshot.push_back({shader, sourcePtr, node});
}
// Phase B of the same link: SPIR-V generation, spirv-opt and the global-UBO routing
@@ -60,6 +60,26 @@ namespace MobileGL::MG_State::GLState {
Vector<SharedPtr<ShaderObject>>& GetAttachedShaders();
const Vector<SharedPtr<ShaderObject>>& GetAttachedShaders() const;
// One shader exactly as this program's last Link() consumed it: the object, the
// source snapshot, and the compile node taken at that link's enqueue. GL 4.6 7.3/7.4
// makes this triple - not the live attach list, not the shader's current compile -
// what a program pipeline stage executes ("as last linked"): glAttachShader and
// glCompileShader take effect only at the program's next link, yet neither moves
// m_linkVersion, so anything keyed on the link generation must consume this
// snapshot rather than re-read the live state.
struct LinkedShaderRef {
SharedPtr<ShaderObject> shader;
SharedPtr<const String> source;
SharedPtr<ShaderCompileTask> node;
};
// The last link's full input set; empty when this program has never linked (or its
// last link had no shaders attached). GL-thread-owned, rebuilt in Link()'s prologue.
const Vector<LinkedShaderRef>& GetLinkedShaderSnapshot() const { return m_linkedShaderSnapshot; }
// Pipeline-composite attach: AttachShader plus a pin that makes THIS program's
// Link() consume ref's (source, node) instead of the shader's current ones, so a
// post-link recompile of the stage program's shader cannot leak into the composite.
bool AttachShaderWithPinnedLinkInput(const LinkedShaderRef& ref);
const String& GetInfoLog() const { return Artifacts().infoLog; }
// glCreateShaderProgramv folds the shader's compile log into the program's log, which
// is the only place a caller can read it from once the shader name is gone.
@@ -786,6 +806,13 @@ namespace MobileGL::MG_State::GLState {
// order - and the name is the only coordinate all three agree on. Absent from the map
// means "never rebound", and the shader's declared binding still stands.
void SetShaderStorageBlockBinding(const String& blockName, Uint binding) {
// Equality bail-out like SetUniformBlockBinding's: the pipeline composite
// mirror replays every override each draw, and without this every replay
// would churn m_blockBindingVersion and rebuild whatever keys on it.
const auto it = Artifacts().shaderStorageBlockBinding.find(blockName);
if (it != Artifacts().shaderStorageBlockBinding.end() && it->second == static_cast<Int>(binding)) {
return;
}
Artifacts().shaderStorageBlockBinding[blockName] = static_cast<Int>(binding);
// Deliberately NOT m_backendStateVersion: Espryt's entry point never forces a
// program build off this, and bumping that version would start doing so. The
@@ -1223,6 +1250,13 @@ namespace MobileGL::MG_State::GLState {
// glGetAttachedShaders / GL_ATTACHED_SHADERS / the orphan-shader sweep need no join.
Vector<SharedPtr<ShaderObject>> m_shaders;
Vector<SharedPtr<ShaderObject>> m_detachedShaders; // Store detached shaders and remove on next link
// See GetLinkedShaderSnapshot. Holding the SharedPtrs here is deliberate: the
// "as last linked" set must survive detach-and-delete of its shaders (the
// glCreateShaderProgramv shape) until the next link replaces it.
Vector<LinkedShaderRef> m_linkedShaderSnapshot;
// See AttachShaderWithPinnedLinkInput. Populated only on pipeline composites,
// which never detach, so entries need no removal path. GL-thread-owned.
UnorderedMap<const ShaderObject*, LinkedShaderRef> m_pinnedLinkInputs;
// Link INPUTS (all "take effect at the next link" per GL): glBindAttribLocation,
// glBindFragDataLocation(Indexed), glTransformFeedbackVaryings, and the draw-buffer
@@ -140,6 +140,13 @@ namespace MobileGL::MG_State::GLState {
}
void ShaderObject::Compile() {
// The compile-environment snapshot is taken HERE, on the GL thread, and handed to
// the job. Everything the pipeline needs to know about the device comes through it,
// never through pActiveBackendObject - that is what makes the body movable.
// Hoisted above the memo check because the memo must be env-disciplined too (below).
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env =
MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
// P0b layer 1, as a tri-state: the memo is "the node in m_compiled was built from
// the string m_source still points at". SetShaderSource only swaps that pointer when
// the text actually differs, so this is a pointer compare, and it covers Pending as
@@ -152,7 +159,18 @@ namespace MobileGL::MG_State::GLState {
// ClaimParsedShader's on-demand re-parse needs - a real recompile would have handed
// the next link a fresh parse, the no-op hands it a fresh re-parse of the identical
// source instead. Same result, one parse either way.
if (HasMemoizedCompile()) return;
//
// The environment joins the check (ShaderSourceKey.h's memo-hazard rule: a memo
// must never be handed back under an environment other than the one it was
// computed against). Layers 2 and 3 key on the fingerprint, but this memo sits
// ABOVE both, so without this compare a node computed against a dead environment
// - e.g. a compute shader rejected against the pre-capability fallback limits -
// would keep answering forever while a fresh object with byte-identical source
// compiles fine. The fingerprint is a content hash, so a republish of identical
// capabilities still hits.
if (HasMemoizedCompile() && m_compiled->env != nullptr && m_compiled->env->fingerprint == env->fingerprint) {
return;
}
// Two reasons to stay on this thread, one rule. Without the async flag the whole
// path must be byte-identical to the synchronous implementation, and a cache-less
@@ -168,12 +186,6 @@ namespace MobileGL::MG_State::GLState {
// glMaxShaderCompilerThreadsKHR(0) and a flag-off build both bypass sharing exactly
// as they bypass the pool, and their behaviour stays byte-identical to pre-stage-6.
const Bool runOnPool = m_preprocessCache && MG_Util::Async::AsyncShaderCompileActive();
// The compile-environment snapshot is taken HERE, on the GL thread, and handed to
// the job. Everything the pipeline needs to know about the device comes through it,
// never through pActiveBackendObject - that is what makes the body movable.
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env =
MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
const Uint64 sourceHash = ShaderPreprocessCache::HashSource(*m_source);
// ---- P1 stage 6: adopt an equivalent compile instead of enqueueing a duplicate ----
@@ -4,6 +4,7 @@ add_executable(
SpirvPassTest
SpirvPassTest.cpp
DeriveNumSubgroupsTest.cpp
FixIterationRPBarrierTest.cpp
FixIterationRPSubgroupScratchTest.cpp
EmulateSubgroupsTest.cpp
DemoteFloat64Test.cpp
@@ -0,0 +1,206 @@
// MobileGL - MobileGL/MG_Test/ShaderTranspiler/FixIterationRPBarrierTest.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#define SPV_ENABLE_UTILITY_CODE
#include "glslang/SPIRV/spirv.hpp11"
#undef SPV_ENABLE_UTILITY_CODE
#include "Includes.h"
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/Types.h>
#include <spirv-tools/libspirv.hpp>
#include <map>
#include <vector>
using namespace MobileGL;
using MobileGL::MG_Util::ShaderTranspiler::ShaderCompiler;
namespace {
constexpr SizeT kSpirvHeaderWordCount = 5u;
template <typename Visitor>
void ForEachInstruction(const Vector<Uint32>& spirv, Visitor&& visit) {
for (SizeT offset = kSpirvHeaderWordCount; offset < spirv.size();) {
const Uint32 wordCount = spirv[offset] >> 16u;
if (wordCount == 0u || offset + wordCount > spirv.size()) break;
visit(static_cast<spv::Op>(spirv[offset] & 0xffffu), &spirv[offset], wordCount);
offset += wordCount;
}
}
Vector<Uint32> CompileCompute(const String& source) {
using namespace MobileGL::MG_Util::ShaderTranspiler;
ShaderAttrib shaderAttrib{.shaderType = GL_COMPUTE_SHADER, .sourceStr = source};
auto shaderResult = ShaderCompiler::CompileShader(shaderAttrib);
EXPECT_TRUE(shaderResult) << (shaderResult ? String{} : shaderResult.error().log);
if (!shaderResult) return {};
ProgramAttrib programAttrib{.shaders = {shaderResult.value()}};
auto programResult = ShaderCompiler::LinkProgram(programAttrib);
EXPECT_TRUE(programResult) << (programResult ? String{} : programResult.error().log);
if (!programResult) return {};
ProgramBinaryAttrib binaryAttrib{.shaderTypes = {GL_COMPUTE_SHADER}, .program = *programResult.value()};
auto binaryResult = ShaderCompiler::GetSpirvBinaryFromProgram(binaryAttrib);
EXPECT_TRUE(binaryResult) << (binaryResult ? String{} : binaryResult.error().log);
if (!binaryResult || binaryResult->empty()) return {};
return binaryResult->front();
}
bool Validates(const Vector<Uint32>& spirv) {
spvtools::SpirvTools tools(SPV_ENV_VULKAN_1_1);
tools.SetMessageConsumer(
[](spv_message_level_t, const char*, const spv_position_t& position, const char* message) {
ADD_FAILURE() << "spirv-val at word " << position.index << ": " << message;
});
return tools.Validate(spirv);
}
Uint32 CountOpcode(const Vector<Uint32>& spirv, spv::Op wanted) {
Uint32 count = 0u;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32*, Uint32) {
if (opcode == wanted) ++count;
});
return count;
}
bool HasWorkgroupBarrierImmediatelyBeforeSecondScan(const Vector<Uint32>& spirv) {
std::map<Uint32, Uint32> uintConstants;
spv::Op previous = spv::Op::OpNop;
Uint32 scanCount = 0u;
bool found = false;
const Uint32* previousWords = nullptr;
Uint32 previousWordCount = 0u;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
if (opcode == spv::Op::OpConstant && wordCount >= 4u) {
uintConstants[words[2]] = words[3];
}
if (opcode == spv::Op::OpGroupNonUniformFAdd && wordCount >= 6u &&
static_cast<spv::GroupOperation>(words[4]) == spv::GroupOperation::InclusiveScan && ++scanCount == 2u &&
previous == spv::Op::OpControlBarrier && previousWordCount == 4u) {
found =
uintConstants[previousWords[1]] == static_cast<Uint32>(spv::Scope::Workgroup) &&
uintConstants[previousWords[2]] == static_cast<Uint32>(spv::Scope::Workgroup) &&
uintConstants[previousWords[3]] == (static_cast<Uint32>(spv::MemorySemanticsMask::AcquireRelease) |
static_cast<Uint32>(spv::MemorySemanticsMask::WorkgroupMemory));
}
previous = opcode;
previousWords = words;
previousWordCount = wordCount;
});
return found;
}
constexpr const char* kProgram203RaceShape = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { vec2 value; } outputData;
shared vec2 prefixSumCache[32];
void main() {
vec2 sampleLuminance = subgroupInclusiveAdd(
vec2(float(gl_LocalInvocationIndex), 1.0));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
barrier();
if (gl_LocalInvocationIndex == 511u)
prefixSumCache[0] = sampleLuminance / 512.0;
barrier();
float avg = prefixSumCache[0].x;
float weight = avg > 0.0 ? float(gl_LocalInvocationIndex + 1u) / avg : 0.0;
vec2 sampleExposure = subgroupInclusiveAdd(vec2(weight, 1.0));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleExposure;
barrier();
if (gl_LocalInvocationIndex == 511u)
outputData.value = sampleExposure;
}
)";
constexpr const char* kAlreadySynchronizedShape = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { vec2 value; } outputData;
shared vec2 prefixSumCache[32];
void main() {
vec2 first = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 1.0));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = first;
barrier();
if (gl_LocalInvocationIndex == 511u) prefixSumCache[0] = first / 512.0;
barrier();
float avg = prefixSumCache[0].x;
barrier();
vec2 second = subgroupInclusiveAdd(vec2(avg, 1.0));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = second;
barrier();
if (gl_LocalInvocationIndex == 511u) outputData.value = second;
}
)";
constexpr const char* kForeignSingleScanShape = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { vec2 value; } outputData;
shared vec2 prefixSumCache[32];
void main() {
vec2 value = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 1.0));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = value;
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = prefixSumCache[0];
}
)";
} // namespace
TEST(FixIterationRPBarrierPass, InsertsWorkgroupBarrierBeforeSecondReduction) {
const Vector<Uint32> input = CompileCompute(kProgram203RaceShape);
ASSERT_FALSE(input.empty());
const Uint32 inputBarrierCount = CountOpcode(input, spv::Op::OpControlBarrier);
EXPECT_FALSE(HasWorkgroupBarrierImmediatelyBeforeSecondScan(input));
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPBarrierForVulkan(input, output, true));
EXPECT_EQ(CountOpcode(output, spv::Op::OpControlBarrier), inputBarrierCount + 1u);
EXPECT_TRUE(HasWorkgroupBarrierImmediatelyBeforeSecondScan(output));
EXPECT_TRUE(Validates(output));
}
TEST(FixIterationRPBarrierPass, LeavesOtherShapesByteIdentical) {
const Vector<Uint32> input = CompileCompute(kForeignSingleScanShape);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPBarrierForVulkan(input, output, true));
EXPECT_EQ(output, input);
}
TEST(FixIterationRPBarrierPass, LeavesAnAlreadySynchronizedShaderByteIdentical) {
const Vector<Uint32> input = CompileCompute(kAlreadySynchronizedShape);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPBarrierForVulkan(input, output, true));
EXPECT_EQ(output, input);
}
TEST(FixIterationRPBarrierPass, IsIdempotent) {
const Vector<Uint32> input = CompileCompute(kProgram203RaceShape);
ASSERT_FALSE(input.empty());
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::FixIterationRPBarrierForVulkan(input, once, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::FixIterationRPBarrierForVulkan(once, twice, true));
EXPECT_EQ(twice, once);
}
@@ -109,10 +109,11 @@ namespace {
return tools.Validate(spirv);
}
// iterationRP's reduction fingerprint: 32x16x1, subgroupInclusiveAdd on a
// vec2, and the pack's own 32-entry gl_SubgroupID-indexed scratch. A second,
// plainly indexed array rides along to prove the patch is surgical.
constexpr const char* kIterationRPShapedSource = R"(#version 450 core
// iterationRP's exposure reduction, as the pack ships it: 32x16 (512
// invocations), subgroupInclusiveAdd on a vec2, and a 32-entry
// gl_SubgroupID-indexed scratch. A second, plainly indexed array rides along
// to prove the patch is surgical.
constexpr const char* kExposureShapedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
@@ -141,85 +142,195 @@ void main() {
}
)";
// Same scratch idiom, different workgroup shape - NOT iterationRP, so the
// fingerprint must refuse it even though it would break identically.
constexpr const char* kWrongWorkgroupShapeSource = R"(#version 450 core
// The pack's OTHER instance of the same bug, which a fingerprint pinned to the
// exposure pass's dimensions walks straight past: the RTW importance warp
// scans a plain float across 1024 invocations into a 64-entry scratch.
constexpr const char* kRtwWarpShapedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 64, local_size_y = 8, local_size_z = 1) in;
layout(local_size_x = 1024) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 prefixSumCache[32];
shared float prefixSumCache[64];
void main() {
vec2 v = subgroupInclusiveAdd(vec2(1.0, 0.0));
float importance = float(gl_LocalInvocationID.x) * 0.5;
float prefixSum = subgroupInclusiveAdd(importance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = v;
prefixSumCache[gl_SubgroupID] = prefixSum;
barrier();
if (gl_LocalInvocationIndex == 0u)
outputData.value = prefixSumCache[0].x;
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
prefixSum += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
}
barrier();
}
if (gl_LocalInvocationID.x == 1023u) outputData.value = prefixSumCache[0];
}
)";
// Right shape, but a float scan and a float[32] scratch - not the pack's
// vec2 accumulator signature.
constexpr const char* kWrongElementTypeSource = R"(#version 450 core
// A subgroup scan, but the scratch is indexed per invocation rather than per
// subgroup: its size is not a subgroup-count assumption, so it is not ours.
constexpr const char* kInvocationIndexedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared float cache[32];
shared vec2 perInvocation[32];
void main() {
float v = subgroupInclusiveAdd(float(gl_LocalInvocationIndex));
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
cache[gl_SubgroupID] = v;
vec2 v = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 0.0));
perInvocation[gl_LocalInvocationIndex & 31u] = v;
barrier();
if (gl_LocalInvocationIndex == 0u)
outputData.value = cache[0];
if (gl_LocalInvocationIndex == 0u) outputData.value = perInvocation[0].x;
}
)";
// gl_SubgroupID-indexed, but no subgroup scan feeds it and the element type is
// not the pack's float accumulator.
constexpr const char* kNonFloatScratchSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { uint value; } outputData;
shared uint tally[32];
void main() {
float scan = subgroupInclusiveAdd(float(gl_LocalInvocationIndex));
tally[gl_SubgroupID] = uint(scan);
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = tally[0];
}
)";
// gl_SubgroupID-indexed, but masked into range: the declaration is bounded by
// construction, not a subgroup-count assumption, so it is not the pack's bug.
constexpr const char* kMaskedSubgroupIndexSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 bounded[8];
void main() {
vec2 v = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 0.0));
bounded[gl_SubgroupID & 7u] = v;
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = bounded[0].x;
}
)";
// Neither of the pack's shapes: a small per-subgroup array in a 256-invocation
// workgroup, used to prove the width gate keeps EVERY module inert at >= 16 lanes.
constexpr const char* kForeignShapeSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 256) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared float partial[4];
void main() {
float v = subgroupInclusiveAdd(float(gl_LocalInvocationID.x));
if (gl_SubgroupID < 4u) partial[gl_SubgroupID] = v;
barrier();
if (gl_LocalInvocationID.x == 0u) outputData.value = partial[0];
}
)";
// No subgroup construct at all.
constexpr const char* kSubgroupFreeSource = R"(#version 450 core
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 scratch[32];
void main() {
scratch[gl_LocalInvocationIndex & 31u] = vec2(float(gl_LocalInvocationIndex), 0.0);
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = scratch[0].x;
}
)";
} // namespace
TEST(FixIterationRPSubgroupScratchPass, GrowsThePacksScratchForNarrowSubgroups) {
const Vector<Uint32> input = CompileCompute(kIterationRPShapedSource);
TEST(FixIterationRPSubgroupScratchPass, GrowsTheExposureScratchForNarrowSubgroups) {
const Vector<Uint32> input = CompileCompute(kExposureShapedSource);
ASSERT_FALSE(input.empty());
ASSERT_EQ(WorkgroupArrayLengths(input), (std::vector<Uint32>{4u, 32u}));
// lavapipe: 8-lane subgroups over 512 invocations need 64 entries; the
// plainly indexed neighbour must keep its 4.
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, true));
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(WorkgroupArrayLengths(output), (std::vector<Uint32>{4u, 64u}));
EXPECT_TRUE(Validates(output));
}
TEST(FixIterationRPSubgroupScratchPass, LeavesPackWidthAssumptionsAloneOnWideDevices) {
const Vector<Uint32> input = CompileCompute(kIterationRPShapedSource);
// The regression the CI retrace caught: patching only the exposure pass leaves
// this one writing 128 subgroups into 64 entries, and the frame stays wrong.
TEST(FixIterationRPSubgroupScratchPass, GrowsTheRtwWarpScratchForNarrowSubgroups) {
const Vector<Uint32> input = CompileCompute(kRtwWarpShapedSource);
ASSERT_FALSE(input.empty());
ASSERT_EQ(WorkgroupArrayLengths(input), (std::vector<Uint32>{64u}));
// >= 16 lanes means at most 32 subgroups: the pack's declared size holds and
// the module must pass through byte-identical.
for (const Uint32 nativeSize : {16u, 32u, 64u, 128u}) {
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, nativeSize, true));
EXPECT_EQ(output, input) << "native width " << nativeSize;
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(WorkgroupArrayLengths(output), (std::vector<Uint32>{128u}));
EXPECT_TRUE(Validates(output));
}
TEST(FixIterationRPSubgroupScratchPass, LeavesPackWidthAssumptionsAloneOnWideDevices) {
// Both shapes are sized for >= 16 lanes (512/16 = 32, 1024/16 = 64), so on
// every such device the modules must pass through byte-identical.
for (const char* source : {kExposureShapedSource, kRtwWarpShapedSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
for (const Uint32 nativeSize : {16u, 32u, 64u, 128u}) {
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
input, output, nativeSize, 32768u, true));
EXPECT_EQ(output, input) << "native width " << nativeSize;
}
}
}
TEST(FixIterationRPSubgroupScratchPass, RefusesAModuleOutsideTheFingerprint) {
for (const char* source : {kWrongWorkgroupShapeSource, kWrongElementTypeSource}) {
TEST(FixIterationRPSubgroupScratchPass, RefusesAModuleOutsideTheIdiom) {
for (const char* source : {kInvocationIndexedSource, kNonFloatScratchSource,
kSubgroupFreeSource, kMaskedSubgroupIndexSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, true));
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(output, input);
}
}
TEST(FixIterationRPSubgroupScratchPass, IsIdempotent) {
const Vector<Uint32> input = CompileCompute(kIterationRPShapedSource);
// A grown array that would not fit the device's shared memory is left alone:
// a pipeline that cannot be created is worse than the pack's own overrun.
// The width gate is what keeps unrelated shaders untouched on the devices the pack
// was written for: at >= 16 lanes nothing is rewritten, whatever its shape.
TEST(FixIterationRPSubgroupScratchPass, LeavesEveryModuleAloneAtThePacksAssumedWidth) {
const Vector<Uint32> input = CompileCompute(kForeignShapeSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, once, 8u, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(once, twice, 8u, true));
EXPECT_EQ(twice, once);
for (const Uint32 nativeSize : {16u, 32u, 64u}) {
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
input, output, nativeSize, 32768u, true));
EXPECT_EQ(output, input) << "native width " << nativeSize;
}
}
TEST(FixIterationRPSubgroupScratchPass, RefusesGrowthThatWouldNotFitSharedMemory) {
const Vector<Uint32> input = CompileCompute(kRtwWarpShapedSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 256u, true));
EXPECT_EQ(output, input);
}
TEST(FixIterationRPSubgroupScratchPass, IsIdempotent) {
for (const char* source : {kExposureShapedSource, kRtwWarpShapedSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, once, 8u, 32768u, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(once, twice, 8u, 32768u, true));
EXPECT_EQ(twice, once);
}
}
@@ -27,6 +27,7 @@
#include "SpirvPasses/ZeroBaseVertexPass.h"
#include "SpirvPasses/DeriveNumSubgroupsPass.h"
#include "SpirvPasses/EmulateSubgroupsPass.h"
#include "SpirvPasses/FixIterationRPBarrierPass.h"
#include "SpirvPasses/FixIterationRPSubgroupScratchPass.h"
#include "SpirvPasses/NormalizeRectCoordinatesPass.h"
#include "SpirvPasses/Lower1DArrayImagesPass.h"
@@ -912,17 +913,29 @@ namespace MobileGL {
bool ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
const Vector<Uint32>& inputBinary, Vector<uint32_t>& outputBinary,
const Uint32 nativeSubgroupSize, const bool enableSpirvValidation) {
const Uint32 nativeSubgroupSize, const Uint32 maxWorkgroupScratchBytes,
const bool enableSpirvValidation) {
using namespace spvtools;
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(
FixIterationRPSubgroupScratchPass::CreateFixIterationRPSubgroupScratchPass(
nativeSubgroupSize));
nativeSubgroupSize, maxWorkgroupScratchBytes));
return RunOptimizerChecked("FixIterationRPSubgroupScratchForVulkan", optimizer,
inputBinary, outputBinary, true, enableSpirvValidation);
}
bool ShaderCompiler::FixIterationRPBarrierForVulkan(
const Vector<Uint32>& inputBinary, Vector<uint32_t>& outputBinary,
const bool enableSpirvValidation) {
using namespace spvtools;
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(FixIterationRPBarrierPass::CreateFixIterationRPBarrierPass());
return RunOptimizerChecked("FixIterationRPBarrierForVulkan", optimizer,
inputBinary, outputBinary, true, enableSpirvValidation);
}
bool ShaderCompiler::DecoratePositionInvariantForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary, const bool enableSpirvValidation) {
using namespace spvtools;
@@ -167,13 +167,24 @@ namespace MobileGL {
Vector<uint32_t>& outputBinary,
Uint32 maxWorkgroupScratchBytes,
bool enableSpirvValidation = false);
// Patches iterationRP's under-declared prefixSumCache[32] on sub-16-lane
// devices, fingerprint-gated to that pack's reduction; every other module
// passes through byte-identical. See FixIterationRPSubgroupScratchPass.
// Grows iterationRP's under-declared gl_SubgroupID-indexed scratch to the
// subgroup count the device actually partitions into, fingerprint-gated to
// that pack's reduction idiom; every other module - and every device whose
// width the pack already assumed - passes through byte-identical.
// maxWorkgroupScratchBytes bounds the growth (pass the device's
// maxComputeSharedMemorySize; 0 falls back to the 16384-byte Vulkan
// minimum). See FixIterationRPSubgroupScratchPass.
static bool FixIterationRPSubgroupScratchForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
Uint32 nativeSubgroupSize,
Uint32 maxWorkgroupScratchBytes,
bool enableSpirvValidation = false);
// Inserts the missing workgroup rendezvous between Program 203's two
// prefixSumCache reductions. Fingerprint-gated to the iterationRP shape;
// unrelated and already-repaired modules pass through byte-identical.
static bool FixIterationRPBarrierForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
bool enableSpirvValidation = false);
// Re-declares 64-bit float vertex inputs as their 32-bit unsigned word pair
// (double -> uvec2, dvec2 -> uvec4) and bitcasts them back to double at entry, so no
// VK_FORMAT_R64*_SFLOAT is needed - lavapipe advertises none of them for vertex
@@ -91,9 +91,15 @@ namespace MobileGL {
BlockRelayout(IRContext* irContext, Bool std140)
: m_irContext(irContext), m_std140(std140) {}
// Size and alignment of `typeId`, applying every stride decoration it implies
// on the way down. Zero size means "not a type this layout knows how to
// describe"; the caller then leaves the block alone rather than guessing.
// Size and alignment of `typeId`, QUEUING every offset/stride decoration it
// implies on the way down. Zero size means "not a type this layout knows how
// to describe"; the caller then leaves the block alone rather than guessing.
// The queue is what makes that fallback honest: measurement must be
// side-effect-free until it is known to succeed, or a mid-struct failure
// would leave the block half-relaid-out - members before the failing one at
// compacted 32-bit offsets, members after it at the original 64-bit ones, a
// layout matching neither convention. Commit() flushes the queue and is
// called only on a successful Measure of the whole block.
struct Extent {
Uint32 size = 0;
Uint32 alignment = 0;
@@ -108,7 +114,29 @@ namespace MobileGL {
return extent;
}
// Flushes the decoration writes a successful Measure queued. Call exactly
// once, only when Measure returned a non-zero size; a failed measurement's
// queue dies with this per-block instance, leaving the module untouched.
void Commit() {
for (const PendingDecoration& pending : m_pendingWrites) {
if (pending.member) {
ApplyMemberDecoration(pending.targetId, pending.memberIndex, pending.decoration,
pending.value);
} else {
ApplyTypeDecoration(pending.targetId, pending.decoration, pending.value);
}
}
m_pendingWrites.clear();
}
private:
struct PendingDecoration {
Bool member = false;
Uint32 targetId = 0;
Uint32 memberIndex = 0;
spv::Decoration decoration = spv::Decoration::Offset;
Uint32 value = 0;
};
Extent MeasureUncached(Uint32 typeId) {
const Instruction* type = m_irContext->get_def_use_mgr()->GetDef(typeId);
if (type == nullptr) return {};
@@ -204,7 +232,17 @@ namespace MobileGL {
return length->GetSingleWordInOperand(0);
}
// Queue-only during measurement; the module is mutated in Commit().
void SetTypeDecoration(Uint32 targetId, spv::Decoration decoration, Uint32 value) {
m_pendingWrites.push_back({false, targetId, 0, decoration, value});
}
void SetMemberDecoration(Uint32 structId, Uint32 member, spv::Decoration decoration,
Uint32 value) {
m_pendingWrites.push_back({true, structId, member, decoration, value});
}
void ApplyTypeDecoration(Uint32 targetId, spv::Decoration decoration, Uint32 value) {
for (Instruction& annotation : m_irContext->annotations()) {
if (annotation.opcode() != spv::Op::OpDecorate) continue;
if (annotation.GetSingleWordInOperand(0) != targetId) continue;
@@ -216,8 +254,8 @@ namespace MobileGL {
}
}
void SetMemberDecoration(Uint32 structId, Uint32 member, spv::Decoration decoration,
Uint32 value) {
void ApplyMemberDecoration(Uint32 structId, Uint32 member, spv::Decoration decoration,
Uint32 value) {
for (Instruction& annotation : m_irContext->annotations()) {
if (annotation.opcode() != spv::Op::OpMemberDecorate) continue;
if (annotation.GetSingleWordInOperand(0) != structId) continue;
@@ -233,6 +271,7 @@ namespace MobileGL {
IRContext* m_irContext = nullptr;
Bool m_std140 = true;
std::unordered_map<Uint32, Extent> m_extents;
std::vector<PendingDecoration> m_pendingWrites;
};
} // namespace
@@ -529,9 +568,12 @@ namespace MobileGL {
// A member shape the layout rules here do not describe. Leaving the block
// at its 64-bit offsets keeps the module valid for Vulkan; SPIRV-Cross will
// decline it for ESSL, which is the same outcome as before the demotion.
// Nothing was written: Measure only queues, and the queue dies here.
MGLOG_D("DemoteFloat64Pass: block %%%u contains a member this pass cannot lay "
"out; its 64-bit offsets are left in place",
blockType->result_id());
} else {
relayout.Commit();
}
}
@@ -0,0 +1,232 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPBarrierPass.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "FixIterationRPBarrierPass.h"
#include "spirv.hpp"
#include "source/opt/constants.h"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/util/make_unique.h"
#include <vector>
namespace MobileGL::MG_Util::ShaderTranspiler {
namespace {
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
using spvtools::opt::Operand;
const Instruction* RootVariable(IRContext* context, uint32_t pointerId) {
const Instruction* def = context->get_def_use_mgr()->GetDef(pointerId);
while (def != nullptr) {
switch (def->opcode()) {
case spv::Op::OpVariable:
return def;
case spv::Op::OpAccessChain:
case spv::Op::OpInBoundsAccessChain:
case spv::Op::OpCopyObject:
def = context->get_def_use_mgr()->GetDef(def->GetSingleWordInOperand(0));
break;
default:
return nullptr;
}
}
return nullptr;
}
bool IsUintConstant(IRContext* context, uint32_t id, uint32_t wanted) {
const Instruction* def = context->get_def_use_mgr()->GetDef(id);
return def != nullptr && def->opcode() == spv::Op::OpConstant && def->NumInOperands() == 1u &&
def->GetSingleWordInOperand(0) == wanted;
}
bool IsZeroElementPointer(IRContext* context, uint32_t pointerId, const Instruction** root) {
const Instruction* pointer = context->get_def_use_mgr()->GetDef(pointerId);
if (pointer == nullptr ||
(pointer->opcode() != spv::Op::OpAccessChain && pointer->opcode() != spv::Op::OpInBoundsAccessChain) ||
pointer->NumInOperands() < 2u) {
return false;
}
for (uint32_t i = 1u; i < pointer->NumInOperands(); ++i) {
if (!IsUintConstant(context, pointer->GetSingleWordInOperand(i), 0u)) return false;
}
*root = RootVariable(context, pointerId);
return *root != nullptr;
}
bool IsWorkgroupVec2Array(IRContext* context, const Instruction* variable) {
if (variable == nullptr || variable->opcode() != spv::Op::OpVariable || variable->NumInOperands() < 1u ||
static_cast<spv::StorageClass>(variable->GetSingleWordInOperand(0)) != spv::StorageClass::Workgroup) {
return false;
}
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* pointerType = defUseMgr->GetDef(variable->type_id());
if (pointerType == nullptr || pointerType->opcode() != spv::Op::OpTypePointer ||
pointerType->NumInOperands() < 2u) {
return false;
}
const Instruction* arrayType = defUseMgr->GetDef(pointerType->GetSingleWordInOperand(1));
if (arrayType == nullptr || arrayType->opcode() != spv::Op::OpTypeArray ||
arrayType->NumInOperands() < 2u) {
return false;
}
const Instruction* length = defUseMgr->GetDef(arrayType->GetSingleWordInOperand(1));
if (length == nullptr || length->opcode() != spv::Op::OpConstant || length->NumInOperands() != 1u) {
return false;
}
const uint32_t arrayLength = length->GetSingleWordInOperand(0);
if (arrayLength < 32u || arrayLength > 512u) return false;
const Instruction* vectorType = defUseMgr->GetDef(arrayType->GetSingleWordInOperand(0));
if (vectorType == nullptr || vectorType->opcode() != spv::Op::OpTypeVector ||
vectorType->NumInOperands() < 2u || vectorType->GetSingleWordInOperand(1) != 2u) {
return false;
}
const Instruction* scalarType = defUseMgr->GetDef(vectorType->GetSingleWordInOperand(0));
return scalarType != nullptr && scalarType->opcode() == spv::Op::OpTypeFloat &&
scalarType->NumInOperands() == 1u && scalarType->GetSingleWordInOperand(0) == 32u;
}
bool IsVec2FloatInclusiveAdd(IRContext* context, const Instruction* inst) {
if (inst->opcode() != spv::Op::OpGroupNonUniformFAdd || inst->NumInOperands() < 3u ||
static_cast<spv::GroupOperation>(inst->GetSingleWordInOperand(1)) !=
spv::GroupOperation::InclusiveScan) {
return false;
}
const Instruction* vectorType = context->get_def_use_mgr()->GetDef(inst->type_id());
if (vectorType == nullptr || vectorType->opcode() != spv::Op::OpTypeVector ||
vectorType->NumInOperands() < 2u || vectorType->GetSingleWordInOperand(1) != 2u) {
return false;
}
const Instruction* scalarType = context->get_def_use_mgr()->GetDef(vectorType->GetSingleWordInOperand(0));
return scalarType != nullptr && scalarType->opcode() == spv::Op::OpTypeFloat &&
scalarType->NumInOperands() == 1u && scalarType->GetSingleWordInOperand(0) == 32u;
}
bool HasProgram203LocalSize(IRContext* context) {
for (const Instruction& entryPoint : context->module()->entry_points()) {
if (static_cast<spv::ExecutionModel>(entryPoint.GetSingleWordInOperand(0)) !=
spv::ExecutionModel::GLCompute) {
return false;
}
}
for (const Instruction& mode : context->module()->execution_modes()) {
if (mode.opcode() == spv::Op::OpExecutionMode && mode.NumInOperands() >= 5u &&
static_cast<spv::ExecutionMode>(mode.GetSingleWordInOperand(1)) == spv::ExecutionMode::LocalSize) {
return mode.GetSingleWordInOperand(2) == 32u && mode.GetSingleWordInOperand(3) == 16u &&
mode.GetSingleWordInOperand(4) == 1u;
}
}
return false;
}
bool IsStoreToRoot(IRContext* context, const Instruction* inst, const Instruction* root) {
return inst->opcode() == spv::Op::OpStore && inst->NumInOperands() >= 2u &&
RootVariable(context, inst->GetSingleWordInOperand(0)) == root;
}
} // namespace
spvtools::opt::Pass::Status FixIterationRPBarrierPass::Process() {
auto* irContext = context();
if (!HasProgram203LocalSize(irContext)) return Status::SuccessWithoutChange;
for (auto& function : *irContext->module()) {
std::vector<Instruction*> instructions;
std::vector<size_t> scans;
for (auto& block : function) {
for (auto& inst : block) {
if (IsVec2FloatInclusiveAdd(irContext, &inst)) scans.push_back(instructions.size());
instructions.push_back(&inst);
}
}
// Program 203 has exactly two vec2 inclusive adds: the luminance reduction
// and the weighted-exposure reduction. More or fewer is not our fingerprint.
if (scans.size() != 2u) continue;
const size_t firstScan = scans[0];
const size_t secondScan = scans[1];
const Instruction* scratch = nullptr;
size_t averageLoad = instructions.size();
for (size_t i = firstScan + 1u; i < secondScan; ++i) {
Instruction* inst = instructions[i];
if (inst->opcode() != spv::Op::OpLoad || inst->NumInOperands() < 1u) continue;
const Instruction* root = nullptr;
if (!IsZeroElementPointer(irContext, inst->GetSingleWordInOperand(0), &root) ||
!IsWorkgroupVec2Array(irContext, root)) {
continue;
}
// The broadcast is read as prefixSumCache[0].x, hence a scalar load.
const Instruction* type = irContext->get_def_use_mgr()->GetDef(inst->type_id());
if (type == nullptr || type->opcode() != spv::Op::OpTypeFloat || type->NumInOperands() != 1u ||
type->GetSingleWordInOperand(0) != 32u) {
continue;
}
scratch = root;
averageLoad = i;
break;
}
if (scratch == nullptr) continue;
bool sawZeroBroadcastStore = false;
bool sawPublishBarrier = false;
for (size_t i = firstScan + 1u; i < averageLoad; ++i) {
const Instruction* root = nullptr;
if (instructions[i]->opcode() == spv::Op::OpStore &&
IsZeroElementPointer(irContext, instructions[i]->GetSingleWordInOperand(0), &root) &&
root == scratch) {
sawZeroBroadcastStore = true;
} else if (sawZeroBroadcastStore && instructions[i]->opcode() == spv::Op::OpControlBarrier) {
sawPublishBarrier = true;
}
}
if (!sawZeroBroadcastStore || !sawPublishBarrier) continue;
bool alreadySynchronized = false;
for (size_t i = averageLoad + 1u; i < secondScan; ++i) {
if (instructions[i]->opcode() == spv::Op::OpControlBarrier) {
alreadySynchronized = true;
break;
}
}
if (alreadySynchronized) return Status::SuccessWithoutChange;
bool secondPhaseReusesScratch = false;
for (size_t i = secondScan + 1u; i < instructions.size(); ++i) {
if (IsStoreToRoot(irContext, instructions[i], scratch)) {
secondPhaseReusesScratch = true;
break;
}
}
if (!secondPhaseReusesScratch) continue;
auto* constantMgr = irContext->get_constant_mgr();
const uint32_t scopeId = constantMgr->GetUIntConstId(static_cast<uint32_t>(spv::Scope::Workgroup));
const uint32_t semanticsId =
constantMgr->GetUIntConstId(static_cast<uint32_t>(spv::MemorySemanticsMask::AcquireRelease) |
static_cast<uint32_t>(spv::MemorySemanticsMask::WorkgroupMemory));
if (scopeId == 0u || semanticsId == 0u) return Status::Failure;
instructions[secondScan]->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpControlBarrier, 0u, 0u,
Instruction::OperandList{Operand{SPV_OPERAND_TYPE_ID, {scopeId}},
Operand{SPV_OPERAND_TYPE_ID, {scopeId}},
Operand{SPV_OPERAND_TYPE_ID, {semanticsId}}}));
irContext->InvalidateAnalysesExceptFor(IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
return Status::SuccessWithoutChange;
}
spvtools::Optimizer::PassToken FixIterationRPBarrierPass::CreateFixIterationRPBarrierPass() {
return spvtools::Optimizer::PassToken(spvtools::MakeUnique<FixIterationRPBarrierPass>());
}
} // namespace MobileGL::MG_Util::ShaderTranspiler
@@ -0,0 +1,28 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPBarrierPass.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
namespace MobileGL::MG_Util::ShaderTranspiler {
// Repairs iterationRP Program 203's missing workgroup rendezvous between two
// reductions that reuse prefixSumCache. The first phase broadcasts its result
// through prefixSumCache[0], but the second phase may overwrite that element before
// every invocation has read it. The pass fingerprints that exact two-scan,
// 512-invocation shape and inserts one Workgroup control barrier immediately before
// the second scan. Unrelated modules and already-repaired modules are byte-identical.
class FixIterationRPBarrierPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "fix-iterationrp-barrier"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateFixIterationRPBarrierPass();
};
} // namespace MobileGL::MG_Util::ShaderTranspiler
@@ -26,13 +26,13 @@ namespace MobileGL {
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
// iterationRP's reduction fingerprint, spelled out.
constexpr uint32_t kIterationRPLocalSizeX = 32u;
constexpr uint32_t kIterationRPLocalSizeY = 16u;
constexpr uint32_t kIterationRPLocalSizeZ = 1u;
constexpr uint32_t kIterationRPInvocations =
kIterationRPLocalSizeX * kIterationRPLocalSizeY * kIterationRPLocalSizeZ;
constexpr uint32_t kIterationRPScratchLength = 32u;
// The Vulkan minimum for maxComputeSharedMemorySize, used when the caller
// could not tell us the device's real limit.
constexpr uint32_t kMinimumSharedMemoryBytes = 16384u;
// The narrowest subgroup width iterationRP's declarations are sized for.
// At or above it both shipped shapes fit and nothing may be rewritten.
constexpr uint32_t kPackAssumedSubgroupWidth = 16u;
Instruction* FindBuiltinDefinition(IRContext* context, spv::BuiltIn builtin) {
auto* defUseMgr = context->get_def_use_mgr();
@@ -73,18 +73,121 @@ namespace MobileGL {
return nullptr;
}
// vec2 of 32-bit float - the type of iterationRP's luminance/exposure
// accumulator and of its prefixSumCache entries.
bool IsVec2Float32(IRContext* context, uint32_t typeId) {
const Instruction* type = context->get_def_use_mgr()->GetDef(typeId);
if (type == nullptr || type->opcode() != spv::Op::OpTypeVector ||
type->GetSingleWordInOperand(1) != 2u) {
// A 32-bit float scalar or vector - the shape of every accumulator the
// pack runs through its scans (float, vec2 and vec4 all appear). Returns
// the component count, or 0 for anything else.
uint32_t Float32ComponentCount(IRContext* context, uint32_t typeId) {
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* type = defUseMgr->GetDef(typeId);
if (type == nullptr) return 0u;
uint32_t components = 1u;
if (type->opcode() == spv::Op::OpTypeVector) {
components = type->GetSingleWordInOperand(1);
if (components < 2u || components > 4u) return 0u;
type = defUseMgr->GetDef(type->GetSingleWordInOperand(0));
if (type == nullptr) return 0u;
}
if (type->opcode() != spv::Op::OpTypeFloat ||
type->GetSingleWordInOperand(0) != 32u) {
return 0u;
}
return components;
}
uint32_t RoundUp(uint32_t value, uint32_t alignment) {
return alignment == 0u ? value : ((value + alignment - 1u) / alignment) * alignment;
}
// Size AND alignment of a workgroup-storage type. Drivers lay shared
// memory out at natural alignment and the limit
// (VUID-RuntimeSpirv-Workgroup-06530) counts the padding that produces,
// so a model that sums unpadded sizes would under-count exactly where the
// budget check matters. Returns false for anything not modelled here,
// which the caller answers by declining to grow at all rather than by
// certifying growth against a total it knows is an underestimate.
bool WorkgroupTypeLayout(IRContext* context, uint32_t typeId, uint32_t* size,
uint32_t* alignment, uint32_t depth = 0u) {
if (depth > 8u) return false;
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* type = defUseMgr->GetDef(typeId);
if (type == nullptr) return false;
switch (type->opcode()) {
case spv::Op::OpTypeBool:
*size = 4u;
*alignment = 4u;
return true;
case spv::Op::OpTypeInt:
case spv::Op::OpTypeFloat: {
const uint32_t width = type->GetSingleWordInOperand(0) / 8u;
if (width == 0u) return false;
*size = width;
*alignment = width;
return true;
}
case spv::Op::OpTypeVector: {
uint32_t componentSize = 0u;
uint32_t componentAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(0),
&componentSize, &componentAlignment, depth + 1u)) {
return false;
}
const uint32_t components = type->GetSingleWordInOperand(1);
if (components < 2u || components > 4u) return false;
*size = componentSize * components;
// A three-component vector aligns like a four-component one.
*alignment = componentSize * (components == 3u ? 4u : components);
return true;
}
case spv::Op::OpTypeMatrix:
case spv::Op::OpTypeArray: {
uint32_t elementSize = 0u;
uint32_t elementAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(0), &elementSize,
&elementAlignment, depth + 1u)) {
return false;
}
uint32_t count = 0u;
if (type->opcode() == spv::Op::OpTypeMatrix) {
count = type->GetSingleWordInOperand(1);
} else {
const Instruction* length =
defUseMgr->GetDef(type->GetSingleWordInOperand(1));
if (length == nullptr || length->opcode() != spv::Op::OpConstant) {
return false; // spec-constant length: not sizeable here
}
count = length->GetSingleWordInOperand(0);
}
*size = RoundUp(elementSize, elementAlignment) * count;
*alignment = elementAlignment;
return true;
}
case spv::Op::OpTypeStruct: {
uint32_t offset = 0u;
uint32_t structAlignment = 1u;
for (uint32_t i = 0; i < type->NumInOperands(); ++i) {
uint32_t memberSize = 0u;
uint32_t memberAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(i),
&memberSize, &memberAlignment, depth + 1u)) {
return false;
}
offset = RoundUp(offset, memberAlignment) + memberSize;
if (memberAlignment > structAlignment) structAlignment = memberAlignment;
}
*size = RoundUp(offset, structAlignment);
*alignment = structAlignment;
return true;
}
default:
return false;
}
const Instruction* component =
context->get_def_use_mgr()->GetDef(type->GetSingleWordInOperand(0));
return component != nullptr && component->opcode() == spv::Op::OpTypeFloat &&
component->GetSingleWordInOperand(0) == 32u;
}
// The group operations the pack's prefix scans use.
bool IsScanOrReduce(spv::GroupOperation operation) {
return operation == spv::GroupOperation::Reduce ||
operation == spv::GroupOperation::InclusiveScan ||
operation == spv::GroupOperation::ExclusiveScan;
}
} // namespace
@@ -92,13 +195,15 @@ namespace MobileGL {
auto* irContext = context();
auto* defUseMgr = irContext->get_def_use_mgr();
// A device whose native width already satisfies the pack's assumption
// (>= 16 lanes -> at most 32 subgroups) needs no patch at all.
if (m_nativeSubgroupSize == 0u || m_nativeSubgroupSize >= 16u) {
// Without a known device width there is no topology to compare against;
// and a width the pack already assumed needs no patch at all. Both of
// iterationRP's shapes are sized for >= 16 lanes (512/16 = 32 entries,
// 1024/16 = 64), so every module on such a device - the pack's or anyone
// else's - must pass through byte-identical. The per-array length test
// further down is the second gate, not a replacement for this one.
if (m_nativeSubgroupSize == 0u || m_nativeSubgroupSize >= kPackAssumedSubgroupWidth) {
return Status::SuccessWithoutChange;
}
const uint32_t requiredLength =
(kIterationRPInvocations + m_nativeSubgroupSize - 1u) / m_nativeSubgroupSize;
for (const Instruction& entryPoint : irContext->module()->entry_points()) {
if (static_cast<spv::ExecutionModel>(entryPoint.GetSingleWordInOperand(0)) !=
@@ -107,7 +212,8 @@ namespace MobileGL {
}
}
// Fingerprint 1: the pack's exposure-pass workgroup shape, 32x16x1.
// Fingerprint 1: a literal workgroup size, so the subgroup count the
// dispatch actually partitions into is known here.
const auto resolveUintConstant = [&](uint32_t id, uint32_t* value) {
const Instruction* def = defUseMgr->GetDef(id);
if (def == nullptr || def->opcode() != spv::Op::OpConstant) return false;
@@ -139,26 +245,40 @@ namespace MobileGL {
}
}
}
if (!haveLocalSize || localSize[0] != kIterationRPLocalSizeX ||
localSize[1] != kIterationRPLocalSizeY || localSize[2] != kIterationRPLocalSizeZ) {
if (!haveLocalSize || localSize[0] == 0u || localSize[1] == 0u || localSize[2] == 0u) {
return Status::SuccessWithoutChange;
}
const uint64_t totalInvocations =
static_cast<uint64_t>(localSize[0]) * localSize[1] * localSize[2];
if (totalInvocations == 0u || totalInvocations > (1u << 20)) {
return Status::SuccessWithoutChange;
}
const uint32_t requiredLength = static_cast<uint32_t>(
(totalInvocations + m_nativeSubgroupSize - 1u) / m_nativeSubgroupSize);
// Fingerprint 2: the reduction's subgroupInclusiveAdd on a vec2.
bool sawVec2InclusiveAdd = false;
// Fingerprint 2: a subgroup scan over a 32-bit float value - the pack's
// prefix-sum reduction, and the reason its scratch is indexed per subgroup.
bool sawFloatSubgroupScan = false;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.opcode() == spv::Op::OpGroupNonUniformFAdd &&
static_cast<spv::GroupOperation>(inst.GetSingleWordInOperand(1)) ==
spv::GroupOperation::InclusiveScan &&
IsVec2Float32(irContext, inst.type_id())) {
sawVec2InclusiveAdd = true;
if (inst.opcode() != spv::Op::OpGroupNonUniformFAdd &&
inst.opcode() != spv::Op::OpGroupNonUniformFMin &&
inst.opcode() != spv::Op::OpGroupNonUniformFMax) {
continue;
}
if (inst.NumInOperands() < 2) continue;
if (!IsScanOrReduce(static_cast<spv::GroupOperation>(
inst.GetSingleWordInOperand(1)))) {
continue;
}
if (Float32ComponentCount(irContext, inst.type_id()) != 0u) {
sawFloatSubgroupScan = true;
}
}
}
}
if (!sawVec2InclusiveAdd) {
if (!sawFloatSubgroupScan) {
return Status::SuccessWithoutChange;
}
@@ -171,60 +291,89 @@ namespace MobileGL {
}
const uint32_t subgroupIdVariableId = subgroupIdVariable->result_id();
// Conservative taint walk over values, and through Function/Private
// temporaries by variable (glslang routinely spills builtin loads into
// locals before they reach an index expression). Over-tainting is safe:
// the candidate filter below still demands the exact vec2[32] shape.
std::unordered_map<uint32_t, bool> valueTainted; // result id -> tainted
std::unordered_map<uint32_t, bool> variableTainted; // variable id -> tainted
bool changedTaint = true;
while (changedTaint) {
changedTaint = false;
// The pack indexes its scratch with gl_SubgroupID ITSELF, so only values
// that ARE that id qualify - not everything computed from it. An index
// that is masked or clamped (cache[gl_SubgroupID & 3u]) is bounded by
// construction and is none of this pass's business; accepting it would
// turn a targeted repair into a general array resizer. Identity survives
// OpCopyObject, a signedness OpBitcast, and the Function/Private spill
// glslang emits for a builtin load - and nothing else. A spill variable
// counts only when EVERY store into it is the id.
std::unordered_map<uint32_t, bool> subgroupIdValues; // result id IS the id
std::unordered_map<uint32_t, bool> subgroupIdVariables; // spill holding only it
bool changedIdentity = true;
while (changedIdentity) {
changedIdentity = false;
std::unordered_map<uint32_t, uint32_t> totalStores;
std::unordered_map<uint32_t, uint32_t> idStores;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
const spv::Op opcode = inst.opcode();
if (opcode == spv::Op::OpStore) {
if (!valueTainted.count(inst.GetSingleWordInOperand(1))) continue;
const Instruction* root =
RootVariable(irContext, inst.GetSingleWordInOperand(0));
if (root == nullptr) continue;
if (!variableTainted.count(root->result_id())) {
variableTainted[root->result_id()] = true;
changedTaint = true;
}
if (inst.opcode() != spv::Op::OpStore) continue;
const uint32_t pointerId = inst.GetSingleWordInOperand(0);
const Instruction* target = defUseMgr->GetDef(pointerId);
if (target == nullptr || target->opcode() != spv::Op::OpVariable) {
continue;
}
if (inst.result_id() == 0 || valueTainted.count(inst.result_id())) {
const auto storageClass = static_cast<spv::StorageClass>(
target->GetSingleWordInOperand(0));
if (storageClass != spv::StorageClass::Function &&
storageClass != spv::StorageClass::Private) {
continue;
}
bool tainted = false;
if (opcode == spv::Op::OpLoad) {
totalStores[pointerId] += 1u;
if (subgroupIdValues.count(inst.GetSingleWordInOperand(1))) {
idStores[pointerId] += 1u;
}
}
}
}
for (const auto& entry : totalStores) {
if (entry.second != 0u && idStores[entry.first] == entry.second &&
!subgroupIdVariables.count(entry.first)) {
subgroupIdVariables[entry.first] = true;
changedIdentity = true;
}
}
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.result_id() == 0 ||
subgroupIdValues.count(inst.result_id())) {
continue;
}
bool isSubgroupId = false;
switch (inst.opcode()) {
case spv::Op::OpLoad: {
const uint32_t pointerId = inst.GetSingleWordInOperand(0);
if (pointerId == subgroupIdVariableId) tainted = true;
const Instruction* root = RootVariable(irContext, pointerId);
if (root != nullptr && variableTainted.count(root->result_id())) {
tainted = true;
}
} else {
inst.ForEachInId([&](const uint32_t* operandId) {
if (valueTainted.count(*operandId)) tainted = true;
});
isSubgroupId = pointerId == subgroupIdVariableId ||
subgroupIdVariables.count(pointerId) != 0u;
break;
}
if (tainted) {
valueTainted[inst.result_id()] = true;
changedTaint = true;
case spv::Op::OpCopyObject:
case spv::Op::OpBitcast:
isSubgroupId =
subgroupIdValues.count(inst.GetSingleWordInOperand(0)) != 0u;
break;
default:
break;
}
if (isSubgroupId) {
subgroupIdValues[inst.result_id()] = true;
changedIdentity = true;
}
}
}
}
}
if (valueTainted.empty()) {
if (subgroupIdValues.empty()) {
return Status::SuccessWithoutChange;
}
// Fingerprint 3: workgroup-shared vec2[32] arrays whose access-chain
// index depends on gl_SubgroupID - the under-declared prefixSumCache.
// Fingerprint 3: workgroup-shared float arrays indexed by gl_SubgroupID
// itself - the under-declared prefixSumCache.
std::map<uint32_t, Instruction*> candidates;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
@@ -234,7 +383,7 @@ namespace MobileGL {
continue;
}
if (inst.NumInOperands() < 2) continue;
if (!valueTainted.count(inst.GetSingleWordInOperand(1))) continue;
if (!subgroupIdValues.count(inst.GetSingleWordInOperand(1))) continue;
Instruction* baseVariable =
defUseMgr->GetDef(inst.GetSingleWordInOperand(0));
if (baseVariable == nullptr ||
@@ -252,7 +401,16 @@ namespace MobileGL {
return Status::SuccessWithoutChange;
}
bool changedModule = false;
// Everything that survives the filter, with the bytes each grown array
// will need. Nothing is mutated until the whole set fits the device's
// shared-memory budget, so a module is never left half-grown.
struct Growth {
Instruction* variable = nullptr;
uint32_t elementTypeId = 0;
uint32_t lengthTypeId = 0;
uint32_t addedBytes = 0;
};
std::vector<Growth> growths;
for (auto& entry : candidates) {
Instruction* variable = entry.second;
@@ -290,18 +448,70 @@ namespace MobileGL {
continue;
}
const uint32_t elementTypeId = arrayType->GetSingleWordInOperand(0);
if (!IsVec2Float32(irContext, elementTypeId)) continue;
const uint32_t components = Float32ComponentCount(irContext, elementTypeId);
if (components == 0u) continue;
const Instruction* lengthConstant =
defUseMgr->GetDef(arrayType->GetSingleWordInOperand(1));
uint32_t currentLength = 0;
if (lengthConstant == nullptr ||
lengthConstant->opcode() != spv::Op::OpConstant ||
!((currentLength = lengthConstant->GetSingleWordInOperand(0),
currentLength == kIterationRPScratchLength))) {
if (lengthConstant == nullptr || lengthConstant->opcode() != spv::Op::OpConstant) {
continue;
}
const uint32_t currentLength = lengthConstant->GetSingleWordInOperand(0);
// The pack's own assumption holds on this device: the declared array
// already covers every subgroup the workgroup partitions into. That is
// every >= 16-lane device for the shapes iterationRP ships, and those
// modules must pass through byte-identical.
if (currentLength >= requiredLength) continue;
// vec3 strides at its 16-byte alignment, so charge the padded stride.
const uint32_t elementStride = (components == 3u ? 4u : components) * 4u;
growths.push_back(Growth{variable, elementTypeId, lengthConstant->type_id(),
(requiredLength - currentLength) * elementStride});
}
if (growths.empty()) {
return Status::SuccessWithoutChange;
}
// Growing must not push the module past what the device can launch: a
// pipeline that fails to create is worse than the pack's own overrun.
{
uint64_t declaredBytes = 0;
bool sawUnsizeable = false;
for (auto& global : irContext->module()->types_values()) {
if (global.opcode() != spv::Op::OpVariable ||
static_cast<spv::StorageClass>(global.GetSingleWordInOperand(0)) !=
spv::StorageClass::Workgroup) {
continue;
}
const Instruction* pointerType = defUseMgr->GetDef(global.type_id());
uint32_t bytes = 0u;
uint32_t alignment = 0u;
if (pointerType == nullptr ||
pointerType->opcode() != spv::Op::OpTypePointer ||
!WorkgroupTypeLayout(irContext, pointerType->GetSingleWordInOperand(1),
&bytes, &alignment)) {
sawUnsizeable = true;
break;
}
declaredBytes = RoundUp(static_cast<uint32_t>(declaredBytes), alignment) + bytes;
}
// A declaration this pass cannot size leaves the total an
// underestimate, so the growth cannot be certified against the device
// limit at all - decline rather than guess.
if (sawUnsizeable) {
return Status::SuccessWithoutChange;
}
for (const Growth& growth : growths) declaredBytes += growth.addedBytes;
const uint32_t deviceBudget = m_maxWorkgroupScratchBytes != 0u
? m_maxWorkgroupScratchBytes
: kMinimumSharedMemoryBytes;
if (declaredBytes > deviceBudget) {
return Status::SuccessWithoutChange;
}
}
for (const Growth& growth : growths) {
// Build the grown array type. All three new instructions are inserted
// immediately BEFORE the variable so definition-before-use holds in the
// module's global section (manager-created instructions append to its
@@ -310,17 +520,17 @@ namespace MobileGL {
// scalar constant is legal SPIR-V); the fresh array type makes the
// pointer type unique by construction, so neither collides with an
// existing declaration.
const uint32_t lengthTypeId = lengthConstant->type_id();
Instruction* variable = growth.variable;
const uint32_t newLengthId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpConstant, lengthTypeId, newLengthId,
irContext, spv::Op::OpConstant, growth.lengthTypeId, newLengthId,
Instruction::OperandList{{SPV_OPERAND_TYPE_TYPED_LITERAL_NUMBER,
{requiredLength}}}));
const uint32_t newArrayTypeId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpTypeArray, 0, newArrayTypeId,
Instruction::OperandList{
{SPV_OPERAND_TYPE_ID, {elementTypeId}},
{SPV_OPERAND_TYPE_ID, {growth.elementTypeId}},
{SPV_OPERAND_TYPE_ID, {newLengthId}}}));
const uint32_t newPointerTypeId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
@@ -331,21 +541,17 @@ namespace MobileGL {
{SPV_OPERAND_TYPE_ID, {newArrayTypeId}}}));
variable->SetResultType(newPointerTypeId);
changedModule = true;
}
if (!changedModule) {
return Status::SuccessWithoutChange;
}
irContext->InvalidateAnalysesExceptFor(IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
spvtools::Optimizer::PassToken
FixIterationRPSubgroupScratchPass::CreateFixIterationRPSubgroupScratchPass(
const Uint32 nativeSubgroupSize) {
return spvtools::Optimizer::PassToken(
MakeUnique<FixIterationRPSubgroupScratchPass>(nativeSubgroupSize));
const Uint32 nativeSubgroupSize, const Uint32 maxWorkgroupScratchBytes) {
return spvtools::Optimizer::PassToken(MakeUnique<FixIterationRPSubgroupScratchPass>(
nativeSubgroupSize, maxWorkgroupScratchBytes));
}
} // namespace ShaderTranspiler
} // namespace MG_Util
@@ -16,48 +16,57 @@
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Patches ONE known shader-pack defect: iterationRP's auto-exposure reduction
// declares `shared vec2 prefixSumCache[32]` for its 512-invocation workgroup
// and stores per-subgroup subtotals through prefixSumCache[gl_SubgroupID].
// The pack hard-sized that scratch for the >=16-lane subgroups desktop GL
// drivers ship; on a narrower Vulkan device (lavapipe's 8 lanes -> 64
// subgroups) every subgroup past entry 31 indexes shared memory out of
// bounds - on a CPU rasterizer that is literal heap corruption. The
// reduction ALGORITHM is width-agnostic (its combine loop is sized by
// gl_NumSubgroups), so the faithful repair is to grow the one under-declared
// array to ceil(512 / native width) and change nothing else. This is the
// pack author's bug, not MobileGL's; the patch is therefore deliberately
// NOT a general mechanism - it only rewrites modules that positively match
// iterationRP's reduction fingerprint:
// - GLCompute entry point with local size exactly 32x16x1;
// - a subgroupInclusiveAdd on a vec2 (OpGroupNonUniformFAdd InclusiveScan,
// the pack's luminance/exposure accumulator signature);
// - a workgroup-shared array of exactly vec2[32] whose access-chain index
// is data-dependent on gl_SubgroupID.
// Matching at the SPIR-V level keeps the recognition robust against
// whitespace/identifier-level drift that made the old source-text template
// rewrite (removed in 7769156) so brittle, while still refusing to touch
// anything that is not this pack's reduction. On devices whose native width
// already satisfies the pack's assumption (>= 16 lanes: desktop GL, Adreno),
// the grown length equals or undershoots the declared 32 and every module
// passes through byte-identical.
// Patches ONE known shader-pack defect: iterationRP hard-sizes the scratch
// its subgroup prefix scans write through prefixSumCache[gl_SubgroupID].
// The pack ships that idiom twice, sized for the >= 16-lane subgroups
// desktop GL drivers give it:
// - the auto-exposure reduction: 32x16 (512 invocations), vec2[32];
// - the RTW importance warp: 1024 invocations, float[64].
// On a narrower Vulkan device (lavapipe's 8 lanes -> 64 and 128 subgroups)
// every subgroup past the last declared entry indexes shared memory out of
// bounds - on a CPU rasterizer that is literal heap corruption. Both
// reduction ALGORITHMS are width-agnostic (their combine loops are sized by
// gl_NumSubgroups), so the faithful repair is to grow the under-declared
// arrays to ceil(invocations / native width) and change nothing else.
//
// The pass never fails a module: anything it cannot prove is this exact
// pattern - or cannot grow safely (a whole-array use, a spec-constant
// length, an initializer) - is left exactly as it was.
// This is the pack author's bug, not MobileGL's, so the patch is
// deliberately NOT a general "resize shared arrays" mechanism. It rewrites
// an array only when the module positively matches the pack's reduction
// idiom AND the device's own topology proves the declaration too small:
// - GLCompute entry point with a literal workgroup size;
// - a subgroup scan/reduce over a 32-bit float scalar or vector
// (OpGroupNonUniformF{Add,Min,Max}), the pack's accumulator signature;
// - a workgroup-shared array of 32-bit float scalars/vectors whose
// access-chain index is data-dependent on gl_SubgroupID;
// - a declared length strictly below ceil(invocations / native width).
// That last clause is what keeps the patch inert wherever the pack is
// correct: on any device whose width satisfies the pack's assumption
// (>= 16 lanes: desktop GL, Adreno) both shapes already fit and every
// module passes through byte-identical. Matching at the SPIR-V level keeps
// recognition robust against the whitespace/identifier drift that made the
// old source-text template rewrite (removed in 7769156) so brittle.
//
// The pass never fails a module: anything it cannot prove is this pattern -
// or cannot grow safely (a whole-array use, a spec-constant length, an
// initializer, or growth that would not fit maxWorkgroupScratchBytes) - is
// left exactly as it was. Pass the device's maxComputeSharedMemorySize as
// maxWorkgroupScratchBytes; 0 falls back to the 16384-byte Vulkan minimum.
class FixIterationRPSubgroupScratchPass : public spvtools::opt::Pass {
public:
explicit FixIterationRPSubgroupScratchPass(Uint32 nativeSubgroupSize)
: m_nativeSubgroupSize(nativeSubgroupSize) {}
FixIterationRPSubgroupScratchPass(Uint32 nativeSubgroupSize,
Uint32 maxWorkgroupScratchBytes)
: m_nativeSubgroupSize(nativeSubgroupSize),
m_maxWorkgroupScratchBytes(maxWorkgroupScratchBytes) {}
const char* name() const override { return "fix-iterationrp-subgroup-scratch"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateFixIterationRPSubgroupScratchPass(
Uint32 nativeSubgroupSize);
Uint32 nativeSubgroupSize, Uint32 maxWorkgroupScratchBytes);
private:
Uint32 m_nativeSubgroupSize;
Uint32 m_maxWorkgroupScratchBytes;
};
} // namespace ShaderTranspiler
} // namespace MG_Util
@@ -164,6 +164,21 @@ bool LoadMobileGL(const Request& request, std::string& error) {
} else {
unsetenv("MOBILEGL_COHERENT_AS_FLUSH");
}
if (request.fixIterationRPSubgroupScratch) {
setenv("MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH", "1", 1);
} else {
unsetenv("MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH");
}
if (request.deriveNumSubgroups) {
setenv("MOBILEGL_DERIVE_NUM_SUBGROUPS", "1", 1);
} else {
unsetenv("MOBILEGL_DERIVE_NUM_SUBGROUPS");
}
if (request.iterationRPFixBarrier) {
setenv("MOBILEGL_ITERATIONRP_FIX_BARRIER", "1", 1);
} else {
unsetenv("MOBILEGL_ITERATIONRP_FIX_BARRIER");
}
if (request.fboAttachmentDumps.empty()) {
unsetenv("MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS");
} else {
@@ -818,6 +833,10 @@ bool WriteResultJson(const Request& request, const Result& result) {
<< (request.avoidAngleLlvmpipeSamplerMipmapMinFilter ? "true" : "false") << ",\n";
file << " \"avoidAngleLlvmpipeExplicitLodBias\": "
<< (request.avoidAngleLlvmpipeExplicitLodBias ? "true" : "false") << ",\n";
file << " \"fixIterationRPSubgroupScratch\": " << (request.fixIterationRPSubgroupScratch ? "true" : "false")
<< ",\n";
file << " \"deriveNumSubgroups\": " << (request.deriveNumSubgroups ? "true" : "false") << ",\n";
file << " \"iterationRPFixBarrier\": " << (request.iterationRPFixBarrier ? "true" : "false") << ",\n";
file << " \"holdMs\": " << request.holdMs << ",\n";
file << " \"mismatchPixels\": " << result.mismatchPixels << "\n";
file << "}\n";
@@ -44,6 +44,9 @@ struct Request {
bool avoidAngleLlvmpipeSamplerMipmapMinFilter = false;
bool avoidAngleLlvmpipeExplicitLodBias = false;
bool coherentAsFlush = false;
bool fixIterationRPSubgroupScratch = false;
bool deriveNumSubgroups = false;
bool iterationRPFixBarrier = false;
int holdMs = 0;
};
@@ -122,6 +122,9 @@ Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunTraceReplay(JNIEnv*
jboolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
jboolean avoidAngleLlvmpipeExplicitLodBias,
jboolean coherentAsFlush,
jboolean fixIterationRPSubgroupScratch,
jboolean deriveNumSubgroups,
jboolean iterationRPFixBarrier,
jstring texture2dDumps) {
mobilegl_trace::Request request;
request.tracePath = ToString(env, tracePath);
@@ -150,6 +153,9 @@ Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunTraceReplay(JNIEnv*
avoidAngleLlvmpipeSamplerMipmapMinFilter == JNI_TRUE;
request.avoidAngleLlvmpipeExplicitLodBias = avoidAngleLlvmpipeExplicitLodBias == JNI_TRUE;
request.coherentAsFlush = coherentAsFlush == JNI_TRUE;
request.fixIterationRPSubgroupScratch = fixIterationRPSubgroupScratch == JNI_TRUE;
request.deriveNumSubgroups = deriveNumSubgroups == JNI_TRUE;
request.iterationRPFixBarrier = iterationRPFixBarrier == JNI_TRUE;
ScopedTraceReplayState replayState;
mobilegl_trace_set_requested_size(request.width, request.height);
@@ -116,6 +116,9 @@ public final class TraceReplayActivity extends Activity {
request.avoidAngleLlvmpipeSamplerMipmapMinFilter,
request.avoidAngleLlvmpipeExplicitLodBias,
request.coherentAsFlush,
request.fixIterationRPSubgroupScratch,
request.deriveNumSubgroups,
request.iterationRPFixBarrier,
request.texture2dDumps
);
Log.i(TAG, result.toString());
@@ -149,6 +152,9 @@ public final class TraceReplayActivity extends Activity {
boolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
boolean avoidAngleLlvmpipeExplicitLodBias,
boolean coherentAsFlush,
boolean fixIterationRPSubgroupScratch,
boolean deriveNumSubgroups,
boolean iterationRPFixBarrier,
String texture2dDumps
);
@@ -174,6 +180,9 @@ public final class TraceReplayActivity extends Activity {
final boolean avoidAngleLlvmpipeSamplerMipmapMinFilter;
final boolean avoidAngleLlvmpipeExplicitLodBias;
final boolean coherentAsFlush;
final boolean fixIterationRPSubgroupScratch;
final boolean deriveNumSubgroups;
final boolean iterationRPFixBarrier;
final String texture2dDumps;
private TraceReplayRequest(
@@ -198,6 +207,9 @@ public final class TraceReplayActivity extends Activity {
boolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
boolean avoidAngleLlvmpipeExplicitLodBias,
boolean coherentAsFlush,
boolean fixIterationRPSubgroupScratch,
boolean deriveNumSubgroups,
boolean iterationRPFixBarrier,
String texture2dDumps
) {
this.tracePath = tracePath;
@@ -221,6 +233,9 @@ public final class TraceReplayActivity extends Activity {
this.avoidAngleLlvmpipeSamplerMipmapMinFilter = avoidAngleLlvmpipeSamplerMipmapMinFilter;
this.avoidAngleLlvmpipeExplicitLodBias = avoidAngleLlvmpipeExplicitLodBias;
this.coherentAsFlush = coherentAsFlush;
this.fixIterationRPSubgroupScratch = fixIterationRPSubgroupScratch;
this.deriveNumSubgroups = deriveNumSubgroups;
this.iterationRPFixBarrier = iterationRPFixBarrier;
this.texture2dDumps = texture2dDumps;
}
@@ -249,6 +264,9 @@ public final class TraceReplayActivity extends Activity {
intent.getBooleanExtra("avoid_angle_llvmpipe_sampler_mipmap_min_filter", false),
intent.getBooleanExtra("avoid_angle_llvmpipe_explicit_lod_bias", false),
intent.getBooleanExtra("coherent_as_flush", false),
intent.getBooleanExtra("fix_iterationrp_subgroup_scratch", false),
intent.getBooleanExtra("derive_num_subgroups", false),
intent.getBooleanExtra("iterationrp_fix_barrier", false),
readString(intent, "texture_2d_dumps", "")
);
}
+12
View File
@@ -40,6 +40,9 @@ Set MOBILEGL_TRACE_ANGLE_VARIANT to the packaged ANGLE short hash used by
DirectGLES replay.
Set MOBILEGL_RETRACE_USE_PBUFFER=1 or pass --use-pbuffer to run DirectGLES
against an offscreen EGL pbuffer instead of the Activity surface.
Set MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH=1,
MOBILEGL_DERIVE_NUM_SUBGROUPS=1, and MOBILEGL_ITERATIONRP_FIX_BARRIER=1 to
forward the corresponding iterationRP SPIR-V repairs into the APK process.
Pass --avoid-angle-llvmpipe-sampler-mipmap-min-filter for DirectGLES traces that
need ANGLE llvmpipe sampler mipmap filters downgraded to avoid driver stalls.
Pass --avoid-angle-llvmpipe-explicit-lod-bias for DirectGLES traces whose shaders
@@ -363,6 +366,15 @@ run_retrace() {
if [ "${coherent_as_flush}" -eq 1 ]; then
set -- "$@" --ez coherent_as_flush true
fi
if [ "${MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH:-}" = "1" ]; then
set -- "$@" --ez fix_iterationrp_subgroup_scratch true
fi
if [ "${MOBILEGL_DERIVE_NUM_SUBGROUPS:-}" = "1" ]; then
set -- "$@" --ez derive_num_subgroups true
fi
if [ "${MOBILEGL_ITERATIONRP_FIX_BARRIER:-}" = "1" ]; then
set -- "$@" --ez iterationrp_fix_barrier true
fi
if [ -n "${texture_2d_dumps}" ]; then
set -- "$@" --es texture_2d_dumps "${texture_2d_dumps}"
fi
+8
View File
@@ -72,8 +72,16 @@ bool ReadDouble(int argc, char **argv, int &index, double &out) {
return true;
}
bool ReadEnvFlag(const char *name) {
const char *value = std::getenv(name);
return value != nullptr && std::string(value) == "1";
}
bool ParseArgs(int argc, char **argv, mobilegl_trace::Request &request) {
request.backend = "DirectGLES";
request.fixIterationRPSubgroupScratch = ReadEnvFlag("MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH");
request.deriveNumSubgroups = ReadEnvFlag("MOBILEGL_DERIVE_NUM_SUBGROUPS");
request.iterationRPFixBarrier = ReadEnvFlag("MOBILEGL_ITERATIONRP_FIX_BARRIER");
for (int i = 1; i < argc; ++i) {
const std::string arg = argv[i];