Compare commits

...
Author SHA1 Message Date
Claude 562076f657 [Fix] (ShaderTranspiler, DirectVulkan, MG_IntegrationTest): patch both of iterationRP's under-declared subgroup scratch arrays
The previous commit's fingerprint was pinned to one array's incidental
dimensions - workgroup exactly 32x16x1, element exactly vec2, length
exactly 32 - which is the auto-exposure reduction and nothing else. The
pack ships the same idiom twice:

  - auto-exposure:  32x16 (512 invocations), shared vec2 prefixSumCache[32]
  - RTW warp:       1024 invocations,        shared float prefixSumCache[64]

so the warp kept writing 128 subgroups into 64 entries on an 8-lane
device and the retrace stayed bit-identically wrong (ssim 0.027902).

Key the fingerprint on the pack's idiom instead of one array's shape: a
workgroup array of 32-bit floats indexed by gl_SubgroupID, fed by a
subgroup scan, whose declared length is below ceil(invocations / native
width). Three properties keep that a targeted repair rather than a
general array resizer:

  - the index must BE gl_SubgroupID (through OpCopyObject, a signedness
    OpBitcast, or a spill whose every store is that id), so an index
    masked or clamped into range is left alone;
  - the >= 16-lane early-out is retained, so every module on the devices
    the pack was written for passes through byte-identical;
  - growth is certified against maxComputeSharedMemorySize using a
    natural-alignment layout model, and declined outright when a
    declaration cannot be sized, so a patched module can never fail
    pipeline creation where the original would not have.

Verified against the shaders the CI trace actually contains: of the 14
compute modules in the fixture exactly these two change, the other
twelve are byte-identical, and all fourteen pass spirv-val. The
integration scenario grows a second case for the 1024-invocation shape;
both abort with heap corruption when the patch is disabled.

Claude-Session: https://claude.ai/code/session_01EXSURVxwp8VVrWrPEQeLCm
2026-08-19 17:00:33 +00:00
swung0x48 d8576a2ed3 [Fix] (DirectVulkan, ShaderTranspiler, MG_IntegrationTest, SelfTest, TraceReplay): use native subgroups and patch iterationRP's under-declared scratch
iterationRP's Program 203 declares shared vec2 prefixSumCache[32] for a
512-invocation workgroup indexed by gl_SubgroupID; any device narrower
than 16 lanes partitions into more than 32 subgroups and the pack writes
shared memory out of bounds (heap corruption on lavapipe's CPU
rasterizer, ssim 0.028 on the CI retrace). Fix it where the fault lies -
in the fixture - and keep the GL contract sound everywhere else:

- FixIterationRPSubgroupScratchPass: fingerprint-gated SPIR-V pass that
  grows exactly that array to ceil(invocations/width) entries on sub-16-lane devices; every other module passes through byte-identical.
- DeriveNumSubgroupsPass stays default-on for the Adreno topology bug
  and is made spec-sound: pipelines request REQUIRE_FULL_SUBGROUPS
  whenever the workgroup shape makes the flag legal (computeFullSubgroups
  enabled, local_size_x a multiple of the native width, subgroup count
  within maxComputeWorkgroupSubgroups).
- EmulateSubgroupsPass: 32-lane virtual-subgroup lowering kept in-tree
  as a last resort, enabled only by MOBILEGL_MAGMA_EMULATE_SUBGROUP=1 on
  devices with no native subgroup support; fails closed on extended
  subgroup instructions and on modules whose added scratch would exceed
  maxComputeSharedMemorySize.
- IterationRPFirstReductionScenario skips gracefully outside the pack's
  16..256-lane source domain; the new IterationRPScratchFixScenario runs
  the fixture-shaped reduction on any width and asserts the exact
  width-independent total. DriverPost keeps reporting FAIL on
  out-of-domain devices.
- Program203 -> IterationRP rename throughout; the per-trace
  num_subgroups_quirk plumbing is removed from the trace replayer, JNI
  chain, and CI workflows.
2026-08-19 09:48:11 -04:00
swung0x48 2b6c2b561c [Fix, Test] (DirectVulkan, ShaderTranspiler, TraceReplay): derive NumSubgroups behind opt-in quirk 2026-08-18 22:31:57 -04:00
swung0x48andClaude Fable 5 12c94111b5 [Fix, Test] (DirectVulkan, SelfTest, MG_IntegrationTest, TraceReplay): snapshot sampler/image feedback and add Program 203 diagnostics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:04:09 -04:00
49 changed files with 7104 additions and 51 deletions
+4
View File
@@ -284,6 +284,9 @@ set(SOURCE_FILES
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/SplitArrayVertexInputsPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/SplitArrayVertexInputsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RebaseInstanceIndexPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RebaseInstanceIndexPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/ZeroBaseVertexPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/ZeroBaseVertexPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DeriveNumSubgroupsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPSubgroupScratchPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EmulateSubgroupsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/NormalizeRectCoordinatesPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/NormalizeRectCoordinatesPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/Lower1DArrayImagesPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/Lower1DArrayImagesPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/BakeImageFormatsPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/BakeImageFormatsPass.cpp
@@ -298,6 +301,7 @@ set(SOURCE_FILES
MobileGL/MG_Util/BackendLoaders/Vulkan/Loader.cpp MobileGL/MG_Util/BackendLoaders/Vulkan/Loader.cpp
MobileGL/MG_Util/SelfTest/DriverPost.cpp MobileGL/MG_Util/SelfTest/DriverPost.cpp
MobileGL/MG_Util/SelfTest/DriverPostIterationRPWitness.cpp
MobileGL/MG_Util/Texture/PixelStoreProcessor.cpp MobileGL/MG_Util/Texture/PixelStoreProcessor.cpp
MobileGL/MG_Util/Texture/TextureFormatProcessor.cpp MobileGL/MG_Util/Texture/TextureFormatProcessor.cpp
+30 -1
View File
@@ -78,8 +78,37 @@ namespace MobileGL::MG_Config {
// MOBILEGL_TRACE_ANGLE_VARIANT: signed trace-APK ANGLE build short hash. // MOBILEGL_TRACE_ANGLE_VARIANT: signed trace-APK ANGLE build short hash.
String TraceAngleVariant; String TraceAngleVariant;
#endif #endif
// MOBILEGL_DISABLE_SUBGROUP: force-disable Vulkan shader subgroup support. // MOBILEGL_DISABLE_SUBGROUP: force-disable Vulkan shader subgroup support,
// including the opt-in emulated compute path below.
Bool DisableSubgroup = false; Bool DisableSubgroup = false;
// MOBILEGL_MAGMA_EMULATE_SUBGROUP: implement GL_KHR_shader_subgroup's compute
// stage on a 32-lane VIRTUAL subgroup lowered to workgroup-shared memory
// (ShaderTranspiler::EmulateSubgroupsPass). Strictly a last resort: it only ever
// engages when this flag is set AND the device has no native subgroup support at
// all - a device with real subgroup operations always uses them natively,
// whatever their width (the known iterationRP defect is patched by
// FixIterationRPSubgroupScratch below instead). Off by default.
Bool MagmaEmulateSubgroup = false;
// MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH: patch iterationRP's own bug - the
// pack declares `shared vec2 prefixSumCache[32]` for a 512-invocation exposure
// reduction and indexes it by gl_SubgroupID, so any device with sub-16-lane
// subgroups (8-lane lavapipe -> 64 subgroups) writes shared memory out of
// bounds. The pass grows that one array to what the device's topology needs and
// touches nothing else; it only rewrites modules positively matching the pack's
// reduction fingerprint (ShaderTranspiler::FixIterationRPSubgroupScratchPass),
// so every other shader passes through byte-identical - as does iterationRP
// itself on >= 16-lane devices. Auto is ON; ForceOff replays the pack's bug
// verbatim.
QuirkOverride FixIterationRPSubgroupScratch = QuirkOverride::Auto;
// MOBILEGL_DERIVE_NUM_SUBGROUPS: replace compute gl_NumSubgroups loads with
// ceil(workgroup invocations / gl_SubgroupSize) on the NATIVE subgroup path
// (ShaderTranspiler::DeriveNumSubgroupsPass). Auto is ON: GL requires
// gl_SubgroupID < gl_NumSubgroups, Adreno's builtin reports 1 while the same
// dispatch emits IDs 0..7, and the derived value is the one Vulkan guarantees
// whenever the pipeline can request REQUIRE_FULL_SUBGROUPS (which the renderer
// does whenever local_size_x is a multiple of the native width). ForceOff returns
// to the raw driver builtin.
QuirkOverride DeriveNumSubgroups = QuirkOverride::Auto;
// MOBILEGL_ADVERTISE_FP64: add GL_ARB_gpu_shader_fp64 to the advertised extension // MOBILEGL_ADVERTISE_FP64: add GL_ARB_gpu_shader_fp64 to the advertised extension
// string. `double` in a shader always WORKS - it is narrowed to 32 bits before any // string. `double` in a shader always WORKS - it is narrowed to 32 bits before any
// module reaches a backend (ShaderTranspiler::DemoteFloat64Pass) - but the extension // module reaches a backend (ShaderTranspiler::DemoteFloat64Pass) - but the extension
+4
View File
@@ -168,6 +168,10 @@ namespace MobileGL::MG_ConfigLoader {
QueryEnvVariable("MOBILEGL_TRACE_ANGLE_VARIANT", features.TraceAngleVariant, ""); QueryEnvVariable("MOBILEGL_TRACE_ANGLE_VARIANT", features.TraceAngleVariant, "");
#endif #endif
features.DisableSubgroup = QueryEnvFlag("MOBILEGL_DISABLE_SUBGROUP"); features.DisableSubgroup = QueryEnvFlag("MOBILEGL_DISABLE_SUBGROUP");
features.MagmaEmulateSubgroup = QueryEnvFlag("MOBILEGL_MAGMA_EMULATE_SUBGROUP");
features.FixIterationRPSubgroupScratch =
QueryEnvQuirkOverride("MOBILEGL_FIX_ITERATIONRP_SUBGROUP_SCRATCH");
features.DeriveNumSubgroups = QueryEnvQuirkOverride("MOBILEGL_DERIVE_NUM_SUBGROUPS");
features.AdvertiseFp64 = QueryEnvFlag("MOBILEGL_ADVERTISE_FP64"); features.AdvertiseFp64 = QueryEnvFlag("MOBILEGL_ADVERTISE_FP64");
features.MagmaR11G11B10FFallback = QueryEnvFlag("MOBILEGL_MAGMA_R11G11B10F_FALLBACK"); features.MagmaR11G11B10FFallback = QueryEnvFlag("MOBILEGL_MAGMA_R11G11B10F_FALLBACK");
features.MagmaFramesInFlight = QueryEnvUint32("MOBILEGL_MAGMA_FRAMESINFLIGHT", 3, 1, 64); features.MagmaFramesInFlight = QueryEnvUint32("MOBILEGL_MAGMA_FRAMESINFLIGHT", 3, 1, 64);
@@ -9,6 +9,7 @@
#include "BackendObject_DirectVulkan.h" #include "BackendObject_DirectVulkan.h"
#include "MG_Backend/BackendObject.h" #include "MG_Backend/BackendObject.h"
#include "DirectVulkan.h" #include "DirectVulkan.h"
#include "SubgroupSupportPolicy.h"
#include "MG_State/GLState/FramebufferState/FramebufferObject.h" #include "MG_State/GLState/FramebufferState/FramebufferObject.h"
#include "MG_State/GLState/Core.h" #include "MG_State/GLState/Core.h"
#include "MG_State/GLState/TextureState/TextureState.h" #include "MG_State/GLState/TextureState/TextureState.h"
@@ -704,8 +705,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// real device timestamp support. ApplyVulkanCapabilitiesForTesting may // real device timestamp support. ApplyVulkanCapabilitiesForTesting may
// run without a renderer; no timer query is advertised then. Rebuilding // run without a renderer; no timer query is advertised then. Rebuilding
// the whole list keeps re-runs idempotent. // the whole list keeps re-runs idempotent.
// The opt-in emulated compute path (SubgroupSupportPolicy.h) carries the
// extension by itself on devices with no native subgroup support at all; a
// device with native subgroups always advertises - and uses - those.
const Bool subgroupSupportAdvertised =
m_vulkanCaps.SupportsShaderSubgroup ||
ShouldEmulateSubgroups(m_vulkanCaps.SupportsShaderSubgroup);
m_rendererInfo.RendererGLInfo.Extensions = BuildAdvertisedExtensions( m_rendererInfo.RendererGLInfo.Extensions = BuildAdvertisedExtensions(
m_vulkanCaps.SupportsShaderSubgroup, pVulkanRenderer && pVulkanRenderer->IsTimerQuerySupported(), subgroupSupportAdvertised, pVulkanRenderer && pVulkanRenderer->IsTimerQuerySupported(),
pVulkanRenderer && pVulkanRenderer->IsSamplerAnisotropySupported(), pVulkanRenderer && pVulkanRenderer->IsSamplerAnisotropySupported(),
pVulkanRenderer && pVulkanRenderer->IsNonZeroIndirectBaseInstanceSupported()); pVulkanRenderer && pVulkanRenderer->IsNonZeroIndirectBaseInstanceSupported());
} }
@@ -941,6 +948,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_dynamicParameters.SubgroupSupportedFeatures = m_dynamicParameters.SubgroupSupportedFeatures =
mapSubgroupFeatures(m_vulkanCaps.SubgroupSupportedOperations); mapSubgroupFeatures(m_vulkanCaps.SubgroupSupportedOperations);
m_dynamicParameters.SubgroupQuadOperationsInAllStages = m_vulkanCaps.SubgroupQuadOperationsInAllStages; m_dynamicParameters.SubgroupQuadOperationsInAllStages = m_vulkanCaps.SubgroupQuadOperationsInAllStages;
} else if (ShouldEmulateSubgroups(m_vulkanCaps.SupportsShaderSubgroup)) {
// MOBILEGL_MAGMA_EMULATE_SUBGROUP on a device with no native subgroups: the
// advertised values describe the 32-lane virtual subgroup the compute
// lowering implements (SubgroupSupportPolicy.h / EmulateSubgroupsPass).
// GL requires the advertisement and the execution to agree, and on this
// path the emulation is what executes; only the compute stage is offered.
m_dynamicParameters.SubgroupSize = kEmulatedSubgroupSize;
m_dynamicParameters.SubgroupSupportedStages = kEmulatedSubgroupStages;
m_dynamicParameters.SubgroupSupportedFeatures = kEmulatedSubgroupFeatures;
m_dynamicParameters.SubgroupQuadOperationsInAllStages = false;
MGLOG_I("DirectVulkan: emulating 32-lane compute subgroups "
"(MOBILEGL_MAGMA_EMULATE_SUBGROUP, no native subgroup support)");
} else { } else {
m_dynamicParameters.SubgroupSize = 0; m_dynamicParameters.SubgroupSize = 0;
m_dynamicParameters.SubgroupSupportedStages = 0; m_dynamicParameters.SubgroupSupportedStages = 0;
@@ -33,6 +33,32 @@ namespace MobileGL::MG_Backend::DirectVulkan {
using SpvcSession = MG_Util::ShaderTranspiler::SpvcSession; using SpvcSession = MG_Util::ShaderTranspiler::SpvcSession;
using SessionUsageBit = MG_Util::ShaderTranspiler::SessionUsageBit; using SessionUsageBit = MG_Util::ShaderTranspiler::SessionUsageBit;
// Local size of a compute module, read from OpExecutionMode LocalSize; all-zero
// when absent. The compile chain pins SPIR-V 1.3, where a literal local size
// always reaches the module as this execution mode (LocalSizeId does not exist
// yet).
struct ComputeLocalSize {
Uint32 x = 0;
Uint32 y = 0;
Uint32 z = 0;
Uint64 Total() const { return static_cast<Uint64>(x) * y * z; }
};
ComputeLocalSize TryGetComputeLocalSize(const Vector<Uint>& spirv) {
constexpr SizeT kHeaderWords = 5;
constexpr Uint32 kOpExecutionMode = 16;
constexpr Uint32 kModeLocalSize = 17;
for (SizeT offset = kHeaderWords; offset < spirv.size();) {
const Uint32 wordCount = spirv[offset] >> 16u;
const Uint32 opcode = spirv[offset] & 0xffffu;
if (wordCount == 0 || offset + wordCount > spirv.size()) break;
if (opcode == kOpExecutionMode && wordCount >= 6 && spirv[offset + 2] == kModeLocalSize) {
return {spirv[offset + 3], spirv[offset + 4], spirv[offset + 5]};
}
offset += wordCount;
}
return {};
}
struct DescriptorKey { struct DescriptorKey {
ProgramFactory::DescriptorBindingKind kind = ProgramFactory::DescriptorBindingKind::None; ProgramFactory::DescriptorBindingKind kind = ProgramFactory::DescriptorBindingKind::None;
String name; String name;
@@ -3163,6 +3189,60 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
} }
// GL_KHR_shader_subgroup handling (SubgroupSupportPolicy.h). Native subgroup
// operations execute natively; two module repairs keep the GL contract intact
// around them. The opt-in emulation path replaces them only on devices with no
// subgroup support at all (MOBILEGL_MAGMA_EMULATE_SUBGROUP).
if (shaders[i] && shaders[i]->GetShaderStage() == ShaderStage::Compute) {
if (m_subgroupPolicy.emulateSubgroups) {
Vector<Uint> emulatedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::EmulateSubgroupsForVulkan(
moduleSpirvs[i], emulatedSpirv,
m_subgroupPolicy.maxComputeSharedMemoryBytes, enableSpirvValidation)) {
moduleSpirvs[i] = std::move(emulatedSpirv);
} else {
MGLOG_E("ProgramFactory: subgroup emulation failed for program %u; the "
"module keeps subgroup operations the device cannot execute",
program.GetExternalIndex());
}
} else {
// iterationRP under-declares its cross-subgroup scratch
// (prefixSumCache[32] for 512 invocations); on a sub-16-lane device
// grow that one fingerprinted array to what the topology needs.
if (m_subgroupPolicy.fixIterationRPSubgroupScratch) {
Vector<Uint> patchedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
moduleSpirvs[i], patchedSpirv, m_subgroupPolicy.nativeSubgroupSize,
m_subgroupPolicy.maxComputeSharedMemoryBytes,
enableSpirvValidation)) {
moduleSpirvs[i] = std::move(patchedSpirv);
} else {
MGLOG_E("ProgramFactory: iterationRP subgroup scratch patch failed for "
"program %u; the pack's declared array sizes stay in effect",
program.GetExternalIndex());
}
}
// gl_NumSubgroups must agree with the gl_SubgroupID range GL promises;
// derive it from the workgroup dimensions and gl_SubgroupSize instead of
// trusting a driver builtin that can disagree with the topology the same
// dispatch emits (Adreno reports 1 while emitting IDs 0..7 for a
// 512-invocation, 64-wide workgroup). The ceil() partition this derives
// is pinned by REQUIRE_FULL_SUBGROUPS at pipeline creation whenever the
// workgroup shape makes that flag legal (see the stage setup below).
if (m_subgroupPolicy.deriveNumSubgroups) {
Vector<Uint> derivedNumSubgroupsSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::DeriveNumSubgroupsForVulkan(
moduleSpirvs[i], derivedNumSubgroupsSpirv, enableSpirvValidation)) {
moduleSpirvs[i] = std::move(derivedNumSubgroupsSpirv);
} else {
MGLOG_E("ProgramFactory: failed to derive gl_NumSubgroups for program %u; "
"compute shaders may observe a driver-inconsistent subgroup count",
program.GetExternalIndex());
}
}
}
}
// Vulkan's SPIR-V environment has no rectangle image dimension, so a // Vulkan's SPIR-V environment has no rectangle image dimension, so a
// GL_TEXTURE_RECTANGLE lookup has to become the 2D one the texture is really // GL_TEXTURE_RECTANGLE lookup has to become the 2D one the texture is really
// stored as - which addresses [0,1] where the application addressed texels. // stored as - which addresses [0,1] where the application addressed texels.
@@ -3308,6 +3388,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
stage.stage = ToVkStage(shaderStage); stage.stage = ToVkStage(shaderStage);
stage.module = module; stage.module = module;
stage.pName = "main"; stage.pName = "main";
// Pin the full-subgroup launch the derived gl_NumSubgroups assumes. Legal
// exactly when the computeFullSubgroups feature is enabled and local_size_x is
// a multiple of the subgroup size (VUID-VkPipelineShaderStageCreateInfo-
// flags-02759/-02785), and only worth requesting while the resulting subgroup
// count fits the device's maxComputeWorkgroupSubgroups (lavapipe caps it at
// 32, below a 512-invocation dispatch's 64). With the bit set, "Full
// Subgroups" guarantees every subgroup launches with all invocations active,
// making the subgroup count exactly invocations / size. Shapes the flag
// cannot cover (e.g. 32x16 on a 64-wide device) fall back to the driver's
// own - spec-encouraged - tight partitioning, which the DriverPost witness
// verifies per device.
if (shaderStage == ShaderStage::Compute && m_subgroupPolicy.requireFullSubgroups &&
!m_subgroupPolicy.emulateSubgroups && m_subgroupPolicy.nativeSubgroupSize != 0) {
const ComputeLocalSize localSize = TryGetComputeLocalSize(moduleSpv);
const Uint64 fullSubgroupCount =
localSize.Total() / m_subgroupPolicy.nativeSubgroupSize;
if (localSize.x != 0 && localSize.x % m_subgroupPolicy.nativeSubgroupSize == 0 &&
fullSubgroupCount <= m_subgroupPolicy.maxComputeWorkgroupSubgroups) {
stage.flags |= VK_PIPELINE_SHADER_STAGE_CREATE_REQUIRE_FULL_SUBGROUPS_BIT;
}
}
entry.modules.push_back(module); entry.modules.push_back(module);
entry.stages.push_back(stage); entry.stages.push_back(stage);
@@ -372,16 +372,38 @@ namespace MobileGL::MG_Backend::DirectVulkan {
virtual void OnProgramEvicted(HashType programHash, VkDescriptorSetLayout descriptorSetLayout) = 0; virtual void OnProgramEvicted(HashType programHash, VkDescriptorSetLayout descriptorSetLayout) = 0;
}; };
// How this factory's compute modules implement GL_KHR_shader_subgroup. Computed
// once at renderer initialization (SubgroupSupportPolicy.h + the device's
// subgroup properties) so lowering can never disagree with the advertised
// capabilities. Native subgroup operations always execute natively; the two
// repair passes patch modules AROUND them, and the emulation only replaces them
// on opted-in devices with no subgroup support at all.
struct SubgroupLoweringPolicy {
Bool emulateSubgroups = false; // MOBILEGL_MAGMA_EMULATE_SUBGROUP, no-native-support devices
Bool fixIterationRPSubgroupScratch = false; // patch iterationRP's under-declared scratch
Bool deriveNumSubgroups = false; // repair the NumSubgroups builtin
Bool requireFullSubgroups = false; // computeFullSubgroups enabled on the device
Uint32 nativeSubgroupSize = 0;
// Full-subgroup launches are bounded by this device limit; a dispatch whose
// workgroup needs more subgroups than this cannot request the flag.
Uint32 maxComputeWorkgroupSubgroups = 0;
// VkPhysicalDeviceLimits::maxComputeSharedMemorySize; bounds the scratch the
// emulation pass may add (0 falls back to the Vulkan minimum, 16384).
Uint32 maxComputeSharedMemoryBytes = 0;
};
explicit ProgramFactory(VkDevice device, const VulkanRendererConfig& config, Uint32 maxBindings, explicit ProgramFactory(VkDevice device, const VulkanRendererConfig& config, Uint32 maxBindings,
Bool shaderDrawParametersEnabled, Bool shaderDrawParametersEnabled,
Bool unformattedFloatStorageImagesEnabled, Bool unformattedFloatStorageImagesEnabled,
Bool enableSpirvValidation, Bool enableSpirvValidation,
UpdateAfterBindLimits updateAfterBindLimits) UpdateAfterBindLimits updateAfterBindLimits,
SubgroupLoweringPolicy subgroupPolicy)
: m_device(device), m_maxBindings(maxBindings), m_config(config), : m_device(device), m_maxBindings(maxBindings), m_config(config),
m_shaderDrawParametersEnabled(shaderDrawParametersEnabled), m_shaderDrawParametersEnabled(shaderDrawParametersEnabled),
m_unformattedFloatStorageImagesEnabled(unformattedFloatStorageImagesEnabled), m_unformattedFloatStorageImagesEnabled(unformattedFloatStorageImagesEnabled),
m_enableSpirvValidation(enableSpirvValidation), m_enableSpirvValidation(enableSpirvValidation),
m_updateAfterBindLimits(updateAfterBindLimits) { m_updateAfterBindLimits(updateAfterBindLimits),
m_subgroupPolicy(subgroupPolicy) {
VkProgramObject::s_device = device; VkProgramObject::s_device = device;
} }
// Destroys the pass-through tessellation control modules. Runs while the device is // Destroys the pass-through tessellation control modules. Runs while the device is
@@ -511,6 +533,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the factory lets each reflected layout choose ordinary descriptors when its // the factory lets each reflected layout choose ordinary descriptors when its
// own counts would exceed the update-after-bind budget. // own counts would exceed the update-after-bind budget.
UpdateAfterBindLimits m_updateAfterBindLimits{}; UpdateAfterBindLimits m_updateAfterBindLimits{};
SubgroupLoweringPolicy m_subgroupPolicy{};
// See SetDefaultFramebufferHeight. 0 means "not known yet"; the FragCoordYFlip bit is // See SetDefaultFramebufferHeight. 0 means "not known yet"; the FragCoordYFlip bit is
// never set before the swapchain exists, so no variant can be compiled against it. // never set before the swapchain exists, so no variant can be compiled against it.
Uint32 m_defaultFramebufferHeight = 0; Uint32 m_defaultFramebufferHeight = 0;
@@ -542,11 +542,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
outImageInfo = { outImageInfo = {
.sampler = m_samplerManager->GetOrCreateSampler(*samplerBindingOverride.sampler, .sampler = m_samplerManager->GetOrCreateSampler(*samplerBindingOverride.sampler,
*samplerBindingOverride.texture), *samplerBindingOverride.texture,
samplerBindingOverride.forceNearestFiltering,
resource->sampledLevelCount),
.imageView = samplerBindingOverride.imageView != VK_NULL_HANDLE ? .imageView = samplerBindingOverride.imageView != VK_NULL_HANDLE ?
samplerBindingOverride.imageView : samplerBindingOverride.imageView :
(resource->sampledView != VK_NULL_HANDLE ? resource->sampledView : resource->fullView), (resource->sampledView != VK_NULL_HANDLE ? resource->sampledView : resource->fullView),
.imageLayout = resource->layout, .imageLayout = samplerBindingOverride.imageLayout != VK_IMAGE_LAYOUT_UNDEFINED ?
samplerBindingOverride.imageLayout : resource->layout,
}; };
return outImageInfo.sampler != VK_NULL_HANDLE; return outImageInfo.sampler != VK_NULL_HANDLE;
} }
@@ -1269,6 +1272,87 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true; return true;
} }
Bool UniformManager::SamplerOverlapsWritableImageSubresource(Int samplerBaseLevel, Int samplerMaxLevel,
GLint imageLevel, GLenum imageAccess) {
return imageAccess != GL_READ_ONLY && imageLevel >= samplerBaseLevel && imageLevel <= samplerMaxLevel;
}
Bool UniformManager::CollectSamplerImageFeedback(
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Vector<SamplerImageFeedbackBinding>& outBindings) const {
outBindings.clear();
MOBILEGL_ASSERT(MG_State::pGLContext != nullptr,
"CollectSamplerImageFeedback: GL context is null");
if (programObj.declinedDescriptors) return true;
for (const Uint32 samplerBinding : programObj.activeBindings) {
if (samplerBinding >= m_maxBindings ||
programObj.bindingKinds[samplerBinding] != ProgramFactory::DescriptorBindingKind::CombinedImageSampler) {
continue;
}
const Uint32 samplerCount = BindingDescriptorCount(programObj, samplerBinding);
for (Uint32 samplerElement = 0; samplerElement < samplerCount; ++samplerElement) {
MG_State::GLState::ITextureObject* sampledTexture = nullptr;
const MG_State::GLState::SamplerObject* sampledSampler = nullptr;
if (!ResolveSampledBinding(program, programObj, samplerBinding, samplerElement,
sampledTexture, sampledSampler) ||
sampledTexture == nullptr || sampledSampler == nullptr ||
MG_State::GLState::SamplesAsIncompleteTexture(sampledTexture, sampledSampler)) {
// ResolveSamplerDescriptor uses a fallback in these cases, which cannot
// alias the image-unit binding of the original texture.
continue;
}
// Multisample source images intentionally omit TRANSFER_SRC usage. Keep their existing
// direct binding instead of turning otherwise valid sampler2DMS/image2DMS dispatches
// into failed dispatches; a correct snapshot for them needs a same-sample-count path.
const TextureTarget sampledTarget = sampledTexture->GetTarget();
if (sampledTarget == TextureTarget::Texture2DMultisample ||
sampledTarget == TextureTarget::Texture2DMultisampleArray) {
continue;
}
const auto& levelRange = sampledTexture->GetLevelRange();
Bool aliasesWritableImage = false;
for (const Uint32 imageBinding : programObj.activeBindings) {
if (imageBinding >= m_maxBindings ||
programObj.bindingKinds[imageBinding] != ProgramFactory::DescriptorBindingKind::StorageImage) {
continue;
}
if (imageBinding >= programObj.samplerUniformLocationByBinding.size()) return false;
const Int baseLocation = programObj.samplerUniformLocationByBinding[imageBinding];
if (baseLocation < 0) return false;
const Uint32 imageCount = BindingDescriptorCount(programObj, imageBinding);
for (Uint32 imageElement = 0; imageElement < imageCount; ++imageElement) {
const Int location = ResolveDescriptorElementLocation(program, baseLocation, imageElement);
if (location < 0) return false;
const Int imageUnit = program.GetUniformSamplerOrImageUnitIndex(static_cast<Uint>(location));
if (imageUnit < 0 || imageUnit >= MG_State::GLState::TextureState::MAX_TEXTURE_IMAGE_UNITS) {
return false;
}
const auto& image = MG_State::pGLContext->GetImageTextureBinding(imageUnit);
// A sampler view exposes all layers of its target; equal texture plus an
// overlapping mip therefore aliases the writable image subresource.
if (image.Texture.get() == sampledTexture &&
SamplerOverlapsWritableImageSubresource(levelRange.x(), levelRange.y(),
image.Level, image.Access)) {
aliasesWritableImage = true;
break;
}
}
if (aliasesWritableImage) break;
}
if (aliasesWritableImage) {
outBindings.push_back({.samplerBinding = samplerBinding,
.samplerElement = samplerElement,
.texture = sampledTexture,
.sampler = sampledSampler,
.numericDomain = programObj.samplerNumericDomainByBinding[samplerBinding]});
}
}
}
return true;
}
Bool UniformManager::ResolveUniformBufferPayload(const MG_State::GLState::ProgramObject& program, Bool UniformManager::ResolveUniformBufferPayload(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding, const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 arrayElement, UboBindResult& out) const { Uint32 arrayElement, UboBindResult& out) const {
@@ -1642,7 +1726,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 frameIndex, Uint32 frameIndex,
VkPipelineBindPoint bindPoint, VkPipelineBindPoint bindPoint,
const SamplerBindingOverride* samplerBindingOverride, const SamplerBindingOverride* samplerBindingOverride,
Bool samplerDescriptorsUnchangedHint) { Bool samplerDescriptorsUnchangedHint,
const Vector<SamplerBindingOverride>* samplerBindingOverrides) {
// This program has a descriptor MobileGL could not resolve (see // This program has a descriptor MobileGL could not resolve (see
// VkProgramObject::declinedDescriptors). Refusing here is the whole of the decline: the // VkProgramObject::declinedDescriptors). Refusing here is the whole of the decline: the
// binding is still declared in the layout, so the pipeline is consistent with the shader // binding is still declared in the layout, so the pipeline is consistent with the shader
@@ -1669,7 +1754,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// sampler binding, and an unchanged (buffer, range) for the single // sampler binding, and an unchanged (buffer, range) for the single
// dynamic UBO covers the rest - except the dynamic offset, which rebinding // dynamic UBO covers the rest - except the dynamic offset, which rebinding
// the SAME set delivers without any descriptor write. // the SAME set delivers without any descriptor write.
const Bool cacheable = (samplerBindingOverride == nullptr); const Bool cacheable = samplerBindingOverride == nullptr &&
(samplerBindingOverrides == nullptr || samplerBindingOverrides->empty());
if (cacheable && samplerDescriptorsUnchangedHint && m_fastRebindMemo.valid && if (cacheable && samplerDescriptorsUnchangedHint && m_fastRebindMemo.valid &&
m_fastRebindMemo.frameIndex == frameIndex && m_fastRebindMemo.frameIndex == frameIndex &&
m_fastRebindMemo.programLifetimeId == program.GetLifetimeId() && m_fastRebindMemo.programLifetimeId == program.GetLifetimeId() &&
@@ -1893,13 +1979,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const SizeT firstImageInfoIndex = imageInfos.size(); const SizeT firstImageInfoIndex = imageInfos.size();
for (Uint32 element = 0; element < descriptorCount; ++element) { for (Uint32 element = 0; element < descriptorCount; ++element) {
VkDescriptorImageInfo imageInfo{}; VkDescriptorImageInfo imageInfo{};
Bool hasImage = false; const SamplerBindingOverride* overrideForElement =
if (overrideThisBinding && element == 0) { overrideThisBinding && element == 0 ? samplerBindingOverride : nullptr;
hasImage = ResolveSamplerDescriptorOverride(*samplerBindingOverride, imageInfo); if (overrideForElement == nullptr && samplerBindingOverrides != nullptr) {
} else { const auto overrideIt = std::find_if(
hasImage = ResolveSamplerDescriptor(commandBuffer, program, programObj, binding, element, samplerBindingOverrides->begin(), samplerBindingOverrides->end(),
imageInfo, samplerDescriptorsUnchangedHint); [binding, element](const SamplerBindingOverride& candidate) {
return candidate.binding == binding && candidate.element == element;
});
if (overrideIt != samplerBindingOverrides->end()) {
overrideForElement = &*overrideIt;
}
} }
const Bool hasImage = overrideForElement != nullptr
? ResolveSamplerDescriptorOverride(*overrideForElement, imageInfo)
: ResolveSamplerDescriptor(commandBuffer, program, programObj, binding,
element, imageInfo,
samplerDescriptorsUnchangedHint);
if (!hasImage) { if (!hasImage) {
MGLOG_E_ONCE( MGLOG_E_ONCE(
"UniformDescriptorBinder::BindProgramUniformBuffers failed: sampler binding %u element %u " "UniformDescriptorBinder::BindProgramUniformBuffers failed: sampler binding %u element %u "
@@ -26,9 +26,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
public: public:
struct SamplerBindingOverride { struct SamplerBindingOverride {
Uint32 binding = 0; Uint32 binding = 0;
Uint32 element = 0;
MG_State::GLState::ITextureObject* texture = nullptr; MG_State::GLState::ITextureObject* texture = nullptr;
const MG_State::GLState::SamplerObject* sampler = nullptr; const MG_State::GLState::SamplerObject* sampler = nullptr;
VkImageView imageView = VK_NULL_HANDLE; VkImageView imageView = VK_NULL_HANDLE;
VkImageLayout imageLayout = VK_IMAGE_LAYOUT_UNDEFINED;
Bool forceNearestFiltering = false;
};
struct SamplerImageFeedbackBinding {
Uint32 samplerBinding = 0;
Uint32 samplerElement = 0;
MG_State::GLState::ITextureObject* texture = nullptr;
const MG_State::GLState::SamplerObject* sampler = nullptr;
SamplerNumericDomain numericDomain = SamplerNumericDomain::Unknown;
}; };
Bool Initialize(VkDevice device, VkBufferManager* bufferManager, Bool Initialize(VkDevice device, VkBufferManager* bufferManager,
@@ -79,6 +90,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool CollectStorageImageTextures(const MG_State::GLState::ProgramObject& program, Bool CollectStorageImageTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, const ProgramFactory::VkProgramObject& programObj,
Vector<MG_State::GLState::ITextureObject*>& outTextures) const; Vector<MG_State::GLState::ITextureObject*>& outTextures) const;
Bool CollectSamplerImageFeedback(
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Vector<SamplerImageFeedbackBinding>& outBindings) const;
static Bool SamplerOverlapsWritableImageSubresource(Int samplerBaseLevel, Int samplerMaxLevel,
GLint imageLevel, GLenum imageAccess);
// samplerDescriptorsUnchangedHint: the caller (SetupDraw fast path) proved that // samplerDescriptorsUnchangedHint: the caller (SetupDraw fast path) proved that
// every input of every combined-image-sampler resolution is unchanged since the // every input of every combined-image-sampler resolution is unchanged since the
// previous draw's resolve - same (texture, sampler) per binding, texture params // previous draw's resolve - same (texture, sampler) per binding, texture params
@@ -91,7 +108,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 frameIndex, Uint32 frameIndex,
VkPipelineBindPoint bindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS, VkPipelineBindPoint bindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
const SamplerBindingOverride* samplerBindingOverride = nullptr, const SamplerBindingOverride* samplerBindingOverride = nullptr,
Bool samplerDescriptorsUnchangedHint = false); Bool samplerDescriptorsUnchangedHint = false,
const Vector<SamplerBindingOverride>* samplerBindingOverrides = nullptr);
// Pure format-policy helper kept public for host regression tests. Formatted storage // Pure format-policy helper kept public for host regression tests. Formatted storage
// images use their shader qualifier; transformed float images use glBindImageTexture's // images use their shader qualifier; transformed float images use glBindImageTexture's
@@ -1291,6 +1291,158 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return ok; return ok;
} }
Bool VkTextureManager::SnapshotTextureForSampling(VkCommandBuffer commandBuffer,
MG_State::GLState::ITextureObject& texture,
SamplerNumericDomain numericDomain,
VkPipelineStageFlags consumerShaderStageMask,
SampledTextureSnapshot& outSnapshot) {
outSnapshot = {};
TextureResource* source = SyncTextureAndGetDescriptor(texture);
if (source == nullptr || source->image == VK_NULL_HANDLE || source->sampleCount != VK_SAMPLE_COUNT_1_BIT ||
source->sampledLevelCount == 0) {
return false;
}
const VkFormat sampledFormat = ResolveSampledImageViewFormat(source->format, numericDomain);
if (sampledFormat == VK_FORMAT_UNDEFINED ||
!AreSampledImageViewFormatsCompatible(source->format, sampledFormat)) {
MGLOG_E_ONCE("SnapshotTextureForSampling: textureId=%d cannot create sampled view format=%d from image format=%d",
texture.GetExternalIndex(), static_cast<Int>(sampledFormat), static_cast<Int>(source->format));
return false;
}
if (sampledFormat != source->format &&
(source->imageCreateFlags & VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT) == 0) {
MGLOG_E_ONCE("SnapshotTextureForSampling: textureId=%d needs unavailable mutable image format=%d for sampled view=%d",
texture.GetExternalIndex(), static_cast<Int>(source->format), static_cast<Int>(sampledFormat));
return false;
}
VkImageType imageType = VK_IMAGE_TYPE_2D;
switch (source->viewType) {
case VK_IMAGE_VIEW_TYPE_1D:
case VK_IMAGE_VIEW_TYPE_1D_ARRAY:
imageType = VK_IMAGE_TYPE_1D;
break;
case VK_IMAGE_VIEW_TYPE_3D:
imageType = VK_IMAGE_TYPE_3D;
break;
default:
break;
}
TextureResource snapshot{};
VkImageCreateInfo imageInfo{};
imageInfo.sType = VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO;
imageInfo.flags = source->imageCreateFlags;
imageInfo.imageType = imageType;
imageInfo.extent = {source->extent.width, source->extent.height, source->depth};
imageInfo.mipLevels = source->mipLevels;
imageInfo.arrayLayers = source->arrayLayers;
imageInfo.format = source->format;
imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
imageInfo.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT;
imageInfo.samples = VK_SAMPLE_COUNT_1_BIT;
imageInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
// Keep the temporary's view-format list just as narrow as the source's sampler use. This
// has no storage-image usage, so unlike an app image binding the exact list is knowable.
Vector<VkFormat> viewFormats;
VkImageFormatListCreateInfo formatListInfo{};
if (m_imageFormatListSupported && (imageInfo.flags & VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT) != 0) {
viewFormats.push_back(source->format);
if (sampledFormat != source->format) {
viewFormats.push_back(sampledFormat);
}
formatListInfo.sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO;
formatListInfo.viewFormatCount = static_cast<Uint32>(viewFormats.size());
formatListInfo.pViewFormats = viewFormats.data();
imageInfo.pNext = &formatListInfo;
}
VmaAllocationCreateInfo allocationInfo{};
allocationInfo.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
allocationInfo.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
const VkResult createResult =
vmaCreateImage(m_allocator, &imageInfo, &allocationInfo, &snapshot.image, &snapshot.allocation, nullptr);
if (createResult != VK_SUCCESS) {
MGLOG_E_ONCE("SnapshotTextureForSampling: vmaCreateImage failed result=%d textureId=%d", createResult,
texture.GetExternalIndex());
return false;
}
snapshot.extent = source->extent;
snapshot.depth = source->depth;
snapshot.arrayLayers = source->arrayLayers;
snapshot.mipLevels = source->mipLevels;
snapshot.sampledBaseMipLevel = source->sampledBaseMipLevel;
snapshot.sampledLevelCount = source->sampledLevelCount;
snapshot.format = source->format;
snapshot.aspect = source->aspect;
snapshot.viewType = source->viewType;
snapshot.sampleCount = VK_SAMPLE_COUNT_1_BIT;
snapshot.imageCreateFlags = imageInfo.flags;
snapshot.usageFlags = imageInfo.usage;
const TextureFormatInfo formatInfo = ResolveTextureFormatInfo(texture.GetFormat());
const VkComponentMapping sampledComponents = ResolveSampledViewComponents(texture, formatInfo);
const VkImageAspectFlags sampledAspect =
ResolveSampledImageViewAspectMask(snapshot.aspect, texture.GetDepthStencilTextureMode());
snapshot.sampledView = CreateImageView(snapshot.image, sampledFormat, sampledAspect, snapshot.viewType,
snapshot.sampledBaseMipLevel, snapshot.sampledLevelCount, 0,
snapshot.arrayLayers, &sampledComponents);
if (snapshot.sampledView == VK_NULL_HANDLE) {
MGLOG_E_ONCE("SnapshotTextureForSampling: failed to create sampled view textureId=%d", texture.GetExternalIndex());
return false;
}
VkPipelineStageFlags sourceStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
VkAccessFlags sourceAccessMask = 0;
const VkImageLayout sourceLayout = source->layout;
GetImageTransitionSourceState(sourceLayout, sourceStageMask, sourceAccessMask);
if (!TransitionImageLayout(commandBuffer, source->image, source->layout, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
sourceStageMask, VK_PIPELINE_STAGE_TRANSFER_BIT, sourceAccessMask,
VK_ACCESS_TRANSFER_READ_BIT, source->aspect, 0, source->mipLevels) ||
!TransitionImageLayout(commandBuffer, snapshot.image, snapshot.layout, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0,
VK_ACCESS_TRANSFER_WRITE_BIT, snapshot.aspect, snapshot.sampledBaseMipLevel,
snapshot.sampledLevelCount)) {
return false;
}
Vector<VkImageCopy> copyRegions;
copyRegions.reserve(snapshot.sampledLevelCount);
for (Uint32 level = snapshot.sampledBaseMipLevel;
level < snapshot.sampledBaseMipLevel + snapshot.sampledLevelCount; ++level) {
VkImageCopy copy{};
copy.srcSubresource = {source->aspect, level, 0, source->arrayLayers};
copy.dstSubresource = {snapshot.aspect, level, 0, snapshot.arrayLayers};
copy.extent = {std::max(source->extent.width >> level, 1u),
std::max(source->extent.height >> level, 1u),
std::max(source->depth >> level, 1u)};
copyRegions.push_back(copy);
}
vkCmdCopyImage(commandBuffer, source->image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, snapshot.image,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, static_cast<Uint32>(copyRegions.size()), copyRegions.data());
if (!TransitionImageLayout(commandBuffer, snapshot.image, snapshot.layout,
ResolveSampledReadOnlyLayout(snapshot.aspect), VK_PIPELINE_STAGE_TRANSFER_BIT,
consumerShaderStageMask, VK_ACCESS_TRANSFER_WRITE_BIT,
VK_ACCESS_SHADER_READ_BIT, snapshot.aspect, snapshot.sampledBaseMipLevel,
snapshot.sampledLevelCount) ||
!TransitionImageLayout(commandBuffer, source->image, source->layout, sourceLayout,
VK_PIPELINE_STAGE_TRANSFER_BIT, consumerShaderStageMask,
VK_ACCESS_TRANSFER_READ_BIT, VK_ACCESS_SHADER_READ_BIT | VK_ACCESS_SHADER_WRITE_BIT,
source->aspect, 0, source->mipLevels)) {
return false;
}
StampResourceRecordingUse(*source);
outSnapshot = {.imageView = snapshot.sampledView, .layout = snapshot.layout};
DeferResourceRelease(Move(snapshot));
return true;
}
void VkTextureManager::MarkStorageImageTexture(MG_State::GLState::ITextureObject& texture) { void VkTextureManager::MarkStorageImageTexture(MG_State::GLState::ITextureObject& texture) {
m_storageImageTextures.insert(MakeTextureIdentity(&texture)); m_storageImageTextures.insert(MakeTextureIdentity(&texture));
} }
@@ -310,6 +310,11 @@ public:
static inline VmaAllocator s_allocator = VK_NULL_HANDLE; static inline VmaAllocator s_allocator = VK_NULL_HANDLE;
}; };
struct SampledTextureSnapshot {
VkImageView imageView = VK_NULL_HANDLE;
VkImageLayout layout = VK_IMAGE_LAYOUT_UNDEFINED;
};
Bool Initialize(const InitInfo& initInfo); Bool Initialize(const InitInfo& initInfo);
void Shutdown(); void Shutdown();
void BeginFrame(Uint32 frameIndex); void BeginFrame(Uint32 frameIndex);
@@ -343,6 +348,13 @@ public:
VkImageLayout newLayout); VkImageLayout newLayout);
Bool TransitionTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture); Bool TransitionTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
Bool TransitionTextureForStorageImage(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture); Bool TransitionTextureForStorageImage(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
// Copies the complete sampler-visible mip range into a transient sampled image. The source is
// restored to its prior layout, so image-store descriptors continue to name the original image.
// The transient ownership is tied to the current frame slot and is safe through its submission.
Bool SnapshotTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture,
SamplerNumericDomain numericDomain,
VkPipelineStageFlags consumerShaderStageMask,
SampledTextureSnapshot& outSnapshot);
// Recording-generation bookkeeping for the pre-pass command stream. The // Recording-generation bookkeeping for the pre-pass command stream. The
// generation advances every time the frame command buffer (re)begins // generation advances every time the frame command buffer (re)begins
@@ -8,6 +8,7 @@
#include "VulkanRenderer.h" #include "VulkanRenderer.h"
#include "MG_Backend/DirectVulkan/SubgroupSupportPolicy.h"
#include "MG_Backend/DirectGLES/Utils.h" #include "MG_Backend/DirectGLES/Utils.h"
#include "VertexInputStateFactory.h" #include "VertexInputStateFactory.h"
#include "VertexInputStateBuilder.h" #include "VertexInputStateBuilder.h"
@@ -3058,11 +3059,22 @@ void main() {
} }
PipelineFactory::SetSuppressBlendedDepthWrite(suppressBlendedDepthWrite); PipelineFactory::SetSuppressBlendedDepthWrite(suppressBlendedDepthWrite);
} }
ProgramFactory::SubgroupLoweringPolicy subgroupPolicy{};
subgroupPolicy.emulateSubgroups = ShouldEmulateSubgroups(m_nativeSubgroupSupported);
subgroupPolicy.fixIterationRPSubgroupScratch =
m_nativeSubgroupSupported && ShouldFixIterationRPSubgroupScratch();
subgroupPolicy.deriveNumSubgroups =
m_nativeSubgroupSupported && ShouldDeriveNumSubgroups();
subgroupPolicy.requireFullSubgroups = m_computeFullSubgroupsFeatureEnabled;
subgroupPolicy.nativeSubgroupSize = m_nativeSubgroupSize;
subgroupPolicy.maxComputeWorkgroupSubgroups = m_maxComputeWorkgroupSubgroups;
subgroupPolicy.maxComputeSharedMemoryBytes =
m_physicalDevice.properties.limits.maxComputeSharedMemorySize;
m_programFactory = MakeUnique<ProgramFactory>(m_device, m_config, maxProgramBindings, m_programFactory = MakeUnique<ProgramFactory>(m_device, m_config, maxProgramBindings,
m_shaderDrawParametersFeatureEnabled, m_shaderDrawParametersFeatureEnabled,
m_unformattedFloatStorageImagesEnabled, m_unformattedFloatStorageImagesEnabled,
MG_Config::Features.EnableSpirvValidation, MG_Config::Features.EnableSpirvValidation,
m_updateAfterBindLimits); m_updateAfterBindLimits, subgroupPolicy);
MOBILEGL_ASSERT(m_programFactory != nullptr, "ProgramFactory creation failed."); MOBILEGL_ASSERT(m_programFactory != nullptr, "ProgramFactory creation failed.");
// The swapchain already exists at this point (Initialize creates it first), so seed the // The swapchain already exists at this point (Initialize creates it first), so seed the
// height the factory could not be told about from CreateSwapchain. // height the factory could not be told about from CreateSwapchain.
@@ -5412,7 +5424,81 @@ void main() {
} }
return true; return true;
} }
Bool VulkanRenderer::PrepareSamplerImageFeedbackSnapshots(
FrameContext::FrameData& frame,
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
VkPipelineStageFlags consumerShaderStageMask) {
auto& feedbackBindings = m_samplerImageFeedbackScratch;
auto& overrides = m_samplerImageBindingOverridesScratch;
overrides.clear();
if (!programObj.hasStorageImages) {
feedbackBindings.clear();
return true;
}
if (!m_uniformManager->CollectSamplerImageFeedback(program, programObj, feedbackBindings)) {
MGLOG_E_ONCE("%s: failed to collect sampler/image feedback for program=%u", __func__,
program.GetExternalIndex());
return false;
}
if (feedbackBindings.empty()) {
return true;
}
// Copy and layout barriers cannot be recorded inside a render pass. A graphics draw only
// gets here after an actual sampled/writable-image mip overlap was found, so ordinary
// graphics draws retain the active pass.
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(frame.commandBuffer);
}
struct SnapshotCacheEntry {
MG_State::GLState::ITextureObject* texture = nullptr;
SamplerNumericDomain numericDomain = SamplerNumericDomain::Unknown;
VkTextureManager::SampledTextureSnapshot snapshot{};
};
Vector<SnapshotCacheEntry> snapshotCache;
snapshotCache.reserve(feedbackBindings.size());
overrides.reserve(feedbackBindings.size());
for (const auto& feedback : feedbackBindings) {
VkTextureManager::SampledTextureSnapshot snapshot{};
const auto existing = std::find_if(
snapshotCache.begin(), snapshotCache.end(), [&feedback](const SnapshotCacheEntry& candidate) {
return candidate.texture == feedback.texture && candidate.numericDomain == feedback.numericDomain;
});
if (existing != snapshotCache.end()) {
snapshot = existing->snapshot;
} else {
if (!m_textureManager->SnapshotTextureForSampling(frame.commandBuffer, *feedback.texture,
feedback.numericDomain, consumerShaderStageMask,
snapshot) ||
snapshot.imageView == VK_NULL_HANDLE) {
MGLOG_E_ONCE("%s: failed to snapshot textureId=%d for sampler binding=%u element=%u", __func__,
feedback.texture != nullptr ? feedback.texture->GetExternalIndex() : 0,
feedback.samplerBinding, feedback.samplerElement);
return false;
}
snapshotCache.push_back({.texture = feedback.texture,
.numericDomain = feedback.numericDomain,
.snapshot = snapshot});
}
overrides.push_back({
.binding = feedback.samplerBinding,
.element = feedback.samplerElement,
.texture = feedback.texture,
.sampler = feedback.sampler,
.imageView = snapshot.imageView,
.imageLayout = snapshot.layout,
.forceNearestFiltering = feedback.numericDomain == SamplerNumericDomain::SignedInteger ||
feedback.numericDomain == SamplerNumericDomain::UnsignedInteger,
});
if (program.GetExternalIndex() == 194 && feedback.texture->GetExternalIndex() == 75) {
MGLOG_D_ONCE("sampler/image feedback snapshot: program=194 texture=75 binding=%u element=%u view=%p",
feedback.samplerBinding, feedback.samplerElement, snapshot.imageView);
}
}
return true;
}
// The scissor rectangle Vulkan needs for ARB_viewport_array index `index`. Vulkan has no // The scissor rectangle Vulkan needs for ARB_viewport_array index `index`. Vulkan has no
// per-viewport scissor-test TOGGLE - a scissor rectangle always applies - so an index whose // per-viewport scissor-test TOGGLE - a scissor rectangle always applies - so an index whose
@@ -6094,6 +6180,11 @@ void main() {
MGLOG_E_ONCE("SetupDraw skipped: storage image preparation failed"); MGLOG_E_ONCE("SetupDraw skipped: storage image preparation failed");
return false; return false;
} }
if (!PrepareSamplerImageFeedbackSnapshots(frame, program, programObj,
VK_PIPELINE_STAGE_ALL_GRAPHICS_BIT)) {
MGLOG_E_ONCE("SetupDraw skipped: sampler/image feedback snapshot failed");
return false;
}
auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass(); auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass();
@@ -6338,7 +6429,9 @@ void main() {
} }
const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers( const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers(
frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex()); frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex(),
VK_PIPELINE_BIND_POINT_GRAPHICS, nullptr, false,
m_samplerImageBindingOverridesScratch.empty() ? nullptr : &m_samplerImageBindingOverridesScratch);
if (!boundUniforms) { if (!boundUniforms) {
MGLOG_E_ONCE("SetupDraw skipped: BindProgramUniformBuffers failed"); MGLOG_E_ONCE("SetupDraw skipped: BindProgramUniformBuffers failed");
return false; return false;
@@ -6461,6 +6554,11 @@ void main() {
MGLOG_E_ONCE("DispatchCompute skipped: storage image preparation failed"); MGLOG_E_ONCE("DispatchCompute skipped: storage image preparation failed");
return; return;
} }
if (!PrepareSamplerImageFeedbackSnapshots(frame, program, programObj,
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT)) {
MGLOG_E_ONCE("DispatchCompute skipped: sampler/image feedback snapshot failed");
return;
}
const VkPipeline pipeline = GetOrCreateComputePipeline(programObj); const VkPipeline pipeline = GetOrCreateComputePipeline(programObj);
if (pipeline == VK_NULL_HANDLE) { if (pipeline == VK_NULL_HANDLE) {
@@ -6472,7 +6570,8 @@ void main() {
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline);
const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers( const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers(
frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex(), frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex(),
VK_PIPELINE_BIND_POINT_COMPUTE); VK_PIPELINE_BIND_POINT_COMPUTE, nullptr, false,
m_samplerImageBindingOverridesScratch.empty() ? nullptr : &m_samplerImageBindingOverridesScratch);
if (!boundUniforms) { if (!boundUniforms) {
MGLOG_E_ONCE("DispatchCompute skipped: BindProgramUniformBuffers failed"); MGLOG_E_ONCE("DispatchCompute skipped: BindProgramUniformBuffers failed");
return; return;
@@ -6507,6 +6606,11 @@ void main() {
MGLOG_E_ONCE("DispatchComputeIndirect skipped: storage image preparation failed"); MGLOG_E_ONCE("DispatchComputeIndirect skipped: storage image preparation failed");
return; return;
} }
if (!PrepareSamplerImageFeedbackSnapshots(frame, program, programObj,
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT)) {
MGLOG_E_ONCE("DispatchComputeIndirect skipped: sampler/image feedback snapshot failed");
return;
}
const VkPipeline pipeline = GetOrCreateComputePipeline(programObj); const VkPipeline pipeline = GetOrCreateComputePipeline(programObj);
if (pipeline == VK_NULL_HANDLE) { if (pipeline == VK_NULL_HANDLE) {
@@ -6518,7 +6622,8 @@ void main() {
vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); vkCmdBindPipeline(frame.commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline);
const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers( const Bool boundUniforms = m_uniformManager->BindProgramUniformBuffers(
frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex(), frame.commandBuffer, program, programObj, m_frameContext.GetCurrentFrameIndex(),
VK_PIPELINE_BIND_POINT_COMPUTE); VK_PIPELINE_BIND_POINT_COMPUTE, nullptr, false,
m_samplerImageBindingOverridesScratch.empty() ? nullptr : &m_samplerImageBindingOverridesScratch);
if (!boundUniforms) { if (!boundUniforms) {
MGLOG_E_ONCE("DispatchComputeIndirect skipped: BindProgramUniformBuffers failed"); MGLOG_E_ONCE("DispatchComputeIndirect skipped: BindProgramUniformBuffers failed");
return; return;
@@ -12647,6 +12752,73 @@ void main() {
} }
} }
// Native subgroup topology, and VK_EXT_subgroup_size_control's
// computeFullSubgroups feature. REQUIRE_FULL_SUBGROUPS on a compute stage is what
// turns the derived gl_NumSubgroups (DeriveNumSubgroupsPass) from
// encouraged-but-unspecified driver behaviour into a spec guarantee: with the bit
// set and local_size_x a multiple of the subgroup size, every subgroup launches
// full, so the subgroup count is exactly invocations / size ("Full Subgroups",
// VUID-VkPipelineShaderStageCreateInfo-flags-02759/-02785).
m_nativeSubgroupSize = 0;
m_nativeSubgroupSupported = false;
m_computeFullSubgroupsFeatureEnabled = false;
if (getPhysicalDeviceProperties2 != nullptr) {
VkPhysicalDeviceSubgroupProperties subgroupProperties{};
subgroupProperties.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_PROPERTIES;
VkPhysicalDeviceProperties2 subgroupPropertyQuery{};
subgroupPropertyQuery.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2;
subgroupPropertyQuery.pNext = &subgroupProperties;
getPhysicalDeviceProperties2(m_physicalDevice.handle, &subgroupPropertyQuery);
// Mirrors the loader's HasUsableShaderSubgroupSupport gate, including the
// MOBILEGL_DISABLE_SUBGROUP escape hatch, so the module lowerings can never
// disagree with the advertised capabilities.
const Bool usableSubgroups =
subgroupProperties.subgroupSize > 0 &&
(subgroupProperties.supportedStages & VK_SHADER_STAGE_COMPUTE_BIT) != 0 &&
(subgroupProperties.supportedOperations & VK_SUBGROUP_FEATURE_BASIC_BIT) != 0;
if (usableSubgroups && !MG_Config::Features.DisableSubgroup) {
m_nativeSubgroupSize = subgroupProperties.subgroupSize;
m_nativeSubgroupSupported = true;
}
}
VkPhysicalDeviceSubgroupSizeControlFeaturesEXT subgroupSizeControlFeatures{};
subgroupSizeControlFeatures.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_SIZE_CONTROL_FEATURES_EXT;
m_maxComputeWorkgroupSubgroups = 0;
if (m_nativeSubgroupSupported &&
IsExtensionSupported(availableExtensions, VK_EXT_SUBGROUP_SIZE_CONTROL_EXTENSION_NAME) &&
getPhysicalDeviceFeatures2 != nullptr) {
VkPhysicalDeviceFeatures2 featureQuery{};
featureQuery.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
featureQuery.pNext = &subgroupSizeControlFeatures;
getPhysicalDeviceFeatures2(m_physicalDevice.handle, &featureQuery);
if (getPhysicalDeviceProperties2 != nullptr) {
VkPhysicalDeviceSubgroupSizeControlPropertiesEXT subgroupSizeControlProperties{};
subgroupSizeControlProperties.sType =
VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_SIZE_CONTROL_PROPERTIES_EXT;
VkPhysicalDeviceProperties2 propertyQuery{};
propertyQuery.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2;
propertyQuery.pNext = &subgroupSizeControlProperties;
getPhysicalDeviceProperties2(m_physicalDevice.handle, &propertyQuery);
m_maxComputeWorkgroupSubgroups =
subgroupSizeControlProperties.maxComputeWorkgroupSubgroups;
}
if (subgroupSizeControlFeatures.computeFullSubgroups == VK_TRUE) {
if (!IsExtensionAlreadyEnabled(enabledDeviceExtensions,
VK_EXT_SUBGROUP_SIZE_CONTROL_EXTENSION_NAME)) {
enabledDeviceExtensions.push_back(VK_EXT_SUBGROUP_SIZE_CONTROL_EXTENSION_NAME);
}
// Only the full-subgroups guarantee is wanted; required/varying subgroup
// sizes stay unrequested.
subgroupSizeControlFeatures.subgroupSizeControl = VK_FALSE;
subgroupSizeControlFeatures.pNext = const_cast<void*>(deviceCreateInfo.pNext);
deviceCreateInfo.pNext = &subgroupSizeControlFeatures;
m_computeFullSubgroupsFeatureEnabled = true;
MGLOG_I("Enabled optional device extension: %s (computeFullSubgroups)",
VK_EXT_SUBGROUP_SIZE_CONTROL_EXTENSION_NAME);
}
}
// VK_EXT_transform_feedback backs GL transform feedback capture. // VK_EXT_transform_feedback backs GL transform feedback capture.
m_transformFeedbackFeatureEnabled = false; m_transformFeedbackFeatureEnabled = false;
VkPhysicalDeviceTransformFeedbackFeaturesEXT transformFeedbackFeatures{}; VkPhysicalDeviceTransformFeedbackFeaturesEXT transformFeedbackFeatures{};
@@ -554,6 +554,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool m_samplerAnisotropyFeatureEnabled = false; Bool m_samplerAnisotropyFeatureEnabled = false;
Bool m_shaderDrawParametersExtensionEnabled = false; Bool m_shaderDrawParametersExtensionEnabled = false;
Bool m_shaderDrawParametersFeatureEnabled = false; Bool m_shaderDrawParametersFeatureEnabled = false;
// Native subgroup topology, queried at device creation for the compute-module
// subgroup repairs (SubgroupSupportPolicy.h) and the REQUIRE_FULL_SUBGROUPS
// stage flag; 0 / false when the device has no usable compute subgroups or
// MOBILEGL_DISABLE_SUBGROUP forced them off.
Uint32 m_nativeSubgroupSize = 0;
Bool m_nativeSubgroupSupported = false;
Bool m_computeFullSubgroupsFeatureEnabled = false;
// VkPhysicalDeviceSubgroupSizeControlProperties::maxComputeWorkgroupSubgroups;
// 0 when the extension (and therefore the full-subgroups flag) is unavailable.
Uint32 m_maxComputeWorkgroupSubgroups = 0;
Bool m_unformattedFloatStorageImagesEnabled = false; Bool m_unformattedFloatStorageImagesEnabled = false;
// Set only after descriptor-indexing feature AND property queries prove that // Set only after descriptor-indexing feature AND property queries prove that
// update-after-bind is legal for every descriptor category this renderer emits. // update-after-bind is legal for every descriptor category this renderer emits.
@@ -929,6 +939,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// already sampleable. // already sampleable.
Vector<VkTextureManager::TextureResource*> m_sampledResourcesScratch; Vector<VkTextureManager::TextureResource*> m_sampledResourcesScratch;
Vector<MG_State::GLState::ITextureObject*> m_storageImageTexturesScratch; Vector<MG_State::GLState::ITextureObject*> m_storageImageTexturesScratch;
Vector<UniformManager::SamplerImageFeedbackBinding> m_samplerImageFeedbackScratch;
Vector<UniformManager::SamplerBindingOverride> m_samplerImageBindingOverridesScratch;
Vector<VkBuffer> m_vertexBuffersScratch; Vector<VkBuffer> m_vertexBuffersScratch;
Vector<VkDeviceSize> m_vertexOffsetsScratch; Vector<VkDeviceSize> m_vertexOffsetsScratch;
Vector<VkVertexInputAttributeDescription> m_patchedAttributesScratch; Vector<VkVertexInputAttributeDescription> m_patchedAttributesScratch;
@@ -1148,6 +1160,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
FrameContext::FrameData& frame, FrameContext::FrameData& frame,
const MG_State::GLState::ProgramObject& program, const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj); const ProgramFactory::VkProgramObject& programObj);
// Vulkan forbids a sampled descriptor and writable storage descriptor from naming the
// same image subresource in one shader operation. Snapshot only the sampler side; the
// storage descriptor continues to name the application texture.
Bool PrepareSamplerImageFeedbackSnapshots(
FrameContext::FrameData& frame,
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
VkPipelineStageFlags consumerShaderStageMask);
// The per-draw dynamic-state tail (viewport, scissor, blend constants, depth // The per-draw dynamic-state tail (viewport, scissor, blend constants, depth
// bias, line width, stencil), gated behind one render-state-parameters-version // bias, line width, stencil), gated behind one render-state-parameters-version
@@ -0,0 +1,57 @@
// MobileGL - MobileGL/MG_Backend/DirectVulkan/SubgroupSupportPolicy.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Config.h>
#include <Includes.h>
namespace MobileGL::MG_Backend::DirectVulkan {
// The single decision point for how DirectVulkan implements GL_KHR_shader_subgroup,
// shared by capability advertisement (BackendObject) and module lowering
// (VulkanRenderer / ProgramFactory) so the two can never disagree.
//
// Native subgroups are the implementation whenever the device has them, whatever
// their width - subgroup operations execute on the hardware paths they were made
// for. Two module-level repairs keep the GL contract intact around them:
// - FixIterationRPSubgroupScratchPass patches the one known pack bug: iterationRP's
// prefixSumCache[32], under-declared for sub-16-lane devices (8-lane lavapipe);
// - DeriveNumSubgroupsPass replaces the one builtin drivers get wrong
// (gl_NumSubgroups) with the value the rest of the topology implies.
// The 32-lane shared-memory emulation (EmulateSubgroupsPass) is a LAST RESORT for
// devices with no subgroup support at all, and only when the user opts in with
// MOBILEGL_MAGMA_EMULATE_SUBGROUP=1; it never replaces available native operations.
inline constexpr Uint32 kEmulatedSubgroupSize = 32u;
inline constexpr Uint32 kEmulatedSubgroupStages = GL_COMPUTE_SHADER_BIT;
inline constexpr Uint32 kEmulatedSubgroupFeatures =
GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_VOTE_BIT_KHR |
GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR | GL_SUBGROUP_FEATURE_BALLOT_BIT_KHR |
GL_SUBGROUP_FEATURE_SHUFFLE_BIT_KHR | GL_SUBGROUP_FEATURE_SHUFFLE_RELATIVE_BIT_KHR |
GL_SUBGROUP_FEATURE_CLUSTERED_BIT_KHR | GL_SUBGROUP_FEATURE_QUAD_BIT_KHR;
inline Bool ShouldEmulateSubgroups(const Bool nativeSubgroupSupported) {
return MG_Config::Features.MagmaEmulateSubgroup && !nativeSubgroupSupported &&
!MG_Config::Features.DisableSubgroup;
}
inline Bool ShouldFixIterationRPSubgroupScratch() {
// Auto is ON: the patch is fingerprint-gated to iterationRP's reduction and
// grows one under-declared array; every other module passes through untouched.
return MG_Config::Features.FixIterationRPSubgroupScratch !=
MG_Config::QuirkOverride::ForceOff;
}
inline Bool ShouldDeriveNumSubgroups() {
// Auto is ON: gl_NumSubgroups must agree with the gl_SubgroupID range for the GL
// contract to hold, and the derived ceil() value is the one the renderer can pin
// with REQUIRE_FULL_SUBGROUPS - the driver builtin is the value with no
// cross-driver guarantee (Adreno returns 1 for an 8-subgroup dispatch).
return MG_Config::Features.DeriveNumSubgroups != MG_Config::QuirkOverride::ForceOff;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -68,6 +68,8 @@ add_executable(MobileGLIntegrationTest
Scenarios/DoublePrecisionScenario.cpp Scenarios/DoublePrecisionScenario.cpp
Scenarios/UniformInitializerScenario.cpp Scenarios/UniformInitializerScenario.cpp
Scenarios/SwizzleAccessRoutineScenario.cpp Scenarios/SwizzleAccessRoutineScenario.cpp
Scenarios/IterationRPFirstReductionScenario.cpp
Scenarios/IterationRPScratchFixScenario.cpp
Scenarios/ProgramPipelineScenario.cpp Scenarios/ProgramPipelineScenario.cpp
Scenarios/ImageLoadStoreSsoScenario.cpp Scenarios/ImageLoadStoreSsoScenario.cpp
Scenarios/ImageTargetKindScenario.cpp Scenarios/ImageTargetKindScenario.cpp
@@ -15,6 +15,11 @@
#include <ostream> #include <ostream>
#include <sstream> #include <sstream>
#if defined(_WIN32)
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#endif
// MobileGL's own headers, in the order MobileGL/Includes.h uses them: GL/gl.h // MobileGL's own headers, in the order MobileGL/Includes.h uses them: GL/gl.h
// first, then glcorearb.h for the 3.x+ entry points. This binary links // first, then glcorearb.h for the 3.x+ entry points. This binary links
// MobileGL_s, so every gl*/egl* below binds to MobileGL's implementation, not // MobileGL_s, so every gl*/egl* below binds to MobileGL's implementation, not
@@ -53,6 +58,37 @@ namespace MGITest {
constexpr int kSurfaceWidth = 128; constexpr int kSurfaceWidth = 128;
constexpr int kSurfaceHeight = 96; constexpr int kSurfaceHeight = 96;
#if defined(_WIN32)
HWND g_testWindow = nullptr;
HWND CreateTestWindow() {
static const wchar_t* const kClassName = L"MobileGLIntegrationTestWindow";
static bool registered = false;
if (!registered) {
WNDCLASSW windowClass{};
windowClass.lpfnWndProc = DefWindowProcW;
windowClass.hInstance = GetModuleHandleW(nullptr);
windowClass.lpszClassName = kClassName;
if (RegisterClassW(&windowClass) == 0 && GetLastError() != ERROR_CLASS_ALREADY_EXISTS) {
return nullptr;
}
registered = true;
}
return CreateWindowExW(0, kClassName, L"MobileGL Integration Test", WS_OVERLAPPEDWINDOW,
CW_USEDEFAULT, CW_USEDEFAULT, kSurfaceWidth, kSurfaceHeight, nullptr, nullptr,
GetModuleHandleW(nullptr), nullptr);
}
#endif
bool UseWindowSurface() {
#if defined(_WIN32)
const char* value = std::getenv("MOBILEGL_ITEST_WINDOW_SURFACE");
return value != nullptr && value[0] != '\0' && std::strcmp(value, "0") != 0;
#else
return false;
#endif
}
std::string EnvOr(const char* name, const char* fallback) { std::string EnvOr(const char* name, const char* fallback) {
const char* value = std::getenv(name); const char* value = std::getenv(name);
return (value != nullptr && value[0] != '\0') ? std::string(value) : std::string(fallback); return (value != nullptr && value[0] != '\0') ? std::string(value) : std::string(fallback);
@@ -134,8 +170,9 @@ namespace MGITest {
return 3; return 3;
} }
const bool useWindowSurface = UseWindowSurface();
const EGLint configAttribs[] = {EGL_SURFACE_TYPE, const EGLint configAttribs[] = {EGL_SURFACE_TYPE,
EGL_PBUFFER_BIT, useWindowSurface ? EGL_WINDOW_BIT : EGL_PBUFFER_BIT,
EGL_RED_SIZE, EGL_RED_SIZE,
8, 8,
EGL_GREEN_SIZE, EGL_GREEN_SIZE,
@@ -152,7 +189,9 @@ namespace MGITest {
EGLConfig config = nullptr; EGLConfig config = nullptr;
EGLint configCount = 0; EGLint configCount = 0;
if (eglChooseConfig(display, configAttribs, &config, 1, &configCount) != EGL_TRUE || configCount < 1) { if (eglChooseConfig(display, configAttribs, &config, 1, &configCount) != EGL_TRUE || configCount < 1) {
outReason = WithEglError("eglChooseConfig found no pbuffer-capable RGBA8/D24 config"); outReason = WithEglError(useWindowSurface
? "eglChooseConfig found no window-capable RGBA8/D24 config"
: "eglChooseConfig found no pbuffer-capable RGBA8/D24 config");
return 4; return 4;
} }
@@ -166,10 +205,23 @@ namespace MGITest {
return 5; return 5;
} }
const EGLint pbufferAttribs[] = {EGL_WIDTH, kSurfaceWidth, EGL_HEIGHT, kSurfaceHeight, EGL_NONE}; EGLSurface surface = EGL_NO_SURFACE;
EGLSurface surface = eglCreatePbufferSurface(display, config, pbufferAttribs); if (useWindowSurface) {
#if defined(_WIN32)
if (g_testWindow == nullptr) g_testWindow = CreateTestWindow();
if (g_testWindow == nullptr) {
outReason = "failed to create the Windows integration-test window";
return 6;
}
surface = eglCreateWindowSurface(display, config, g_testWindow, nullptr);
#endif
} else {
const EGLint pbufferAttribs[] = {EGL_WIDTH, kSurfaceWidth, EGL_HEIGHT, kSurfaceHeight, EGL_NONE};
surface = eglCreatePbufferSurface(display, config, pbufferAttribs);
}
if (surface == EGL_NO_SURFACE) { if (surface == EGL_NO_SURFACE) {
outReason = WithEglError("eglCreatePbufferSurface failed"); outReason = WithEglError(useWindowSurface ? "eglCreateWindowSurface failed"
: "eglCreatePbufferSurface failed");
return 6; return 6;
} }
// The step that brings the whole backend up (DirectVulkan creates its // The step that brings the whole backend up (DirectVulkan creates its
@@ -491,6 +543,12 @@ namespace MGITest {
if (m_context != nullptr) eglDestroyContext(display, static_cast<EGLContext>(m_context)); if (m_context != nullptr) eglDestroyContext(display, static_cast<EGLContext>(m_context));
if (m_surface != nullptr) eglDestroySurface(display, static_cast<EGLSurface>(m_surface)); if (m_surface != nullptr) eglDestroySurface(display, static_cast<EGLSurface>(m_surface));
eglTerminate(display); eglTerminate(display);
#if defined(_WIN32)
if (g_testWindow != nullptr) {
DestroyWindow(g_testWindow);
g_testWindow = nullptr;
}
#endif
m_context = nullptr; m_context = nullptr;
m_surface = nullptr; m_surface = nullptr;
m_display = nullptr; m_display = nullptr;
@@ -0,0 +1,898 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/IterationRPFirstReductionScenario.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario - ITERATIONRP'S FIRST SUBGROUP REDUCTION.
//
// iterationRP reduces a 32 x 16 exposure tile with a vector subgroup inclusive add,
// then a shared-memory scan of subgroup totals. The source assumes that every
// subgroup has a last lane, that there are 2..32 subgroups, and that local index
// 511 belongs to the last subgroup and its last lane. Those are source assumptions,
// not API contracts. This probe intentionally does not repair them: it records the
// observed topology and makes each handoff independently observable.
#include <algorithm>
#include <array>
#include <bit>
#include <cstddef>
#include <cstdint>
#include <cstdlib>
#include <cstring>
#include <iomanip>
#include <iostream>
#include <limits>
#include <sstream>
#include <string>
#include <type_traits>
#include <utility>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
namespace MGITest {
namespace {
constexpr std::size_t kInvocationCount = 512;
constexpr std::size_t kScanStageCount = 6;
constexpr std::uint32_t kQuietNanBits = 0x7fc00000u;
constexpr std::size_t kNoSlot = std::numeric_limits<std::size_t>::max();
struct UVec4 {
std::uint32_t x;
std::uint32_t y;
std::uint32_t z;
std::uint32_t w;
};
struct Vec4 {
float x;
float y;
float z;
float w;
};
// Matches the std430 block exactly. uvec4/vec4 arrays have a 16-byte
// stride, floats are a dense scalar array, and the outer scan array is
// stage-major in both GLSL and C++.
struct ProbeOutput {
std::array<UVec4, kInvocationCount> invocation;
std::array<UVec4, kInvocationCount> subgroup;
std::array<Vec4, kInvocationCount> reduction;
std::array<float, kInvocationCount> finalAverage;
std::array<std::array<float, kInvocationCount>, kScanStageCount> scanAfter;
};
static_assert(sizeof(UVec4) == 16);
static_assert(sizeof(Vec4) == 16);
static_assert(std::is_standard_layout_v<ProbeOutput>);
static_assert(offsetof(ProbeOutput, invocation) == 0);
static_assert(offsetof(ProbeOutput, subgroup) == 8192);
static_assert(offsetof(ProbeOutput, reduction) == 16384);
static_assert(offsetof(ProbeOutput, finalAverage) == 24576);
static_assert(offsetof(ProbeOutput, scanAfter) == 26624);
static_assert(sizeof(ProbeOutput) == 38912);
enum class InputMode {
SampledRgba32f,
IndexedSsbo,
};
const char* InputModeName(InputMode mode) {
return mode == InputMode::SampledRgba32f ? "sampled RGBA32F" : "indexed SSBO";
}
std::uint32_t FloatBits(float value) {
return std::bit_cast<std::uint32_t>(value);
}
bool SameBits(float lhs, float rhs) {
return FloatBits(lhs) == FloatBits(rhs);
}
bool IsQuietNanSentinel(float value) {
return FloatBits(value) == kQuietNanBits;
}
bool DrainGlErrors() {
bool hadError = false;
while (glGetError() != GL_NO_ERROR) hadError = true;
return hadError;
}
bool HasExtension(const char* wanted) {
GLint extensionCount = 0;
glGetIntegerv(GL_NUM_EXTENSIONS, &extensionCount);
for (GLint i = 0; i < extensionCount; ++i) {
const auto* extension = reinterpret_cast<const char*>(glGetStringi(GL_EXTENSIONS, static_cast<GLuint>(i)));
if (extension != nullptr && std::string(extension) == wanted) return true;
}
return false;
}
struct CapabilityInfo {
bool subgroupExtension = false;
GLint subgroupSize = 0;
GLint supportedStages = 0;
GLint supportedFeatures = 0;
GLint maxComputeStorageBlocks = 0;
GLint maxStorageBindings = 0;
GLint maxWorkGroupInvocations = 0;
std::array<GLint, 3> maxWorkGroupSize{};
bool queryHadError = false;
// iterationRP's source contract needs gl_NumSubgroups in [2, 32] for its 512
// invocations, i.e. an advertised subgroup width in [16, 256]. A device
// outside that window (lavapipe's 8-lane subgroups give 64 subgroups) cannot
// run the fixture's verbatim reduction at all, so the scenario SKIPS there -
// the pack itself replays through the FixIterationRPSubgroupScratch patch, which
// this probe deliberately does not model. The width only gates the domain;
// lane placement and group counts still come from observed values alone.
bool SubgroupWidthInSourceDomain() const {
return subgroupSize >= 16 && subgroupSize <= 256;
}
bool SupportsProbe() const {
const auto stages = static_cast<GLbitfield>(supportedStages);
const auto features = static_cast<GLbitfield>(supportedFeatures);
return !queryHadError && subgroupExtension &&
(stages & GL_COMPUTE_SHADER_BIT) != 0 &&
(features & (GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR)) ==
(GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR) &&
SubgroupWidthInSourceDomain() &&
maxComputeStorageBlocks >= 2 && maxStorageBindings >= 2 &&
maxWorkGroupInvocations >= static_cast<GLint>(kInvocationCount) && maxWorkGroupSize[0] >= 32 &&
maxWorkGroupSize[1] >= 16 && maxWorkGroupSize[2] >= 1;
}
std::string MissingRequirements() const {
std::vector<std::string> missing;
const auto stages = static_cast<GLbitfield>(supportedStages);
const auto features = static_cast<GLbitfield>(supportedFeatures);
if (queryHadError) missing.emplace_back("a subgroup/compute capability query generated GL error");
if (!subgroupExtension) missing.emplace_back("GL_KHR_shader_subgroup");
if ((stages & GL_COMPUTE_SHADER_BIT) == 0) {
missing.emplace_back("GL_COMPUTE_SHADER_BIT in GL_SUBGROUP_SUPPORTED_STAGES_KHR");
}
const auto requiredFeatures =
GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR;
if ((features & requiredFeatures) != requiredFeatures) {
missing.emplace_back("basic|arithmetic in GL_SUBGROUP_SUPPORTED_FEATURES_KHR");
}
if (!SubgroupWidthInSourceDomain()) {
missing.emplace_back(
"GL_SUBGROUP_SIZE_KHR in [16, 256] (iterationRP's source contract needs "
"gl_NumSubgroups in [2, 32] for 512 invocations; width " +
std::to_string(subgroupSize) + " is outside the fixture's domain)");
}
if (maxComputeStorageBlocks < 2 || maxStorageBindings < 2) {
missing.emplace_back("two compute SSBO bindings");
}
if (maxWorkGroupInvocations < static_cast<GLint>(kInvocationCount) || maxWorkGroupSize[0] < 32 ||
maxWorkGroupSize[1] < 16 || maxWorkGroupSize[2] < 1) {
missing.emplace_back("a 32x16x1 / 512-invocation compute workgroup");
}
std::ostringstream message;
for (std::size_t i = 0; i < missing.size(); ++i) {
if (i != 0) message << ", ";
message << missing[i];
}
return message.str();
}
};
CapabilityInfo QueryCapabilities() {
CapabilityInfo info;
DrainGlErrors();
info.subgroupExtension = HasExtension("GL_KHR_shader_subgroup");
glGetIntegerv(GL_SUBGROUP_SIZE_KHR, &info.subgroupSize);
glGetIntegerv(GL_SUBGROUP_SUPPORTED_STAGES_KHR, &info.supportedStages);
glGetIntegerv(GL_SUBGROUP_SUPPORTED_FEATURES_KHR, &info.supportedFeatures);
glGetIntegerv(GL_MAX_COMPUTE_SHADER_STORAGE_BLOCKS, &info.maxComputeStorageBlocks);
glGetIntegerv(GL_MAX_SHADER_STORAGE_BUFFER_BINDINGS, &info.maxStorageBindings);
glGetIntegerv(GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, &info.maxWorkGroupInvocations);
for (GLuint axis = 0; axis < info.maxWorkGroupSize.size(); ++axis) {
glGetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, axis, &info.maxWorkGroupSize[axis]);
}
info.queryHadError = DrainGlErrors();
return info;
}
void PrintMetadata(const CapabilityInfo& info, std::ostream& output) {
output << "IterationRPFirstReductionScenario metadata: "
<< "GL_SUBGROUP_SIZE_KHR=" << info.subgroupSize
<< ", GL_SUBGROUP_SUPPORTED_STAGES_KHR=0x" << std::hex
<< static_cast<GLbitfield>(info.supportedStages)
<< ", GL_SUBGROUP_SUPPORTED_FEATURES_KHR=0x"
<< static_cast<GLbitfield>(info.supportedFeatures) << std::dec
<< ", subgroupExtension=" << info.subgroupExtension
<< ", GL_MAX_COMPUTE_SHADER_STORAGE_BLOCKS=" << info.maxComputeStorageBlocks
<< ", GL_MAX_SHADER_STORAGE_BUFFER_BINDINGS=" << info.maxStorageBindings
<< ", GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS=" << info.maxWorkGroupInvocations
<< ", GL_MAX_COMPUTE_WORK_GROUP_SIZE=" << info.maxWorkGroupSize[0] << 'x'
<< info.maxWorkGroupSize[1] << 'x' << info.maxWorkGroupSize[2]
<< ", queryHadError=" << info.queryHadError << '\n';
}
bool DumpRequested() {
const char* value = std::getenv("MOBILEGL_ITEST_SUBGROUP_PROBE_DUMP");
return value != nullptr && std::string(value) == "1";
}
constexpr const char* kShaderPreamble = R"(#version 430 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 1) buffer SubgroupProbeOutput {
uvec4 invocation[512];
uvec4 subgroup[512];
vec4 reduction[512];
float finalAverage[512];
float scanAfter[6][512];
} outProbe;
shared vec2 prefixSumCache[32];
)";
constexpr const char* kSampledInput = R"(
uniform sampler2D colortex2;
uniform vec2 pixelSize;
)";
constexpr const char* kIndexedInput = R"(
layout(std430, binding = 0) readonly buffer Input {
float value[512];
} inputData;
)";
// Only the expression producing tileExposure differs between the two
// tests. The remainder is the iterationRP first reduction, with stores
// placed after its existing barriers to expose each handoff.
constexpr const char* kSampledTileExposure = R"(
vec2 texCoord = (vec2(gl_GlobalInvocationID.xy) + 0.5) *
vec2(1.0 / 32.0, 1.0 / 16.0);
vec2 sampleCoord = texCoord * (1.0 / 64.0);
sampleCoord.x += (15.0 / 32.0) + pixelSize.x * 12.0;
float tileExposure = dot(
textureLod(colortex2, sampleCoord, 0.0).rgb,
vec3(0.2125, 0.7154, 0.0721));
)";
constexpr const char* kIndexedTileExposure = R"(
float tileExposure = inputData.value[gl_LocalInvocationIndex];
)";
constexpr const char* kReductionBody = R"(
vec2 sampleLuminance = vec2(tileExposure, 0.0);
sampleLuminance = subgroupInclusiveAdd(sampleLuminance);
float nativeInclusive = sampleLuminance.x;
// This is a uniform, safety-only branch: it leaves an invalid source
// contract visible without indexing past the 32-entry cache or underflowing
// loopLength - 1. It is deliberately a failure on the CPU, not a skip.
bool sourceDomain = gl_NumSubgroups >= 2u && gl_NumSubgroups <= 32u;
if (!sourceDomain) {
float qNaN = uintBitsToFloat(0x7fc00000u);
uint localIndex = gl_LocalInvocationIndex;
outProbe.invocation[localIndex] = uvec4(localIndex, gl_LocalInvocationID);
outProbe.subgroup[localIndex] = uvec4(gl_SubgroupSize, gl_NumSubgroups, gl_SubgroupID,
gl_SubgroupInvocationID);
outProbe.reduction[localIndex] = vec4(tileExposure, nativeInclusive, qNaN, qNaN);
outProbe.finalAverage[localIndex] = qNaN;
for (uint stage = 0u; stage < 6u; ++stage)
outProbe.scanAfter[stage][localIndex] = qNaN;
return;
}
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
barrier();
float sourceRawSubtotal = prefixSumCache[gl_SubgroupID].x;
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
sampleLuminance += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
}
barrier();
outProbe.scanAfter[scanStage][gl_LocalInvocationIndex] = sampleLuminance.x;
}
float sourceMergedPrefix = sampleLuminance.x;
if (gl_LocalInvocationIndex == 511u)
prefixSumCache[0] = sampleLuminance / 512.0;
barrier();
float avg = prefixSumCache[0].x;
uint localIndex = gl_LocalInvocationIndex;
outProbe.invocation[localIndex] = uvec4(localIndex, gl_LocalInvocationID);
outProbe.subgroup[localIndex] = uvec4(gl_SubgroupSize, gl_NumSubgroups, gl_SubgroupID,
gl_SubgroupInvocationID);
outProbe.reduction[localIndex] = vec4(tileExposure, nativeInclusive, sourceRawSubtotal, sourceMergedPrefix);
outProbe.finalAverage[localIndex] = avg;
}
)";
std::string BuildProbeShader(InputMode mode) {
std::string source = kShaderPreamble;
source += mode == InputMode::SampledRgba32f ? kSampledInput : kIndexedInput;
source += "\nvoid main() {\n";
source += mode == InputMode::SampledRgba32f ? kSampledTileExposure : kIndexedTileExposure;
source += kReductionBody;
return source;
}
std::string FormatFloat(float value) {
std::ostringstream text;
text << std::hexfloat << value;
return text.str();
}
struct ValidationResult {
bool ok = true;
std::string phase;
std::string message;
bool scanStageMismatch = false;
int scanStage = -1;
bool ownerEvaluated = false;
bool index511IsSourceLastLaneWriter = false;
bool index511IsHighestSubgroupMember = false;
std::uint32_t highestObservedSubgroup = 0;
};
ValidationResult Failure(std::string phase, std::string message) {
ValidationResult result;
result.ok = false;
result.phase = std::move(phase);
result.message = std::move(message);
return result;
}
constexpr float kSampledLuminance = 0.2125f + 0.7154f + 0.0721f;
float ExpectedInput(InputMode mode, std::uint32_t localIndex) {
return mode == InputMode::SampledRgba32f ? kSampledLuminance : static_cast<float>(localIndex + 1u);
}
ValidationResult ValidateProbe(const ProbeOutput& output, InputMode mode) {
std::array<std::size_t, kInvocationCount> slotForLocal{};
slotForLocal.fill(kNoSlot);
// 1. Record identity. Slots are only used to locate each reported
// local index; all subgroup behavior below groups recorded IDs/lanes.
for (std::size_t slot = 0; slot < kInvocationCount; ++slot) {
const std::uint32_t localIndex = output.invocation[slot].x;
if (localIndex >= kInvocationCount) {
std::ostringstream message;
message << "output slot " << slot << " reports localIndex " << localIndex << " outside [0, 511]";
return Failure("record identity", message.str());
}
if (slotForLocal[localIndex] != kNoSlot) {
std::ostringstream message;
message << "localIndex " << localIndex << " appears in output slots " << slotForLocal[localIndex]
<< " and " << slot;
return Failure("record identity", message.str());
}
slotForLocal[localIndex] = slot;
}
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
if (slotForLocal[localIndex] == kNoSlot) {
std::ostringstream message;
message << "localIndex " << localIndex << " is missing from all 512 records";
return Failure("record identity", message.str());
}
}
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
const std::size_t slot = slotForLocal[localIndex];
const UVec4& invocation = output.invocation[slot];
const std::uint32_t expectedX = static_cast<std::uint32_t>(localIndex % 32u);
const std::uint32_t expectedY = static_cast<std::uint32_t>(localIndex / 32u);
if (invocation.y != expectedX || invocation.z != expectedY || invocation.w != 0u) {
std::ostringstream message;
message << "localIndex " << localIndex << " reports local invocation (" << invocation.y << ','
<< invocation.z << ',' << invocation.w << "), expected (" << expectedX << ',' << expectedY
<< ",0)";
return Failure("record identity", message.str());
}
const float expectedInput = ExpectedInput(mode, static_cast<std::uint32_t>(localIndex));
const float actualInput = output.reduction[slot].x;
if (!SameBits(actualInput, expectedInput)) {
std::ostringstream message;
message << "localIndex " << localIndex << " input was " << FormatFloat(actualInput) << ", expected "
<< FormatFloat(expectedInput);
return Failure("input", message.str());
}
}
// 2. Observed topology. Do not derive lanes or subgroup membership
// from local invocation indices: only the values the shader recorded
// participate in grouping.
const std::uint32_t reportedNumSubgroups = output.subgroup[slotForLocal[0]].y;
if (reportedNumSubgroups == 0u) {
return Failure("observed topology", "localIndex 0 reported gl_NumSubgroups == 0");
}
if (reportedNumSubgroups > kInvocationCount) {
std::ostringstream message;
message << "reported gl_NumSubgroups=" << reportedNumSubgroups
<< " exceeds the 512 recorded invocations, so at least one subgroup ID is missing";
return Failure("observed topology", message.str());
}
std::vector<std::vector<std::size_t>> subgroupSlots(reportedNumSubgroups);
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
const std::size_t slot = slotForLocal[localIndex];
const UVec4& subgroup = output.subgroup[slot];
if (subgroup.x == 0u || subgroup.y == 0u) {
std::ostringstream message;
message << "localIndex " << localIndex << " reported subgroupSize=" << subgroup.x
<< ", numSubgroups=" << subgroup.y;
return Failure("observed topology", message.str());
}
if (subgroup.y != reportedNumSubgroups) {
std::ostringstream message;
message << "localIndex " << localIndex << " reported numSubgroups=" << subgroup.y
<< ", while localIndex 0 reported " << reportedNumSubgroups;
return Failure("observed topology", message.str());
}
if (subgroup.z >= reportedNumSubgroups) {
std::ostringstream message;
message << "localIndex " << localIndex << " reported subgroupID=" << subgroup.z
<< " outside [0, " << (reportedNumSubgroups - 1u) << ']';
return Failure("observed topology", message.str());
}
if (subgroup.w >= subgroup.x) {
std::ostringstream message;
message << "localIndex " << localIndex << " reported laneID=" << subgroup.w
<< " outside its subgroupSize=" << subgroup.x;
return Failure("observed topology", message.str());
}
subgroupSlots[subgroup.z].push_back(slot);
}
for (std::uint32_t subgroupID = 0; subgroupID < reportedNumSubgroups; ++subgroupID) {
if (subgroupSlots[subgroupID].empty()) {
std::ostringstream message;
message << "reported gl_NumSubgroups=" << reportedNumSubgroups
<< " but subgroupID " << subgroupID << " has no recorded members";
return Failure("observed topology", message.str());
}
auto& members = subgroupSlots[subgroupID];
std::sort(members.begin(), members.end(), [&output](std::size_t lhs, std::size_t rhs) {
return output.subgroup[lhs].w < output.subgroup[rhs].w;
});
for (std::size_t i = 1; i < members.size(); ++i) {
if (output.subgroup[members[i - 1]].w == output.subgroup[members[i]].w) {
std::ostringstream message;
message << "subgroupID " << subgroupID << " contains duplicate laneID "
<< output.subgroup[members[i]].w;
return Failure("observed topology", message.str());
}
}
}
// 3. Native subgroup arithmetic, in the actual lane ordering emitted
// by the driver. The fixture values and all partial sums are exactly
// representable binary32 values, so compare representation, not epsilon.
std::array<float, kInvocationCount> nativePrefix{};
std::vector<float> nativeSubtotal(reportedNumSubgroups, 0.0f);
for (std::uint32_t subgroupID = 0; subgroupID < reportedNumSubgroups; ++subgroupID) {
float inclusive = 0.0f;
for (const std::size_t slot : subgroupSlots[subgroupID]) {
const std::uint32_t localIndex = output.invocation[slot].x;
inclusive += ExpectedInput(mode, localIndex);
nativePrefix[slot] = inclusive;
const float actualNative = output.reduction[slot].y;
if (!SameBits(actualNative, inclusive)) {
std::ostringstream message;
message << "subgroupID " << subgroupID << ", laneID " << output.subgroup[slot].w
<< ", localIndex " << localIndex << " nativeInclusive was " << FormatFloat(actualNative)
<< ", expected " << FormatFloat(inclusive);
return Failure("native subgroup arithmetic", message.str());
}
}
nativeSubtotal[subgroupID] = inclusive;
}
// sourceDomain is the narrow source-side safety branch. It is checked
// after native arithmetic so an unsupported source topology still
// reports native subgroup behavior before failing explicitly.
if (reportedNumSubgroups < 2u || reportedNumSubgroups > 32u) {
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
const std::size_t slot = slotForLocal[localIndex];
const Vec4& reduction = output.reduction[slot];
if (!IsQuietNanSentinel(reduction.z) || !IsQuietNanSentinel(reduction.w) ||
!IsQuietNanSentinel(output.finalAverage[slot])) {
std::ostringstream message;
message << "iterationRP source reduction has no valid contract for gl_NumSubgroups="
<< reportedNumSubgroups << "; localIndex " << localIndex
<< " did not preserve its qNaN source-reduction sentinel";
return Failure("source domain", message.str());
}
for (std::size_t stage = 0; stage < kScanStageCount; ++stage) {
if (!IsQuietNanSentinel(output.scanAfter[stage][slot])) {
std::ostringstream message;
message << "iterationRP source reduction has no valid contract for gl_NumSubgroups="
<< reportedNumSubgroups << "; localIndex " << localIndex << ", scan stage " << stage
<< " did not preserve its qNaN source-reduction sentinel";
return Failure("source domain", message.str());
}
}
}
std::ostringstream message;
message << "iterationRP source reduction has no valid contract for observed gl_NumSubgroups="
<< reportedNumSubgroups << " (requires 2..32); native subgroup results were recorded";
return Failure("source domain", message.str());
}
// 4. iterationRP source writer and first shared-memory handoff.
std::vector<std::size_t> sourceWriter(reportedNumSubgroups, kNoSlot);
for (std::uint32_t subgroupID = 0; subgroupID < reportedNumSubgroups; ++subgroupID) {
std::size_t writerCount = 0;
for (const std::size_t slot : subgroupSlots[subgroupID]) {
const UVec4& subgroup = output.subgroup[slot];
if (subgroup.w == subgroup.x - 1u) {
sourceWriter[subgroupID] = slot;
++writerCount;
}
}
if (writerCount != 1u) {
std::ostringstream message;
message << "subgroupID " << subgroupID << " has " << writerCount
<< " recorded lane(s) where laneID == subgroupSize - 1; iterationRP leaves that "
"shared-cache entry unwritten";
return Failure("source writer", message.str());
}
for (const std::size_t slot : subgroupSlots[subgroupID]) {
const float actualRawSubtotal = output.reduction[slot].z;
if (!SameBits(actualRawSubtotal, nativeSubtotal[subgroupID])) {
std::ostringstream message;
message << "subgroupID " << subgroupID << ", localIndex " << output.invocation[slot].x
<< " sourceRawSubtotal was " << FormatFloat(actualRawSubtotal) << ", expected "
<< FormatFloat(nativeSubtotal[subgroupID]);
return Failure("source raw subtotal", message.str());
}
}
}
// 5. Reproduce the source loop exactly, including the redundant final
// scan iteration on power-of-two subgroup counts. Reads and writes in
// one iteration target disjoint cache entries, so update the cache at
// the CPU equivalent of the source barrier.
std::array<float, kInvocationCount> mergedPrefix = nativePrefix;
std::vector<float> cache = nativeSubtotal;
std::uint32_t loopLength = std::bit_width(reportedNumSubgroups) - 1u;
loopLength +=
static_cast<std::uint32_t>(reportedNumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (std::uint32_t scanStage = 0u; scanStage < loopLength; ++scanStage) {
std::vector<float> cacheAfterStage = cache;
for (std::uint32_t subgroupID = 0; subgroupID < reportedNumSubgroups; ++subgroupID) {
if ((subgroupID & (1u << scanStage)) == 0u) continue;
const std::uint32_t sourceCacheIndex = (subgroupID >> scanStage << scanStage) - 1u;
const float sourcePrefix = cache[sourceCacheIndex];
for (const std::size_t slot : subgroupSlots[subgroupID]) {
mergedPrefix[slot] += sourcePrefix;
}
cacheAfterStage[subgroupID] = mergedPrefix[sourceWriter[subgroupID]];
}
cache.swap(cacheAfterStage);
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
const std::size_t slot = slotForLocal[localIndex];
const float actualAfterStage = output.scanAfter[scanStage][slot];
if (!SameBits(actualAfterStage, mergedPrefix[slot])) {
std::ostringstream message;
message << "scanStage " << scanStage << ", subgroupID " << output.subgroup[slot].z
<< ", laneID " << output.subgroup[slot].w << ", localIndex " << localIndex
<< " scanAfter was " << FormatFloat(actualAfterStage) << ", expected "
<< FormatFloat(mergedPrefix[slot]);
ValidationResult result = Failure("source scan", message.str());
result.scanStageMismatch = true;
result.scanStage = static_cast<int>(scanStage);
return result;
}
}
}
for (std::size_t localIndex = 0; localIndex < kInvocationCount; ++localIndex) {
const std::size_t slot = slotForLocal[localIndex];
const float actualMergedPrefix = output.reduction[slot].w;
if (!SameBits(actualMergedPrefix, mergedPrefix[slot])) {
std::ostringstream message;
message << "localIndex " << localIndex << " sourceMergedPrefix was "
<< FormatFloat(actualMergedPrefix) << ", expected " << FormatFloat(mergedPrefix[slot]);
return Failure("source scan", message.str());
}
}
// 6. Final owner and average. The uniformity check is intentionally
// separate from the source's topology contract at local index 511.
const float firstAverage = output.finalAverage[slotForLocal[0]];
for (std::size_t localIndex = 1; localIndex < kInvocationCount; ++localIndex) {
const float actualAverage = output.finalAverage[slotForLocal[localIndex]];
if (!SameBits(actualAverage, firstAverage)) {
std::ostringstream message;
message << "finalAverage differs: localIndex 0 has " << FormatFloat(firstAverage)
<< ", localIndex " << localIndex << " has " << FormatFloat(actualAverage);
return Failure("final average", message.str());
}
}
ValidationResult ownerResult;
ownerResult.ownerEvaluated = true;
for (std::uint32_t subgroupID = 0; subgroupID < reportedNumSubgroups; ++subgroupID) {
if (!subgroupSlots[subgroupID].empty()) {
ownerResult.highestObservedSubgroup = std::max(ownerResult.highestObservedSubgroup, subgroupID);
}
}
const std::size_t index511Slot = slotForLocal[kInvocationCount - 1u];
const UVec4& index511Subgroup = output.subgroup[index511Slot];
ownerResult.index511IsSourceLastLaneWriter =
index511Subgroup.w == index511Subgroup.x - 1u;
ownerResult.index511IsHighestSubgroupMember =
index511Subgroup.z == ownerResult.highestObservedSubgroup;
if (!ownerResult.index511IsSourceLastLaneWriter || !ownerResult.index511IsHighestSubgroupMember) {
std::ostringstream message;
message << "iterationRP topology incompatibility: localIndex 511 is sourceLastLaneWriter="
<< ownerResult.index511IsSourceLastLaneWriter << ", highestSubgroupMember="
<< ownerResult.index511IsHighestSubgroupMember << " (subgroupID=" << index511Subgroup.z
<< ", highest observed subgroupID=" << ownerResult.highestObservedSubgroup << ')';
ownerResult.ok = false;
ownerResult.phase = "final average";
ownerResult.message = message.str();
return ownerResult;
}
float total = 0.0f;
for (const float subtotal : nativeSubtotal) total += subtotal;
float sampledExpectedTotal = 0.0f;
for (std::size_t i = 0; i < kInvocationCount; ++i) sampledExpectedTotal += kSampledLuminance;
const float expectedTotal = mode == InputMode::IndexedSsbo ? 131328.0f : sampledExpectedTotal;
if (!SameBits(total, expectedTotal) || !SameBits(mergedPrefix[index511Slot], expectedTotal)) {
std::ostringstream message;
message << "iterationRP source total was " << FormatFloat(mergedPrefix[index511Slot])
<< " (native total " << FormatFloat(total) << "), expected " << FormatFloat(expectedTotal);
ownerResult.ok = false;
ownerResult.phase = "final average";
ownerResult.message = message.str();
return ownerResult;
}
const float expectedAverage = mode == InputMode::IndexedSsbo ? 256.5f : sampledExpectedTotal / 512.0f;
if (!SameBits(firstAverage, expectedAverage)) {
std::ostringstream message;
message << "finalAverage was " << FormatFloat(firstAverage) << ", expected "
<< FormatFloat(expectedAverage);
ownerResult.ok = false;
ownerResult.phase = "final average";
ownerResult.message = message.str();
return ownerResult;
}
return ownerResult;
}
void DumpProbe(const ProbeOutput& output, const CapabilityInfo& capabilities, const ValidationResult& validation,
bool includeScanStages) {
PrintMetadata(capabilities, std::cout);
if (validation.ok) {
std::cout << "IterationRPFirstReductionScenario firstFailure=none\n";
} else {
std::cout << "IterationRPFirstReductionScenario firstFailure=" << validation.phase << ": "
<< validation.message << '\n';
}
std::cout << "localIndex,localX,localY,localZ,subgroupSize,numSubgroups,subgroupID,laneID,input,"
"nativeInclusive,subgroupSubtotal,mergedPrefix,finalAverage\n";
for (std::size_t slot = 0; slot < kInvocationCount; ++slot) {
const UVec4& invocation = output.invocation[slot];
const UVec4& subgroup = output.subgroup[slot];
const Vec4& reduction = output.reduction[slot];
std::cout << invocation.x << ',' << invocation.y << ',' << invocation.z << ',' << invocation.w << ','
<< subgroup.x << ',' << subgroup.y << ',' << subgroup.z << ',' << subgroup.w << ','
<< std::hexfloat << reduction.x << ',' << reduction.y << ',' << reduction.z << ','
<< reduction.w << ',' << output.finalAverage[slot] << std::defaultfloat << '\n';
}
if (includeScanStages) {
std::cout << "scanStage,localIndex,scanAfter\n";
for (std::size_t scanStage = 0; scanStage < kScanStageCount; ++scanStage) {
for (std::size_t slot = 0; slot < kInvocationCount; ++slot) {
std::cout << scanStage << ',' << output.invocation[slot].x << ',' << std::hexfloat
<< output.scanAfter[scanStage][slot] << std::defaultfloat << '\n';
}
}
}
}
class IterationRPFirstReductionScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
m_capabilities = QueryCapabilities();
// GL_SUBGROUP_SIZE_KHR gates only whether the fixture's source contract
// can hold on this device (SubgroupWidthInSourceDomain); it is
// deliberately never used to infer lane placement or an expected group
// count - those come from observed values alone.
PrintMetadata(m_capabilities, std::cout);
RecordProperty("iterationrp_gl_subgroup_size_khr", std::to_string(m_capabilities.subgroupSize));
if (!m_capabilities.SupportsProbe()) {
GTEST_SKIP() << "subgroup probe requires " << m_capabilities.MissingRequirements();
}
}
void TearDown() override {
if (!Ready()) return;
glUseProgram(0);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, 0);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, 0);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, 0);
glActiveTexture(GL_TEXTURE3);
glBindTexture(GL_TEXTURE_2D, 0);
glActiveTexture(GL_TEXTURE0);
if (m_texture != 0) glDeleteTextures(1, &m_texture);
if (m_inputBuffer != 0) glDeleteBuffers(1, &m_inputBuffer);
if (m_outputBuffer != 0) glDeleteBuffers(1, &m_outputBuffer);
if (m_program != 0) glDeleteProgram(m_program);
m_texture = 0;
m_inputBuffer = 0;
m_outputBuffer = 0;
m_program = 0;
}
GLuint CompileComputeProgram(const std::string& source, std::string* outError) {
const char* text = source.c_str();
const GLuint shader = glCreateShader(GL_COMPUTE_SHADER);
if (shader == 0) {
*outError = "glCreateShader(GL_COMPUTE_SHADER) returned 0";
return 0;
}
glShaderSource(shader, 1, &text, nullptr);
glCompileShader(shader);
GLint compiled = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
if (compiled == GL_FALSE) {
char log[8192] = {};
glGetShaderInfoLog(shader, sizeof(log) - 1, nullptr, log);
*outError = std::string("the subgroup probe compute shader did not compile: ") + log;
glDeleteShader(shader);
return 0;
}
const GLuint program = glCreateProgram();
glAttachShader(program, shader);
glLinkProgram(program);
glDeleteShader(shader);
GLint linked = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &linked);
if (linked == GL_FALSE) {
char log[8192] = {};
glGetProgramInfoLog(program, sizeof(log) - 1, nullptr, log);
*outError = std::string("the subgroup probe compute program did not link: ") + log;
glDeleteProgram(program);
return 0;
}
return program;
}
bool RunProbe(InputMode mode, ProbeOutput* output, std::string* outError) {
m_program = CompileComputeProgram(BuildProbeShader(mode), outError);
if (m_program == 0) return false;
ProbeOutput poison{};
std::memset(&poison, 0xa5, sizeof(poison));
glGenBuffers(1, &m_outputBuffer);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_outputBuffer);
glBufferData(GL_SHADER_STORAGE_BUFFER, sizeof(ProbeOutput), &poison, GL_DYNAMIC_COPY);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, m_outputBuffer);
if (mode == InputMode::IndexedSsbo) {
std::array<float, kInvocationCount> values{};
for (std::size_t i = 0; i < values.size(); ++i) values[i] = static_cast<float>(i + 1u);
glGenBuffers(1, &m_inputBuffer);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_inputBuffer);
glBufferData(GL_SHADER_STORAGE_BUFFER, sizeof(values), values.data(), GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, m_inputBuffer);
} else {
constexpr std::array<float, 4> kOneTexel = {1.0f, 1.0f, 1.0f, 1.0f};
glGenTextures(1, &m_texture);
glActiveTexture(GL_TEXTURE3);
glBindTexture(GL_TEXTURE_2D, m_texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F, 1, 1, 0, GL_RGBA, GL_FLOAT, kOneTexel.data());
}
if (const GLenum error = FirstGLError(); error != GL_NO_ERROR) {
std::ostringstream message;
message << "subgroup probe resource setup left " << GLErrorName(error);
*outError = message.str();
return false;
}
glUseProgram(m_program);
if (mode == InputMode::SampledRgba32f) {
const GLint sampler = glGetUniformLocation(m_program, "colortex2");
const GLint pixelSize = glGetUniformLocation(m_program, "pixelSize");
if (sampler == -1 || pixelSize == -1) {
*outError = "the sampled probe uniforms were optimized away or not reflected";
return false;
}
glUniform1i(sampler, 3);
glUniform2f(pixelSize, 1.0f / 854.0f, 1.0f / 480.0f);
}
glDispatchCompute(1, 1, 1);
glMemoryBarrier(GL_ALL_BARRIER_BITS);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_outputBuffer);
glGetBufferSubData(GL_SHADER_STORAGE_BUFFER, 0, sizeof(ProbeOutput), output);
if (const GLenum error = FirstGLError(); error != GL_NO_ERROR) {
std::ostringstream message;
message << "subgroup probe dispatch/readback left " << GLErrorName(error);
*outError = message.str();
return false;
}
return true;
}
void RunAndValidate(InputMode mode) {
ProbeOutput output{};
std::string error;
ASSERT_TRUE(RunProbe(mode, &output, &error)) << InputModeName(mode) << ": " << error;
const ValidationResult validation = ValidateProbe(output, mode);
if (validation.ownerEvaluated) {
RecordProperty("iterationrp_index511_source_last_lane_writer",
validation.index511IsSourceLastLaneWriter ? "true" : "false");
RecordProperty("iterationrp_index511_highest_subgroup_member",
validation.index511IsHighestSubgroupMember ? "true" : "false");
RecordProperty("iterationrp_highest_observed_subgroup",
std::to_string(validation.highestObservedSubgroup));
std::cout << "IterationRPFirstReductionScenario owner: localIndex511 sourceLastLaneWriter="
<< validation.index511IsSourceLastLaneWriter << ", highestSubgroupMember="
<< validation.index511IsHighestSubgroupMember << ", highestObservedSubgroup="
<< validation.highestObservedSubgroup << '\n';
}
if (!validation.ok || DumpRequested()) {
DumpProbe(output, m_capabilities, validation, validation.scanStageMismatch || DumpRequested());
}
EXPECT_TRUE(validation.ok) << validation.phase << ": " << validation.message;
}
CapabilityInfo m_capabilities;
GLuint m_program = 0;
GLuint m_inputBuffer = 0;
GLuint m_outputBuffer = 0;
GLuint m_texture = 0;
};
} // namespace
TEST_F(IterationRPFirstReductionScenario, SampledRgba32fFirstAverage) {
if (!Ready() || IsSkipped()) return;
RunAndValidate(InputMode::SampledRgba32f);
}
TEST_F(IterationRPFirstReductionScenario, IndexedInputTopologyAndReduction) {
if (!Ready() || IsSkipped()) return;
RunAndValidate(InputMode::IndexedSsbo);
}
} // namespace MGITest
@@ -0,0 +1,302 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/IterationRPScratchFixScenario.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario - THE FIXTURE-SHAPED SUBGROUP REDUCTION, ON WHATEVER WIDTH THE DEVICE HAS.
//
// iterationRP hard-sizes the scratch its subgroup prefix scans write through
// prefixSumCache[gl_SubgroupID], and ships that idiom twice: the auto-exposure pass
// declares `shared vec2 prefixSumCache[32]` for a 512-invocation workgroup, and the
// RTW importance warp declares `shared float prefixSumCache[64]` for a 1024-invocation
// one. Both algorithms are width-agnostic; only the static lengths bake in "at most 32
// (respectively 64) subgroups", which every desktop capture satisfies and an 8-lane
// device (lavapipe: 64 and 128 subgroups) does not. DirectVulkan patches exactly that with
// FixIterationRPSubgroupScratchPass, growing the array to ceil(invocations / native
// width) on the modules that match the pack's reduction fingerprint.
//
// This scenario replays the fixture's reduction shape verbatim - the same 32-entry
// declaration, the same last-lane handoff, the same findMSB combine loop, and NO
// domain guard - and asserts only the width-independent result: the workgroup total.
// The inputs are small integers, so the fp32 sum is exact under any lane order and any
// association; a correct run produces the exact constant on a 4-lane device and a
// 128-lane device alike. Without the patch, a sub-16-lane device indexes the
// 32-entry array out of bounds - on lavapipe that is literal heap corruption - and
// this scenario is the regression test that keeps the patch working, and it runs on every device that
// has basic+arithmetic compute subgroups (unlike IterationRPFirstReductionScenario,
// which probes the UNREPAIRED source contract and must skip outside [16, 256]).
#include <cstdint>
#include <cstring>
#include <string>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
namespace MGITest {
namespace {
constexpr std::uint32_t kInvocationCount = 512u;
// sum of 0..511, exactly representable and associativity-proof in fp32.
constexpr float kExpectedTotal = 130816.0f;
// The RTW warp's shape: 1024 invocations into a 64-entry float scratch.
constexpr std::uint32_t kWideInvocationCount = 1024u;
// sum of 0..1023, likewise exact in fp32.
constexpr float kWideExpectedTotal = 523776.0f;
constexpr const char* kComputeSource = R"(#version 430 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output {
float total;
uint numSubgroups;
uint maxSubgroupId;
} outputData;
shared vec2 prefixSumCache[32];
void main() {
vec2 sampleLuminance = vec2(float(gl_LocalInvocationIndex), 0.0);
sampleLuminance = subgroupInclusiveAdd(sampleLuminance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
sampleLuminance += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
}
barrier();
}
if (gl_LocalInvocationIndex == 511u) {
outputData.total = sampleLuminance.x;
outputData.numSubgroups = gl_NumSubgroups;
}
atomicMax(outputData.maxSubgroupId, gl_SubgroupID);
}
)";
// The RTW importance warp's shape: a plain float scan over 1024 invocations
// into a 64-entry scratch. Same idiom, different dimensions - which is exactly
// what a fingerprint pinned to the exposure pass's shape walks past.
constexpr const char* kWideComputeSource = R"(#version 430 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 1024) in;
layout(std430, binding = 0) buffer Output {
float total;
uint numSubgroups;
uint maxSubgroupId;
} outputData;
shared float prefixSumCache[64];
void main() {
float importance = float(gl_LocalInvocationID.x);
float prefixSum = subgroupInclusiveAdd(importance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
prefixSum += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
}
barrier();
}
if (gl_LocalInvocationID.x == 1023u) {
outputData.total = prefixSum;
outputData.numSubgroups = gl_NumSubgroups;
}
atomicMax(outputData.maxSubgroupId, gl_SubgroupID);
}
)";
struct OutputBlock {
float total = -1.0f;
std::uint32_t numSubgroups = 0;
std::uint32_t maxSubgroupId = 0;
};
bool HasExtension(const char* wanted) {
GLint extensionCount = 0;
glGetIntegerv(GL_NUM_EXTENSIONS, &extensionCount);
for (GLint i = 0; i < extensionCount; ++i) {
const auto* extension =
reinterpret_cast<const char*>(glGetStringi(GL_EXTENSIONS, static_cast<GLuint>(i)));
if (extension != nullptr && std::string(extension) == wanted) return true;
}
return false;
}
class IterationRPScratchFixScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
GLint stages = 0;
GLint features = 0;
GLint invocations = 0;
const bool subgroupExtension = HasExtension("GL_KHR_shader_subgroup");
if (subgroupExtension) {
glGetIntegerv(GL_SUBGROUP_SUPPORTED_STAGES_KHR, &stages);
glGetIntegerv(GL_SUBGROUP_SUPPORTED_FEATURES_KHR, &features);
}
glGetIntegerv(GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, &invocations);
const GLbitfield requiredFeatures =
GL_SUBGROUP_FEATURE_BASIC_BIT_KHR | GL_SUBGROUP_FEATURE_ARITHMETIC_BIT_KHR;
if (!subgroupExtension || (static_cast<GLbitfield>(stages) & GL_COMPUTE_SHADER_BIT) == 0 ||
(static_cast<GLbitfield>(features) & requiredFeatures) != requiredFeatures ||
invocations < static_cast<GLint>(kInvocationCount)) {
GTEST_SKIP() << "needs GL_KHR_shader_subgroup basic+arithmetic in compute and a "
"512-invocation workgroup";
}
m_maxInvocations = static_cast<std::uint32_t>(invocations);
glGenBuffers(1, &m_output);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_output);
// maxSubgroupId starts at zero HOST-side: the word is touched only by
// atomicMax during the dispatch, since a plain shader-side zeroing store
// would race the other invocations' atomics (barrier() orders shared
// memory, not SSBO stores).
const OutputBlock poison{-1.0f, 0xa5a5a5a5u, 0u};
glBufferData(GL_SHADER_STORAGE_BUFFER, sizeof(OutputBlock), &poison, GL_DYNAMIC_READ);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, m_output);
}
void TearDown() override {
if (!Ready()) return;
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, 0);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, 0);
if (m_output != 0) glDeleteBuffers(1, &m_output);
if (m_program != 0) glDeleteProgram(m_program);
}
unsigned int CompileComputeProgram(const char* source) {
const GLuint shader = glCreateShader(GL_COMPUTE_SHADER);
glShaderSource(shader, 1, &source, nullptr);
glCompileShader(shader);
GLint compiled = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &compiled);
if (compiled == GL_FALSE) {
char log[2048] = {};
glGetShaderInfoLog(shader, sizeof(log) - 1, nullptr, log);
m_buildLog = std::string("compute shader did not compile: ") + log;
glDeleteShader(shader);
return 0;
}
const GLuint program = glCreateProgram();
glAttachShader(program, shader);
glLinkProgram(program);
glDeleteShader(shader);
GLint linked = 0;
glGetProgramiv(program, GL_LINK_STATUS, &linked);
if (linked == GL_FALSE) {
char log[2048] = {};
glGetProgramInfoLog(program, sizeof(log) - 1, nullptr, log);
m_buildLog = std::string("compute program did not link: ") + log;
glDeleteProgram(program);
return 0;
}
return program;
}
// Re-poisons the block, compiles the shape under test and runs it once.
OutputBlock Dispatch(const char* source) {
const OutputBlock poison{-1.0f, 0xa5a5a5a5u, 0u};
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_output);
glBufferSubData(GL_SHADER_STORAGE_BUFFER, 0, sizeof(OutputBlock), &poison);
m_program = CompileComputeProgram(source);
EXPECT_NE(m_program, 0u) << m_buildLog;
if (m_program == 0u) return OutputBlock{};
glUseProgram(m_program);
glDispatchCompute(1, 1, 1);
glMemoryBarrier(GL_BUFFER_UPDATE_BARRIER_BIT);
OutputBlock block{};
glBindBuffer(GL_SHADER_STORAGE_BUFFER, m_output);
glGetBufferSubData(GL_SHADER_STORAGE_BUFFER, 0, sizeof(OutputBlock), &block);
return block;
}
GLuint m_program = 0;
GLuint m_output = 0;
std::uint32_t m_maxInvocations = 0;
std::string m_buildLog;
};
} // namespace
TEST_F(IterationRPScratchFixScenario, FixtureShapedReductionSumsEveryInvocation) {
const OutputBlock block = Dispatch(kComputeSource);
EXPECT_EQ(glGetError(), static_cast<GLenum>(GL_NO_ERROR));
// The topology diagnostics catch the failure modes by name before the sum does:
// an out-of-bounds handoff corrupts the total, a wrong gl_NumSubgroups breaks
// the combine loop's length.
ASSERT_NE(block.numSubgroups, 0xa5a5a5a5u) << "invocation 511 never reached its store";
EXPECT_GE(block.numSubgroups, 1u);
EXPECT_LE(block.numSubgroups, kInvocationCount);
EXPECT_LT(block.maxSubgroupId, block.numSubgroups)
<< "gl_SubgroupID exceeds gl_NumSubgroups - the inconsistency "
"DeriveNumSubgroupsPass exists to repair";
// Integer-valued fp32 inputs: the workgroup total is exact under any subgroup
// width, lane order, and association. This is the value iterationRP's exposure
// average is built from; without FixIterationRPSubgroupScratchPass an 8-lane
// device writes prefixSumCache[32..63] out of bounds and this comparison fails.
EXPECT_EQ(block.total, kExpectedTotal)
<< "workgroup reduction produced " << block.total << " with gl_NumSubgroups="
<< block.numSubgroups;
}
// The pack's second instance of the same bug, and the one that kept the CI
// retrace red after the exposure pass alone was patched.
TEST_F(IterationRPScratchFixScenario, WideFixtureShapedReductionSumsEveryInvocation) {
if (m_maxInvocations < kWideInvocationCount) {
GTEST_SKIP() << "needs a " << kWideInvocationCount << "-invocation workgroup";
}
const OutputBlock block = Dispatch(kWideComputeSource);
EXPECT_EQ(glGetError(), static_cast<GLenum>(GL_NO_ERROR));
ASSERT_NE(block.numSubgroups, 0xa5a5a5a5u) << "invocation 1023 never reached its store";
EXPECT_GE(block.numSubgroups, 1u);
EXPECT_LE(block.numSubgroups, kWideInvocationCount);
EXPECT_LT(block.maxSubgroupId, block.numSubgroups)
<< "gl_SubgroupID exceeds gl_NumSubgroups - the inconsistency "
"DeriveNumSubgroupsPass exists to repair";
// Without the patch an 8-lane device writes prefixSumCache[64..127] out of
// bounds and this comparison fails.
EXPECT_EQ(block.total, kWideExpectedTotal)
<< "workgroup reduction produced " << block.total << " with gl_NumSubgroups="
<< block.numSubgroups;
}
} // namespace MGITest
@@ -428,15 +428,15 @@ void main() { fragColor = vec4(float(gsIndex) * 16.0 / 255.0, 0.0, 0.0, 1.0); }
std::vector<GLfloat> pixels(static_cast<size_t>(kWidth) * kHeight, 0.0f); std::vector<GLfloat> pixels(static_cast<size_t>(kWidth) * kHeight, 0.0f);
glReadPixels(0, 0, kWidth, kHeight, GL_RED, GL_FLOAT, pixels.data()); glReadPixels(0, 0, kWidth, kHeight, GL_RED, GL_FLOAT, pixels.data());
for (int i = 0; i < kViewportCount; ++i) { for (int i = 0; i < kViewportCount; ++i) {
const float near = static_cast<float>(i) / 16.0f; const float nearDepth = static_cast<float>(i) / 16.0f;
const float far = 1.0f - static_cast<float>(i) / 16.0f; const float farDepth = 1.0f - static_cast<float>(i) / 16.0f;
// The tolerance covers depth-buffer-free rasterization of gl_FragCoord.z on a // The tolerance covers depth-buffer-free rasterization of gl_FragCoord.z on a
// software rasterizer; the per-index values are 1/16 apart, so it cannot let a // software rasterizer; the per-index values are 1/16 apart, so it cannot let a
// neighbouring viewport's range through, and viewport 0's range (0, 1) differs // neighbouring viewport's range through, and viewport 0's range (0, 1) differs
// from every other index by at least 1/16. // from every other index by at least 1/16.
EXPECT_NEAR(pixels[i], near, 1.0e-3f) EXPECT_NEAR(pixels[i], nearDepth, 1.0e-3f)
<< "viewport " << i << " near-plane depth; got viewport 0's range if this is 0"; << "viewport " << i << " near-plane depth; got viewport 0's range if this is 0";
EXPECT_NEAR(pixels[static_cast<size_t>(kWidth) + i], far, 1.0e-3f) EXPECT_NEAR(pixels[static_cast<size_t>(kWidth) + i], farDepth, 1.0e-3f)
<< "viewport " << i << " far-plane depth; got viewport 0's range if this is 1"; << "viewport " << i << " far-plane depth; got viewport 0's range if this is 1";
} }
+1
View File
@@ -78,6 +78,7 @@ add_subdirectory(Query)
add_subdirectory(Pipeline) add_subdirectory(Pipeline)
add_subdirectory(ShaderTranspiler) add_subdirectory(ShaderTranspiler)
add_subdirectory(Util) add_subdirectory(Util)
add_subdirectory(SelfTest)
# The DirectGLES post-transpile ESSL passes are pure String -> String, so unlike the # The DirectGLES post-transpile ESSL passes are pure String -> String, so unlike the
# DirectVulkan suite below this one needs no device and always builds. # DirectVulkan suite below this one needs no device and always builds.
add_subdirectory(Backend/DirectGLES) add_subdirectory(Backend/DirectGLES)
+10
View File
@@ -2387,3 +2387,13 @@ TEST(DirectGLESTextureSync, UnitMemoRefusesToDriveATwinFromAnotherTexture) {
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, 0); MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, 0);
} }
TEST(DirectVulkanSanity, GraphicsSamplerFeedbackOnlyAliasesWritableOverlappingMip) {
using MobileGL::MG_Backend::DirectVulkan::UniformManager;
EXPECT_TRUE(UniformManager::SamplerOverlapsWritableImageSubresource(1, 3, 2, GL_WRITE_ONLY));
EXPECT_TRUE(UniformManager::SamplerOverlapsWritableImageSubresource(1, 3, 3, GL_READ_WRITE));
EXPECT_FALSE(UniformManager::SamplerOverlapsWritableImageSubresource(1, 3, 2, GL_READ_ONLY));
EXPECT_FALSE(UniformManager::SamplerOverlapsWritableImageSubresource(1, 3, 0, GL_WRITE_ONLY));
EXPECT_FALSE(UniformManager::SamplerOverlapsWritableImageSubresource(1, 3, 4, GL_WRITE_ONLY));
}
+19
View File
@@ -0,0 +1,19 @@
# MobileGL - MobileGL/MG_Test/SelfTest/CMakeLists.txt
add_executable(
DriverPostIterationRPWitnessTest
DriverPostIterationRPWitnessTest.cpp
)
target_include_directories(DriverPostIterationRPWitnessTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(DriverPostIterationRPWitnessTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
include(GoogleTest)
gtest_discover_tests(DriverPostIterationRPWitnessTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
@@ -0,0 +1,188 @@
// MobileGL - MobileGL/MG_Test/SelfTest/DriverPostIterationRPWitnessTest.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#include <string>
#include "MG_Util/SelfTest/DriverPostIterationRPWitness.h"
namespace MobileGL::MG_Util::SelfTest {
namespace {
IterationRPWitnessOutput MakeValidWitness(std::uint32_t numSubgroups) {
IterationRPWitnessOutput output{};
output.magic = kIterationRPWitnessMagic;
output.numSubgroups = numSubgroups;
output.loopLength = ComputeIterationRPWitnessLoopLength(numSubgroups);
output.seenSubgroupMask =
numSubgroups == kIterationRPWitnessMaxSubgroups ? 0xffffffffu : (1u << numSubgroups) - 1u;
// Valid test layouts use equal contiguous groups of the indexed
// 1..512 input. The compact witness only needs their independent sums.
const std::uint32_t subgroupSize = kIterationRPWitnessInvocationCount / numSubgroups;
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
const std::uint32_t first = subgroup * subgroupSize + 1u;
const std::uint32_t last = first + subgroupSize - 1u;
output.lastLaneWriterCount[subgroup] = 1u;
output.indexedInputTotal[subgroup] = subgroupSize * (first + last) / 2u;
output.rawPrefix[subgroup] = {static_cast<float>(output.indexedInputTotal[subgroup]), 0.0f};
}
output.owner511 = {subgroupSize, numSubgroups, numSubgroups - 1u, subgroupSize - 1u};
auto cache = output.rawPrefix;
for (std::uint32_t scanStage = 0u; scanStage < output.loopLength; ++scanStage) {
auto cacheAfterStage = cache;
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
if ((subgroup & (1u << scanStage)) == 0u) continue;
const std::uint32_t sourceCacheIndex = (subgroup >> scanStage << scanStage) - 1u;
cacheAfterStage[subgroup].x += cache[sourceCacheIndex].x;
cacheAfterStage[subgroup].y += cache[sourceCacheIndex].y;
}
cache = cacheAfterStage;
output.scanCache[scanStage] = cache;
}
output.finalAverage = {256.5f, 0.0f};
return output;
}
IterationRPWitnessLimits MakeSufficientLimits() {
IterationRPWitnessLimits limits;
limits.computeStageSupported = true;
limits.basicSubgroupSupported = true;
limits.arithmeticSubgroupSupported = true;
limits.subgroupSize = 32u;
limits.maxComputeWorkGroupInvocations = kIterationRPWitnessInvocationCount;
limits.maxComputeWorkGroupSize = {32u, 16u, 1u};
limits.maxComputeSharedMemorySize = kIterationRPWitnessSharedMemoryBytes;
limits.maxPerStageDescriptorStorageBuffers = 1u;
limits.maxDescriptorSetStorageBuffers = 1u;
limits.maxBoundDescriptorSets = 1u;
limits.maxStorageBufferRange = sizeof(IterationRPWitnessOutput);
return limits;
}
} // namespace
TEST(DriverPostIterationRPWitnessTest, ValidTwoSubgroupWitness) {
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(MakeValidWitness(2u));
ASSERT_TRUE(validation.ok) << validation.detail;
EXPECT_EQ(validation.detail, "N=2, owner511=id1/lane255, 2 scan stages, average=(256.5,0)");
}
TEST(DriverPostIterationRPWitnessTest, ValidThirtyTwoSubgroupWitness) {
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(MakeValidWitness(32u));
ASSERT_TRUE(validation.ok) << validation.detail;
EXPECT_EQ(validation.detail, "N=32, owner511=id31/lane15, 6 scan stages, average=(256.5,0)");
}
TEST(DriverPostIterationRPWitnessTest, RejectsNonuniformNumSubgroups) {
IterationRPWitnessOutput output = MakeValidWitness(16u);
output.topologyFlags |= IterationRPWitnessNonuniformNumSubgroups;
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(output);
EXPECT_FALSE(validation.ok);
EXPECT_EQ(validation.failure, IterationRPWitnessValidationFailure::Topology);
EXPECT_NE(validation.detail.find("gl_NumSubgroups differed"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, RejectsMissingAndOutOfRangeSubgroupIds) {
IterationRPWitnessOutput missing = MakeValidWitness(16u);
missing.seenSubgroupMask &= ~(1u << 7u);
IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(missing);
EXPECT_FALSE(validation.ok);
EXPECT_NE(validation.detail.find("seen subgroup-ID mask"), std::string::npos);
IterationRPWitnessOutput outOfRange = MakeValidWitness(16u);
outOfRange.topologyFlags |= IterationRPWitnessInvalidSubgroupId;
validation = ValidateIterationRPWitness(outOfRange);
EXPECT_FALSE(validation.ok);
EXPECT_NE(validation.detail.find("invalid gl_SubgroupID"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, RejectsInvalidMultipleAndMissingLastLaneWriters) {
IterationRPWitnessOutput invalidLane = MakeValidWitness(16u);
invalidLane.topologyFlags |= IterationRPWitnessInvalidSubgroupLane;
IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(invalidLane);
EXPECT_FALSE(validation.ok);
EXPECT_NE(validation.detail.find("invalid subgroup lane"), std::string::npos);
IterationRPWitnessOutput multiple = MakeValidWitness(16u);
multiple.lastLaneWriterCount[4] = 2u;
validation = ValidateIterationRPWitness(multiple);
EXPECT_FALSE(validation.ok);
EXPECT_NE(validation.detail.find("subgroup 4 has 2 source last-lane writers"), std::string::npos);
IterationRPWitnessOutput missing = MakeValidWitness(16u);
missing.lastLaneWriterCount[6] = 0u;
validation = ValidateIterationRPWitness(missing);
EXPECT_FALSE(validation.ok);
EXPECT_NE(validation.detail.find("subgroup 6 has 0 source last-lane writers"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, ReportsEarliestCorruptSourceScanStage) {
IterationRPWitnessOutput output = MakeValidWitness(32u);
output.scanCache[0][1].x += 1.0f;
output.scanCache[3][5].x += 1.0f;
IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(output);
EXPECT_FALSE(validation.ok);
EXPECT_EQ(validation.failure, IterationRPWitnessValidationFailure::SourceScan);
EXPECT_EQ(validation.scanStage, 0u);
EXPECT_NE(validation.detail.find("source scan stage 0, subgroup 1"), std::string::npos);
output = MakeValidWitness(32u);
output.scanCache[3][5].x += 1.0f;
validation = ValidateIterationRPWitness(output);
EXPECT_FALSE(validation.ok);
EXPECT_EQ(validation.failure, IterationRPWitnessValidationFailure::SourceScan);
EXPECT_EQ(validation.scanStage, 3u);
EXPECT_NE(validation.detail.find("source scan stage 3, subgroup 5"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, RejectsOwner511OutsideHighestFinalLane) {
IterationRPWitnessOutput output = MakeValidWitness(16u);
output.owner511.z = 14u;
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(output);
EXPECT_FALSE(validation.ok);
EXPECT_EQ(validation.failure, IterationRPWitnessValidationFailure::FinalOwner);
EXPECT_NE(validation.detail.find("not in the highest subgroup"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, RejectsIncorrectVectorFinalAverage) {
IterationRPWitnessOutput output = MakeValidWitness(16u);
output.finalAverage.y = 1.0f;
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(output);
EXPECT_FALSE(validation.ok);
EXPECT_EQ(validation.failure, IterationRPWitnessValidationFailure::FinalAverage);
EXPECT_NE(validation.detail.find("final average"), std::string::npos);
}
TEST(DriverPostIterationRPWitnessTest, MissingNativeFeatureIsTheOnlySkipCondition) {
for (const auto toggleMissingFeature : {0u, 1u, 2u}) {
IterationRPWitnessLimits limits = MakeSufficientLimits();
if (toggleMissingFeature == 0u) limits.computeStageSupported = false;
if (toggleMissingFeature == 1u) limits.basicSubgroupSupported = false;
if (toggleMissingFeature == 2u) limits.arithmeticSubgroupSupported = false;
const IterationRPWitnessEligibilityResult eligibility = EvaluateIterationRPWitnessEligibility(limits);
EXPECT_EQ(eligibility.eligibility, IterationRPWitnessEligibility::SkipUnsupportedNativeFeatureSet)
<< eligibility.detail;
}
IterationRPWitnessLimits zeroSubgroupSize = MakeSufficientLimits();
zeroSubgroupSize.subgroupSize = 0u;
IterationRPWitnessEligibilityResult eligibility = EvaluateIterationRPWitnessEligibility(zeroSubgroupSize);
EXPECT_EQ(eligibility.eligibility, IterationRPWitnessEligibility::FailInadequateLimits) << eligibility.detail;
IterationRPWitnessLimits limits = MakeSufficientLimits();
limits.maxComputeWorkGroupInvocations = 511u;
eligibility = EvaluateIterationRPWitnessEligibility(limits);
EXPECT_EQ(eligibility.eligibility, IterationRPWitnessEligibility::FailInadequateLimits) << eligibility.detail;
limits = MakeSufficientLimits();
limits.maxStorageBufferRange = sizeof(IterationRPWitnessOutput) - 1u;
eligibility = EvaluateIterationRPWitnessEligibility(limits);
EXPECT_EQ(eligibility.eligibility, IterationRPWitnessEligibility::FailInadequateLimits) << eligibility.detail;
}
} // namespace MobileGL::MG_Util::SelfTest
@@ -3,6 +3,9 @@ cmake_minimum_required(VERSION 3.14)
add_executable( add_executable(
SpirvPassTest SpirvPassTest
SpirvPassTest.cpp SpirvPassTest.cpp
DeriveNumSubgroupsTest.cpp
FixIterationRPSubgroupScratchTest.cpp
EmulateSubgroupsTest.cpp
DemoteFloat64Test.cpp DemoteFloat64Test.cpp
FlattenXfbInterfaceBlocksTest.cpp FlattenXfbInterfaceBlocksTest.cpp
) )
@@ -0,0 +1,147 @@
// MobileGL - MobileGL/MG_Test/ShaderTranspiler/DeriveNumSubgroupsTest.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#define SPV_ENABLE_UTILITY_CODE
#include "glslang/SPIRV/spirv.hpp11"
#undef SPV_ENABLE_UTILITY_CODE
#include "Includes.h"
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/Types.h>
#include <spirv-tools/libspirv.hpp>
using namespace MobileGL;
using MobileGL::MG_Util::ShaderTranspiler::ShaderCompiler;
namespace {
constexpr SizeT kSpirvHeaderWordCount = 5u;
template <typename Visitor>
void ForEachInstruction(const Vector<Uint32>& spirv, Visitor&& visit) {
for (SizeT offset = kSpirvHeaderWordCount; offset < spirv.size();) {
const Uint32 wordCount = spirv[offset] >> 16u;
if (wordCount == 0u || offset + wordCount > spirv.size()) break;
visit(static_cast<spv::Op>(spirv[offset] & 0xffffu), &spirv[offset], wordCount);
offset += wordCount;
}
}
Vector<Uint32> CompileCompute(const String& source) {
using namespace MobileGL::MG_Util::ShaderTranspiler;
ShaderAttrib shaderAttrib{.shaderType = GL_COMPUTE_SHADER, .sourceStr = source};
auto shaderResult = ShaderCompiler::CompileShader(shaderAttrib);
EXPECT_TRUE(shaderResult) << (shaderResult ? String{} : shaderResult.error().log);
if (!shaderResult) return {};
ProgramAttrib programAttrib{.shaders = {shaderResult.value()}};
auto programResult = ShaderCompiler::LinkProgram(programAttrib);
EXPECT_TRUE(programResult) << (programResult ? String{} : programResult.error().log);
if (!programResult) return {};
ProgramBinaryAttrib binaryAttrib{.shaderTypes = {GL_COMPUTE_SHADER}, .program = *programResult.value()};
auto binaryResult = ShaderCompiler::GetSpirvBinaryFromProgram(binaryAttrib);
EXPECT_TRUE(binaryResult) << (binaryResult ? String{} : binaryResult.error().log);
if (!binaryResult || binaryResult->empty()) return {};
return binaryResult->front();
}
Uint32 FindBuiltinTarget(const Vector<Uint32>& spirv, spv::BuiltIn builtin) {
Uint32 target = 0u;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
if (opcode == spv::Op::OpDecorate && wordCount >= 4u &&
static_cast<spv::Decoration>(words[2]) == spv::Decoration::BuiltIn &&
static_cast<spv::BuiltIn>(words[3]) == builtin) {
target = words[1];
}
});
return target;
}
Uint32 CountLoadsFrom(const Vector<Uint32>& spirv, Uint32 pointerId) {
Uint32 count = 0u;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
if (opcode == spv::Op::OpLoad && wordCount >= 4u && words[3] == pointerId) ++count;
});
return count;
}
Uint32 CountOpcode(const Vector<Uint32>& spirv, spv::Op wanted) {
Uint32 count = 0u;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32*, Uint32) {
if (opcode == wanted) ++count;
});
return count;
}
bool Validates(const Vector<Uint32>& spirv) {
spvtools::SpirvTools tools(SPV_ENV_VULKAN_1_1);
return tools.Validate(spirv);
}
constexpr const char* kNumSubgroupsOnlySource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { uint value; } outputData;
void main() {
if (gl_LocalInvocationIndex == 0u)
outputData.value = gl_NumSubgroups;
}
)";
constexpr const char* kNoNumSubgroupsSource = R"(#version 450 core
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { uint value; } outputData;
void main() {
if (gl_LocalInvocationIndex == 0u)
outputData.value = gl_WorkGroupSize.x;
}
)";
} // namespace
TEST(DeriveNumSubgroupsPass, ReplacesBuiltinLoadAndSynthesizesSubgroupSize) {
const Vector<Uint32> input = CompileCompute(kNumSubgroupsOnlySource);
ASSERT_FALSE(input.empty());
const Uint32 inputNumSubgroups = FindBuiltinTarget(input, spv::BuiltIn::NumSubgroups);
ASSERT_NE(inputNumSubgroups, 0u);
EXPECT_EQ(CountLoadsFrom(input, inputNumSubgroups), 1u);
EXPECT_EQ(FindBuiltinTarget(input, spv::BuiltIn::SubgroupSize), 0u);
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::DeriveNumSubgroupsForVulkan(input, output, true));
ASSERT_TRUE(Validates(output));
const Uint32 outputNumSubgroups = FindBuiltinTarget(output, spv::BuiltIn::NumSubgroups);
const Uint32 outputSubgroupSize = FindBuiltinTarget(output, spv::BuiltIn::SubgroupSize);
ASSERT_NE(outputNumSubgroups, 0u);
ASSERT_NE(outputSubgroupSize, 0u);
EXPECT_EQ(CountLoadsFrom(output, outputNumSubgroups), 0u);
EXPECT_EQ(CountLoadsFrom(output, outputSubgroupSize), 1u);
EXPECT_EQ(CountOpcode(output, spv::Op::OpCompositeExtract), 3u);
EXPECT_EQ(CountOpcode(output, spv::Op::OpIMul), 2u);
EXPECT_EQ(CountOpcode(output, spv::Op::OpUDiv), 1u);
}
TEST(DeriveNumSubgroupsPass, IsIdempotent) {
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::DeriveNumSubgroupsForVulkan(CompileCompute(kNumSubgroupsOnlySource), once, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::DeriveNumSubgroupsForVulkan(once, twice, true));
EXPECT_EQ(twice, once);
}
TEST(DeriveNumSubgroupsPass, LeavesUnrelatedComputeShaderUntouched) {
const Vector<Uint32> input = CompileCompute(kNoNumSubgroupsSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::DeriveNumSubgroupsForVulkan(input, output, true));
EXPECT_EQ(output, input);
EXPECT_EQ(FindBuiltinTarget(output, spv::BuiltIn::SubgroupSize), 0u);
}
@@ -0,0 +1,248 @@
// MobileGL - MobileGL/MG_Test/ShaderTranspiler/EmulateSubgroupsTest.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#define SPV_ENABLE_UTILITY_CODE
#include "glslang/SPIRV/spirv.hpp11"
#undef SPV_ENABLE_UTILITY_CODE
#include "Includes.h"
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/Types.h>
#include <spirv-tools/libspirv.hpp>
using namespace MobileGL;
using MobileGL::MG_Util::ShaderTranspiler::ShaderCompiler;
namespace {
constexpr SizeT kSpirvHeaderWordCount = 5u;
template <typename Visitor>
void ForEachInstruction(const Vector<Uint32>& spirv, Visitor&& visit) {
for (SizeT offset = kSpirvHeaderWordCount; offset < spirv.size();) {
const Uint32 wordCount = spirv[offset] >> 16u;
if (wordCount == 0u || offset + wordCount > spirv.size()) break;
visit(static_cast<spv::Op>(spirv[offset] & 0xffffu), &spirv[offset], wordCount);
offset += wordCount;
}
}
Vector<Uint32> CompileStage(GLenum stage, const String& source) {
using namespace MobileGL::MG_Util::ShaderTranspiler;
ShaderAttrib shaderAttrib{.shaderType = stage, .sourceStr = source};
auto shaderResult = ShaderCompiler::CompileShader(shaderAttrib);
EXPECT_TRUE(shaderResult) << (shaderResult ? String{} : shaderResult.error().log);
if (!shaderResult) return {};
ProgramAttrib programAttrib{.shaders = {shaderResult.value()}};
auto programResult = ShaderCompiler::LinkProgram(programAttrib);
EXPECT_TRUE(programResult) << (programResult ? String{} : programResult.error().log);
if (!programResult) return {};
ProgramBinaryAttrib binaryAttrib{.shaderTypes = {stage}, .program = *programResult.value()};
auto binaryResult = ShaderCompiler::GetSpirvBinaryFromProgram(binaryAttrib);
EXPECT_TRUE(binaryResult) << (binaryResult ? String{} : binaryResult.error().log);
if (!binaryResult || binaryResult->empty()) return {};
return binaryResult->front();
}
Uint32 CountGroupNonUniform(const Vector<Uint32>& spirv) {
Uint32 count = 0;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32*, Uint32) {
if (opcode >= spv::Op::OpGroupNonUniformElect && opcode <= spv::Op::OpGroupNonUniformQuadSwap) {
++count;
}
});
return count;
}
Uint32 CountGroupNonUniformCapabilities(const Vector<Uint32>& spirv) {
Uint32 count = 0;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
if (opcode != spv::Op::OpCapability || wordCount < 2u) return;
const auto capability = static_cast<spv::Capability>(words[1]);
if (capability >= spv::Capability::GroupNonUniform &&
capability <= spv::Capability::GroupNonUniformQuad) {
++count;
}
});
return count;
}
Uint32 CountOpcode(const Vector<Uint32>& spirv, spv::Op wanted) {
Uint32 count = 0;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32*, Uint32) {
if (opcode == wanted) ++count;
});
return count;
}
bool HasWorkgroupVariable(const Vector<Uint32>& spirv) {
bool found = false;
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
if (opcode == spv::Op::OpVariable && wordCount >= 4u &&
static_cast<spv::StorageClass>(words[3]) == spv::StorageClass::Workgroup) {
found = true;
}
});
return found;
}
bool Validates(const Vector<Uint32>& spirv) {
spvtools::SpirvTools tools(SPV_ENV_VULKAN_1_1);
tools.SetMessageConsumer([](spv_message_level_t, const char*, const spv_position_t& position,
const char* message) {
ADD_FAILURE() << "spirv-val at word " << position.index << ": " << message;
});
return tools.Validate(spirv);
}
// One shader touching every lowered category: builtins, vote, arithmetic
// scans, ballot math, shuffles, clustered and quad operations.
constexpr const char* kEveryCategorySource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_vote : require
#extension GL_KHR_shader_subgroup_arithmetic : require
#extension GL_KHR_shader_subgroup_ballot : require
#extension GL_KHR_shader_subgroup_shuffle : require
#extension GL_KHR_shader_subgroup_shuffle_relative : require
#extension GL_KHR_shader_subgroup_clustered : require
#extension GL_KHR_shader_subgroup_quad : require
layout(local_size_x = 48, local_size_y = 1, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value[]; } outputData;
void main() {
uint slot = gl_LocalInvocationIndex * 24u;
float v = float(gl_LocalInvocationIndex + 1u);
outputData.value[slot + 0u] = float(gl_SubgroupSize);
outputData.value[slot + 1u] = float(gl_NumSubgroups);
outputData.value[slot + 2u] = float(gl_SubgroupID);
outputData.value[slot + 3u] = float(gl_SubgroupInvocationID);
outputData.value[slot + 4u] = float(gl_SubgroupEqMask.x + gl_SubgroupLtMask.x);
outputData.value[slot + 5u] = subgroupElect() ? 1.0 : 0.0;
outputData.value[slot + 6u] = subgroupAll(v > 0.0) ? 1.0 : 0.0;
outputData.value[slot + 7u] = subgroupAny(v > 40.0) ? 1.0 : 0.0;
outputData.value[slot + 8u] = subgroupAllEqual(gl_WorkGroupID.x) ? 1.0 : 0.0;
outputData.value[slot + 9u] = subgroupAdd(v);
outputData.value[slot + 10u] = subgroupInclusiveAdd(v);
outputData.value[slot + 11u] = subgroupExclusiveMax(v);
outputData.value[slot + 12u] = float(subgroupMin(gl_LocalInvocationIndex));
uvec4 ballot = subgroupBallot((gl_LocalInvocationIndex & 1u) == 0u);
outputData.value[slot + 13u] = float(subgroupBallotBitCount(ballot));
outputData.value[slot + 14u] = float(subgroupBallotFindLSB(ballot));
outputData.value[slot + 15u] = float(subgroupBallotFindMSB(ballot));
outputData.value[slot + 16u] = subgroupInverseBallot(ballot) ? 1.0 : 0.0;
outputData.value[slot + 17u] = subgroupBallotBitExtract(ballot, 3u) ? 1.0 : 0.0;
outputData.value[slot + 18u] = subgroupBroadcast(v, 2u);
outputData.value[slot + 19u] = subgroupBroadcastFirst(v);
outputData.value[slot + 20u] = subgroupShuffle(v, gl_SubgroupInvocationID ^ 5u);
outputData.value[slot + 21u] = subgroupShuffleXor(v, 1u) + subgroupShuffleUp(v, 1u) +
subgroupShuffleDown(v, 1u);
outputData.value[slot + 22u] = subgroupClusteredAdd(v, 4u);
outputData.value[slot + 23u] = subgroupQuadBroadcast(v, 1u) + subgroupQuadSwapHorizontal(v);
subgroupBarrier();
subgroupMemoryBarrierShared();
}
)";
constexpr const char* kNoSubgroupSource = R"(#version 450 core
layout(local_size_x = 64) in;
layout(std430, binding = 0) buffer Output { uint value; } outputData;
void main() {
if (gl_LocalInvocationIndex == 0u) outputData.value = gl_WorkGroupSize.x;
}
)";
// An extended subgroup instruction (SPV_KHR_subgroup_rotate) alongside core
// ones: outside the lowered set, so the pass must fail rather than emit
// "subgroup-free" output that still rotates.
constexpr const char* kRotateSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
#extension GL_KHR_shader_subgroup_rotate : require
layout(local_size_x = 64) in;
layout(std430, binding = 0) buffer Output { float value[]; } outputData;
void main() {
float v = subgroupAdd(float(gl_SubgroupInvocationID));
outputData.value[gl_LocalInvocationIndex] = subgroupRotate(v, 1u);
}
)";
// A 1024-invocation workgroup exchanging a vec4 and a float: the lowering
// would need 16 KiB + 4 KiB of scratch, past the Vulkan-minimum shared
// budget of 16384 bytes.
constexpr const char* kScratchHungrySource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 1024) in;
layout(std430, binding = 0) buffer Output { vec4 value[]; } outputData;
void main() {
vec4 wide = subgroupAdd(vec4(float(gl_LocalInvocationIndex)));
wide.x += subgroupInclusiveAdd(float(gl_SubgroupInvocationID));
outputData.value[gl_LocalInvocationIndex] = wide;
}
)";
} // namespace
TEST(EmulateSubgroupsPass, LowersEveryCategoryToSharedMemory) {
const Vector<Uint32> input = CompileStage(GL_COMPUTE_SHADER, kEveryCategorySource);
ASSERT_FALSE(input.empty());
ASSERT_GT(CountGroupNonUniform(input), 0u);
ASSERT_GT(CountGroupNonUniformCapabilities(input), 0u);
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::EmulateSubgroupsForVulkan(input, output, 16384u, true));
ASSERT_TRUE(Validates(output));
// The whole point: nothing subgroup-shaped survives, so the module runs on a
// device with no subgroup support at all.
EXPECT_EQ(CountGroupNonUniform(output), 0u);
EXPECT_EQ(CountGroupNonUniformCapabilities(output), 0u);
// The exchanges go through workgroup-shared scratch behind control barriers.
EXPECT_TRUE(HasWorkgroupVariable(output));
EXPECT_GT(CountOpcode(output, spv::Op::OpControlBarrier), CountOpcode(input, spv::Op::OpControlBarrier));
}
TEST(EmulateSubgroupsPass, IsIdempotent) {
const Vector<Uint32> input = CompileStage(GL_COMPUTE_SHADER, kEveryCategorySource);
ASSERT_FALSE(input.empty());
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::EmulateSubgroupsForVulkan(input, once, 16384u, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::EmulateSubgroupsForVulkan(once, twice, 16384u, true));
EXPECT_EQ(twice, once);
}
TEST(EmulateSubgroupsPass, LeavesSubgroupFreeComputeUntouched) {
const Vector<Uint32> input = CompileStage(GL_COMPUTE_SHADER, kNoSubgroupSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::EmulateSubgroupsForVulkan(input, output, 16384u, true));
EXPECT_EQ(output, input);
}
TEST(EmulateSubgroupsPass, RefusesExtendedSubgroupInstructions) {
const Vector<Uint32> input = CompileStage(GL_COMPUTE_SHADER, kRotateSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
EXPECT_FALSE(ShaderCompiler::EmulateSubgroupsForVulkan(input, output, 16384u, false));
}
TEST(EmulateSubgroupsPass, RefusesAModuleOverTheScratchBudget) {
const Vector<Uint32> input = CompileStage(GL_COMPUTE_SHADER, kScratchHungrySource);
ASSERT_FALSE(input.empty());
// vec4 scratch (1024 slots * 16 bytes) plus float scratch (4 KiB) exceeds
// the 16 KiB Vulkan-minimum budget.
Vector<Uint32> output;
EXPECT_FALSE(ShaderCompiler::EmulateSubgroupsForVulkan(input, output, 16384u, false));
// A device advertising more shared memory takes the same module fine.
Vector<Uint32> roomier;
EXPECT_TRUE(ShaderCompiler::EmulateSubgroupsForVulkan(input, roomier, 32768u, true));
EXPECT_TRUE(Validates(roomier));
}
@@ -0,0 +1,336 @@
// MobileGL - MobileGL/MG_Test/ShaderTranspiler/FixIterationRPSubgroupScratchTest.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#define SPV_ENABLE_UTILITY_CODE
#include "glslang/SPIRV/spirv.hpp11"
#undef SPV_ENABLE_UTILITY_CODE
#include "Includes.h"
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/Types.h>
#include <spirv-tools/libspirv.hpp>
#include <algorithm>
#include <map>
#include <vector>
using namespace MobileGL;
using MobileGL::MG_Util::ShaderTranspiler::ShaderCompiler;
namespace {
constexpr SizeT kSpirvHeaderWordCount = 5u;
template <typename Visitor>
void ForEachInstruction(const Vector<Uint32>& spirv, Visitor&& visit) {
for (SizeT offset = kSpirvHeaderWordCount; offset < spirv.size();) {
const Uint32 wordCount = spirv[offset] >> 16u;
if (wordCount == 0u || offset + wordCount > spirv.size()) break;
visit(static_cast<spv::Op>(spirv[offset] & 0xffffu), &spirv[offset], wordCount);
offset += wordCount;
}
}
Vector<Uint32> CompileCompute(const String& source) {
using namespace MobileGL::MG_Util::ShaderTranspiler;
ShaderAttrib shaderAttrib{.shaderType = GL_COMPUTE_SHADER, .sourceStr = source};
auto shaderResult = ShaderCompiler::CompileShader(shaderAttrib);
EXPECT_TRUE(shaderResult) << (shaderResult ? String{} : shaderResult.error().log);
if (!shaderResult) return {};
ProgramAttrib programAttrib{.shaders = {shaderResult.value()}};
auto programResult = ShaderCompiler::LinkProgram(programAttrib);
EXPECT_TRUE(programResult) << (programResult ? String{} : programResult.error().log);
if (!programResult) return {};
ProgramBinaryAttrib binaryAttrib{.shaderTypes = {GL_COMPUTE_SHADER}, .program = *programResult.value()};
auto binaryResult = ShaderCompiler::GetSpirvBinaryFromProgram(binaryAttrib);
EXPECT_TRUE(binaryResult) << (binaryResult ? String{} : binaryResult.error().log);
if (!binaryResult || binaryResult->empty()) return {};
return binaryResult->front();
}
// The declared lengths of every Workgroup-storage array variable, sorted.
std::vector<Uint32> WorkgroupArrayLengths(const Vector<Uint32>& spirv) {
std::map<Uint32, Uint32> constantValues; // constant id -> value
std::map<Uint32, Uint32> arrayLengthIds; // array type id -> length constant id
std::map<Uint32, Uint32> pointerPointees; // pointer type id -> pointee type id
std::vector<Uint32> workgroupPointerTypes; // type ids of Workgroup variables
ForEachInstruction(spirv, [&](spv::Op opcode, const Uint32* words, Uint32 wordCount) {
switch (opcode) {
case spv::Op::OpConstant:
if (wordCount >= 4u) constantValues[words[2]] = words[3];
break;
case spv::Op::OpTypeArray:
if (wordCount >= 4u) arrayLengthIds[words[1]] = words[3];
break;
case spv::Op::OpTypePointer:
if (wordCount >= 4u &&
static_cast<spv::StorageClass>(words[2]) == spv::StorageClass::Workgroup) {
pointerPointees[words[1]] = words[3];
}
break;
case spv::Op::OpVariable:
if (wordCount >= 4u &&
static_cast<spv::StorageClass>(words[3]) == spv::StorageClass::Workgroup) {
workgroupPointerTypes.push_back(words[1]);
}
break;
default:
break;
}
});
std::vector<Uint32> lengths;
for (const Uint32 pointerTypeId : workgroupPointerTypes) {
const auto pointee = pointerPointees.find(pointerTypeId);
if (pointee == pointerPointees.end()) continue;
const auto lengthId = arrayLengthIds.find(pointee->second);
if (lengthId == arrayLengthIds.end()) continue;
const auto value = constantValues.find(lengthId->second);
if (value != constantValues.end()) lengths.push_back(value->second);
}
std::sort(lengths.begin(), lengths.end());
return lengths;
}
bool Validates(const Vector<Uint32>& spirv) {
spvtools::SpirvTools tools(SPV_ENV_VULKAN_1_1);
tools.SetMessageConsumer([](spv_message_level_t, const char*, const spv_position_t& position,
const char* message) {
ADD_FAILURE() << "spirv-val at word " << position.index << ": " << message;
});
return tools.Validate(spirv);
}
// iterationRP's exposure reduction, as the pack ships it: 32x16 (512
// invocations), subgroupInclusiveAdd on a vec2, and a 32-entry
// gl_SubgroupID-indexed scratch. A second, plainly indexed array rides along
// to prove the patch is surgical.
constexpr const char* kExposureShapedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 prefixSumCache[32];
shared float plainScratch[4];
void main() {
vec2 sampleLuminance = vec2(float(gl_LocalInvocationIndex), 0.0);
sampleLuminance = subgroupInclusiveAdd(sampleLuminance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
plainScratch[gl_LocalInvocationIndex & 3u] = sampleLuminance.x;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
sampleLuminance += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
}
barrier();
}
if (gl_LocalInvocationIndex == 511u)
outputData.value = prefixSumCache[0].x / 512.0 + plainScratch[0];
}
)";
// The pack's OTHER instance of the same bug, which a fingerprint pinned to the
// exposure pass's dimensions walks straight past: the RTW importance warp
// scans a plain float across 1024 invocations into a 64-entry scratch.
constexpr const char* kRtwWarpShapedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 1024) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared float prefixSumCache[64];
void main() {
float importance = float(gl_LocalInvocationID.x) * 0.5;
float prefixSum = subgroupInclusiveAdd(importance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
prefixSum += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = prefixSum;
}
barrier();
}
if (gl_LocalInvocationID.x == 1023u) outputData.value = prefixSumCache[0];
}
)";
// A subgroup scan, but the scratch is indexed per invocation rather than per
// subgroup: its size is not a subgroup-count assumption, so it is not ours.
constexpr const char* kInvocationIndexedSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 perInvocation[32];
void main() {
vec2 v = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 0.0));
perInvocation[gl_LocalInvocationIndex & 31u] = v;
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = perInvocation[0].x;
}
)";
// gl_SubgroupID-indexed, but no subgroup scan feeds it and the element type is
// not the pack's float accumulator.
constexpr const char* kNonFloatScratchSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { uint value; } outputData;
shared uint tally[32];
void main() {
float scan = subgroupInclusiveAdd(float(gl_LocalInvocationIndex));
tally[gl_SubgroupID] = uint(scan);
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = tally[0];
}
)";
// gl_SubgroupID-indexed, but masked into range: the declaration is bounded by
// construction, not a subgroup-count assumption, so it is not the pack's bug.
constexpr const char* kMaskedSubgroupIndexSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 bounded[8];
void main() {
vec2 v = subgroupInclusiveAdd(vec2(float(gl_LocalInvocationIndex), 0.0));
bounded[gl_SubgroupID & 7u] = v;
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = bounded[0].x;
}
)";
// Neither of the pack's shapes: a small per-subgroup array in a 256-invocation
// workgroup, used to prove the width gate keeps EVERY module inert at >= 16 lanes.
constexpr const char* kForeignShapeSource = R"(#version 450 core
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 256) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared float partial[4];
void main() {
float v = subgroupInclusiveAdd(float(gl_LocalInvocationID.x));
if (gl_SubgroupID < 4u) partial[gl_SubgroupID] = v;
barrier();
if (gl_LocalInvocationID.x == 0u) outputData.value = partial[0];
}
)";
// No subgroup construct at all.
constexpr const char* kSubgroupFreeSource = R"(#version 450 core
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
layout(std430, binding = 0) buffer Output { float value; } outputData;
shared vec2 scratch[32];
void main() {
scratch[gl_LocalInvocationIndex & 31u] = vec2(float(gl_LocalInvocationIndex), 0.0);
barrier();
if (gl_LocalInvocationIndex == 0u) outputData.value = scratch[0].x;
}
)";
} // namespace
TEST(FixIterationRPSubgroupScratchPass, GrowsTheExposureScratchForNarrowSubgroups) {
const Vector<Uint32> input = CompileCompute(kExposureShapedSource);
ASSERT_FALSE(input.empty());
ASSERT_EQ(WorkgroupArrayLengths(input), (std::vector<Uint32>{4u, 32u}));
// lavapipe: 8-lane subgroups over 512 invocations need 64 entries; the
// plainly indexed neighbour must keep its 4.
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(WorkgroupArrayLengths(output), (std::vector<Uint32>{4u, 64u}));
EXPECT_TRUE(Validates(output));
}
// The regression the CI retrace caught: patching only the exposure pass leaves
// this one writing 128 subgroups into 64 entries, and the frame stays wrong.
TEST(FixIterationRPSubgroupScratchPass, GrowsTheRtwWarpScratchForNarrowSubgroups) {
const Vector<Uint32> input = CompileCompute(kRtwWarpShapedSource);
ASSERT_FALSE(input.empty());
ASSERT_EQ(WorkgroupArrayLengths(input), (std::vector<Uint32>{64u}));
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(WorkgroupArrayLengths(output), (std::vector<Uint32>{128u}));
EXPECT_TRUE(Validates(output));
}
TEST(FixIterationRPSubgroupScratchPass, LeavesPackWidthAssumptionsAloneOnWideDevices) {
// Both shapes are sized for >= 16 lanes (512/16 = 32, 1024/16 = 64), so on
// every such device the modules must pass through byte-identical.
for (const char* source : {kExposureShapedSource, kRtwWarpShapedSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
for (const Uint32 nativeSize : {16u, 32u, 64u, 128u}) {
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
input, output, nativeSize, 32768u, true));
EXPECT_EQ(output, input) << "native width " << nativeSize;
}
}
}
TEST(FixIterationRPSubgroupScratchPass, RefusesAModuleOutsideTheIdiom) {
for (const char* source : {kInvocationIndexedSource, kNonFloatScratchSource,
kSubgroupFreeSource, kMaskedSubgroupIndexSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 32768u, true));
EXPECT_EQ(output, input);
}
}
// A grown array that would not fit the device's shared memory is left alone:
// a pipeline that cannot be created is worse than the pack's own overrun.
// The width gate is what keeps unrelated shaders untouched on the devices the pack
// was written for: at >= 16 lanes nothing is rewritten, whatever its shape.
TEST(FixIterationRPSubgroupScratchPass, LeavesEveryModuleAloneAtThePacksAssumedWidth) {
const Vector<Uint32> input = CompileCompute(kForeignShapeSource);
ASSERT_FALSE(input.empty());
for (const Uint32 nativeSize : {16u, 32u, 64u}) {
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
input, output, nativeSize, 32768u, true));
EXPECT_EQ(output, input) << "native width " << nativeSize;
}
}
TEST(FixIterationRPSubgroupScratchPass, RefusesGrowthThatWouldNotFitSharedMemory) {
const Vector<Uint32> input = CompileCompute(kRtwWarpShapedSource);
ASSERT_FALSE(input.empty());
Vector<Uint32> output;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, output, 8u, 256u, true));
EXPECT_EQ(output, input);
}
TEST(FixIterationRPSubgroupScratchPass, IsIdempotent) {
for (const char* source : {kExposureShapedSource, kRtwWarpShapedSource}) {
const Vector<Uint32> input = CompileCompute(source);
ASSERT_FALSE(input.empty());
Vector<Uint32> once;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(input, once, 8u, 32768u, true));
Vector<Uint32> twice;
ASSERT_TRUE(ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(once, twice, 8u, 32768u, true));
EXPECT_EQ(twice, once);
}
}
+449 -1
View File
@@ -7,6 +7,8 @@
// End of Source File Header // End of Source File Header
#include "DriverPost.h" #include "DriverPost.h"
#include "DriverPostIterationRPWitness.h"
#include "DriverPostIterationRPWitnessSpv.h"
#include "MG_Util/BackendLoaders/OpenGL/Loader.h" #include "MG_Util/BackendLoaders/OpenGL/Loader.h"
#include <Config.h> #include <Config.h>
#include <MGGitHash.h> #include <MGGitHash.h>
@@ -24,6 +26,8 @@
#include <MG_Util/Texture/TextureFormatProcessor.h> #include <MG_Util/Texture/TextureFormatProcessor.h>
#include <MG_Util/Async/ShaderCompilePool.h> #include <MG_Util/Async/ShaderCompilePool.h>
#include <chrono> #include <chrono>
#include <cstring>
#include <limits>
#include <thread> #include <thread>
#if !defined(_WIN32) #if !defined(_WIN32)
@@ -1454,6 +1458,437 @@ namespace MobileGL::MG_Util::SelfTest {
disabledNote); disabledNote);
} }
// Native iterationRP compute witness. This deliberately uses a separate
// throwaway Vulkan device rather than the real renderer's queues, and it
// treats MOBILEGL_DISABLE_SUBGROUP as irrelevant: the row reports what the
// driver does, not what MobileGL elects to advertise to applications.
void ProbeVulkanIterationRPWitness(ReportBuilder& builder, PFN_vkGetInstanceProcAddr getInstanceProcAddr,
VkInstance instance, VkPhysicalDevice physicalDevice,
Uint32 computeQueueFamilyIndex,
const VkPhysicalDeviceProperties& properties,
Bool subgroupPropertiesAvailable,
const VkPhysicalDeviceSubgroupProperties& subgroupProperties) {
constexpr const char* RowName = "Subgroup first-reduction witness";
const auto fail = [&](String detail) { builder.Fail(RowName, Move(detail)); };
if (!subgroupPropertiesAvailable) {
fail("vkGetPhysicalDeviceProperties2 could not provide raw Vulkan subgroup properties");
return;
}
IterationRPWitnessLimits limits{};
limits.computeStageSupported =
(subgroupProperties.supportedStages & VK_SHADER_STAGE_COMPUTE_BIT) != 0;
limits.basicSubgroupSupported =
(subgroupProperties.supportedOperations & VK_SUBGROUP_FEATURE_BASIC_BIT) != 0;
limits.arithmeticSubgroupSupported =
(subgroupProperties.supportedOperations & VK_SUBGROUP_FEATURE_ARITHMETIC_BIT) != 0;
limits.subgroupSize = subgroupProperties.subgroupSize;
limits.maxComputeWorkGroupInvocations = properties.limits.maxComputeWorkGroupInvocations;
limits.maxComputeWorkGroupSize = {properties.limits.maxComputeWorkGroupSize[0],
properties.limits.maxComputeWorkGroupSize[1],
properties.limits.maxComputeWorkGroupSize[2]};
limits.maxComputeSharedMemorySize = properties.limits.maxComputeSharedMemorySize;
limits.maxPerStageDescriptorStorageBuffers = properties.limits.maxPerStageDescriptorStorageBuffers;
limits.maxDescriptorSetStorageBuffers = properties.limits.maxDescriptorSetStorageBuffers;
limits.maxBoundDescriptorSets = properties.limits.maxBoundDescriptorSets;
limits.maxStorageBufferRange = properties.limits.maxStorageBufferRange;
const IterationRPWitnessEligibilityResult eligibility = EvaluateIterationRPWitnessEligibility(limits);
if (eligibility.eligibility == IterationRPWitnessEligibility::SkipUnsupportedNativeFeatureSet) {
builder.Info(RowName, eligibility.detail);
return;
}
if (eligibility.eligibility == IterationRPWitnessEligibility::FailInadequateLimits) {
fail(eligibility.detail);
return;
}
if (computeQueueFamilyIndex == std::numeric_limits<Uint32>::max()) {
fail("no compute queue family is available for the native Vulkan witness");
return;
}
const auto vkGetPhysicalDeviceMemoryPropertiesFn =
reinterpret_cast<PFN_vkGetPhysicalDeviceMemoryProperties>(
getInstanceProcAddr(instance, "vkGetPhysicalDeviceMemoryProperties"));
const auto vkCreateDeviceFn =
reinterpret_cast<PFN_vkCreateDevice>(getInstanceProcAddr(instance, "vkCreateDevice"));
const auto vkDestroyDeviceFn =
reinterpret_cast<PFN_vkDestroyDevice>(getInstanceProcAddr(instance, "vkDestroyDevice"));
const auto vkGetDeviceQueueFn =
reinterpret_cast<PFN_vkGetDeviceQueue>(getInstanceProcAddr(instance, "vkGetDeviceQueue"));
const auto vkCreateBufferFn =
reinterpret_cast<PFN_vkCreateBuffer>(getInstanceProcAddr(instance, "vkCreateBuffer"));
const auto vkDestroyBufferFn =
reinterpret_cast<PFN_vkDestroyBuffer>(getInstanceProcAddr(instance, "vkDestroyBuffer"));
const auto vkGetBufferMemoryRequirementsFn = reinterpret_cast<PFN_vkGetBufferMemoryRequirements>(
getInstanceProcAddr(instance, "vkGetBufferMemoryRequirements"));
const auto vkAllocateMemoryFn =
reinterpret_cast<PFN_vkAllocateMemory>(getInstanceProcAddr(instance, "vkAllocateMemory"));
const auto vkFreeMemoryFn =
reinterpret_cast<PFN_vkFreeMemory>(getInstanceProcAddr(instance, "vkFreeMemory"));
const auto vkBindBufferMemoryFn =
reinterpret_cast<PFN_vkBindBufferMemory>(getInstanceProcAddr(instance, "vkBindBufferMemory"));
const auto vkMapMemoryFn =
reinterpret_cast<PFN_vkMapMemory>(getInstanceProcAddr(instance, "vkMapMemory"));
const auto vkUnmapMemoryFn =
reinterpret_cast<PFN_vkUnmapMemory>(getInstanceProcAddr(instance, "vkUnmapMemory"));
const auto vkCreateDescriptorSetLayoutFn = reinterpret_cast<PFN_vkCreateDescriptorSetLayout>(
getInstanceProcAddr(instance, "vkCreateDescriptorSetLayout"));
const auto vkDestroyDescriptorSetLayoutFn = reinterpret_cast<PFN_vkDestroyDescriptorSetLayout>(
getInstanceProcAddr(instance, "vkDestroyDescriptorSetLayout"));
const auto vkCreateDescriptorPoolFn =
reinterpret_cast<PFN_vkCreateDescriptorPool>(getInstanceProcAddr(instance, "vkCreateDescriptorPool"));
const auto vkDestroyDescriptorPoolFn = reinterpret_cast<PFN_vkDestroyDescriptorPool>(
getInstanceProcAddr(instance, "vkDestroyDescriptorPool"));
const auto vkAllocateDescriptorSetsFn = reinterpret_cast<PFN_vkAllocateDescriptorSets>(
getInstanceProcAddr(instance, "vkAllocateDescriptorSets"));
const auto vkUpdateDescriptorSetsFn =
reinterpret_cast<PFN_vkUpdateDescriptorSets>(getInstanceProcAddr(instance, "vkUpdateDescriptorSets"));
const auto vkCreateShaderModuleFn =
reinterpret_cast<PFN_vkCreateShaderModule>(getInstanceProcAddr(instance, "vkCreateShaderModule"));
const auto vkDestroyShaderModuleFn =
reinterpret_cast<PFN_vkDestroyShaderModule>(getInstanceProcAddr(instance, "vkDestroyShaderModule"));
const auto vkCreatePipelineLayoutFn =
reinterpret_cast<PFN_vkCreatePipelineLayout>(getInstanceProcAddr(instance, "vkCreatePipelineLayout"));
const auto vkDestroyPipelineLayoutFn = reinterpret_cast<PFN_vkDestroyPipelineLayout>(
getInstanceProcAddr(instance, "vkDestroyPipelineLayout"));
const auto vkCreateComputePipelinesFn = reinterpret_cast<PFN_vkCreateComputePipelines>(
getInstanceProcAddr(instance, "vkCreateComputePipelines"));
const auto vkDestroyPipelineFn =
reinterpret_cast<PFN_vkDestroyPipeline>(getInstanceProcAddr(instance, "vkDestroyPipeline"));
const auto vkCreateCommandPoolFn =
reinterpret_cast<PFN_vkCreateCommandPool>(getInstanceProcAddr(instance, "vkCreateCommandPool"));
const auto vkDestroyCommandPoolFn =
reinterpret_cast<PFN_vkDestroyCommandPool>(getInstanceProcAddr(instance, "vkDestroyCommandPool"));
const auto vkAllocateCommandBuffersFn = reinterpret_cast<PFN_vkAllocateCommandBuffers>(
getInstanceProcAddr(instance, "vkAllocateCommandBuffers"));
const auto vkBeginCommandBufferFn =
reinterpret_cast<PFN_vkBeginCommandBuffer>(getInstanceProcAddr(instance, "vkBeginCommandBuffer"));
const auto vkEndCommandBufferFn =
reinterpret_cast<PFN_vkEndCommandBuffer>(getInstanceProcAddr(instance, "vkEndCommandBuffer"));
const auto vkCmdBindPipelineFn =
reinterpret_cast<PFN_vkCmdBindPipeline>(getInstanceProcAddr(instance, "vkCmdBindPipeline"));
const auto vkCmdBindDescriptorSetsFn = reinterpret_cast<PFN_vkCmdBindDescriptorSets>(
getInstanceProcAddr(instance, "vkCmdBindDescriptorSets"));
const auto vkCmdDispatchFn =
reinterpret_cast<PFN_vkCmdDispatch>(getInstanceProcAddr(instance, "vkCmdDispatch"));
const auto vkCmdPipelineBarrierFn =
reinterpret_cast<PFN_vkCmdPipelineBarrier>(getInstanceProcAddr(instance, "vkCmdPipelineBarrier"));
const auto vkCreateFenceFn =
reinterpret_cast<PFN_vkCreateFence>(getInstanceProcAddr(instance, "vkCreateFence"));
const auto vkDestroyFenceFn =
reinterpret_cast<PFN_vkDestroyFence>(getInstanceProcAddr(instance, "vkDestroyFence"));
const auto vkQueueSubmitFn =
reinterpret_cast<PFN_vkQueueSubmit>(getInstanceProcAddr(instance, "vkQueueSubmit"));
const auto vkWaitForFencesFn =
reinterpret_cast<PFN_vkWaitForFences>(getInstanceProcAddr(instance, "vkWaitForFences"));
const auto vkDeviceWaitIdleFn =
reinterpret_cast<PFN_vkDeviceWaitIdle>(getInstanceProcAddr(instance, "vkDeviceWaitIdle"));
if (vkGetPhysicalDeviceMemoryPropertiesFn == nullptr || vkCreateDeviceFn == nullptr ||
vkDestroyDeviceFn == nullptr || vkGetDeviceQueueFn == nullptr || vkCreateBufferFn == nullptr ||
vkDestroyBufferFn == nullptr || vkGetBufferMemoryRequirementsFn == nullptr ||
vkAllocateMemoryFn == nullptr || vkFreeMemoryFn == nullptr || vkBindBufferMemoryFn == nullptr ||
vkMapMemoryFn == nullptr || vkUnmapMemoryFn == nullptr || vkCreateDescriptorSetLayoutFn == nullptr ||
vkDestroyDescriptorSetLayoutFn == nullptr || vkCreateDescriptorPoolFn == nullptr ||
vkDestroyDescriptorPoolFn == nullptr || vkAllocateDescriptorSetsFn == nullptr ||
vkUpdateDescriptorSetsFn == nullptr || vkCreateShaderModuleFn == nullptr ||
vkDestroyShaderModuleFn == nullptr || vkCreatePipelineLayoutFn == nullptr ||
vkDestroyPipelineLayoutFn == nullptr || vkCreateComputePipelinesFn == nullptr ||
vkDestroyPipelineFn == nullptr || vkCreateCommandPoolFn == nullptr || vkDestroyCommandPoolFn == nullptr ||
vkAllocateCommandBuffersFn == nullptr || vkBeginCommandBufferFn == nullptr ||
vkEndCommandBufferFn == nullptr || vkCmdBindPipelineFn == nullptr ||
vkCmdBindDescriptorSetsFn == nullptr || vkCmdDispatchFn == nullptr ||
vkCmdPipelineBarrierFn == nullptr || vkCreateFenceFn == nullptr || vkDestroyFenceFn == nullptr ||
vkQueueSubmitFn == nullptr || vkWaitForFencesFn == nullptr || vkDeviceWaitIdleFn == nullptr) {
fail("vkGetInstanceProcAddr could not resolve the Vulkan entry points required for the witness");
return;
}
const Float queuePriority = 1.0f;
VkDeviceQueueCreateInfo queueInfo{};
queueInfo.sType = VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO;
queueInfo.queueFamilyIndex = computeQueueFamilyIndex;
queueInfo.queueCount = 1;
queueInfo.pQueuePriorities = &queuePriority;
VkDeviceCreateInfo deviceInfo{};
deviceInfo.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO;
deviceInfo.queueCreateInfoCount = 1;
deviceInfo.pQueueCreateInfos = &queueInfo;
VkDevice device = VK_NULL_HANDLE;
VkResult result = vkCreateDeviceFn(physicalDevice, &deviceInfo, nullptr, &device);
if (result != VK_SUCCESS || device == VK_NULL_HANDLE) {
fail(format("vkCreateDevice failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkBuffer outputBuffer = VK_NULL_HANDLE;
VkDeviceMemory outputMemory = VK_NULL_HANDLE;
VkDescriptorSetLayout descriptorSetLayout = VK_NULL_HANDLE;
VkDescriptorPool descriptorPool = VK_NULL_HANDLE;
VkShaderModule shaderModule = VK_NULL_HANDLE;
VkPipelineLayout pipelineLayout = VK_NULL_HANDLE;
VkPipeline pipeline = VK_NULL_HANDLE;
VkCommandPool commandPool = VK_NULL_HANDLE;
VkFence fence = VK_NULL_HANDLE;
void* mappedOutput = nullptr;
Bool fenceWaitTimedOut = false;
const ScopeGuard destroyDeviceObjects([&]() {
if (fenceWaitTimedOut) {
// Match ProbeVulkanTimerQuery: the command may still execute
// after a timeout, so intentionally retain every device-owned
// resource rather than risking a forever wait or UAF in the ICD.
return;
}
vkDeviceWaitIdleFn(device);
if (fence != VK_NULL_HANDLE) vkDestroyFenceFn(device, fence, nullptr);
if (commandPool != VK_NULL_HANDLE) vkDestroyCommandPoolFn(device, commandPool, nullptr);
if (pipeline != VK_NULL_HANDLE) vkDestroyPipelineFn(device, pipeline, nullptr);
if (pipelineLayout != VK_NULL_HANDLE) vkDestroyPipelineLayoutFn(device, pipelineLayout, nullptr);
if (shaderModule != VK_NULL_HANDLE) vkDestroyShaderModuleFn(device, shaderModule, nullptr);
if (descriptorPool != VK_NULL_HANDLE) vkDestroyDescriptorPoolFn(device, descriptorPool, nullptr);
if (descriptorSetLayout != VK_NULL_HANDLE) {
vkDestroyDescriptorSetLayoutFn(device, descriptorSetLayout, nullptr);
}
if (mappedOutput != nullptr) vkUnmapMemoryFn(device, outputMemory);
if (outputBuffer != VK_NULL_HANDLE) vkDestroyBufferFn(device, outputBuffer, nullptr);
if (outputMemory != VK_NULL_HANDLE) vkFreeMemoryFn(device, outputMemory, nullptr);
vkDestroyDeviceFn(device, nullptr);
});
VkQueue queue = VK_NULL_HANDLE;
vkGetDeviceQueueFn(device, computeQueueFamilyIndex, 0, &queue);
if (queue == VK_NULL_HANDLE) {
fail("vkGetDeviceQueue returned a null compute queue");
return;
}
VkBufferCreateInfo bufferInfo{};
bufferInfo.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
bufferInfo.size = sizeof(IterationRPWitnessOutput);
bufferInfo.usage = VK_BUFFER_USAGE_STORAGE_BUFFER_BIT;
bufferInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
result = vkCreateBufferFn(device, &bufferInfo, nullptr, &outputBuffer);
if (result != VK_SUCCESS) {
fail(format("vkCreateBuffer(output SSBO) failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkMemoryRequirements memoryRequirements{};
vkGetBufferMemoryRequirementsFn(device, outputBuffer, &memoryRequirements);
VkPhysicalDeviceMemoryProperties memoryProperties{};
vkGetPhysicalDeviceMemoryPropertiesFn(physicalDevice, &memoryProperties);
Uint32 memoryTypeIndex = std::numeric_limits<Uint32>::max();
for (Uint32 index = 0; index < memoryProperties.memoryTypeCount; ++index) {
const Bool compatible = (memoryRequirements.memoryTypeBits & (1u << index)) != 0u;
const VkMemoryPropertyFlags required = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT |
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
if (compatible && (memoryProperties.memoryTypes[index].propertyFlags & required) == required) {
memoryTypeIndex = index;
break;
}
}
if (memoryTypeIndex == std::numeric_limits<Uint32>::max()) {
fail("no host-visible/coherent memory type is compatible with the output SSBO");
return;
}
VkMemoryAllocateInfo memoryInfo{};
memoryInfo.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
memoryInfo.allocationSize = memoryRequirements.size;
memoryInfo.memoryTypeIndex = memoryTypeIndex;
result = vkAllocateMemoryFn(device, &memoryInfo, nullptr, &outputMemory);
if (result != VK_SUCCESS) {
fail(format("vkAllocateMemory(output SSBO) failed (VkResult = {})", static_cast<Int>(result)));
return;
}
result = vkBindBufferMemoryFn(device, outputBuffer, outputMemory, 0);
if (result != VK_SUCCESS) {
fail(format("vkBindBufferMemory(output SSBO) failed (VkResult = {})", static_cast<Int>(result)));
return;
}
result = vkMapMemoryFn(device, outputMemory, 0, sizeof(IterationRPWitnessOutput), 0, &mappedOutput);
if (result != VK_SUCCESS || mappedOutput == nullptr) {
fail(format("vkMapMemory(output SSBO) failed (VkResult = {})", static_cast<Int>(result)));
return;
}
std::memset(mappedOutput, 0xa5, sizeof(IterationRPWitnessOutput));
VkDescriptorSetLayoutBinding outputBinding{};
outputBinding.binding = 0;
outputBinding.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER;
outputBinding.descriptorCount = 1;
outputBinding.stageFlags = VK_SHADER_STAGE_COMPUTE_BIT;
VkDescriptorSetLayoutCreateInfo descriptorSetLayoutInfo{};
descriptorSetLayoutInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO;
descriptorSetLayoutInfo.bindingCount = 1;
descriptorSetLayoutInfo.pBindings = &outputBinding;
result = vkCreateDescriptorSetLayoutFn(device, &descriptorSetLayoutInfo, nullptr, &descriptorSetLayout);
if (result != VK_SUCCESS) {
fail(format("vkCreateDescriptorSetLayout failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkDescriptorPoolSize poolSize{};
poolSize.type = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER;
poolSize.descriptorCount = 1;
VkDescriptorPoolCreateInfo descriptorPoolInfo{};
descriptorPoolInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO;
descriptorPoolInfo.maxSets = 1;
descriptorPoolInfo.poolSizeCount = 1;
descriptorPoolInfo.pPoolSizes = &poolSize;
result = vkCreateDescriptorPoolFn(device, &descriptorPoolInfo, nullptr, &descriptorPool);
if (result != VK_SUCCESS) {
fail(format("vkCreateDescriptorPool failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkDescriptorSet descriptorSet = VK_NULL_HANDLE;
VkDescriptorSetAllocateInfo descriptorSetInfo{};
descriptorSetInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO;
descriptorSetInfo.descriptorPool = descriptorPool;
descriptorSetInfo.descriptorSetCount = 1;
descriptorSetInfo.pSetLayouts = &descriptorSetLayout;
result = vkAllocateDescriptorSetsFn(device, &descriptorSetInfo, &descriptorSet);
if (result != VK_SUCCESS) {
fail(format("vkAllocateDescriptorSets failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkDescriptorBufferInfo outputDescriptor{};
outputDescriptor.buffer = outputBuffer;
outputDescriptor.offset = 0;
outputDescriptor.range = sizeof(IterationRPWitnessOutput);
VkWriteDescriptorSet descriptorWrite{};
descriptorWrite.sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET;
descriptorWrite.dstSet = descriptorSet;
descriptorWrite.dstBinding = 0;
descriptorWrite.descriptorCount = 1;
descriptorWrite.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER;
descriptorWrite.pBufferInfo = &outputDescriptor;
vkUpdateDescriptorSetsFn(device, 1, &descriptorWrite, 0, nullptr);
VkShaderModuleCreateInfo shaderModuleInfo{};
shaderModuleInfo.sType = VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO;
shaderModuleInfo.codeSize = sizeof(kDriverPostIterationRPWitnessSpv);
shaderModuleInfo.pCode = kDriverPostIterationRPWitnessSpv;
result = vkCreateShaderModuleFn(device, &shaderModuleInfo, nullptr, &shaderModule);
if (result != VK_SUCCESS) {
fail(format("vkCreateShaderModule failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkPipelineLayoutCreateInfo pipelineLayoutInfo{};
pipelineLayoutInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO;
pipelineLayoutInfo.setLayoutCount = 1;
pipelineLayoutInfo.pSetLayouts = &descriptorSetLayout;
result = vkCreatePipelineLayoutFn(device, &pipelineLayoutInfo, nullptr, &pipelineLayout);
if (result != VK_SUCCESS) {
fail(format("vkCreatePipelineLayout failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkPipelineShaderStageCreateInfo shaderStage{};
shaderStage.sType = VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO;
shaderStage.stage = VK_SHADER_STAGE_COMPUTE_BIT;
shaderStage.module = shaderModule;
shaderStage.pName = "main";
VkComputePipelineCreateInfo pipelineInfo{};
pipelineInfo.sType = VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO;
pipelineInfo.stage = shaderStage;
pipelineInfo.layout = pipelineLayout;
result = vkCreateComputePipelinesFn(device, VK_NULL_HANDLE, 1, &pipelineInfo, nullptr, &pipeline);
if (result != VK_SUCCESS) {
fail(format("vkCreateComputePipelines failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkCommandPoolCreateInfo commandPoolInfo{};
commandPoolInfo.sType = VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO;
commandPoolInfo.queueFamilyIndex = computeQueueFamilyIndex;
result = vkCreateCommandPoolFn(device, &commandPoolInfo, nullptr, &commandPool);
if (result != VK_SUCCESS) {
fail(format("vkCreateCommandPool failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkCommandBufferAllocateInfo commandBufferInfo{};
commandBufferInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
commandBufferInfo.commandPool = commandPool;
commandBufferInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
commandBufferInfo.commandBufferCount = 1;
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
result = vkAllocateCommandBuffersFn(device, &commandBufferInfo, &commandBuffer);
if (result != VK_SUCCESS) {
fail(format("vkAllocateCommandBuffers failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkCommandBufferBeginInfo commandBufferBeginInfo{};
commandBufferBeginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
commandBufferBeginInfo.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
result = vkBeginCommandBufferFn(commandBuffer, &commandBufferBeginInfo);
if (result != VK_SUCCESS) {
fail(format("vkBeginCommandBuffer failed (VkResult = {})", static_cast<Int>(result)));
return;
}
vkCmdBindPipelineFn(commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline);
vkCmdBindDescriptorSetsFn(commandBuffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipelineLayout, 0, 1,
&descriptorSet, 0, nullptr);
vkCmdDispatchFn(commandBuffer, 1, 1, 1);
VkBufferMemoryBarrier hostReadBarrier{};
hostReadBarrier.sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER;
hostReadBarrier.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT;
hostReadBarrier.dstAccessMask = VK_ACCESS_HOST_READ_BIT;
hostReadBarrier.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
hostReadBarrier.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
hostReadBarrier.buffer = outputBuffer;
hostReadBarrier.offset = 0;
hostReadBarrier.size = sizeof(IterationRPWitnessOutput);
vkCmdPipelineBarrierFn(commandBuffer, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0,
0, nullptr, 1, &hostReadBarrier, 0, nullptr);
result = vkEndCommandBufferFn(commandBuffer);
if (result != VK_SUCCESS) {
fail(format("vkEndCommandBuffer failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkFenceCreateInfo fenceInfo{};
fenceInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO;
result = vkCreateFenceFn(device, &fenceInfo, nullptr, &fence);
if (result != VK_SUCCESS) {
fail(format("vkCreateFence failed (VkResult = {})", static_cast<Int>(result)));
return;
}
VkSubmitInfo submitInfo{};
submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &commandBuffer;
result = vkQueueSubmitFn(queue, 1, &submitInfo, fence);
if (result != VK_SUCCESS) {
fail(format("vkQueueSubmit failed (VkResult = {})", static_cast<Int>(result)));
return;
}
constexpr Uint64 FenceTimeoutNs = 5'000'000'000ull;
result = vkWaitForFencesFn(device, 1, &fence, VK_TRUE, FenceTimeoutNs);
if (result != VK_SUCCESS) {
fenceWaitTimedOut = true;
fail(format("vkWaitForFences did not signal within 5 s (VkResult = {})", static_cast<Int>(result)));
return;
}
IterationRPWitnessOutput output{};
std::memcpy(&output, mappedOutput, sizeof(output));
const IterationRPWitnessValidationResult validation = ValidateIterationRPWitness(output);
if (!validation.ok) {
fail(validation.detail);
return;
}
builder.Pass(RowName, validation.detail);
}
// Everything the "MobileGL reported ..." rows need from the Vulkan device probe. // Everything the "MobileGL reported ..." rows need from the Vulkan device probe.
struct VulkanProbeSummary { struct VulkanProbeSummary {
Bool devicePropsValid = false; Bool devicePropsValid = false;
@@ -1669,6 +2104,7 @@ namespace MobileGL::MG_Util::SelfTest {
VkPhysicalDevice physicalDevice = VK_NULL_HANDLE; VkPhysicalDevice physicalDevice = VK_NULL_HANDLE;
Uint32 graphicsQueueFamilyIndex = 0; Uint32 graphicsQueueFamilyIndex = 0;
Uint32 graphicsQueueTimestampValidBits = 0; Uint32 graphicsQueueTimestampValidBits = 0;
Uint32 computeQueueFamilyIndex = std::numeric_limits<Uint32>::max();
for (VkPhysicalDevice candidate : devices) { for (VkPhysicalDevice candidate : devices) {
Uint32 queueFamilyCount = 0; Uint32 queueFamilyCount = 0;
vkGetPhysicalDeviceQueueFamilyPropertiesFn(candidate, &queueFamilyCount, nullptr); vkGetPhysicalDeviceQueueFamilyPropertiesFn(candidate, &queueFamilyCount, nullptr);
@@ -1684,6 +2120,13 @@ namespace MobileGL::MG_Util::SelfTest {
} }
} }
if (physicalDevice != VK_NULL_HANDLE) { if (physicalDevice != VK_NULL_HANDLE) {
for (Uint32 familyIndex = 0; familyIndex < queueFamilyCount; ++familyIndex) {
const VkQueueFamilyProperties& family = queueFamilies[familyIndex];
if (family.queueCount > 0 && (family.queueFlags & VK_QUEUE_COMPUTE_BIT) != 0) {
computeQueueFamilyIndex = familyIndex;
break;
}
}
break; break;
} }
} }
@@ -2020,13 +2463,15 @@ namespace MobileGL::MG_Util::SelfTest {
"change every N instances change every one"); "change every N instances change every one");
} }
VkPhysicalDeviceSubgroupProperties subgroupProperties{};
Bool subgroupPropertiesAvailable = false;
if (vkGetPhysicalDeviceProperties2Fn != nullptr && properties.apiVersion >= VK_API_VERSION_1_1) { if (vkGetPhysicalDeviceProperties2Fn != nullptr && properties.apiVersion >= VK_API_VERSION_1_1) {
VkPhysicalDeviceSubgroupProperties subgroupProperties{};
subgroupProperties.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_PROPERTIES; subgroupProperties.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SUBGROUP_PROPERTIES;
VkPhysicalDeviceProperties2 properties2{}; VkPhysicalDeviceProperties2 properties2{};
properties2.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2; properties2.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2;
properties2.pNext = &subgroupProperties; properties2.pNext = &subgroupProperties;
vkGetPhysicalDeviceProperties2Fn(physicalDevice, &properties2); vkGetPhysicalDeviceProperties2Fn(physicalDevice, &properties2);
subgroupPropertiesAvailable = true;
const Bool subgroupUsable = subgroupProperties.subgroupSize > 0 && const Bool subgroupUsable = subgroupProperties.subgroupSize > 0 &&
(subgroupProperties.supportedStages & VK_SHADER_STAGE_COMPUTE_BIT) != 0 && (subgroupProperties.supportedStages & VK_SHADER_STAGE_COMPUTE_BIT) != 0 &&
(subgroupProperties.supportedOperations & VK_SUBGROUP_FEATURE_BASIC_BIT) != 0; (subgroupProperties.supportedOperations & VK_SUBGROUP_FEATURE_BASIC_BIT) != 0;
@@ -2046,6 +2491,9 @@ namespace MobileGL::MG_Util::SelfTest {
builder.Warn("Compute shader subgroup", "subgroup properties could not be queried"); builder.Warn("Compute shader subgroup", "subgroup properties could not be queried");
} }
ProbeVulkanIterationRPWitness(builder, getInstanceProcAddr, instance, physicalDevice, computeQueueFamilyIndex,
properties, subgroupPropertiesAvailable, subgroupProperties);
if (HasVkExtension(deviceExtensions, VK_KHR_DRAW_INDIRECT_COUNT_EXTENSION_NAME)) { if (HasVkExtension(deviceExtensions, VK_KHR_DRAW_INDIRECT_COUNT_EXTENSION_NAME)) {
builder.Pass("VK_KHR_draw_indirect_count", builder.Pass("VK_KHR_draw_indirect_count",
"supported (count-buffer indirect draws run as single native " "supported (count-buffer indirect draws run as single native "
@@ -0,0 +1,164 @@
// MobileGL - MobileGL/MG_Util/SelfTest/DriverPostIterationRPWitness.comp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Native Vulkan GLSL 450 witness for iterationRP's first subgroup reduction.
// It is intentionally independent of the GL 430 integration scenario. The body
// below preserves iterationRP's source reduction; the surrounding diagnostics
// only observe its topology and cache handoffs.
#version 450
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_arithmetic : require
layout(local_size_x = 32, local_size_y = 16, local_size_z = 1) in;
const uint kTopologyNonuniformNumSubgroups = 1u << 0u;
const uint kTopologyInvalidNumSubgroups = 1u << 1u;
const uint kTopologyInvalidSubgroupId = 1u << 2u;
const uint kTopologyInvalidSubgroupLane = 1u << 3u;
const uint kWitnessMagic = 0x50323033u;
layout(std430, set = 0, binding = 0) buffer IterationRPWitnessOutput {
uint magic;
uint topologyFlags;
uint numSubgroups;
uint loopLength;
uint seenSubgroupMask;
uvec4 owner511;
uint lastLaneWriterCount[32];
uint indexedInputTotal[32];
vec2 rawPrefix[32];
vec2 scanCache[6][32];
vec2 finalAverage;
} outWitness;
// iterationRP's cache stays separate from all diagnostic shared state. In
// particular, no instrumentation stores through prefixSumCache except source
// writes retained below.
shared vec2 prefixSumCache[32];
shared uint canonicalNumSubgroups;
shared uint topologyFlagsShared;
shared uint seenSubgroupMaskShared;
shared uint lastLaneWriterCountShared[32];
shared uint indexedInputTotalShared[32];
void main() {
const uint localInvocationIndex = gl_LocalInvocationIndex;
// Host memory is deliberately poisoned before dispatch. Initialize only
// shared atomic diagnostic state; owner, average, and magic remain poisoned
// until their required post-source-barrier writes below.
if (localInvocationIndex == 0u) {
canonicalNumSubgroups = 0u;
topologyFlagsShared = 0u;
seenSubgroupMaskShared = 0u;
}
if (localInvocationIndex < 32u) {
lastLaneWriterCountShared[localInvocationIndex] = 0u;
indexedInputTotalShared[localInvocationIndex] = 0u;
}
memoryBarrierShared();
barrier();
// Invocation zero defines the canonical domain. It is broadcast through
// shared memory before every invocation records its own raw observation.
if (localInvocationIndex == 0u) {
canonicalNumSubgroups = gl_NumSubgroups;
outWitness.numSubgroups = gl_NumSubgroups;
}
barrier();
const uint canonicalN = canonicalNumSubgroups;
if (gl_NumSubgroups != canonicalN)
atomicOr(topologyFlagsShared, kTopologyNonuniformNumSubgroups);
if (gl_NumSubgroups < 2u || gl_NumSubgroups > 32u)
atomicOr(topologyFlagsShared, kTopologyInvalidNumSubgroups);
if (gl_SubgroupID >= canonicalN || gl_SubgroupID >= 32u)
atomicOr(topologyFlagsShared, kTopologyInvalidSubgroupId);
if (gl_SubgroupSize == 0u || gl_SubgroupInvocationID >= gl_SubgroupSize)
atomicOr(topologyFlagsShared, kTopologyInvalidSubgroupLane);
// Keep all atomic collection bounded by the canonical valid domain. A
// nonuniform/broken report reaches the uniform safety branch below instead
// of making some lanes return before a barrier.
const bool canonicalDomain = canonicalN >= 2u && canonicalN <= 32u;
const bool idInCanonicalDomain = canonicalDomain && gl_SubgroupID < canonicalN;
if (idInCanonicalDomain) {
atomicOr(seenSubgroupMaskShared, 1u << gl_SubgroupID);
atomicAdd(indexedInputTotalShared[gl_SubgroupID], localInvocationIndex + 1u);
if (gl_SubgroupSize != 0u && gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
atomicAdd(lastLaneWriterCountShared[gl_SubgroupID], 1u);
}
memoryBarrierShared();
barrier();
if (localInvocationIndex == 0u)
outWitness.seenSubgroupMask = seenSubgroupMaskShared;
if (localInvocationIndex < 32u) {
outWitness.lastLaneWriterCount[localInvocationIndex] = lastLaneWriterCountShared[localInvocationIndex];
outWitness.indexedInputTotal[localInvocationIndex] = indexedInputTotalShared[localInvocationIndex];
}
// This branch is uniform after collection and is solely a safety guard for
// broken topology reports. The valid side retains iterationRP verbatim.
const bool sourceDomain = canonicalDomain && topologyFlagsShared == 0u;
if (sourceDomain) {
vec2 sampleLuminance = vec2(float(gl_LocalInvocationIndex + 1u), 0.0);
sampleLuminance = subgroupInclusiveAdd(sampleLuminance);
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
barrier();
if (gl_LocalInvocationIndex < gl_NumSubgroups)
outWitness.rawPrefix[gl_LocalInvocationIndex] = prefixSumCache[gl_LocalInvocationIndex];
barrier();
uint loopLength = uint(findMSB(gl_NumSubgroups));
loopLength += uint(gl_NumSubgroups - (1u << (loopLength - 1u)) > 0u);
if (gl_LocalInvocationIndex == 0u)
outWitness.loopLength = loopLength;
for (uint scanStage = 0u; scanStage < loopLength; ++scanStage) {
if ((gl_SubgroupID & (1u << scanStage)) > 0u) {
sampleLuminance += prefixSumCache[(gl_SubgroupID >> scanStage << scanStage) - 1u];
if (gl_SubgroupInvocationID == gl_SubgroupSize - 1u)
prefixSumCache[gl_SubgroupID] = sampleLuminance;
}
barrier();
if (gl_LocalInvocationIndex < gl_NumSubgroups)
outWitness.scanCache[scanStage][gl_LocalInvocationIndex] =
prefixSumCache[gl_LocalInvocationIndex];
// A second, diagnostic-only barrier prevents a faster invocation
// from entering the next source stage while another reads this cache.
barrier();
}
if (gl_LocalInvocationIndex == 511u)
prefixSumCache[0] = sampleLuminance / 512.0;
barrier();
if (gl_LocalInvocationIndex == 511u) {
outWitness.owner511 = uvec4(gl_SubgroupSize, gl_NumSubgroups, gl_SubgroupID,
gl_SubgroupInvocationID);
outWitness.finalAverage = prefixSumCache[0];
}
}
// Both sides of the uniform branch reach this barrier. The magic is the
// completion latch and therefore cannot be written before the final barrier.
barrier();
if (localInvocationIndex == 0u) {
outWitness.topologyFlags = topologyFlagsShared;
outWitness.magic = kWitnessMagic;
}
}
@@ -0,0 +1,277 @@
// MobileGL - MobileGL/MG_Util/SelfTest/DriverPostIterationRPWitness.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DriverPostIterationRPWitness.h"
#include <bit>
#include <sstream>
#include <utility>
#include <vector>
namespace MobileGL::MG_Util::SelfTest {
namespace {
[[nodiscard]] IterationRPWitnessValidationResult Failure(IterationRPWitnessValidationFailure failure,
std::string detail,
std::uint32_t scanStage = 0u,
std::uint32_t subgroup = 0u) {
IterationRPWitnessValidationResult result;
result.ok = false;
result.failure = failure;
result.scanStage = scanStage;
result.subgroup = subgroup;
result.detail = std::move(detail);
return result;
}
[[nodiscard]] std::uint32_t FloatBits(float value) {
return std::bit_cast<std::uint32_t>(value);
}
[[nodiscard]] bool SameBits(float lhs, float rhs) {
return FloatBits(lhs) == FloatBits(rhs);
}
[[nodiscard]] bool SameBits(const IterationRPWitnessVec2& lhs, const IterationRPWitnessVec2& rhs) {
return SameBits(lhs.x, rhs.x) && SameBits(lhs.y, rhs.y);
}
[[nodiscard]] std::string Vec2String(const IterationRPWitnessVec2& value) {
std::ostringstream output;
output << '(' << value.x << ',' << value.y << ')';
return output.str();
}
[[nodiscard]] std::uint32_t ExpectedSeenSubgroupMask(std::uint32_t numSubgroups) {
return numSubgroups == kIterationRPWitnessMaxSubgroups ? 0xffffffffu : (1u << numSubgroups) - 1u;
}
[[nodiscard]] std::string JoinRequirements(const std::vector<std::string>& requirements) {
std::ostringstream output;
for (std::size_t i = 0; i < requirements.size(); ++i) {
if (i != 0u) output << "; ";
output << requirements[i];
}
return output.str();
}
} // namespace
IterationRPWitnessEligibilityResult
EvaluateIterationRPWitnessEligibility(const IterationRPWitnessLimits& limits) {
// This classification deliberately precedes numeric limits. An absent native
// compute/basic/arithmetic subgroup contract means there is nothing to witness,
// whereas every resource/entry-point failure on a capable device is a POST FAIL.
if (!limits.computeStageSupported || !limits.basicSubgroupSupported || !limits.arithmeticSubgroupSupported) {
std::vector<std::string> missing;
if (!limits.computeStageSupported) missing.emplace_back("VK_SHADER_STAGE_COMPUTE_BIT");
if (!limits.basicSubgroupSupported) missing.emplace_back("VK_SUBGROUP_FEATURE_BASIC_BIT");
if (!limits.arithmeticSubgroupSupported) missing.emplace_back("VK_SUBGROUP_FEATURE_ARITHMETIC_BIT");
return {IterationRPWitnessEligibility::SkipUnsupportedNativeFeatureSet,
"skipped because the native compute/basic/arithmetic subgroup feature set is unsupported (missing " +
JoinRequirements(missing) + ')'};
}
std::vector<std::string> inadequate;
if (limits.subgroupSize == 0u) {
inadequate.emplace_back("subgroupSize == 0");
}
if (limits.maxComputeWorkGroupInvocations < kIterationRPWitnessInvocationCount) {
inadequate.emplace_back("maxComputeWorkGroupInvocations < 512");
}
if (limits.maxComputeWorkGroupSize[0] < 32u || limits.maxComputeWorkGroupSize[1] < 16u ||
limits.maxComputeWorkGroupSize[2] < 1u) {
inadequate.emplace_back("maxComputeWorkGroupSize does not cover 32x16x1");
}
if (limits.maxComputeSharedMemorySize < kIterationRPWitnessSharedMemoryBytes) {
inadequate.emplace_back("maxComputeSharedMemorySize < " +
std::to_string(kIterationRPWitnessSharedMemoryBytes));
}
if (limits.maxPerStageDescriptorStorageBuffers < 1u) {
inadequate.emplace_back("maxPerStageDescriptorStorageBuffers < 1");
}
if (limits.maxDescriptorSetStorageBuffers < 1u) {
inadequate.emplace_back("maxDescriptorSetStorageBuffers < 1");
}
if (limits.maxBoundDescriptorSets < 1u) {
inadequate.emplace_back("maxBoundDescriptorSets < 1");
}
if (limits.maxStorageBufferRange < sizeof(IterationRPWitnessOutput)) {
inadequate.emplace_back("maxStorageBufferRange < " +
std::to_string(sizeof(IterationRPWitnessOutput)));
}
if (!inadequate.empty()) {
return {IterationRPWitnessEligibility::FailInadequateLimits,
"insufficient Vulkan limits for a 32x16x1 workgroup, one output SSBO, and " +
std::to_string(kIterationRPWitnessSharedMemoryBytes) + " bytes of shared memory: " +
JoinRequirements(inadequate)};
}
return {IterationRPWitnessEligibility::Execute, {}};
}
std::uint32_t ComputeIterationRPWitnessLoopLength(std::uint32_t numSubgroups) {
if (numSubgroups < 2u || numSubgroups > kIterationRPWitnessMaxSubgroups) return 0u;
// Exact C++ spelling of the source's findMSB-based calculation. In
// particular, its final iteration for powers of two is intentional.
std::uint32_t loopLength = 0u;
for (std::uint32_t value = numSubgroups; value > 1u; value >>= 1u) {
++loopLength;
}
loopLength += static_cast<std::uint32_t>(numSubgroups - (1u << (loopLength - 1u)) > 0u);
return loopLength;
}
IterationRPWitnessValidationResult ValidateIterationRPWitness(const IterationRPWitnessOutput& output) {
// 1. Completion. A poisoned or unwritten result must never turn into a
// topology diagnosis, because it says nothing about execution.
if (output.magic != kIterationRPWitnessMagic) {
std::ostringstream detail;
detail << "completion: magic was 0x" << std::hex << output.magic << ", expected 0x"
<< kIterationRPWitnessMagic;
return Failure(IterationRPWitnessValidationFailure::Completion, detail.str());
}
// 2. Observed topology. All checks consume observations written by the
// shader, rather than inferring subgroup layout from invocation indices.
const std::uint32_t numSubgroups = output.numSubgroups;
if (numSubgroups < 2u || numSubgroups > kIterationRPWitnessMaxSubgroups) {
std::ostringstream detail;
detail << "topology: canonical gl_NumSubgroups=" << numSubgroups << " is outside [2, 32]";
return Failure(IterationRPWitnessValidationFailure::Topology, detail.str());
}
if ((output.topologyFlags & IterationRPWitnessNonuniformNumSubgroups) != 0u) {
return Failure(IterationRPWitnessValidationFailure::Topology,
"topology: gl_NumSubgroups differed across workgroup");
}
if ((output.topologyFlags & IterationRPWitnessInvalidNumSubgroups) != 0u) {
return Failure(IterationRPWitnessValidationFailure::Topology,
"topology: an invocation reported gl_NumSubgroups outside [2, 32]");
}
if ((output.topologyFlags & IterationRPWitnessInvalidSubgroupId) != 0u) {
return Failure(IterationRPWitnessValidationFailure::Topology,
"topology: an invocation reported an invalid gl_SubgroupID");
}
if ((output.topologyFlags & IterationRPWitnessInvalidSubgroupLane) != 0u) {
return Failure(IterationRPWitnessValidationFailure::Topology,
"topology: an invocation reported an invalid subgroup lane");
}
if ((output.topologyFlags & ~(IterationRPWitnessNonuniformNumSubgroups |
IterationRPWitnessInvalidNumSubgroups |
IterationRPWitnessInvalidSubgroupId |
IterationRPWitnessInvalidSubgroupLane)) != 0u) {
std::ostringstream detail;
detail << "topology: unknown topology flags 0x" << std::hex << output.topologyFlags;
return Failure(IterationRPWitnessValidationFailure::Topology, detail.str());
}
const std::uint32_t expectedMask = ExpectedSeenSubgroupMask(numSubgroups);
if (output.seenSubgroupMask != expectedMask) {
std::ostringstream detail;
detail << "topology: seen subgroup-ID mask was 0x" << std::hex << output.seenSubgroupMask
<< ", expected 0x" << expectedMask;
return Failure(IterationRPWitnessValidationFailure::Topology, detail.str());
}
const std::uint32_t expectedLoopLength = ComputeIterationRPWitnessLoopLength(numSubgroups);
if (output.loopLength != expectedLoopLength) {
std::ostringstream detail;
detail << "topology: loopLength was " << std::dec << output.loopLength << ", expected "
<< expectedLoopLength;
return Failure(IterationRPWitnessValidationFailure::Topology, detail.str());
}
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
if (output.lastLaneWriterCount[subgroup] != 1u) {
std::ostringstream detail;
detail << "topology: subgroup " << subgroup << " has "
<< output.lastLaneWriterCount[subgroup] << " source last-lane writers, expected exactly 1";
return Failure(IterationRPWitnessValidationFailure::Topology, detail.str(), 0u, subgroup);
}
}
if (output.owner511.y != numSubgroups) {
std::ostringstream detail;
detail << "final owner: invocation 511 reported gl_NumSubgroups=" << output.owner511.y << ", expected "
<< numSubgroups;
return Failure(IterationRPWitnessValidationFailure::FinalOwner, detail.str());
}
if (output.owner511.z != numSubgroups - 1u) {
std::ostringstream detail;
detail << "final owner: invocation 511 is not in the highest subgroup (id" << output.owner511.z
<< ", expected id" << (numSubgroups - 1u) << ')';
return Failure(IterationRPWitnessValidationFailure::FinalOwner, detail.str());
}
if (output.owner511.x == 0u || output.owner511.w != output.owner511.x - 1u) {
std::ostringstream detail;
detail << "final owner: invocation 511 is not the last lane of highest subgroup (size "
<< output.owner511.x << ", lane " << output.owner511.w << ')';
return Failure(IterationRPWitnessValidationFailure::FinalOwner, detail.str());
}
// 3. Initial subgroup handoff. The atomic scalar totals are independent
// of subgroupInclusiveAdd; their sum and the cache values establish that
// the final lanes handed off the native vector inclusive-add results.
std::uint64_t indexedTotal = 0u;
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
indexedTotal += output.indexedInputTotal[subgroup];
}
if (indexedTotal != 131328u) {
std::ostringstream detail;
detail << "initial subgroup handoff: indexed input total was " << indexedTotal << ", expected 131328";
return Failure(IterationRPWitnessValidationFailure::InitialSubgroupHandoff, detail.str());
}
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
const IterationRPWitnessVec2 expected = {static_cast<float>(output.indexedInputTotal[subgroup]), 0.0f};
if (!SameBits(output.rawPrefix[subgroup], expected)) {
std::ostringstream detail;
detail << "initial subgroup handoff: subgroup " << subgroup << " rawPrefix observed "
<< Vec2String(output.rawPrefix[subgroup]) << ", expected " << Vec2String(expected);
return Failure(IterationRPWitnessValidationFailure::InitialSubgroupHandoff, detail.str(), 0u,
subgroup);
}
}
// 4. Source scan. Do not substitute a conventional scan: this reproduces
// the source cache index expression and stage ordering word for word.
std::array<IterationRPWitnessVec2, kIterationRPWitnessMaxSubgroups> expectedCache = output.rawPrefix;
for (std::uint32_t scanStage = 0u; scanStage < expectedLoopLength; ++scanStage) {
auto cacheAfterStage = expectedCache;
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
if ((subgroup & (1u << scanStage)) > 0u) {
const std::uint32_t sourceCacheIndex = (subgroup >> scanStage << scanStage) - 1u;
cacheAfterStage[subgroup].x += expectedCache[sourceCacheIndex].x;
cacheAfterStage[subgroup].y += expectedCache[sourceCacheIndex].y;
}
}
expectedCache = cacheAfterStage;
for (std::uint32_t subgroup = 0u; subgroup < numSubgroups; ++subgroup) {
if (!SameBits(output.scanCache[scanStage][subgroup], expectedCache[subgroup])) {
std::ostringstream detail;
detail << "source scan stage " << scanStage << ", subgroup " << subgroup << ": observed "
<< Vec2String(output.scanCache[scanStage][subgroup]) << ", expected "
<< Vec2String(expectedCache[subgroup]);
return Failure(IterationRPWitnessValidationFailure::SourceScan, detail.str(), scanStage, subgroup);
}
}
}
// 5. The owner contract was checked above with the other topology facts;
// this final result remains a separate exact-vector check.
const IterationRPWitnessVec2 expectedAverage = {256.5f, 0.0f};
if (!SameBits(output.finalAverage, expectedAverage)) {
std::ostringstream detail;
detail << "final average: observed " << Vec2String(output.finalAverage) << ", expected "
<< Vec2String(expectedAverage);
return Failure(IterationRPWitnessValidationFailure::FinalAverage, detail.str());
}
std::ostringstream detail;
detail << "N=" << numSubgroups << ", owner511=id" << output.owner511.z << "/lane" << output.owner511.w
<< ", " << expectedLoopLength << " scan stages, average=" << Vec2String(output.finalAverage);
IterationRPWitnessValidationResult result;
result.ok = true;
result.failure = IterationRPWitnessValidationFailure::None;
result.detail = detail.str();
return result;
}
} // namespace MobileGL::MG_Util::SelfTest
@@ -0,0 +1,153 @@
// MobileGL - MobileGL/MG_Util/SelfTest/DriverPostIterationRPWitness.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Compact, native-Vulkan iterationRP first-reduction witness ABI and its pure
// validator. The types below deliberately mirror DriverPostIterationRPWitness.comp's
// single std430 storage block; changing either side requires updating the static
// layout assertions here.
#pragma once
#include <array>
#include <cstddef>
#include <cstdint>
#include <string>
#include <type_traits>
namespace MobileGL::MG_Util::SelfTest {
// "P203": the pack's trace program id, kept stable so the checked-in witness
// SPIR-V (DriverPostIterationRPWitnessSpv.h) needs no regeneration.
constexpr std::uint32_t kIterationRPWitnessMagic = 0x50323033u;
constexpr std::uint32_t kIterationRPWitnessInvocationCount = 512u;
constexpr std::uint32_t kIterationRPWitnessMaxSubgroups = 32u;
constexpr std::uint32_t kIterationRPWitnessMaxScanStages = 6u;
// These bit values are shared with the GLSL source. They document failures in
// topology observations rather than guessing a topology from local IDs on the host.
enum IterationRPWitnessTopologyFlag : std::uint32_t {
IterationRPWitnessNonuniformNumSubgroups = 1u << 0u,
IterationRPWitnessInvalidNumSubgroups = 1u << 1u,
IterationRPWitnessInvalidSubgroupId = 1u << 2u,
IterationRPWitnessInvalidSubgroupLane = 1u << 3u,
};
struct alignas(8) IterationRPWitnessVec2 {
float x;
float y;
};
struct alignas(16) IterationRPWitnessUVec4 {
std::uint32_t x;
std::uint32_t y;
std::uint32_t z;
std::uint32_t w;
};
// std430 layout of DriverPostIterationRPWitness.comp's IterationRPWitnessOutput block.
struct alignas(16) IterationRPWitnessOutput {
std::uint32_t magic;
std::uint32_t topologyFlags;
std::uint32_t numSubgroups;
std::uint32_t loopLength;
std::uint32_t seenSubgroupMask;
IterationRPWitnessUVec4 owner511;
std::array<std::uint32_t, kIterationRPWitnessMaxSubgroups> lastLaneWriterCount;
std::array<std::uint32_t, kIterationRPWitnessMaxSubgroups> indexedInputTotal;
std::array<IterationRPWitnessVec2, kIterationRPWitnessMaxSubgroups> rawPrefix;
std::array<std::array<IterationRPWitnessVec2, kIterationRPWitnessMaxSubgroups>,
kIterationRPWitnessMaxScanStages>
scanCache;
IterationRPWitnessVec2 finalAverage;
};
static_assert(std::is_standard_layout_v<IterationRPWitnessVec2>);
static_assert(std::is_standard_layout_v<IterationRPWitnessUVec4>);
static_assert(std::is_standard_layout_v<IterationRPWitnessOutput>);
static_assert(sizeof(IterationRPWitnessVec2) == 8u);
static_assert(alignof(IterationRPWitnessVec2) == 8u);
static_assert(sizeof(IterationRPWitnessUVec4) == 16u);
static_assert(alignof(IterationRPWitnessUVec4) == 16u);
static_assert(offsetof(IterationRPWitnessOutput, magic) == 0u);
static_assert(offsetof(IterationRPWitnessOutput, topologyFlags) == 4u);
static_assert(offsetof(IterationRPWitnessOutput, numSubgroups) == 8u);
static_assert(offsetof(IterationRPWitnessOutput, loopLength) == 12u);
static_assert(offsetof(IterationRPWitnessOutput, seenSubgroupMask) == 16u);
static_assert(offsetof(IterationRPWitnessOutput, owner511) == 32u);
static_assert(offsetof(IterationRPWitnessOutput, lastLaneWriterCount) == 48u);
static_assert(offsetof(IterationRPWitnessOutput, indexedInputTotal) == 176u);
static_assert(offsetof(IterationRPWitnessOutput, rawPrefix) == 304u);
static_assert(offsetof(IterationRPWitnessOutput, scanCache) == 560u);
static_assert(offsetof(IterationRPWitnessOutput, finalAverage) == 2096u);
static_assert(sizeof(IterationRPWitnessOutput) == 2112u);
// The witness uses prefixSumCache[32], three scalar shared diagnostics, and
// two 32-entry scalar diagnostic arrays in the GLSL source. Keep this
// independent of the output SSBO size.
constexpr std::uint32_t kIterationRPWitnessSharedMemoryBytes =
kIterationRPWitnessMaxSubgroups * sizeof(IterationRPWitnessVec2) +
3u * sizeof(std::uint32_t) +
2u * kIterationRPWitnessMaxSubgroups * sizeof(std::uint32_t);
enum class IterationRPWitnessEligibility {
Execute,
SkipUnsupportedNativeFeatureSet,
FailInadequateLimits,
};
// The raw physical-device conditions needed by the native witness. This is
// intentionally distinct from MobileGL's advertised-extension policy.
struct IterationRPWitnessLimits {
bool computeStageSupported = false;
bool basicSubgroupSupported = false;
bool arithmeticSubgroupSupported = false;
std::uint32_t subgroupSize = 0u;
std::uint32_t maxComputeWorkGroupInvocations = 0u;
std::array<std::uint32_t, 3> maxComputeWorkGroupSize{};
std::uint32_t maxComputeSharedMemorySize = 0u;
std::uint32_t maxPerStageDescriptorStorageBuffers = 0u;
std::uint32_t maxDescriptorSetStorageBuffers = 0u;
std::uint32_t maxBoundDescriptorSets = 0u;
std::uint64_t maxStorageBufferRange = 0u;
};
struct IterationRPWitnessEligibilityResult {
IterationRPWitnessEligibility eligibility = IterationRPWitnessEligibility::FailInadequateLimits;
std::string detail;
};
enum class IterationRPWitnessValidationFailure {
None,
Completion,
Topology,
InitialSubgroupHandoff,
SourceScan,
FinalOwner,
FinalAverage,
};
struct IterationRPWitnessValidationResult {
bool ok = false;
IterationRPWitnessValidationFailure failure = IterationRPWitnessValidationFailure::Completion;
std::uint32_t scanStage = 0u;
std::uint32_t subgroup = 0u;
std::string detail;
};
[[nodiscard]] IterationRPWitnessEligibilityResult
EvaluateIterationRPWitnessEligibility(const IterationRPWitnessLimits& limits);
// Mirrors the source's findMSB expression for valid N in [2, 32].
[[nodiscard]] std::uint32_t ComputeIterationRPWitnessLoopLength(std::uint32_t numSubgroups);
[[nodiscard]] IterationRPWitnessValidationResult
ValidateIterationRPWitness(const IterationRPWitnessOutput& output);
} // namespace MobileGL::MG_Util::SelfTest
@@ -0,0 +1,297 @@
// MobileGL - MobileGL/MG_Util/SelfTest/DriverPostIterationRPWitnessSpv.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Generated from DriverPostIterationRPWitness.comp with:
// glslangValidator --target-env vulkan1.1 -V DriverPostIterationRPWitness.comp
// Validated with spirv-val --target-env vulkan1.1. Do not edit words by hand.
//
// The stored words predate the Program203 -> IterationRP source rename, so their
// embedded OpName debug strings still spell the old identifiers; regeneration from
// the renamed source produces semantically identical code differing only in those
// strings. The witness magic stays 0x50323033 ("P203" - the trace's program id) so
// these words remain valid without regeneration.
#pragma once
#include <cstddef>
#include <cstdint>
namespace MobileGL::MG_Util::SelfTest {
inline constexpr std::uint32_t kDriverPostIterationRPWitnessSpv[] = {
0x07230203u, 0x00010300u, 0x0008000bu, 0x00000145u, 0x00000000u, 0x00020011u, 0x00000001u, 0x00020011u,
0x0000003du, 0x00020011u, 0x0000003fu, 0x0006000bu, 0x00000001u, 0x4c534c47u, 0x6474732eu, 0x3035342eu,
0x00000000u, 0x0003000eu, 0x00000000u, 0x00000001u, 0x000a000fu, 0x00000005u, 0x00000004u, 0x6e69616du,
0x00000000u, 0x0000000au, 0x0000002au, 0x00000050u, 0x0000005eu, 0x00000064u, 0x00060010u, 0x00000004u,
0x00000011u, 0x00000020u, 0x00000010u, 0x00000001u, 0x00030003u, 0x00000002u, 0x000001c2u, 0x000a0004u,
0x4b5f4c47u, 0x735f5248u, 0x65646168u, 0x75735f72u, 0x6f726762u, 0x615f7075u, 0x68746972u, 0x6974656du,
0x00000063u, 0x00090004u, 0x4b5f4c47u, 0x735f5248u, 0x65646168u, 0x75735f72u, 0x6f726762u, 0x625f7075u,
0x63697361u, 0x00000000u, 0x00040005u, 0x00000004u, 0x6e69616du, 0x00000000u, 0x00080005u, 0x00000008u,
0x61636f6cu, 0x766e496cu, 0x7461636fu, 0x496e6f69u, 0x7865646eu, 0x00000000u, 0x00080005u, 0x0000000au,
0x4c5f6c67u, 0x6c61636fu, 0x6f766e49u, 0x69746163u, 0x6e496e6fu, 0x00786564u, 0x00080005u, 0x00000013u,
0x6f6e6163u, 0x6163696eu, 0x6d754e6cu, 0x67627553u, 0x70756f72u, 0x00000073u, 0x00070005u, 0x00000014u,
0x6f706f74u, 0x79676f6cu, 0x67616c46u, 0x61685373u, 0x00646572u, 0x00080005u, 0x00000015u, 0x6e656573u,
0x67627553u, 0x70756f72u, 0x6b73614du, 0x72616853u, 0x00006465u, 0x00090005u, 0x0000001du, 0x7473616cu,
0x656e614cu, 0x74697257u, 0x6f437265u, 0x53746e75u, 0x65726168u, 0x00000064u, 0x00080005u, 0x00000020u,
0x65646e69u, 0x49646578u, 0x7475706eu, 0x61746f54u, 0x6168536cu, 0x00646572u, 0x00060005u, 0x0000002au,
0x4e5f6c67u, 0x75536d75u, 0x6f726762u, 0x00737075u, 0x00080005u, 0x00000035u, 0x676f7250u, 0x326d6172u,
0x69573330u, 0x73656e74u, 0x74754f73u, 0x00747570u, 0x00050006u, 0x00000035u, 0x00000000u, 0x6967616du,
0x00000063u, 0x00070006u, 0x00000035u, 0x00000001u, 0x6f706f74u, 0x79676f6cu, 0x67616c46u, 0x00000073u,
0x00070006u, 0x00000035u, 0x00000002u, 0x536d756eu, 0x72676275u, 0x7370756fu, 0x00000000u, 0x00060006u,
0x00000035u, 0x00000003u, 0x706f6f6cu, 0x676e654cu, 0x00006874u, 0x00080006u, 0x00000035u, 0x00000004u,
0x6e656573u, 0x67627553u, 0x70756f72u, 0x6b73614du, 0x00000000u, 0x00060006u, 0x00000035u, 0x00000005u,
0x656e776fu, 0x31313572u, 0x00000000u, 0x00080006u, 0x00000035u, 0x00000006u, 0x7473616cu, 0x656e614cu,
0x74697257u, 0x6f437265u, 0x00746e75u, 0x00080006u, 0x00000035u, 0x00000007u, 0x65646e69u, 0x49646578u,
0x7475706eu, 0x61746f54u, 0x0000006cu, 0x00060006u, 0x00000035u, 0x00000008u, 0x50776172u, 0x69666572u,
0x00000078u, 0x00060006u, 0x00000035u, 0x00000009u, 0x6e616373u, 0x68636143u, 0x00000065u, 0x00070006u,
0x00000035u, 0x0000000au, 0x616e6966u, 0x6576416cu, 0x65676172u, 0x00000000u, 0x00050005u, 0x00000037u,
0x5774756fu, 0x656e7469u, 0x00007373u, 0x00050005u, 0x0000003du, 0x6f6e6163u, 0x6163696eu, 0x00004e6cu,
0x00060005u, 0x00000050u, 0x535f6c67u, 0x72676275u, 0x4970756fu, 0x00000044u, 0x00060005u, 0x0000005eu,
0x535f6c67u, 0x72676275u, 0x5370756fu, 0x00657a69u, 0x00080005u, 0x00000064u, 0x535f6c67u, 0x72676275u,
0x4970756fu, 0x636f766eu, 0x6f697461u, 0x0044496eu, 0x00060005u, 0x0000006eu, 0x6f6e6163u, 0x6163696eu,
0x6d6f446cu, 0x006e6961u, 0x00070005u, 0x00000074u, 0x6e496469u, 0x6f6e6143u, 0x6163696eu, 0x6d6f446cu,
0x006e6961u, 0x00060005u, 0x000000acu, 0x72756f73u, 0x6f446563u, 0x6e69616du, 0x00000000u, 0x00060005u,
0x000000b7u, 0x706d6173u, 0x754c656cu, 0x616e696du, 0x0065636eu, 0x00060005u, 0x000000c8u, 0x66657270u,
0x75537869u, 0x6361436du, 0x00006568u, 0x00050005u, 0x000000d9u, 0x706f6f6cu, 0x676e654cu, 0x00006874u,
0x00050005u, 0x000000edu, 0x6e616373u, 0x67617453u, 0x00000065u, 0x00040047u, 0x0000000au, 0x0000000bu,
0x0000001du, 0x00040047u, 0x0000002au, 0x0000000bu, 0x00000026u, 0x00040047u, 0x0000002du, 0x00000006u,
0x00000004u, 0x00040047u, 0x0000002eu, 0x00000006u, 0x00000004u, 0x00040047u, 0x00000031u, 0x00000006u,
0x00000008u, 0x00040047u, 0x00000032u, 0x00000006u, 0x00000008u, 0x00040047u, 0x00000034u, 0x00000006u,
0x00000100u, 0x00030047u, 0x00000035u, 0x00000002u, 0x00050048u, 0x00000035u, 0x00000000u, 0x00000023u,
0x00000000u, 0x00050048u, 0x00000035u, 0x00000001u, 0x00000023u, 0x00000004u, 0x00050048u, 0x00000035u,
0x00000002u, 0x00000023u, 0x00000008u, 0x00050048u, 0x00000035u, 0x00000003u, 0x00000023u, 0x0000000cu,
0x00050048u, 0x00000035u, 0x00000004u, 0x00000023u, 0x00000010u, 0x00050048u, 0x00000035u, 0x00000005u,
0x00000023u, 0x00000020u, 0x00050048u, 0x00000035u, 0x00000006u, 0x00000023u, 0x00000030u, 0x00050048u,
0x00000035u, 0x00000007u, 0x00000023u, 0x000000b0u, 0x00050048u, 0x00000035u, 0x00000008u, 0x00000023u,
0x00000130u, 0x00050048u, 0x00000035u, 0x00000009u, 0x00000023u, 0x00000230u, 0x00050048u, 0x00000035u,
0x0000000au, 0x00000023u, 0x00000830u, 0x00040047u, 0x00000037u, 0x00000021u, 0x00000000u, 0x00040047u,
0x00000037u, 0x00000022u, 0x00000000u, 0x00040047u, 0x00000050u, 0x0000000bu, 0x00000028u, 0x00030047u,
0x0000005eu, 0x00000000u, 0x00040047u, 0x0000005eu, 0x0000000bu, 0x00000024u, 0x00030047u, 0x0000005fu,
0x00000000u, 0x00030047u, 0x00000064u, 0x00000000u, 0x00040047u, 0x00000064u, 0x0000000bu, 0x00000029u,
0x00030047u, 0x00000065u, 0x00000000u, 0x00030047u, 0x00000066u, 0x00000000u, 0x00030047u, 0x00000087u,
0x00000000u, 0x00030047u, 0x0000008bu, 0x00000000u, 0x00030047u, 0x0000008cu, 0x00000000u, 0x00030047u,
0x0000008du, 0x00000000u, 0x00030047u, 0x000000c0u, 0x00000000u, 0x00030047u, 0x000000c1u, 0x00000000u,
0x00030047u, 0x000000c2u, 0x00000000u, 0x00030047u, 0x00000107u, 0x00000000u, 0x00030047u, 0x00000108u,
0x00000000u, 0x00030047u, 0x00000109u, 0x00000000u, 0x00030047u, 0x0000012fu, 0x00000000u, 0x00030047u,
0x00000132u, 0x00000000u, 0x00040047u, 0x00000144u, 0x0000000bu, 0x00000019u, 0x00020013u, 0x00000002u,
0x00030021u, 0x00000003u, 0x00000002u, 0x00040015u, 0x00000006u, 0x00000020u, 0x00000000u, 0x00040020u,
0x00000007u, 0x00000007u, 0x00000006u, 0x00040020u, 0x00000009u, 0x00000001u, 0x00000006u, 0x0004003bu,
0x00000009u, 0x0000000au, 0x00000001u, 0x0004002bu, 0x00000006u, 0x0000000du, 0x00000000u, 0x00020014u,
0x0000000eu, 0x00040020u, 0x00000012u, 0x00000004u, 0x00000006u, 0x0004003bu, 0x00000012u, 0x00000013u,
0x00000004u, 0x0004003bu, 0x00000012u, 0x00000014u, 0x00000004u, 0x0004003bu, 0x00000012u, 0x00000015u,
0x00000004u, 0x0004002bu, 0x00000006u, 0x00000017u, 0x00000020u, 0x0004001cu, 0x0000001bu, 0x00000006u,
0x00000017u, 0x00040020u, 0x0000001cu, 0x00000004u, 0x0000001bu, 0x0004003bu, 0x0000001cu, 0x0000001du,
0x00000004u, 0x0004003bu, 0x0000001cu, 0x00000020u, 0x00000004u, 0x0004002bu, 0x00000006u, 0x00000023u,
0x00000001u, 0x0004002bu, 0x00000006u, 0x00000024u, 0x00000108u, 0x0004002bu, 0x00000006u, 0x00000025u,
0x00000002u, 0x0004003bu, 0x00000009u, 0x0000002au, 0x00000001u, 0x00040017u, 0x0000002cu, 0x00000006u,
0x00000004u, 0x0004001cu, 0x0000002du, 0x00000006u, 0x00000017u, 0x0004001cu, 0x0000002eu, 0x00000006u,
0x00000017u, 0x00030016u, 0x0000002fu, 0x00000020u, 0x00040017u, 0x00000030u, 0x0000002fu, 0x00000002u,
0x0004001cu, 0x00000031u, 0x00000030u, 0x00000017u, 0x0004001cu, 0x00000032u, 0x00000030u, 0x00000017u,
0x0004002bu, 0x00000006u, 0x00000033u, 0x00000006u, 0x0004001cu, 0x00000034u, 0x00000032u, 0x00000033u,
0x000d001eu, 0x00000035u, 0x00000006u, 0x00000006u, 0x00000006u, 0x00000006u, 0x00000006u, 0x0000002cu,
0x0000002du, 0x0000002eu, 0x00000031u, 0x00000034u, 0x00000030u, 0x00040020u, 0x00000036u, 0x0000000cu,
0x00000035u, 0x0004003bu, 0x00000036u, 0x00000037u, 0x0000000cu, 0x00040015u, 0x00000038u, 0x00000020u,
0x00000001u, 0x0004002bu, 0x00000038u, 0x00000039u, 0x00000002u, 0x00040020u, 0x0000003bu, 0x0000000cu,
0x00000006u, 0x0004003bu, 0x00000009u, 0x00000050u, 0x00000001u, 0x0004002bu, 0x00000006u, 0x0000005cu,
0x00000004u, 0x0004003bu, 0x00000009u, 0x0000005eu, 0x00000001u, 0x0004003bu, 0x00000009u, 0x00000064u,
0x00000001u, 0x0004002bu, 0x00000006u, 0x0000006bu, 0x00000008u, 0x00040020u, 0x0000006du, 0x00000007u,
0x0000000eu, 0x0004002bu, 0x00000038u, 0x00000099u, 0x00000004u, 0x0004002bu, 0x00000038u, 0x000000a0u,
0x00000006u, 0x0004002bu, 0x00000038u, 0x000000a6u, 0x00000007u, 0x00040020u, 0x000000b6u, 0x00000007u,
0x00000030u, 0x0004002bu, 0x0000002fu, 0x000000bbu, 0x00000000u, 0x0004002bu, 0x00000006u, 0x000000beu,
0x00000003u, 0x0004001cu, 0x000000c6u, 0x00000030u, 0x00000017u, 0x00040020u, 0x000000c7u, 0x00000004u,
0x000000c6u, 0x0004003bu, 0x000000c7u, 0x000000c8u, 0x00000004u, 0x00040020u, 0x000000cbu, 0x00000004u,
0x00000030u, 0x0004002bu, 0x00000038u, 0x000000d2u, 0x00000008u, 0x00040020u, 0x000000d7u, 0x0000000cu,
0x00000030u, 0x0004002bu, 0x00000038u, 0x000000eau, 0x00000003u, 0x0004002bu, 0x00000038u, 0x00000115u,
0x00000009u, 0x0004002bu, 0x00000038u, 0x0000011du, 0x00000001u, 0x0004002bu, 0x00000006u, 0x00000120u,
0x000001ffu, 0x0004002bu, 0x00000038u, 0x00000124u, 0x00000000u, 0x0004002bu, 0x0000002fu, 0x00000126u,
0x44000000u, 0x0004002bu, 0x00000038u, 0x0000012eu, 0x00000005u, 0x00040020u, 0x00000134u, 0x0000000cu,
0x0000002cu, 0x0004002bu, 0x00000038u, 0x00000136u, 0x0000000au, 0x0004002bu, 0x00000006u, 0x00000140u,
0x50323033u, 0x00040017u, 0x00000142u, 0x00000006u, 0x00000003u, 0x0004002bu, 0x00000006u, 0x00000143u,
0x00000010u, 0x0006002cu, 0x00000142u, 0x00000144u, 0x00000017u, 0x00000143u, 0x00000023u, 0x00050036u,
0x00000002u, 0x00000004u, 0x00000000u, 0x00000003u, 0x000200f8u, 0x00000005u, 0x0004003bu, 0x00000007u,
0x00000008u, 0x00000007u, 0x0004003bu, 0x00000007u, 0x0000003du, 0x00000007u, 0x0004003bu, 0x0000006du,
0x0000006eu, 0x00000007u, 0x0004003bu, 0x0000006du, 0x00000074u, 0x00000007u, 0x0004003bu, 0x0000006du,
0x000000acu, 0x00000007u, 0x0004003bu, 0x000000b6u, 0x000000b7u, 0x00000007u, 0x0004003bu, 0x00000007u,
0x000000d9u, 0x00000007u, 0x0004003bu, 0x00000007u, 0x000000edu, 0x00000007u, 0x0004003du, 0x00000006u,
0x0000000bu, 0x0000000au, 0x0003003eu, 0x00000008u, 0x0000000bu, 0x0004003du, 0x00000006u, 0x0000000cu,
0x00000008u, 0x000500aau, 0x0000000eu, 0x0000000fu, 0x0000000cu, 0x0000000du, 0x000300f7u, 0x00000011u,
0x00000000u, 0x000400fau, 0x0000000fu, 0x00000010u, 0x00000011u, 0x000200f8u, 0x00000010u, 0x0003003eu,
0x00000013u, 0x0000000du, 0x0003003eu, 0x00000014u, 0x0000000du, 0x0003003eu, 0x00000015u, 0x0000000du,
0x000200f9u, 0x00000011u, 0x000200f8u, 0x00000011u, 0x0004003du, 0x00000006u, 0x00000016u, 0x00000008u,
0x000500b0u, 0x0000000eu, 0x00000018u, 0x00000016u, 0x00000017u, 0x000300f7u, 0x0000001au, 0x00000000u,
0x000400fau, 0x00000018u, 0x00000019u, 0x0000001au, 0x000200f8u, 0x00000019u, 0x0004003du, 0x00000006u,
0x0000001eu, 0x00000008u, 0x00050041u, 0x00000012u, 0x0000001fu, 0x0000001du, 0x0000001eu, 0x0003003eu,
0x0000001fu, 0x0000000du, 0x0004003du, 0x00000006u, 0x00000021u, 0x00000008u, 0x00050041u, 0x00000012u,
0x00000022u, 0x00000020u, 0x00000021u, 0x0003003eu, 0x00000022u, 0x0000000du, 0x000200f9u, 0x0000001au,
0x000200f8u, 0x0000001au, 0x000300e1u, 0x00000023u, 0x00000024u, 0x000400e0u, 0x00000025u, 0x00000025u,
0x00000024u, 0x0004003du, 0x00000006u, 0x00000026u, 0x00000008u, 0x000500aau, 0x0000000eu, 0x00000027u,
0x00000026u, 0x0000000du, 0x000300f7u, 0x00000029u, 0x00000000u, 0x000400fau, 0x00000027u, 0x00000028u,
0x00000029u, 0x000200f8u, 0x00000028u, 0x0004003du, 0x00000006u, 0x0000002bu, 0x0000002au, 0x0003003eu,
0x00000013u, 0x0000002bu, 0x0004003du, 0x00000006u, 0x0000003au, 0x0000002au, 0x00050041u, 0x0000003bu,
0x0000003cu, 0x00000037u, 0x00000039u, 0x0003003eu, 0x0000003cu, 0x0000003au, 0x000200f9u, 0x00000029u,
0x000200f8u, 0x00000029u, 0x000400e0u, 0x00000025u, 0x00000025u, 0x00000024u, 0x0004003du, 0x00000006u,
0x0000003eu, 0x00000013u, 0x0003003eu, 0x0000003du, 0x0000003eu, 0x0004003du, 0x00000006u, 0x0000003fu,
0x0000002au, 0x0004003du, 0x00000006u, 0x00000040u, 0x0000003du, 0x000500abu, 0x0000000eu, 0x00000041u,
0x0000003fu, 0x00000040u, 0x000300f7u, 0x00000043u, 0x00000000u, 0x000400fau, 0x00000041u, 0x00000042u,
0x00000043u, 0x000200f8u, 0x00000042u, 0x000700f1u, 0x00000006u, 0x00000044u, 0x00000014u, 0x00000023u,
0x0000000du, 0x00000023u, 0x000200f9u, 0x00000043u, 0x000200f8u, 0x00000043u, 0x0004003du, 0x00000006u,
0x00000045u, 0x0000002au, 0x000500b0u, 0x0000000eu, 0x00000046u, 0x00000045u, 0x00000025u, 0x000400a8u,
0x0000000eu, 0x00000047u, 0x00000046u, 0x000300f7u, 0x00000049u, 0x00000000u, 0x000400fau, 0x00000047u,
0x00000048u, 0x00000049u, 0x000200f8u, 0x00000048u, 0x0004003du, 0x00000006u, 0x0000004au, 0x0000002au,
0x000500acu, 0x0000000eu, 0x0000004bu, 0x0000004au, 0x00000017u, 0x000200f9u, 0x00000049u, 0x000200f8u,
0x00000049u, 0x000700f5u, 0x0000000eu, 0x0000004cu, 0x00000046u, 0x00000043u, 0x0000004bu, 0x00000048u,
0x000300f7u, 0x0000004eu, 0x00000000u, 0x000400fau, 0x0000004cu, 0x0000004du, 0x0000004eu, 0x000200f8u,
0x0000004du, 0x000700f1u, 0x00000006u, 0x0000004fu, 0x00000014u, 0x00000023u, 0x0000000du, 0x00000025u,
0x000200f9u, 0x0000004eu, 0x000200f8u, 0x0000004eu, 0x0004003du, 0x00000006u, 0x00000051u, 0x00000050u,
0x0004003du, 0x00000006u, 0x00000052u, 0x0000003du, 0x000500aeu, 0x0000000eu, 0x00000053u, 0x00000051u,
0x00000052u, 0x000400a8u, 0x0000000eu, 0x00000054u, 0x00000053u, 0x000300f7u, 0x00000056u, 0x00000000u,
0x000400fau, 0x00000054u, 0x00000055u, 0x00000056u, 0x000200f8u, 0x00000055u, 0x0004003du, 0x00000006u,
0x00000057u, 0x00000050u, 0x000500aeu, 0x0000000eu, 0x00000058u, 0x00000057u, 0x00000017u, 0x000200f9u,
0x00000056u, 0x000200f8u, 0x00000056u, 0x000700f5u, 0x0000000eu, 0x00000059u, 0x00000053u, 0x0000004eu,
0x00000058u, 0x00000055u, 0x000300f7u, 0x0000005bu, 0x00000000u, 0x000400fau, 0x00000059u, 0x0000005au,
0x0000005bu, 0x000200f8u, 0x0000005au, 0x000700f1u, 0x00000006u, 0x0000005du, 0x00000014u, 0x00000023u,
0x0000000du, 0x0000005cu, 0x000200f9u, 0x0000005bu, 0x000200f8u, 0x0000005bu, 0x0004003du, 0x00000006u,
0x0000005fu, 0x0000005eu, 0x000500aau, 0x0000000eu, 0x00000060u, 0x0000005fu, 0x0000000du, 0x000400a8u,
0x0000000eu, 0x00000061u, 0x00000060u, 0x000300f7u, 0x00000063u, 0x00000000u, 0x000400fau, 0x00000061u,
0x00000062u, 0x00000063u, 0x000200f8u, 0x00000062u, 0x0004003du, 0x00000006u, 0x00000065u, 0x00000064u,
0x0004003du, 0x00000006u, 0x00000066u, 0x0000005eu, 0x000500aeu, 0x0000000eu, 0x00000067u, 0x00000065u,
0x00000066u, 0x000200f9u, 0x00000063u, 0x000200f8u, 0x00000063u, 0x000700f5u, 0x0000000eu, 0x00000068u,
0x00000060u, 0x0000005bu, 0x00000067u, 0x00000062u, 0x000300f7u, 0x0000006au, 0x00000000u, 0x000400fau,
0x00000068u, 0x00000069u, 0x0000006au, 0x000200f8u, 0x00000069u, 0x000700f1u, 0x00000006u, 0x0000006cu,
0x00000014u, 0x00000023u, 0x0000000du, 0x0000006bu, 0x000200f9u, 0x0000006au, 0x000200f8u, 0x0000006au,
0x0004003du, 0x00000006u, 0x0000006fu, 0x0000003du, 0x000500aeu, 0x0000000eu, 0x00000070u, 0x0000006fu,
0x00000025u, 0x0004003du, 0x00000006u, 0x00000071u, 0x0000003du, 0x000500b2u, 0x0000000eu, 0x00000072u,
0x00000071u, 0x00000017u, 0x000500a7u, 0x0000000eu, 0x00000073u, 0x00000070u, 0x00000072u, 0x0003003eu,
0x0000006eu, 0x00000073u, 0x0004003du, 0x0000000eu, 0x00000075u, 0x0000006eu, 0x000300f7u, 0x00000077u,
0x00000000u, 0x000400fau, 0x00000075u, 0x00000076u, 0x00000077u, 0x000200f8u, 0x00000076u, 0x0004003du,
0x00000006u, 0x00000078u, 0x00000050u, 0x0004003du, 0x00000006u, 0x00000079u, 0x0000003du, 0x000500b0u,
0x0000000eu, 0x0000007au, 0x00000078u, 0x00000079u, 0x000200f9u, 0x00000077u, 0x000200f8u, 0x00000077u,
0x000700f5u, 0x0000000eu, 0x0000007bu, 0x00000075u, 0x0000006au, 0x0000007au, 0x00000076u, 0x0003003eu,
0x00000074u, 0x0000007bu, 0x0004003du, 0x0000000eu, 0x0000007cu, 0x00000074u, 0x000300f7u, 0x0000007eu,
0x00000000u, 0x000400fau, 0x0000007cu, 0x0000007du, 0x0000007eu, 0x000200f8u, 0x0000007du, 0x0004003du,
0x00000006u, 0x0000007fu, 0x00000050u, 0x000500c4u, 0x00000006u, 0x00000080u, 0x00000023u, 0x0000007fu,
0x000700f1u, 0x00000006u, 0x00000081u, 0x00000015u, 0x00000023u, 0x0000000du, 0x00000080u, 0x0004003du,
0x00000006u, 0x00000082u, 0x00000050u, 0x00050041u, 0x00000012u, 0x00000083u, 0x00000020u, 0x00000082u,
0x0004003du, 0x00000006u, 0x00000084u, 0x00000008u, 0x00050080u, 0x00000006u, 0x00000085u, 0x00000084u,
0x00000023u, 0x000700eau, 0x00000006u, 0x00000086u, 0x00000083u, 0x00000023u, 0x0000000du, 0x00000085u,
0x0004003du, 0x00000006u, 0x00000087u, 0x0000005eu, 0x000500abu, 0x0000000eu, 0x00000088u, 0x00000087u,
0x0000000du, 0x000300f7u, 0x0000008au, 0x00000000u, 0x000400fau, 0x00000088u, 0x00000089u, 0x0000008au,
0x000200f8u, 0x00000089u, 0x0004003du, 0x00000006u, 0x0000008bu, 0x00000064u, 0x0004003du, 0x00000006u,
0x0000008cu, 0x0000005eu, 0x00050082u, 0x00000006u, 0x0000008du, 0x0000008cu, 0x00000023u, 0x000500aau,
0x0000000eu, 0x0000008eu, 0x0000008bu, 0x0000008du, 0x000200f9u, 0x0000008au, 0x000200f8u, 0x0000008au,
0x000700f5u, 0x0000000eu, 0x0000008fu, 0x00000088u, 0x0000007du, 0x0000008eu, 0x00000089u, 0x000300f7u,
0x00000091u, 0x00000000u, 0x000400fau, 0x0000008fu, 0x00000090u, 0x00000091u, 0x000200f8u, 0x00000090u,
0x0004003du, 0x00000006u, 0x00000092u, 0x00000050u, 0x00050041u, 0x00000012u, 0x00000093u, 0x0000001du,
0x00000092u, 0x000700eau, 0x00000006u, 0x00000094u, 0x00000093u, 0x00000023u, 0x0000000du, 0x00000023u,
0x000200f9u, 0x00000091u, 0x000200f8u, 0x00000091u, 0x000200f9u, 0x0000007eu, 0x000200f8u, 0x0000007eu,
0x000300e1u, 0x00000023u, 0x00000024u, 0x000400e0u, 0x00000025u, 0x00000025u, 0x00000024u, 0x0004003du,
0x00000006u, 0x00000095u, 0x00000008u, 0x000500aau, 0x0000000eu, 0x00000096u, 0x00000095u, 0x0000000du,
0x000300f7u, 0x00000098u, 0x00000000u, 0x000400fau, 0x00000096u, 0x00000097u, 0x00000098u, 0x000200f8u,
0x00000097u, 0x0004003du, 0x00000006u, 0x0000009au, 0x00000015u, 0x00050041u, 0x0000003bu, 0x0000009bu,
0x00000037u, 0x00000099u, 0x0003003eu, 0x0000009bu, 0x0000009au, 0x000200f9u, 0x00000098u, 0x000200f8u,
0x00000098u, 0x0004003du, 0x00000006u, 0x0000009cu, 0x00000008u, 0x000500b0u, 0x0000000eu, 0x0000009du,
0x0000009cu, 0x00000017u, 0x000300f7u, 0x0000009fu, 0x00000000u, 0x000400fau, 0x0000009du, 0x0000009eu,
0x0000009fu, 0x000200f8u, 0x0000009eu, 0x0004003du, 0x00000006u, 0x000000a1u, 0x00000008u, 0x0004003du,
0x00000006u, 0x000000a2u, 0x00000008u, 0x00050041u, 0x00000012u, 0x000000a3u, 0x0000001du, 0x000000a2u,
0x0004003du, 0x00000006u, 0x000000a4u, 0x000000a3u, 0x00060041u, 0x0000003bu, 0x000000a5u, 0x00000037u,
0x000000a0u, 0x000000a1u, 0x0003003eu, 0x000000a5u, 0x000000a4u, 0x0004003du, 0x00000006u, 0x000000a7u,
0x00000008u, 0x0004003du, 0x00000006u, 0x000000a8u, 0x00000008u, 0x00050041u, 0x00000012u, 0x000000a9u,
0x00000020u, 0x000000a8u, 0x0004003du, 0x00000006u, 0x000000aau, 0x000000a9u, 0x00060041u, 0x0000003bu,
0x000000abu, 0x00000037u, 0x000000a6u, 0x000000a7u, 0x0003003eu, 0x000000abu, 0x000000aau, 0x000200f9u,
0x0000009fu, 0x000200f8u, 0x0000009fu, 0x0004003du, 0x0000000eu, 0x000000adu, 0x0000006eu, 0x000300f7u,
0x000000afu, 0x00000000u, 0x000400fau, 0x000000adu, 0x000000aeu, 0x000000afu, 0x000200f8u, 0x000000aeu,
0x0004003du, 0x00000006u, 0x000000b0u, 0x00000014u, 0x000500aau, 0x0000000eu, 0x000000b1u, 0x000000b0u,
0x0000000du, 0x000200f9u, 0x000000afu, 0x000200f8u, 0x000000afu, 0x000700f5u, 0x0000000eu, 0x000000b2u,
0x000000adu, 0x0000009fu, 0x000000b1u, 0x000000aeu, 0x0003003eu, 0x000000acu, 0x000000b2u, 0x0004003du,
0x0000000eu, 0x000000b3u, 0x000000acu, 0x000300f7u, 0x000000b5u, 0x00000000u, 0x000400fau, 0x000000b3u,
0x000000b4u, 0x000000b5u, 0x000200f8u, 0x000000b4u, 0x0004003du, 0x00000006u, 0x000000b8u, 0x0000000au,
0x00050080u, 0x00000006u, 0x000000b9u, 0x000000b8u, 0x00000023u, 0x00040070u, 0x0000002fu, 0x000000bau,
0x000000b9u, 0x00050050u, 0x00000030u, 0x000000bcu, 0x000000bau, 0x000000bbu, 0x0003003eu, 0x000000b7u,
0x000000bcu, 0x0004003du, 0x00000030u, 0x000000bdu, 0x000000b7u, 0x0006015eu, 0x00000030u, 0x000000bfu,
0x000000beu, 0x00000001u, 0x000000bdu, 0x0003003eu, 0x000000b7u, 0x000000bfu, 0x0004003du, 0x00000006u,
0x000000c0u, 0x00000064u, 0x0004003du, 0x00000006u, 0x000000c1u, 0x0000005eu, 0x00050082u, 0x00000006u,
0x000000c2u, 0x000000c1u, 0x00000023u, 0x000500aau, 0x0000000eu, 0x000000c3u, 0x000000c0u, 0x000000c2u,
0x000300f7u, 0x000000c5u, 0x00000000u, 0x000400fau, 0x000000c3u, 0x000000c4u, 0x000000c5u, 0x000200f8u,
0x000000c4u, 0x0004003du, 0x00000006u, 0x000000c9u, 0x00000050u, 0x0004003du, 0x00000030u, 0x000000cau,
0x000000b7u, 0x00050041u, 0x000000cbu, 0x000000ccu, 0x000000c8u, 0x000000c9u, 0x0003003eu, 0x000000ccu,
0x000000cau, 0x000200f9u, 0x000000c5u, 0x000200f8u, 0x000000c5u, 0x000400e0u, 0x00000025u, 0x00000025u,
0x00000024u, 0x0004003du, 0x00000006u, 0x000000cdu, 0x0000000au, 0x0004003du, 0x00000006u, 0x000000ceu,
0x0000002au, 0x000500b0u, 0x0000000eu, 0x000000cfu, 0x000000cdu, 0x000000ceu, 0x000300f7u, 0x000000d1u,
0x00000000u, 0x000400fau, 0x000000cfu, 0x000000d0u, 0x000000d1u, 0x000200f8u, 0x000000d0u, 0x0004003du,
0x00000006u, 0x000000d3u, 0x0000000au, 0x0004003du, 0x00000006u, 0x000000d4u, 0x0000000au, 0x00050041u,
0x000000cbu, 0x000000d5u, 0x000000c8u, 0x000000d4u, 0x0004003du, 0x00000030u, 0x000000d6u, 0x000000d5u,
0x00060041u, 0x000000d7u, 0x000000d8u, 0x00000037u, 0x000000d2u, 0x000000d3u, 0x0003003eu, 0x000000d8u,
0x000000d6u, 0x000200f9u, 0x000000d1u, 0x000200f8u, 0x000000d1u, 0x000400e0u, 0x00000025u, 0x00000025u,
0x00000024u, 0x0004003du, 0x00000006u, 0x000000dau, 0x0000002au, 0x0006000cu, 0x00000038u, 0x000000dbu,
0x00000001u, 0x0000004bu, 0x000000dau, 0x0004007cu, 0x00000006u, 0x000000dcu, 0x000000dbu, 0x0003003eu,
0x000000d9u, 0x000000dcu, 0x0004003du, 0x00000006u, 0x000000ddu, 0x0000002au, 0x0004003du, 0x00000006u,
0x000000deu, 0x000000d9u, 0x00050082u, 0x00000006u, 0x000000dfu, 0x000000deu, 0x00000023u, 0x000500c4u,
0x00000006u, 0x000000e0u, 0x00000023u, 0x000000dfu, 0x00050082u, 0x00000006u, 0x000000e1u, 0x000000ddu,
0x000000e0u, 0x000500acu, 0x0000000eu, 0x000000e2u, 0x000000e1u, 0x0000000du, 0x000600a9u, 0x00000006u,
0x000000e3u, 0x000000e2u, 0x00000023u, 0x0000000du, 0x0004003du, 0x00000006u, 0x000000e4u, 0x000000d9u,
0x00050080u, 0x00000006u, 0x000000e5u, 0x000000e4u, 0x000000e3u, 0x0003003eu, 0x000000d9u, 0x000000e5u,
0x0004003du, 0x00000006u, 0x000000e6u, 0x0000000au, 0x000500aau, 0x0000000eu, 0x000000e7u, 0x000000e6u,
0x0000000du, 0x000300f7u, 0x000000e9u, 0x00000000u, 0x000400fau, 0x000000e7u, 0x000000e8u, 0x000000e9u,
0x000200f8u, 0x000000e8u, 0x0004003du, 0x00000006u, 0x000000ebu, 0x000000d9u, 0x00050041u, 0x0000003bu,
0x000000ecu, 0x00000037u, 0x000000eau, 0x0003003eu, 0x000000ecu, 0x000000ebu, 0x000200f9u, 0x000000e9u,
0x000200f8u, 0x000000e9u, 0x0003003eu, 0x000000edu, 0x0000000du, 0x000200f9u, 0x000000eeu, 0x000200f8u,
0x000000eeu, 0x000400f6u, 0x000000f0u, 0x000000f1u, 0x00000000u, 0x000200f9u, 0x000000f2u, 0x000200f8u,
0x000000f2u, 0x0004003du, 0x00000006u, 0x000000f3u, 0x000000edu, 0x0004003du, 0x00000006u, 0x000000f4u,
0x000000d9u, 0x000500b0u, 0x0000000eu, 0x000000f5u, 0x000000f3u, 0x000000f4u, 0x000400fau, 0x000000f5u,
0x000000efu, 0x000000f0u, 0x000200f8u, 0x000000efu, 0x0004003du, 0x00000006u, 0x000000f6u, 0x00000050u,
0x0004003du, 0x00000006u, 0x000000f7u, 0x000000edu, 0x000500c4u, 0x00000006u, 0x000000f8u, 0x00000023u,
0x000000f7u, 0x000500c7u, 0x00000006u, 0x000000f9u, 0x000000f6u, 0x000000f8u, 0x000500acu, 0x0000000eu,
0x000000fau, 0x000000f9u, 0x0000000du, 0x000300f7u, 0x000000fcu, 0x00000000u, 0x000400fau, 0x000000fau,
0x000000fbu, 0x000000fcu, 0x000200f8u, 0x000000fbu, 0x0004003du, 0x00000006u, 0x000000fdu, 0x00000050u,
0x0004003du, 0x00000006u, 0x000000feu, 0x000000edu, 0x000500c2u, 0x00000006u, 0x000000ffu, 0x000000fdu,
0x000000feu, 0x0004003du, 0x00000006u, 0x00000100u, 0x000000edu, 0x000500c4u, 0x00000006u, 0x00000101u,
0x000000ffu, 0x00000100u, 0x00050082u, 0x00000006u, 0x00000102u, 0x00000101u, 0x00000023u, 0x00050041u,
0x000000cbu, 0x00000103u, 0x000000c8u, 0x00000102u, 0x0004003du, 0x00000030u, 0x00000104u, 0x00000103u,
0x0004003du, 0x00000030u, 0x00000105u, 0x000000b7u, 0x00050081u, 0x00000030u, 0x00000106u, 0x00000105u,
0x00000104u, 0x0003003eu, 0x000000b7u, 0x00000106u, 0x0004003du, 0x00000006u, 0x00000107u, 0x00000064u,
0x0004003du, 0x00000006u, 0x00000108u, 0x0000005eu, 0x00050082u, 0x00000006u, 0x00000109u, 0x00000108u,
0x00000023u, 0x000500aau, 0x0000000eu, 0x0000010au, 0x00000107u, 0x00000109u, 0x000300f7u, 0x0000010cu,
0x00000000u, 0x000400fau, 0x0000010au, 0x0000010bu, 0x0000010cu, 0x000200f8u, 0x0000010bu, 0x0004003du,
0x00000006u, 0x0000010du, 0x00000050u, 0x0004003du, 0x00000030u, 0x0000010eu, 0x000000b7u, 0x00050041u,
0x000000cbu, 0x0000010fu, 0x000000c8u, 0x0000010du, 0x0003003eu, 0x0000010fu, 0x0000010eu, 0x000200f9u,
0x0000010cu, 0x000200f8u, 0x0000010cu, 0x000200f9u, 0x000000fcu, 0x000200f8u, 0x000000fcu, 0x000400e0u,
0x00000025u, 0x00000025u, 0x00000024u, 0x0004003du, 0x00000006u, 0x00000110u, 0x0000000au, 0x0004003du,
0x00000006u, 0x00000111u, 0x0000002au, 0x000500b0u, 0x0000000eu, 0x00000112u, 0x00000110u, 0x00000111u,
0x000300f7u, 0x00000114u, 0x00000000u, 0x000400fau, 0x00000112u, 0x00000113u, 0x00000114u, 0x000200f8u,
0x00000113u, 0x0004003du, 0x00000006u, 0x00000116u, 0x000000edu, 0x0004003du, 0x00000006u, 0x00000117u,
0x0000000au, 0x0004003du, 0x00000006u, 0x00000118u, 0x0000000au, 0x00050041u, 0x000000cbu, 0x00000119u,
0x000000c8u, 0x00000118u, 0x0004003du, 0x00000030u, 0x0000011au, 0x00000119u, 0x00070041u, 0x000000d7u,
0x0000011bu, 0x00000037u, 0x00000115u, 0x00000116u, 0x00000117u, 0x0003003eu, 0x0000011bu, 0x0000011au,
0x000200f9u, 0x00000114u, 0x000200f8u, 0x00000114u, 0x000400e0u, 0x00000025u, 0x00000025u, 0x00000024u,
0x000200f9u, 0x000000f1u, 0x000200f8u, 0x000000f1u, 0x0004003du, 0x00000006u, 0x0000011cu, 0x000000edu,
0x00050080u, 0x00000006u, 0x0000011eu, 0x0000011cu, 0x0000011du, 0x0003003eu, 0x000000edu, 0x0000011eu,
0x000200f9u, 0x000000eeu, 0x000200f8u, 0x000000f0u, 0x0004003du, 0x00000006u, 0x0000011fu, 0x0000000au,
0x000500aau, 0x0000000eu, 0x00000121u, 0x0000011fu, 0x00000120u, 0x000300f7u, 0x00000123u, 0x00000000u,
0x000400fau, 0x00000121u, 0x00000122u, 0x00000123u, 0x000200f8u, 0x00000122u, 0x0004003du, 0x00000030u,
0x00000125u, 0x000000b7u, 0x00050050u, 0x00000030u, 0x00000127u, 0x00000126u, 0x00000126u, 0x00050088u,
0x00000030u, 0x00000128u, 0x00000125u, 0x00000127u, 0x00050041u, 0x000000cbu, 0x00000129u, 0x000000c8u,
0x00000124u, 0x0003003eu, 0x00000129u, 0x00000128u, 0x000200f9u, 0x00000123u, 0x000200f8u, 0x00000123u,
0x000400e0u, 0x00000025u, 0x00000025u, 0x00000024u, 0x0004003du, 0x00000006u, 0x0000012au, 0x0000000au,
0x000500aau, 0x0000000eu, 0x0000012bu, 0x0000012au, 0x00000120u, 0x000300f7u, 0x0000012du, 0x00000000u,
0x000400fau, 0x0000012bu, 0x0000012cu, 0x0000012du, 0x000200f8u, 0x0000012cu, 0x0004003du, 0x00000006u,
0x0000012fu, 0x0000005eu, 0x0004003du, 0x00000006u, 0x00000130u, 0x0000002au, 0x0004003du, 0x00000006u,
0x00000131u, 0x00000050u, 0x0004003du, 0x00000006u, 0x00000132u, 0x00000064u, 0x00070050u, 0x0000002cu,
0x00000133u, 0x0000012fu, 0x00000130u, 0x00000131u, 0x00000132u, 0x00050041u, 0x00000134u, 0x00000135u,
0x00000037u, 0x0000012eu, 0x0003003eu, 0x00000135u, 0x00000133u, 0x00050041u, 0x000000cbu, 0x00000137u,
0x000000c8u, 0x00000124u, 0x0004003du, 0x00000030u, 0x00000138u, 0x00000137u, 0x00050041u, 0x000000d7u,
0x00000139u, 0x00000037u, 0x00000136u, 0x0003003eu, 0x00000139u, 0x00000138u, 0x000200f9u, 0x0000012du,
0x000200f8u, 0x0000012du, 0x000200f9u, 0x000000b5u, 0x000200f8u, 0x000000b5u, 0x000400e0u, 0x00000025u,
0x00000025u, 0x00000024u, 0x0004003du, 0x00000006u, 0x0000013au, 0x00000008u, 0x000500aau, 0x0000000eu,
0x0000013bu, 0x0000013au, 0x0000000du, 0x000300f7u, 0x0000013du, 0x00000000u, 0x000400fau, 0x0000013bu,
0x0000013cu, 0x0000013du, 0x000200f8u, 0x0000013cu, 0x0004003du, 0x00000006u, 0x0000013eu, 0x00000014u,
0x00050041u, 0x0000003bu, 0x0000013fu, 0x00000037u, 0x0000011du, 0x0003003eu, 0x0000013fu, 0x0000013eu,
0x00050041u, 0x0000003bu, 0x00000141u, 0x00000037u, 0x00000124u, 0x0003003eu, 0x00000141u, 0x00000140u,
0x000200f9u, 0x0000013du, 0x000200f8u, 0x0000013du, 0x000100fdu, 0x00010038u,
};
inline constexpr std::size_t kDriverPostIterationRPWitnessSpvWordCount =
sizeof(kDriverPostIterationRPWitnessSpv) / sizeof(kDriverPostIterationRPWitnessSpv[0]);
} // namespace MobileGL::MG_Util::SelfTest
@@ -25,6 +25,9 @@
#include "SpirvPasses/SplitArrayVertexInputsPass.h" #include "SpirvPasses/SplitArrayVertexInputsPass.h"
#include "SpirvPasses/RebaseInstanceIndexPass.h" #include "SpirvPasses/RebaseInstanceIndexPass.h"
#include "SpirvPasses/ZeroBaseVertexPass.h" #include "SpirvPasses/ZeroBaseVertexPass.h"
#include "SpirvPasses/DeriveNumSubgroupsPass.h"
#include "SpirvPasses/EmulateSubgroupsPass.h"
#include "SpirvPasses/FixIterationRPSubgroupScratchPass.h"
#include "SpirvPasses/NormalizeRectCoordinatesPass.h" #include "SpirvPasses/NormalizeRectCoordinatesPass.h"
#include "SpirvPasses/Lower1DArrayImagesPass.h" #include "SpirvPasses/Lower1DArrayImagesPass.h"
#include "SpirvPasses/BakeImageFormatsPass.h" #include "SpirvPasses/BakeImageFormatsPass.h"
@@ -883,6 +886,44 @@ namespace MobileGL {
return RunOptimizerChecked("ZeroBaseVertexForVulkan", optimizer, inputBinary, outputBinary, true, enableSpirvValidation); return RunOptimizerChecked("ZeroBaseVertexForVulkan", optimizer, inputBinary, outputBinary, true, enableSpirvValidation);
} }
bool ShaderCompiler::DeriveNumSubgroupsForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
const bool enableSpirvValidation) {
using namespace spvtools;
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(DeriveNumSubgroupsPass::CreateDeriveNumSubgroupsPass());
return RunOptimizerChecked("DeriveNumSubgroupsForVulkan", optimizer, inputBinary,
outputBinary, true, enableSpirvValidation);
}
bool ShaderCompiler::EmulateSubgroupsForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
const Uint32 maxWorkgroupScratchBytes,
const bool enableSpirvValidation) {
using namespace spvtools;
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(
EmulateSubgroupsPass::CreateEmulateSubgroupsPass(maxWorkgroupScratchBytes));
return RunOptimizerChecked("EmulateSubgroupsForVulkan", optimizer, inputBinary,
outputBinary, true, enableSpirvValidation);
}
bool ShaderCompiler::FixIterationRPSubgroupScratchForVulkan(
const Vector<Uint32>& inputBinary, Vector<uint32_t>& outputBinary,
const Uint32 nativeSubgroupSize, const Uint32 maxWorkgroupScratchBytes,
const bool enableSpirvValidation) {
using namespace spvtools;
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(
FixIterationRPSubgroupScratchPass::CreateFixIterationRPSubgroupScratchPass(
nativeSubgroupSize, maxWorkgroupScratchBytes));
return RunOptimizerChecked("FixIterationRPSubgroupScratchForVulkan", optimizer,
inputBinary, outputBinary, true, enableSpirvValidation);
}
bool ShaderCompiler::DecoratePositionInvariantForVulkan(const Vector<Uint32>& inputBinary, bool ShaderCompiler::DecoratePositionInvariantForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary, const bool enableSpirvValidation) { Vector<uint32_t>& outputBinary, const bool enableSpirvValidation) {
using namespace spvtools; using namespace spvtools;
@@ -145,6 +145,40 @@ namespace MobileGL {
static bool ZeroBaseVertexForVulkan(const Vector<Uint32>& inputBinary, static bool ZeroBaseVertexForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary, Vector<uint32_t>& outputBinary,
bool enableSpirvValidation = false); bool enableSpirvValidation = false);
// Replaces compute gl_NumSubgroups loads with ceil(workgroup invocations /
// gl_SubgroupSize). DirectVulkan only; this repairs drivers whose builtin
// disagrees with the subgroup IDs the same dispatch emits (Adreno reports 1
// while emitting IDs 0..7). The ceil() partition is only spec-guaranteed
// under VK_PIPELINE_SHADER_STAGE_CREATE_REQUIRE_FULL_SUBGROUPS_BIT, which
// the caller requests whenever it is legal for the workgroup shape; see
// DeriveNumSubgroupsPass.
static bool DeriveNumSubgroupsForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
bool enableSpirvValidation = false);
// Lowers every GL_KHR_shader_subgroup construct in a compute module onto a
// 32-lane virtual subgroup built from workgroup-shared memory. Last-resort
// path for devices with NO native subgroup support, opt-in via
// MOBILEGL_MAGMA_EMULATE_SUBGROUP=1; a device with native subgroup
// operations always uses them. maxWorkgroupScratchBytes bounds the shared
// scratch the lowering may add (pass the device's
// maxComputeSharedMemorySize; 0 falls back to the 16384-byte Vulkan
// minimum). See EmulateSubgroupsPass.
static bool EmulateSubgroupsForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
Uint32 maxWorkgroupScratchBytes,
bool enableSpirvValidation = false);
// Grows iterationRP's under-declared gl_SubgroupID-indexed scratch to the
// subgroup count the device actually partitions into, fingerprint-gated to
// that pack's reduction idiom; every other module - and every device whose
// width the pack already assumed - passes through byte-identical.
// maxWorkgroupScratchBytes bounds the growth (pass the device's
// maxComputeSharedMemorySize; 0 falls back to the 16384-byte Vulkan
// minimum). See FixIterationRPSubgroupScratchPass.
static bool FixIterationRPSubgroupScratchForVulkan(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary,
Uint32 nativeSubgroupSize,
Uint32 maxWorkgroupScratchBytes,
bool enableSpirvValidation = false);
// Re-declares 64-bit float vertex inputs as their 32-bit unsigned word pair // Re-declares 64-bit float vertex inputs as their 32-bit unsigned word pair
// (double -> uvec2, dvec2 -> uvec4) and bitcasts them back to double at entry, so no // (double -> uvec2, dvec2 -> uvec4) and bitcasts them back to double at entry, so no
// VK_FORMAT_R64*_SFLOAT is needed - lavapipe advertises none of them for vertex // VK_FORMAT_R64*_SFLOAT is needed - lavapipe advertises none of them for vertex
@@ -0,0 +1,257 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DeriveNumSubgroupsPass.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DeriveNumSubgroupsPass.h"
#include "spirv.hpp"
#include "source/opt/constants.h"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/util/make_unique.h"
#include <vector>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
namespace {
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
using spvtools::opt::Operand;
Instruction* FindBuiltinDefinition(IRContext* context, spv::BuiltIn builtin) {
auto* defUseMgr = context->get_def_use_mgr();
for (auto& annotation : context->annotations()) {
if (annotation.opcode() != spv::Op::OpDecorate || annotation.NumInOperands() < 3) {
continue;
}
if (static_cast<spv::Decoration>(annotation.GetSingleWordInOperand(1)) !=
spv::Decoration::BuiltIn) {
continue;
}
if (static_cast<spv::BuiltIn>(annotation.GetSingleWordInOperand(2)) != builtin) {
continue;
}
return defUseMgr->GetDef(annotation.GetSingleWordInOperand(0));
}
return nullptr;
}
bool IsInputPointerTo(IRContext* context, const Instruction* variable, uint32_t pointeeTypeId) {
if (variable == nullptr || variable->opcode() != spv::Op::OpVariable ||
variable->NumInOperands() < 1 ||
static_cast<spv::StorageClass>(variable->GetSingleWordInOperand(0)) !=
spv::StorageClass::Input) {
return false;
}
const Instruction* pointerType = context->get_def_use_mgr()->GetDef(variable->type_id());
return pointerType != nullptr && pointerType->opcode() == spv::Op::OpTypePointer &&
pointerType->NumInOperands() >= 2 &&
static_cast<spv::StorageClass>(pointerType->GetSingleWordInOperand(0)) ==
spv::StorageClass::Input &&
pointerType->GetSingleWordInOperand(1) == pointeeTypeId;
}
bool IsUnsignedInt32(IRContext* context, uint32_t typeId) {
const Instruction* type = context->get_def_use_mgr()->GetDef(typeId);
return type != nullptr && type->opcode() == spv::Op::OpTypeInt &&
type->NumInOperands() >= 2 && type->GetSingleWordInOperand(0) == 32u &&
type->GetSingleWordInOperand(1) == 0u;
}
uint32_t SynthesizeSubgroupSizeVariable(IRContext* context, uint32_t pointerTypeId) {
const uint32_t variableId = context->TakeNextId();
context->AddGlobalValue(spvtools::MakeUnique<Instruction>(
context, spv::Op::OpVariable, pointerTypeId, variableId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_STORAGE_CLASS,
{static_cast<uint32_t>(spv::StorageClass::Input)}}}));
context->AddAnnotationInst(spvtools::MakeUnique<Instruction>(
context, spv::Op::OpDecorate, 0, 0,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {variableId}},
{SPV_OPERAND_TYPE_DECORATION,
{static_cast<uint32_t>(spv::Decoration::BuiltIn)}},
{SPV_OPERAND_TYPE_LITERAL_INTEGER,
{static_cast<uint32_t>(spv::BuiltIn::SubgroupSize)}}}));
for (Instruction& entryPoint : context->module()->entry_points()) {
entryPoint.AddOperand({SPV_OPERAND_TYPE_ID, {variableId}});
}
return variableId;
}
} // namespace
spvtools::opt::Pass::Status DeriveNumSubgroupsPass::Process() {
auto* irContext = context();
auto* defUseMgr = irContext->get_def_use_mgr();
Instruction* numSubgroupsVar = FindBuiltinDefinition(irContext, spv::BuiltIn::NumSubgroups);
if (numSubgroupsVar == nullptr) {
return Status::SuccessWithoutChange;
}
std::vector<Instruction*> numSubgroupsLoads;
bool sawUnexpectedUser = false;
const uint32_t numSubgroupsVarId = numSubgroupsVar->result_id();
defUseMgr->ForEachUser(numSubgroupsVar, [&](Instruction* user) {
switch (user->opcode()) {
case spv::Op::OpLoad:
if (user->NumInOperands() >= 1 &&
user->GetSingleWordInOperand(0) == numSubgroupsVarId) {
numSubgroupsLoads.push_back(user);
} else {
sawUnexpectedUser = true;
}
return;
case spv::Op::OpDecorate:
case spv::Op::OpDecorateId:
case spv::Op::OpDecorateString:
case spv::Op::OpName:
case spv::Op::OpEntryPoint:
return;
default:
sawUnexpectedUser = true;
return;
}
});
if (sawUnexpectedUser) {
return Status::Failure;
}
if (numSubgroupsLoads.empty()) {
return Status::SuccessWithoutChange;
}
const uint32_t valueTypeId = numSubgroupsLoads.front()->type_id();
if (!IsUnsignedInt32(irContext, valueTypeId) ||
!IsInputPointerTo(irContext, numSubgroupsVar, valueTypeId)) {
return Status::Failure;
}
for (const Instruction* load : numSubgroupsLoads) {
if (load->type_id() != valueTypeId) {
return Status::Failure;
}
}
Instruction* workgroupSize = FindBuiltinDefinition(irContext, spv::BuiltIn::WorkgroupSize);
if (workgroupSize == nullptr ||
(workgroupSize->opcode() != spv::Op::OpConstantComposite &&
workgroupSize->opcode() != spv::Op::OpSpecConstantComposite)) {
return Status::Failure;
}
const Instruction* workgroupSizeType = defUseMgr->GetDef(workgroupSize->type_id());
if (workgroupSizeType == nullptr || workgroupSizeType->opcode() != spv::Op::OpTypeVector ||
workgroupSizeType->NumInOperands() < 2 ||
workgroupSizeType->GetSingleWordInOperand(0) != valueTypeId ||
workgroupSizeType->GetSingleWordInOperand(1) != 3u) {
return Status::Failure;
}
Instruction* subgroupSizeVar = FindBuiltinDefinition(irContext, spv::BuiltIn::SubgroupSize);
if (subgroupSizeVar != nullptr &&
!IsInputPointerTo(irContext, subgroupSizeVar, valueTypeId)) {
return Status::Failure;
}
auto* constantMgr = irContext->get_constant_mgr();
auto* typeMgr = irContext->get_type_mgr();
const auto* valueType = typeMgr->GetType(valueTypeId);
if (valueType == nullptr) {
return Status::Failure;
}
const auto* one = constantMgr->GetConstant(valueType, {1u});
const Instruction* oneInst =
one != nullptr ? constantMgr->GetDefiningInstruction(one, valueTypeId) : nullptr;
if (oneInst == nullptr) {
return Status::Failure;
}
const uint32_t oneId = oneInst->result_id();
const uint32_t subgroupSizeVarId = subgroupSizeVar != nullptr
? subgroupSizeVar->result_id()
: SynthesizeSubgroupSizeVariable(irContext, numSubgroupsVar->type_id());
const uint32_t workgroupSizeId = workgroupSize->result_id();
// ceil(local invocation count / SubgroupSize): the subgroup count of a
// full-subgroup launch. Vulkan only guarantees that partition under
// REQUIRE_FULL_SUBGROUPS - which ProgramFactory requests whenever
// local_size_x is a multiple of the subgroup size makes it legal
// (VUID-VkPipelineShaderStageCreateInfo-flags-02759) - and calls the
// tighter behaviour "encouraged" everywhere else; the DriverPost witness
// verifies it per device where the flag cannot be set. The absence of
// ALLOW_VARYING_SUBGROUP_SIZE pins only the SubgroupSize builtin itself.
// `(count - 1) / size + 1` avoids an addition overflow at count + size - 1.
for (Instruction* load : numSubgroupsLoads) {
const uint32_t localSizeXId = irContext->TakeNextId();
const uint32_t localSizeYId = irContext->TakeNextId();
const uint32_t localSizeZId = irContext->TakeNextId();
const uint32_t localSizeXYId = irContext->TakeNextId();
const uint32_t invocationCountId = irContext->TakeNextId();
const uint32_t adjustedCountId = irContext->TakeNextId();
const uint32_t subgroupSizeId = irContext->TakeNextId();
const uint32_t quotientId = irContext->TakeNextId();
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpCompositeExtract, valueTypeId, localSizeXId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {workgroupSizeId}},
{SPV_OPERAND_TYPE_LITERAL_INTEGER, {0u}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpCompositeExtract, valueTypeId, localSizeYId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {workgroupSizeId}},
{SPV_OPERAND_TYPE_LITERAL_INTEGER, {1u}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpCompositeExtract, valueTypeId, localSizeZId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {workgroupSizeId}},
{SPV_OPERAND_TYPE_LITERAL_INTEGER, {2u}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpIMul, valueTypeId, localSizeXYId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {localSizeXId}},
{SPV_OPERAND_TYPE_ID, {localSizeYId}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpIMul, valueTypeId, invocationCountId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {localSizeXYId}},
{SPV_OPERAND_TYPE_ID, {localSizeZId}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpISub, valueTypeId, adjustedCountId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {invocationCountId}},
{SPV_OPERAND_TYPE_ID, {oneId}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpLoad, valueTypeId, subgroupSizeId,
std::initializer_list<Operand>{{SPV_OPERAND_TYPE_ID, {subgroupSizeVarId}}}));
load->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpUDiv, valueTypeId, quotientId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {adjustedCountId}},
{SPV_OPERAND_TYPE_ID, {subgroupSizeId}}}));
// Preserve the original result id so every downstream use automatically sees
// the derived value instead of the driver's NumSubgroups builtin.
load->SetOpcode(spv::Op::OpIAdd);
load->SetInOperands(Instruction::OperandList{
{SPV_OPERAND_TYPE_ID, {quotientId}},
{SPV_OPERAND_TYPE_ID, {oneId}}});
}
irContext->InvalidateAnalysesExceptFor(IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
spvtools::Optimizer::PassToken DeriveNumSubgroupsPass::CreateDeriveNumSubgroupsPass() {
return spvtools::Optimizer::PassToken(MakeUnique<DeriveNumSubgroupsPass>());
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -0,0 +1,39 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DeriveNumSubgroupsPass.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Replaces compute-stage NumSubgroups builtin loads with
// ceil(WorkgroupSize.x * WorkgroupSize.y * WorkgroupSize.z / SubgroupSize).
//
// That is the subgroup count of a full-subgroup launch - guaranteed by Vulkan
// under REQUIRE_FULL_SUBGROUPS (which ProgramFactory requests whenever the
// workgroup shape makes it legal), spec-"encouraged" and witness-verified
// (DriverPost) elsewhere. Deriving it repairs drivers that expose the real
// SubgroupId topology but return an inconsistent NumSubgroups value, breaking
// GL's gl_SubgroupID < gl_NumSubgroups contract. This is a DirectVulkan
// semantic repair, not a source-shader rewrite; the application's subgroup
// arithmetic and shared-memory logic remain unchanged.
class DeriveNumSubgroupsPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "derive-num-subgroups"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateDeriveNumSubgroupsPass();
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,73 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EmulateSubgroupsPass.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Lowers every GL_KHR_shader_subgroup construct in a compute module onto a
// 32-lane VIRTUAL subgroup implemented with workgroup-shared memory. Virtual
// subgroups partition the workgroup by gl_LocalInvocationIndex:
// lane = index & 31, id = index >> 5, count = ceil(invocations / 32).
//
// This is a LAST-RESORT path, never a substitute for real subgroups: it only
// runs when MOBILEGL_MAGMA_EMULATE_SUBGROUP=1 is set explicitly and the device
// has no native subgroup support at all (SubgroupSupportPolicy.h). A device
// with native subgroup operations - however narrow - uses them natively, with
// FixIterationRPSubgroupScratchPass patching the known pack bug instead.
//
// Lowered constructs:
// - the builtins gl_SubgroupSize / gl_SubgroupInvocationID / gl_SubgroupID /
// gl_NumSubgroups and the five gl_Subgroup*Mask ballot builtins;
// - OpGroupNonUniform{Elect,All,Any,AllEqual,Broadcast,BroadcastFirst,
// Ballot,InverseBallot,BallotBitExtract,BallotBitCount,BallotFind{L,M}SB,
// Shuffle,ShuffleXor,ShuffleUp,ShuffleDown,
// <arithmetic/min/max/bitwise/logical reduce+scans+clustered>,
// QuadBroadcast,QuadSwap};
// - subgroupBarrier()/subgroupMemoryBarrier*() (their Subgroup scopes widen
// to Workgroup, which is strictly stronger).
// The output uses no GroupNonUniform* instruction or capability at all, which
// is what lets it run on devices with no subgroup feature bits.
//
// Semantic contract, narrower than native subgroups in exactly one way: every
// emulated exchange synchronizes through OpControlBarrier, so subgroup
// operations must sit in WORKGROUP-uniform control flow (the shape every
// Iris-style pack reduction has). GLSL already imposes this for barrier();
// a subgroup op in divergent flow - legal on native subgroups - is undefined
// here.
//
// Fails (Status::Failure, leaving the input module unchanged) on anything it
// cannot lower faithfully: extended subgroup ops (partitioned-NV, rotate,
// quad-all/any), non-32-bit participating types, spec-constant workgroup
// sizes, a subgroup builtin reached by anything but a direct OpLoad, or a
// module whose lowering would add more workgroup scratch than
// maxWorkgroupScratchBytes (pass the device's maxComputeSharedMemorySize;
// 0 falls back to the 16384-byte Vulkan minimum).
class EmulateSubgroupsPass : public spvtools::opt::Pass {
public:
explicit EmulateSubgroupsPass(Uint32 maxWorkgroupScratchBytes)
: m_maxWorkgroupScratchBytes(maxWorkgroupScratchBytes) {}
const char* name() const override { return "emulate-subgroups"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateEmulateSubgroupsPass(
Uint32 maxWorkgroupScratchBytes);
private:
Uint32 m_maxWorkgroupScratchBytes;
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -0,0 +1,558 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPSubgroupScratchPass.cpp
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "FixIterationRPSubgroupScratchPass.h"
#include "spirv.hpp"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/util/make_unique.h"
#include <map>
#include <unordered_map>
#include <vector>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
namespace {
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
// The Vulkan minimum for maxComputeSharedMemorySize, used when the caller
// could not tell us the device's real limit.
constexpr uint32_t kMinimumSharedMemoryBytes = 16384u;
// The narrowest subgroup width iterationRP's declarations are sized for.
// At or above it both shipped shapes fit and nothing may be rewritten.
constexpr uint32_t kPackAssumedSubgroupWidth = 16u;
Instruction* FindBuiltinDefinition(IRContext* context, spv::BuiltIn builtin) {
auto* defUseMgr = context->get_def_use_mgr();
for (auto& annotation : context->annotations()) {
if (annotation.opcode() != spv::Op::OpDecorate || annotation.NumInOperands() < 3) {
continue;
}
if (static_cast<spv::Decoration>(annotation.GetSingleWordInOperand(1)) !=
spv::Decoration::BuiltIn) {
continue;
}
if (static_cast<spv::BuiltIn>(annotation.GetSingleWordInOperand(2)) != builtin) {
continue;
}
return defUseMgr->GetDef(annotation.GetSingleWordInOperand(0));
}
return nullptr;
}
// Walks an access-chain pointer expression back to the variable it is
// rooted at; returns nullptr for anything that is not a plain chain.
const Instruction* RootVariable(IRContext* context, uint32_t pointerId) {
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* def = defUseMgr->GetDef(pointerId);
while (def != nullptr) {
switch (def->opcode()) {
case spv::Op::OpVariable:
return def;
case spv::Op::OpAccessChain:
case spv::Op::OpInBoundsAccessChain:
case spv::Op::OpCopyObject:
def = defUseMgr->GetDef(def->GetSingleWordInOperand(0));
break;
default:
return nullptr;
}
}
return nullptr;
}
// A 32-bit float scalar or vector - the shape of every accumulator the
// pack runs through its scans (float, vec2 and vec4 all appear). Returns
// the component count, or 0 for anything else.
uint32_t Float32ComponentCount(IRContext* context, uint32_t typeId) {
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* type = defUseMgr->GetDef(typeId);
if (type == nullptr) return 0u;
uint32_t components = 1u;
if (type->opcode() == spv::Op::OpTypeVector) {
components = type->GetSingleWordInOperand(1);
if (components < 2u || components > 4u) return 0u;
type = defUseMgr->GetDef(type->GetSingleWordInOperand(0));
if (type == nullptr) return 0u;
}
if (type->opcode() != spv::Op::OpTypeFloat ||
type->GetSingleWordInOperand(0) != 32u) {
return 0u;
}
return components;
}
uint32_t RoundUp(uint32_t value, uint32_t alignment) {
return alignment == 0u ? value : ((value + alignment - 1u) / alignment) * alignment;
}
// Size AND alignment of a workgroup-storage type. Drivers lay shared
// memory out at natural alignment and the limit
// (VUID-RuntimeSpirv-Workgroup-06530) counts the padding that produces,
// so a model that sums unpadded sizes would under-count exactly where the
// budget check matters. Returns false for anything not modelled here,
// which the caller answers by declining to grow at all rather than by
// certifying growth against a total it knows is an underestimate.
bool WorkgroupTypeLayout(IRContext* context, uint32_t typeId, uint32_t* size,
uint32_t* alignment, uint32_t depth = 0u) {
if (depth > 8u) return false;
auto* defUseMgr = context->get_def_use_mgr();
const Instruction* type = defUseMgr->GetDef(typeId);
if (type == nullptr) return false;
switch (type->opcode()) {
case spv::Op::OpTypeBool:
*size = 4u;
*alignment = 4u;
return true;
case spv::Op::OpTypeInt:
case spv::Op::OpTypeFloat: {
const uint32_t width = type->GetSingleWordInOperand(0) / 8u;
if (width == 0u) return false;
*size = width;
*alignment = width;
return true;
}
case spv::Op::OpTypeVector: {
uint32_t componentSize = 0u;
uint32_t componentAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(0),
&componentSize, &componentAlignment, depth + 1u)) {
return false;
}
const uint32_t components = type->GetSingleWordInOperand(1);
if (components < 2u || components > 4u) return false;
*size = componentSize * components;
// A three-component vector aligns like a four-component one.
*alignment = componentSize * (components == 3u ? 4u : components);
return true;
}
case spv::Op::OpTypeMatrix:
case spv::Op::OpTypeArray: {
uint32_t elementSize = 0u;
uint32_t elementAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(0), &elementSize,
&elementAlignment, depth + 1u)) {
return false;
}
uint32_t count = 0u;
if (type->opcode() == spv::Op::OpTypeMatrix) {
count = type->GetSingleWordInOperand(1);
} else {
const Instruction* length =
defUseMgr->GetDef(type->GetSingleWordInOperand(1));
if (length == nullptr || length->opcode() != spv::Op::OpConstant) {
return false; // spec-constant length: not sizeable here
}
count = length->GetSingleWordInOperand(0);
}
*size = RoundUp(elementSize, elementAlignment) * count;
*alignment = elementAlignment;
return true;
}
case spv::Op::OpTypeStruct: {
uint32_t offset = 0u;
uint32_t structAlignment = 1u;
for (uint32_t i = 0; i < type->NumInOperands(); ++i) {
uint32_t memberSize = 0u;
uint32_t memberAlignment = 0u;
if (!WorkgroupTypeLayout(context, type->GetSingleWordInOperand(i),
&memberSize, &memberAlignment, depth + 1u)) {
return false;
}
offset = RoundUp(offset, memberAlignment) + memberSize;
if (memberAlignment > structAlignment) structAlignment = memberAlignment;
}
*size = RoundUp(offset, structAlignment);
*alignment = structAlignment;
return true;
}
default:
return false;
}
}
// The group operations the pack's prefix scans use.
bool IsScanOrReduce(spv::GroupOperation operation) {
return operation == spv::GroupOperation::Reduce ||
operation == spv::GroupOperation::InclusiveScan ||
operation == spv::GroupOperation::ExclusiveScan;
}
} // namespace
spvtools::opt::Pass::Status FixIterationRPSubgroupScratchPass::Process() {
auto* irContext = context();
auto* defUseMgr = irContext->get_def_use_mgr();
// Without a known device width there is no topology to compare against;
// and a width the pack already assumed needs no patch at all. Both of
// iterationRP's shapes are sized for >= 16 lanes (512/16 = 32 entries,
// 1024/16 = 64), so every module on such a device - the pack's or anyone
// else's - must pass through byte-identical. The per-array length test
// further down is the second gate, not a replacement for this one.
if (m_nativeSubgroupSize == 0u || m_nativeSubgroupSize >= kPackAssumedSubgroupWidth) {
return Status::SuccessWithoutChange;
}
for (const Instruction& entryPoint : irContext->module()->entry_points()) {
if (static_cast<spv::ExecutionModel>(entryPoint.GetSingleWordInOperand(0)) !=
spv::ExecutionModel::GLCompute) {
return Status::SuccessWithoutChange;
}
}
// Fingerprint 1: a literal workgroup size, so the subgroup count the
// dispatch actually partitions into is known here.
const auto resolveUintConstant = [&](uint32_t id, uint32_t* value) {
const Instruction* def = defUseMgr->GetDef(id);
if (def == nullptr || def->opcode() != spv::Op::OpConstant) return false;
*value = def->GetSingleWordInOperand(0);
return true;
};
uint32_t localSize[3] = {0, 0, 0};
bool haveLocalSize = false;
if (Instruction* workgroupSize =
FindBuiltinDefinition(irContext, spv::BuiltIn::WorkgroupSize)) {
if (workgroupSize->opcode() == spv::Op::OpConstantComposite &&
workgroupSize->NumInOperands() == 3) {
haveLocalSize =
resolveUintConstant(workgroupSize->GetSingleWordInOperand(0), &localSize[0]) &&
resolveUintConstant(workgroupSize->GetSingleWordInOperand(1), &localSize[1]) &&
resolveUintConstant(workgroupSize->GetSingleWordInOperand(2), &localSize[2]);
}
}
if (!haveLocalSize) {
for (const Instruction& mode : irContext->module()->execution_modes()) {
if (mode.opcode() == spv::Op::OpExecutionMode &&
static_cast<spv::ExecutionMode>(mode.GetSingleWordInOperand(1)) ==
spv::ExecutionMode::LocalSize) {
localSize[0] = mode.GetSingleWordInOperand(2);
localSize[1] = mode.GetSingleWordInOperand(3);
localSize[2] = mode.GetSingleWordInOperand(4);
haveLocalSize = true;
break;
}
}
}
if (!haveLocalSize || localSize[0] == 0u || localSize[1] == 0u || localSize[2] == 0u) {
return Status::SuccessWithoutChange;
}
const uint64_t totalInvocations =
static_cast<uint64_t>(localSize[0]) * localSize[1] * localSize[2];
if (totalInvocations == 0u || totalInvocations > (1u << 20)) {
return Status::SuccessWithoutChange;
}
const uint32_t requiredLength = static_cast<uint32_t>(
(totalInvocations + m_nativeSubgroupSize - 1u) / m_nativeSubgroupSize);
// Fingerprint 2: a subgroup scan over a 32-bit float value - the pack's
// prefix-sum reduction, and the reason its scratch is indexed per subgroup.
bool sawFloatSubgroupScan = false;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.opcode() != spv::Op::OpGroupNonUniformFAdd &&
inst.opcode() != spv::Op::OpGroupNonUniformFMin &&
inst.opcode() != spv::Op::OpGroupNonUniformFMax) {
continue;
}
if (inst.NumInOperands() < 2) continue;
if (!IsScanOrReduce(static_cast<spv::GroupOperation>(
inst.GetSingleWordInOperand(1)))) {
continue;
}
if (Float32ComponentCount(irContext, inst.type_id()) != 0u) {
sawFloatSubgroupScan = true;
}
}
}
}
if (!sawFloatSubgroupScan) {
return Status::SuccessWithoutChange;
}
// gl_SubgroupID, whose value range the pack's scratch size bakes in.
const Instruction* subgroupIdVariable =
FindBuiltinDefinition(irContext, spv::BuiltIn::SubgroupId);
if (subgroupIdVariable == nullptr ||
subgroupIdVariable->opcode() != spv::Op::OpVariable) {
return Status::SuccessWithoutChange;
}
const uint32_t subgroupIdVariableId = subgroupIdVariable->result_id();
// The pack indexes its scratch with gl_SubgroupID ITSELF, so only values
// that ARE that id qualify - not everything computed from it. An index
// that is masked or clamped (cache[gl_SubgroupID & 3u]) is bounded by
// construction and is none of this pass's business; accepting it would
// turn a targeted repair into a general array resizer. Identity survives
// OpCopyObject, a signedness OpBitcast, and the Function/Private spill
// glslang emits for a builtin load - and nothing else. A spill variable
// counts only when EVERY store into it is the id.
std::unordered_map<uint32_t, bool> subgroupIdValues; // result id IS the id
std::unordered_map<uint32_t, bool> subgroupIdVariables; // spill holding only it
bool changedIdentity = true;
while (changedIdentity) {
changedIdentity = false;
std::unordered_map<uint32_t, uint32_t> totalStores;
std::unordered_map<uint32_t, uint32_t> idStores;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.opcode() != spv::Op::OpStore) continue;
const uint32_t pointerId = inst.GetSingleWordInOperand(0);
const Instruction* target = defUseMgr->GetDef(pointerId);
if (target == nullptr || target->opcode() != spv::Op::OpVariable) {
continue;
}
const auto storageClass = static_cast<spv::StorageClass>(
target->GetSingleWordInOperand(0));
if (storageClass != spv::StorageClass::Function &&
storageClass != spv::StorageClass::Private) {
continue;
}
totalStores[pointerId] += 1u;
if (subgroupIdValues.count(inst.GetSingleWordInOperand(1))) {
idStores[pointerId] += 1u;
}
}
}
}
for (const auto& entry : totalStores) {
if (entry.second != 0u && idStores[entry.first] == entry.second &&
!subgroupIdVariables.count(entry.first)) {
subgroupIdVariables[entry.first] = true;
changedIdentity = true;
}
}
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.result_id() == 0 ||
subgroupIdValues.count(inst.result_id())) {
continue;
}
bool isSubgroupId = false;
switch (inst.opcode()) {
case spv::Op::OpLoad: {
const uint32_t pointerId = inst.GetSingleWordInOperand(0);
isSubgroupId = pointerId == subgroupIdVariableId ||
subgroupIdVariables.count(pointerId) != 0u;
break;
}
case spv::Op::OpCopyObject:
case spv::Op::OpBitcast:
isSubgroupId =
subgroupIdValues.count(inst.GetSingleWordInOperand(0)) != 0u;
break;
default:
break;
}
if (isSubgroupId) {
subgroupIdValues[inst.result_id()] = true;
changedIdentity = true;
}
}
}
}
}
if (subgroupIdValues.empty()) {
return Status::SuccessWithoutChange;
}
// Fingerprint 3: workgroup-shared float arrays indexed by gl_SubgroupID
// itself - the under-declared prefixSumCache.
std::map<uint32_t, Instruction*> candidates;
for (auto& function : *irContext->module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.opcode() != spv::Op::OpAccessChain &&
inst.opcode() != spv::Op::OpInBoundsAccessChain) {
continue;
}
if (inst.NumInOperands() < 2) continue;
if (!subgroupIdValues.count(inst.GetSingleWordInOperand(1))) continue;
Instruction* baseVariable =
defUseMgr->GetDef(inst.GetSingleWordInOperand(0));
if (baseVariable == nullptr ||
baseVariable->opcode() != spv::Op::OpVariable ||
static_cast<spv::StorageClass>(
baseVariable->GetSingleWordInOperand(0)) !=
spv::StorageClass::Workgroup) {
continue;
}
candidates.emplace(baseVariable->result_id(), baseVariable);
}
}
}
if (candidates.empty()) {
return Status::SuccessWithoutChange;
}
// Everything that survives the filter, with the bytes each grown array
// will need. Nothing is mutated until the whole set fits the device's
// shared-memory budget, so a module is never left half-grown.
struct Growth {
Instruction* variable = nullptr;
uint32_t elementTypeId = 0;
uint32_t lengthTypeId = 0;
uint32_t addedBytes = 0;
};
std::vector<Growth> growths;
for (auto& entry : candidates) {
Instruction* variable = entry.second;
// The variable must be reached exclusively through access chains (plus
// debug/decoration instructions): a whole-array load, store, or copy
// would change type with the array and is left alone.
bool onlyAccessChains = true;
const uint32_t variableId = variable->result_id();
defUseMgr->ForEachUser(variable, [&](Instruction* user) {
switch (user->opcode()) {
case spv::Op::OpAccessChain:
case spv::Op::OpInBoundsAccessChain:
if (user->GetSingleWordInOperand(0) != variableId) {
onlyAccessChains = false;
}
return;
case spv::Op::OpName:
case spv::Op::OpDecorate:
return;
default:
onlyAccessChains = false;
return;
}
});
if (!onlyAccessChains) continue;
if (variable->NumInOperands() > 1) continue; // initializer: leave alone
const Instruction* pointerType = defUseMgr->GetDef(variable->type_id());
if (pointerType == nullptr || pointerType->opcode() != spv::Op::OpTypePointer) {
continue;
}
const Instruction* arrayType =
defUseMgr->GetDef(pointerType->GetSingleWordInOperand(1));
if (arrayType == nullptr || arrayType->opcode() != spv::Op::OpTypeArray) {
continue;
}
const uint32_t elementTypeId = arrayType->GetSingleWordInOperand(0);
const uint32_t components = Float32ComponentCount(irContext, elementTypeId);
if (components == 0u) continue;
const Instruction* lengthConstant =
defUseMgr->GetDef(arrayType->GetSingleWordInOperand(1));
if (lengthConstant == nullptr || lengthConstant->opcode() != spv::Op::OpConstant) {
continue;
}
const uint32_t currentLength = lengthConstant->GetSingleWordInOperand(0);
// The pack's own assumption holds on this device: the declared array
// already covers every subgroup the workgroup partitions into. That is
// every >= 16-lane device for the shapes iterationRP ships, and those
// modules must pass through byte-identical.
if (currentLength >= requiredLength) continue;
// vec3 strides at its 16-byte alignment, so charge the padded stride.
const uint32_t elementStride = (components == 3u ? 4u : components) * 4u;
growths.push_back(Growth{variable, elementTypeId, lengthConstant->type_id(),
(requiredLength - currentLength) * elementStride});
}
if (growths.empty()) {
return Status::SuccessWithoutChange;
}
// Growing must not push the module past what the device can launch: a
// pipeline that fails to create is worse than the pack's own overrun.
{
uint64_t declaredBytes = 0;
bool sawUnsizeable = false;
for (auto& global : irContext->module()->types_values()) {
if (global.opcode() != spv::Op::OpVariable ||
static_cast<spv::StorageClass>(global.GetSingleWordInOperand(0)) !=
spv::StorageClass::Workgroup) {
continue;
}
const Instruction* pointerType = defUseMgr->GetDef(global.type_id());
uint32_t bytes = 0u;
uint32_t alignment = 0u;
if (pointerType == nullptr ||
pointerType->opcode() != spv::Op::OpTypePointer ||
!WorkgroupTypeLayout(irContext, pointerType->GetSingleWordInOperand(1),
&bytes, &alignment)) {
sawUnsizeable = true;
break;
}
declaredBytes = RoundUp(static_cast<uint32_t>(declaredBytes), alignment) + bytes;
}
// A declaration this pass cannot size leaves the total an
// underestimate, so the growth cannot be certified against the device
// limit at all - decline rather than guess.
if (sawUnsizeable) {
return Status::SuccessWithoutChange;
}
for (const Growth& growth : growths) declaredBytes += growth.addedBytes;
const uint32_t deviceBudget = m_maxWorkgroupScratchBytes != 0u
? m_maxWorkgroupScratchBytes
: kMinimumSharedMemoryBytes;
if (declaredBytes > deviceBudget) {
return Status::SuccessWithoutChange;
}
}
for (const Growth& growth : growths) {
// Build the grown array type. All three new instructions are inserted
// immediately BEFORE the variable so definition-before-use holds in the
// module's global section (manager-created instructions append to its
// end, after the variable). The new length constant reuses the old
// one's integer type, whatever signedness glslang gave it (a duplicate
// scalar constant is legal SPIR-V); the fresh array type makes the
// pointer type unique by construction, so neither collides with an
// existing declaration.
Instruction* variable = growth.variable;
const uint32_t newLengthId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpConstant, growth.lengthTypeId, newLengthId,
Instruction::OperandList{{SPV_OPERAND_TYPE_TYPED_LITERAL_NUMBER,
{requiredLength}}}));
const uint32_t newArrayTypeId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpTypeArray, 0, newArrayTypeId,
Instruction::OperandList{
{SPV_OPERAND_TYPE_ID, {growth.elementTypeId}},
{SPV_OPERAND_TYPE_ID, {newLengthId}}}));
const uint32_t newPointerTypeId = irContext->TakeNextId();
variable->InsertBefore(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpTypePointer, 0, newPointerTypeId,
Instruction::OperandList{
{SPV_OPERAND_TYPE_STORAGE_CLASS,
{static_cast<uint32_t>(spv::StorageClass::Workgroup)}},
{SPV_OPERAND_TYPE_ID, {newArrayTypeId}}}));
variable->SetResultType(newPointerTypeId);
}
irContext->InvalidateAnalysesExceptFor(IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
spvtools::Optimizer::PassToken
FixIterationRPSubgroupScratchPass::CreateFixIterationRPSubgroupScratchPass(
const Uint32 nativeSubgroupSize, const Uint32 maxWorkgroupScratchBytes) {
return spvtools::Optimizer::PassToken(MakeUnique<FixIterationRPSubgroupScratchPass>(
nativeSubgroupSize, maxWorkgroupScratchBytes));
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -0,0 +1,73 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FixIterationRPSubgroupScratchPass.h
// Copyright (c) 2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Patches ONE known shader-pack defect: iterationRP hard-sizes the scratch
// its subgroup prefix scans write through prefixSumCache[gl_SubgroupID].
// The pack ships that idiom twice, sized for the >= 16-lane subgroups
// desktop GL drivers give it:
// - the auto-exposure reduction: 32x16 (512 invocations), vec2[32];
// - the RTW importance warp: 1024 invocations, float[64].
// On a narrower Vulkan device (lavapipe's 8 lanes -> 64 and 128 subgroups)
// every subgroup past the last declared entry indexes shared memory out of
// bounds - on a CPU rasterizer that is literal heap corruption. Both
// reduction ALGORITHMS are width-agnostic (their combine loops are sized by
// gl_NumSubgroups), so the faithful repair is to grow the under-declared
// arrays to ceil(invocations / native width) and change nothing else.
//
// This is the pack author's bug, not MobileGL's, so the patch is
// deliberately NOT a general "resize shared arrays" mechanism. It rewrites
// an array only when the module positively matches the pack's reduction
// idiom AND the device's own topology proves the declaration too small:
// - GLCompute entry point with a literal workgroup size;
// - a subgroup scan/reduce over a 32-bit float scalar or vector
// (OpGroupNonUniformF{Add,Min,Max}), the pack's accumulator signature;
// - a workgroup-shared array of 32-bit float scalars/vectors whose
// access-chain index is data-dependent on gl_SubgroupID;
// - a declared length strictly below ceil(invocations / native width).
// That last clause is what keeps the patch inert wherever the pack is
// correct: on any device whose width satisfies the pack's assumption
// (>= 16 lanes: desktop GL, Adreno) both shapes already fit and every
// module passes through byte-identical. Matching at the SPIR-V level keeps
// recognition robust against the whitespace/identifier drift that made the
// old source-text template rewrite (removed in 7769156) so brittle.
//
// The pass never fails a module: anything it cannot prove is this pattern -
// or cannot grow safely (a whole-array use, a spec-constant length, an
// initializer, or growth that would not fit maxWorkgroupScratchBytes) - is
// left exactly as it was. Pass the device's maxComputeSharedMemorySize as
// maxWorkgroupScratchBytes; 0 falls back to the 16384-byte Vulkan minimum.
class FixIterationRPSubgroupScratchPass : public spvtools::opt::Pass {
public:
FixIterationRPSubgroupScratchPass(Uint32 nativeSubgroupSize,
Uint32 maxWorkgroupScratchBytes)
: m_nativeSubgroupSize(nativeSubgroupSize),
m_maxWorkgroupScratchBytes(maxWorkgroupScratchBytes) {}
const char* name() const override { return "fix-iterationrp-subgroup-scratch"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateFixIterationRPSubgroupScratchPass(
Uint32 nativeSubgroupSize, Uint32 maxWorkgroupScratchBytes);
private:
Uint32 m_nativeSubgroupSize;
Uint32 m_maxWorkgroupScratchBytes;
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -223,7 +223,8 @@ target_include_directories(glretrace_common PUBLIC
"${APITRACE_GENERATED_DIR}" "${APITRACE_GENERATED_DIR}"
"${APITRACE_ROOT}/dispatch" "${APITRACE_ROOT}/dispatch"
"${APITRACE_ROOT}/helpers" "${APITRACE_ROOT}/helpers"
"${APITRACE_ROOT}/retrace") "${APITRACE_ROOT}/retrace"
"${CMAKE_CURRENT_LIST_DIR}/../../../../../tools/trace_replay")
target_compile_definitions(glretrace_common PRIVATE target_compile_definitions(glretrace_common PRIVATE
main=mobilegl_apitrace_main) main=mobilegl_apitrace_main)
target_redirect_exit(glretrace_common) target_redirect_exit(glretrace_common)
@@ -231,14 +232,16 @@ target_link_libraries(glretrace_common PUBLIC retrace_common glhelpers glproc)
add_library(trace_replay_runner SHARED add_library(trace_replay_runner SHARED
trace_replay_core.cpp trace_replay_core.cpp
trace_replay_jni.cpp) trace_replay_jni.cpp
"${CMAKE_CURRENT_LIST_DIR}/../../../../../tools/trace_replay/apitrace_fbo_dump.cpp")
target_compile_features(trace_replay_runner PRIVATE cxx_std_17) target_compile_features(trace_replay_runner PRIVATE cxx_std_17)
target_compile_definitions(trace_replay_runner PRIVATE target_compile_definitions(trace_replay_runner PRIVATE
MOBILEGL_APITRACE_RETRACE_MAIN=mobilegl_apitrace_main) MOBILEGL_APITRACE_RETRACE_MAIN=mobilegl_apitrace_main)
target_include_directories(trace_replay_runner PRIVATE target_include_directories(trace_replay_runner PRIVATE
"${APITRACE_ROOT}/lib/image") "${APITRACE_ROOT}/lib/image"
"${CMAKE_CURRENT_LIST_DIR}/../../../../../tools/trace_replay")
target_link_libraries(trace_replay_runner target_link_libraries(trace_replay_runner
glretrace_common glretrace_common
@@ -1,3 +1,4 @@
#include "apitrace_fbo_dump.hpp"
#include "glws.hpp" #include "glws.hpp"
#include "retrace.hpp" #include "retrace.hpp"
@@ -375,6 +376,7 @@ bool makeCurrentInternal(Drawable *drawable, Drawable *readable, Context *contex
} }
gCurrentDrawable = drawable; gCurrentDrawable = drawable;
gCurrentContext = eglContext; gCurrentContext = eglContext;
mobilegl_trace_dump::InstallIfRequested();
return true; return true;
} }
@@ -176,6 +176,18 @@ bool LoadMobileGL(const Request& request, std::string& error) {
} }
setenv("MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS", dumpPoints.c_str(), 1); setenv("MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS", dumpPoints.c_str(), 1);
} }
if (request.texture2dDumps.empty()) {
unsetenv("MOBILEGL_TRACE_DUMP_TEXTURE_2D");
} else {
std::string dumpPoints;
for (const std::string& dumpPoint : request.texture2dDumps) {
if (!dumpPoints.empty()) {
dumpPoints += ';';
}
dumpPoints += dumpPoint;
}
setenv("MOBILEGL_TRACE_DUMP_TEXTURE_2D", dumpPoints.c_str(), 1);
}
void* handle = dlopen(request.mobileGlLibrary.c_str(), RTLD_NOW | RTLD_GLOBAL); void* handle = dlopen(request.mobileGlLibrary.c_str(), RTLD_NOW | RTLD_GLOBAL);
if (handle == nullptr) { if (handle == nullptr) {
@@ -383,6 +395,13 @@ std::string SnapshotCallSet(const Request& request) {
callSet += "," + call; callSet += "," + call;
} }
} }
for (const std::string& dumpPoint : request.texture2dDumps) {
const std::size_t separator = dumpPoint.find(',');
const std::string call = dumpPoint.substr(0, separator);
if (!call.empty() && call != std::to_string(request.targetCall)) {
callSet += "," + call;
}
}
return callSet; return callSet;
} }
@@ -27,6 +27,9 @@ struct Request {
// Framebuffer-attachment dump points, each `CALL:DIR[:FBO,FBO,...]`. Debug-only; the // Framebuffer-attachment dump points, each `CALL:DIR[:FBO,FBO,...]`. Debug-only; the
// replay behaves exactly as before when this is empty. // replay behaves exactly as before when this is empty.
std::vector<std::string> fboAttachmentDumps; std::vector<std::string> fboAttachmentDumps;
// Named GL_TEXTURE_2D dump points, each `CALL,TEXTURE,LEVEL,DIR`. Debug-only; the replay
// behaves exactly as before when this is empty.
std::vector<std::string> texture2dDumps;
int targetFrame = -1; int targetFrame = -1;
long long targetCall = -1; long long targetCall = -1;
int width = 0; int width = 0;
@@ -6,6 +6,7 @@
#include <exception> #include <exception>
#include <string> #include <string>
#include <vector>
#include <sys/stat.h> #include <sys/stat.h>
extern "C" void mobilegl_trace_set_native_window(ANativeWindow *window); extern "C" void mobilegl_trace_set_native_window(ANativeWindow *window);
@@ -25,6 +26,23 @@ std::string ToString(JNIEnv* env, jstring value) {
return out; return out;
} }
std::vector<std::string> SplitSemicolonList(const std::string& value) {
std::vector<std::string> values;
std::size_t begin = 0;
while (begin < value.size()) {
const std::size_t end = value.find(';', begin);
const std::string entry = value.substr(begin, end - begin);
if (!entry.empty()) {
values.push_back(entry);
}
if (end == std::string::npos) {
break;
}
begin = end + 1;
}
return values;
}
jobject MakeResult(JNIEnv* env, const mobilegl_trace::Result& result) { jobject MakeResult(JNIEnv* env, const mobilegl_trace::Result& result) {
jclass clazz = env->FindClass("top/mobilegl/plugin/trace/TraceReplayActivity$TraceReplayResult"); jclass clazz = env->FindClass("top/mobilegl/plugin/trace/TraceReplayActivity$TraceReplayResult");
if (clazz == nullptr) { if (clazz == nullptr) {
@@ -103,7 +121,8 @@ Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunTraceReplay(JNIEnv*
jboolean usePbuffer, jboolean usePbuffer,
jboolean avoidAngleLlvmpipeSamplerMipmapMinFilter, jboolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
jboolean avoidAngleLlvmpipeExplicitLodBias, jboolean avoidAngleLlvmpipeExplicitLodBias,
jboolean coherentAsFlush) { jboolean coherentAsFlush,
jstring texture2dDumps) {
mobilegl_trace::Request request; mobilegl_trace::Request request;
request.tracePath = ToString(env, tracePath); request.tracePath = ToString(env, tracePath);
request.goldenPath = ToString(env, goldenPath); request.goldenPath = ToString(env, goldenPath);
@@ -115,6 +134,7 @@ Java_top_mobilegl_plugin_trace_TraceReplayActivity_nativeRunTraceReplay(JNIEnv*
request.diffPath = ToString(env, diffPath); request.diffPath = ToString(env, diffPath);
request.backend = ToString(env, backend); request.backend = ToString(env, backend);
request.angleVariant = ToString(env, angleVariant); request.angleVariant = ToString(env, angleVariant);
request.texture2dDumps = SplitSemicolonList(ToString(env, texture2dDumps));
request.targetFrame = targetFrame; request.targetFrame = targetFrame;
request.targetCall = targetCall; request.targetCall = targetCall;
request.width = width; request.width = width;
@@ -115,7 +115,8 @@ public final class TraceReplayActivity extends Activity {
request.usePbuffer, request.usePbuffer,
request.avoidAngleLlvmpipeSamplerMipmapMinFilter, request.avoidAngleLlvmpipeSamplerMipmapMinFilter,
request.avoidAngleLlvmpipeExplicitLodBias, request.avoidAngleLlvmpipeExplicitLodBias,
request.coherentAsFlush request.coherentAsFlush,
request.texture2dDumps
); );
Log.i(TAG, result.toString()); Log.i(TAG, result.toString());
TraceReplayResult finalResult = result; TraceReplayResult finalResult = result;
@@ -147,7 +148,8 @@ public final class TraceReplayActivity extends Activity {
boolean usePbuffer, boolean usePbuffer,
boolean avoidAngleLlvmpipeSamplerMipmapMinFilter, boolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
boolean avoidAngleLlvmpipeExplicitLodBias, boolean avoidAngleLlvmpipeExplicitLodBias,
boolean coherentAsFlush boolean coherentAsFlush,
String texture2dDumps
); );
private static final class TraceReplayRequest { private static final class TraceReplayRequest {
@@ -172,6 +174,7 @@ public final class TraceReplayActivity extends Activity {
final boolean avoidAngleLlvmpipeSamplerMipmapMinFilter; final boolean avoidAngleLlvmpipeSamplerMipmapMinFilter;
final boolean avoidAngleLlvmpipeExplicitLodBias; final boolean avoidAngleLlvmpipeExplicitLodBias;
final boolean coherentAsFlush; final boolean coherentAsFlush;
final String texture2dDumps;
private TraceReplayRequest( private TraceReplayRequest(
String tracePath, String tracePath,
@@ -194,7 +197,8 @@ public final class TraceReplayActivity extends Activity {
boolean usePbuffer, boolean usePbuffer,
boolean avoidAngleLlvmpipeSamplerMipmapMinFilter, boolean avoidAngleLlvmpipeSamplerMipmapMinFilter,
boolean avoidAngleLlvmpipeExplicitLodBias, boolean avoidAngleLlvmpipeExplicitLodBias,
boolean coherentAsFlush boolean coherentAsFlush,
String texture2dDumps
) { ) {
this.tracePath = tracePath; this.tracePath = tracePath;
this.goldenPath = goldenPath; this.goldenPath = goldenPath;
@@ -217,6 +221,7 @@ public final class TraceReplayActivity extends Activity {
this.avoidAngleLlvmpipeSamplerMipmapMinFilter = avoidAngleLlvmpipeSamplerMipmapMinFilter; this.avoidAngleLlvmpipeSamplerMipmapMinFilter = avoidAngleLlvmpipeSamplerMipmapMinFilter;
this.avoidAngleLlvmpipeExplicitLodBias = avoidAngleLlvmpipeExplicitLodBias; this.avoidAngleLlvmpipeExplicitLodBias = avoidAngleLlvmpipeExplicitLodBias;
this.coherentAsFlush = coherentAsFlush; this.coherentAsFlush = coherentAsFlush;
this.texture2dDumps = texture2dDumps;
} }
static TraceReplayRequest from(Intent intent, File filesDir, String defaultBackend) { static TraceReplayRequest from(Intent intent, File filesDir, String defaultBackend) {
@@ -243,7 +248,8 @@ public final class TraceReplayActivity extends Activity {
intent.getBooleanExtra("use_pbuffer", false), intent.getBooleanExtra("use_pbuffer", false),
intent.getBooleanExtra("avoid_angle_llvmpipe_sampler_mipmap_min_filter", false), intent.getBooleanExtra("avoid_angle_llvmpipe_sampler_mipmap_min_filter", false),
intent.getBooleanExtra("avoid_angle_llvmpipe_explicit_lod_bias", false), intent.getBooleanExtra("avoid_angle_llvmpipe_explicit_lod_bias", false),
intent.getBooleanExtra("coherent_as_flush", false) intent.getBooleanExtra("coherent_as_flush", false),
readString(intent, "texture_2d_dumps", "")
); );
} }
+28
View File
@@ -31,6 +31,7 @@ Usage:
[--avoid-angle-llvmpipe-sampler-mipmap-min-filter] \ [--avoid-angle-llvmpipe-sampler-mipmap-min-filter] \
[--avoid-angle-llvmpipe-explicit-lod-bias] \ [--avoid-angle-llvmpipe-explicit-lod-bias] \
[--coherent-as-flush] \ [--coherent-as-flush] \
[--dump-texture-2d CALL,TEXTURE,LEVEL,DIR] \
--timeout-seconds N --timeout-seconds N
Set MOBILEGL_USE_ANGLE=1 to run DirectGLES replay with packaged ANGLE Set MOBILEGL_USE_ANGLE=1 to run DirectGLES replay with packaged ANGLE
@@ -104,6 +105,7 @@ use_pbuffer=0
avoid_angle_llvmpipe_sampler_mipmap_min_filter=0 avoid_angle_llvmpipe_sampler_mipmap_min_filter=0
avoid_angle_llvmpipe_explicit_lod_bias=0 avoid_angle_llvmpipe_explicit_lod_bias=0
coherent_as_flush=0 coherent_as_flush=0
texture_2d_dumps=""
timeout_seconds="" timeout_seconds=""
while [ "$#" -gt 0 ]; do while [ "$#" -gt 0 ]; do
@@ -144,6 +146,7 @@ while [ "$#" -gt 0 ]; do
shift 1 shift 1
;; ;;
--coherent-as-flush) coherent_as_flush=1; shift 1 ;; --coherent-as-flush) coherent_as_flush=1; shift 1 ;;
--dump-texture-2d) texture_2d_dumps="$(next_arg "$@")"; shift 2 ;;
--timeout-seconds) timeout_seconds="$(next_arg "$@")"; shift 2 ;; --timeout-seconds) timeout_seconds="$(next_arg "$@")"; shift 2 ;;
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
*) die "unknown argument: $1" ;; *) die "unknown argument: $1" ;;
@@ -262,6 +265,27 @@ copy_app_artifact() {
fi fi
} }
copy_texture_2d_dumps() {
[ -n "${texture_2d_dumps}" ] || return 0
saved_ifs="${IFS}"
IFS=';'
set -- ${texture_2d_dumps}
IFS="${saved_ifs}"
for dump_point in "$@"; do
dump_dir="${dump_point#*,}"
dump_dir="${dump_dir#*,}"
dump_dir="${dump_dir#*,}"
[ -n "${dump_dir}" ] || continue
dump_name="$(basename "${dump_dir}")"
destination_dir="${result_dir}/${dump_name}"
mkdir -p "${destination_dir}"
if ! adb_device_path exec-out run-as "${package_name}" tar -C "${dump_dir}" -cf - . | tar -xf - -C "${destination_dir}"; then
echo "trace-replay-ci.sh: warning: failed to copy texture dump ${dump_dir}" >&2
rm -rf "${destination_dir}"
fi
done
}
prepare_fixture() { prepare_fixture() {
fixture_dir="${fixture_root}/${safe_case}" fixture_dir="${fixture_root}/${safe_case}"
rm -rf "${fixture_dir}" rm -rf "${fixture_dir}"
@@ -339,6 +363,9 @@ run_retrace() {
if [ "${coherent_as_flush}" -eq 1 ]; then if [ "${coherent_as_flush}" -eq 1 ]; then
set -- "$@" --ez coherent_as_flush true set -- "$@" --ez coherent_as_flush true
fi fi
if [ -n "${texture_2d_dumps}" ]; then
set -- "$@" --es texture_2d_dumps "${texture_2d_dumps}"
fi
set -- "$@" \ set -- "$@" \
--es output_dir "${app_dir}/output" \ --es output_dir "${app_dir}/output" \
--es diff_path "${app_dir}/output/${safe_case}-diff.png" \ --es diff_path "${app_dir}/output/${safe_case}-diff.png" \
@@ -399,6 +426,7 @@ run_retrace() {
copy_app_artifact "${app_dir}/output/${safe_case}-diff.png" "${result_dir}/${safe_case}-${backend}-diff.png" copy_app_artifact "${app_dir}/output/${safe_case}-diff.png" "${result_dir}/${safe_case}-${backend}-diff.png"
copy_app_artifact "${app_dir}/output/retrace.log" "${result_dir}/retrace.log" copy_app_artifact "${app_dir}/output/retrace.log" "${result_dir}/retrace.log"
copy_app_artifact "${app_dir}/output/mobilegl.log" "${result_dir}/mobilegl.log" copy_app_artifact "${app_dir}/output/mobilegl.log" "${result_dir}/mobilegl.log"
copy_texture_2d_dumps
# A replay that wrote result.json but did not pass used to print nothing but # A replay that wrote result.json but did not pass used to print nothing but
# the JSON, which for a non-zero statusCode says only "retrace failed with # the JSON, which for a non-zero statusCode says only "retrace failed with
+235 -10
View File
@@ -13,8 +13,13 @@
#include <cstring> #include <cstring>
#include <fstream> #include <fstream>
#include <iostream> #include <iostream>
#include <limits>
#include <string> #include <string>
#if defined(_WIN32)
#include <direct.h>
#else
#include <sys/stat.h> #include <sys/stat.h>
#endif
#include <vector> #include <vector>
// Dumps every colour attachment (and the depth attachment) of every live framebuffer // Dumps every colour attachment (and the depth attachment) of every live framebuffer
@@ -34,6 +39,7 @@ using PfnGetIntegerv = void (*)(GLenum, GLint *);
using PfnGetError = GLenum (*)(void); using PfnGetError = GLenum (*)(void);
constexpr const char *kDumpPointsEnv = "MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS"; constexpr const char *kDumpPointsEnv = "MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS";
constexpr const char *kTexture2dDumpPointsEnv = "MOBILEGL_TRACE_DUMP_TEXTURE_2D";
constexpr const char *kScanLimitEnv = "MOBILEGL_TRACE_DUMP_FBO_SCAN_LIMIT"; constexpr const char *kScanLimitEnv = "MOBILEGL_TRACE_DUMP_FBO_SCAN_LIMIT";
constexpr unsigned kDefaultScanLimit = 1024; constexpr unsigned kDefaultScanLimit = 1024;
@@ -45,6 +51,16 @@ struct DumpPoint {
bool done = false; bool done = false;
}; };
// CALL,TEXTURE,LEVEL,DIR entries, separated by ';'. This deliberately does not share the
// framebuffer dump grammar: an absolute Windows directory contains ':' but not ','.
struct Texture2dDumpPoint {
unsigned call = 0;
unsigned texture = 0;
unsigned level = 0;
std::string directory;
bool done = false;
};
struct AttachmentDesc { struct AttachmentDesc {
GLint objectType = GL_NONE; GLint objectType = GL_NONE;
GLint objectName = 0; GLint objectName = 0;
@@ -56,6 +72,7 @@ struct AttachmentDesc {
}; };
std::vector<DumpPoint> gDumpPoints; std::vector<DumpPoint> gDumpPoints;
std::vector<Texture2dDumpPoint> gTexture2dDumpPoints;
bool gInstalled = false; bool gInstalled = false;
bool gConfigured = false; bool gConfigured = false;
retrace::Dumper *gInnerDumper = nullptr; retrace::Dumper *gInnerDumper = nullptr;
@@ -105,13 +122,27 @@ bool MakeDirectories(const std::string &path) {
for (std::size_t i = 0; i < path.size(); ++i) { for (std::size_t i = 0; i < path.size(); ++i) {
partial.push_back(path[i]); partial.push_back(path[i]);
const bool last = i + 1 == path.size(); const bool last = i + 1 == path.size();
if (path[i] != '/' && !last) { #if defined(_WIN32)
const bool separator = path[i] == '/' || path[i] == '\\';
#else
const bool separator = path[i] == '/';
#endif
if (!separator && !last) {
continue; continue;
} }
if (partial == "/") { if (partial == "/") {
continue; continue;
} }
#if defined(_WIN32)
if (separator && partial.size() == 3 && partial[1] == ':') {
continue;
}
#endif
#if defined(_WIN32)
if (_mkdir(partial.c_str()) != 0 && errno != EEXIST) {
#else
if (mkdir(partial.c_str(), 0755) != 0 && errno != EEXIST) { if (mkdir(partial.c_str(), 0755) != 0 && errno != EEXIST) {
#endif
return false; return false;
} }
} }
@@ -161,6 +192,52 @@ void ParseDumpPoints(const char *spec) {
} }
} }
bool IsDecimal(const std::string &value) {
return !value.empty() && value.find_first_not_of("0123456789") == std::string::npos;
}
void ParseTexture2dDumpPoints(const char *spec) {
for (const std::string &entry : Split(spec, ';')) {
if (entry.empty()) {
continue;
}
const std::vector<std::string> fields = Split(entry, ',');
if (fields.size() != 4 || !IsDecimal(fields[0]) || !IsDecimal(fields[1]) ||
!IsDecimal(fields[2]) || fields[3].empty()) {
std::cerr << "warning: ignoring malformed " << kTexture2dDumpPointsEnv
<< " entry: " << entry << "\n";
continue;
}
char *end = nullptr;
const unsigned long call = std::strtoul(fields[0].c_str(), &end, 10);
if (*end != '\0' || call > std::numeric_limits<unsigned>::max()) {
std::cerr << "warning: ignoring malformed " << kTexture2dDumpPointsEnv
<< " call: " << entry << "\n";
continue;
}
const unsigned long texture = std::strtoul(fields[1].c_str(), &end, 10);
if (*end != '\0' || texture == 0 || texture > std::numeric_limits<unsigned>::max()) {
std::cerr << "warning: ignoring malformed " << kTexture2dDumpPointsEnv
<< " texture: " << entry << "\n";
continue;
}
const unsigned long level = std::strtoul(fields[2].c_str(), &end, 10);
if (*end != '\0' || level > static_cast<unsigned long>(std::numeric_limits<GLint>::max())) {
std::cerr << "warning: ignoring malformed " << kTexture2dDumpPointsEnv
<< " level: " << entry << "\n";
continue;
}
Texture2dDumpPoint point;
point.call = static_cast<unsigned>(call);
point.texture = static_cast<unsigned>(texture);
point.level = static_cast<unsigned>(level);
point.directory = fields[3];
gTexture2dDumpPoints.push_back(point);
}
}
unsigned ScanLimit() { unsigned ScanLimit() {
const char *value = std::getenv(kScanLimitEnv); const char *value = std::getenv(kScanLimitEnv);
if (value == nullptr || value[0] == '\0') { if (value == nullptr || value[0] == '\0') {
@@ -235,6 +312,45 @@ bool DescribeAttachment(GLenum attachment, AttachmentDesc &desc) {
return desc.width > 0 && desc.height > 0; return desc.width > 0 && desc.height > 0;
} }
bool DescribeTexture2D(GLuint texture, GLint level, AttachmentDesc &desc) {
const GLint savedTexture = GetInteger(GL_TEXTURE_BINDING_2D);
glBindTexture(GL_TEXTURE_2D, texture);
desc.objectType = GL_TEXTURE;
desc.objectName = static_cast<GLint>(texture);
desc.level = level;
glGetTexLevelParameteriv(GL_TEXTURE_2D, level, GL_TEXTURE_WIDTH, &desc.width);
glGetTexLevelParameteriv(GL_TEXTURE_2D, level, GL_TEXTURE_HEIGHT, &desc.height);
glGetTexLevelParameteriv(GL_TEXTURE_2D, level, GL_TEXTURE_INTERNAL_FORMAT, &desc.internalFormat);
glGetTexLevelParameteriv(GL_TEXTURE_2D, level, GL_TEXTURE_RED_TYPE, &desc.componentType);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(savedTexture));
return DrainErrors() == 0 && desc.width > 0 && desc.height > 0;
}
bool ReadTexture2DFloats(const AttachmentDesc &desc, std::vector<float> &pixels) {
const std::size_t count = static_cast<std::size_t>(desc.width) * desc.height * 4;
pixels.assign(count, 0.0f);
const GLint savedTexture = GetInteger(GL_TEXTURE_BINDING_2D);
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(desc.objectName));
if (desc.componentType == GL_INT || desc.componentType == GL_UNSIGNED_INT) {
std::vector<std::int32_t> raw(count, 0);
const GLenum type = desc.componentType == GL_INT ? GL_INT : GL_UNSIGNED_INT;
glGetTexImage(GL_TEXTURE_2D, desc.level, GL_RGBA_INTEGER, type, raw.data());
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(savedTexture));
if (DrainErrors() != 0) {
return false;
}
for (std::size_t i = 0; i < count; ++i) {
pixels[i] = desc.componentType == GL_INT
? static_cast<float>(raw[i])
: static_cast<float>(static_cast<std::uint32_t>(raw[i]));
}
return true;
}
glGetTexImage(GL_TEXTURE_2D, desc.level, GL_RGBA, GL_FLOAT, pixels.data());
glBindTexture(GL_TEXTURE_2D, static_cast<GLuint>(savedTexture));
return DrainErrors() == 0;
}
// Reads the attachment as floats regardless of its storage: normalised and float targets // Reads the attachment as floats regardless of its storage: normalised and float targets
// convert on the way out, integer targets are read as integers and widened. The float view // convert on the way out, integer targets are read as integers and widened. The float view
// keeps out-of-[0,1] accumulation buffers legible in the statistics even though the PNG // keeps out-of-[0,1] accumulation buffers legible in the statistics even though the PNG
@@ -308,6 +424,17 @@ std::string FormatStatistics(const std::vector<float> &pixels, unsigned channels
static_cast<double>(minimum[c]), static_cast<double>(maximum[c]), mean); static_cast<double>(minimum[c]), static_cast<double>(maximum[c]), mean);
text += buffer; text += buffer;
} }
if (pixelCount > 0) {
text += " first=(";
for (unsigned c = 0; c < channels; ++c) {
if (c > 0) {
text += ",";
}
std::snprintf(buffer, sizeof(buffer), "%.9g", static_cast<double>(pixels[c]));
text += buffer;
}
text += ")";
}
std::snprintf(buffer, sizeof(buffer), " nonfinite=%zu hash=%016llx", nonFinite, std::snprintf(buffer, sizeof(buffer), " nonfinite=%zu hash=%016llx", nonFinite,
static_cast<unsigned long long>(hash)); static_cast<unsigned long long>(hash));
text += buffer; text += buffer;
@@ -325,8 +452,8 @@ bool WriteFloatPng(const std::string &path, const AttachmentDesc &desc, unsigned
return snapshot.writePNG(path.c_str()); return snapshot.writePNG(path.c_str());
} }
void DumpOneAttachment(std::ofstream &manifest, const std::string &directory, unsigned framebuffer, void DumpOneAttachment(std::ofstream &manifest, const std::string &directory,
GLenum attachment, const char *label, bool depth) { const std::string &identity, GLenum attachment, const char *label, bool depth) {
AttachmentDesc desc; AttachmentDesc desc;
if (!DescribeAttachment(attachment, desc)) { if (!DescribeAttachment(attachment, desc)) {
return; return;
@@ -343,14 +470,13 @@ void DumpOneAttachment(std::ofstream &manifest, const std::string &directory, un
std::vector<float> pixels; std::vector<float> pixels;
const bool read = ReadAttachmentFloats(desc, depth, channels, pixels); const bool read = ReadAttachmentFloats(desc, depth, channels, pixels);
const std::string path = const std::string path = directory + "/" + identity + "-" + label + ".png";
directory + "/fbo" + std::to_string(framebuffer) + "-" + label + ".png";
const bool wrote = read && WriteFloatPng(path, desc, channels, pixels); const bool wrote = read && WriteFloatPng(path, desc, channels, pixels);
char header[512]; char header[512];
std::snprintf(header, sizeof(header), std::snprintf(header, sizeof(header),
"fbo %u %s object=%s name=%d level=%d size=%dx%d internalformat=0x%04x component=%s", "%s %s object=%s name=%d level=%d size=%dx%d internalformat=0x%04x component=%s",
framebuffer, label, identity.c_str(), label,
desc.objectType == GL_RENDERBUFFER ? "renderbuffer" : "texture", desc.objectName, desc.objectType == GL_RENDERBUFFER ? "renderbuffer" : "texture", desc.objectName,
desc.level, desc.width, desc.height, static_cast<unsigned>(desc.internalFormat), desc.level, desc.width, desc.height, static_cast<unsigned>(desc.internalFormat),
ComponentTypeName(desc.componentType)); ComponentTypeName(desc.componentType));
@@ -381,13 +507,14 @@ void DumpFramebuffer(std::ofstream &manifest, const std::string &directory, unsi
} }
const GLint savedReadBuffer = GetInteger(GL_READ_BUFFER); const GLint savedReadBuffer = GetInteger(GL_READ_BUFFER);
const std::string identity = "fbo" + std::to_string(framebuffer);
for (GLint index = 0; index < maxColorAttachments; ++index) { for (GLint index = 0; index < maxColorAttachments; ++index) {
char label[32]; char label[32];
std::snprintf(label, sizeof(label), "att%d", index); std::snprintf(label, sizeof(label), "att%d", index);
DumpOneAttachment(manifest, directory, framebuffer, DumpOneAttachment(manifest, directory, identity,
static_cast<GLenum>(GL_COLOR_ATTACHMENT0 + index), label, false); static_cast<GLenum>(GL_COLOR_ATTACHMENT0 + index), label, false);
} }
DumpOneAttachment(manifest, directory, framebuffer, GL_DEPTH_ATTACHMENT, "depth", true); DumpOneAttachment(manifest, directory, identity, GL_DEPTH_ATTACHMENT, "depth", true);
// The read buffer is per-framebuffer state the trace goes on using; put it back. // The read buffer is per-framebuffer state the trace goes on using; put it back.
if (framebuffer != 0 && savedReadBuffer != 0) { if (framebuffer != 0 && savedReadBuffer != 0) {
@@ -416,6 +543,7 @@ void RunDumpPoint(DumpPoint &point) {
const GLint savedPackSkipRows = GetInteger(GL_PACK_SKIP_ROWS); const GLint savedPackSkipRows = GetInteger(GL_PACK_SKIP_ROWS);
const GLint savedPackImageHeight = GetInteger(GL_PACK_IMAGE_HEIGHT); const GLint savedPackImageHeight = GetInteger(GL_PACK_IMAGE_HEIGHT);
const GLint savedPackSkipImages = GetInteger(GL_PACK_SKIP_IMAGES); const GLint savedPackSkipImages = GetInteger(GL_PACK_SKIP_IMAGES);
const GLint savedPackSwapBytes = GetInteger(GL_PACK_SWAP_BYTES);
DrainErrors(); DrainErrors();
if (savedPackBuffer != 0) { if (savedPackBuffer != 0) {
@@ -427,6 +555,7 @@ void RunDumpPoint(DumpPoint &point) {
glPixelStorei(GL_PACK_SKIP_ROWS, 0); glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, 0); glPixelStorei(GL_PACK_IMAGE_HEIGHT, 0);
glPixelStorei(GL_PACK_SKIP_IMAGES, 0); glPixelStorei(GL_PACK_SKIP_IMAGES, 0);
glPixelStorei(GL_PACK_SWAP_BYTES, GL_FALSE);
DrainErrors(); DrainErrors();
const GLint maxColorAttachments = GetInteger(GL_MAX_COLOR_ATTACHMENTS); const GLint maxColorAttachments = GetInteger(GL_MAX_COLOR_ATTACHMENTS);
@@ -462,6 +591,7 @@ void RunDumpPoint(DumpPoint &point) {
glPixelStorei(GL_PACK_SKIP_ROWS, savedPackSkipRows); glPixelStorei(GL_PACK_SKIP_ROWS, savedPackSkipRows);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, savedPackImageHeight); glPixelStorei(GL_PACK_IMAGE_HEIGHT, savedPackImageHeight);
glPixelStorei(GL_PACK_SKIP_IMAGES, savedPackSkipImages); glPixelStorei(GL_PACK_SKIP_IMAGES, savedPackSkipImages);
glPixelStorei(GL_PACK_SWAP_BYTES, savedPackSwapBytes);
DrainErrors(); DrainErrors();
std::cerr << "MOBILEGL_TRACE_FBO_DUMP: call " << retrace::callNo << " -> " << manifestPath std::cerr << "MOBILEGL_TRACE_FBO_DUMP: call " << retrace::callNo << " -> " << manifestPath
@@ -469,12 +599,98 @@ void RunDumpPoint(DumpPoint &point) {
point.done = true; point.done = true;
} }
void RunTexture2dDumpPoint(Texture2dDumpPoint &point) {
if (!MakeDirectories(point.directory)) {
std::cerr << "warning: failed to create texture dump directory " << point.directory << "\n";
point.done = true;
return;
}
// glGetError is destructive. This debug-only snapshot hook deliberately starts from a
// clean error state so diagnostics below identify the dump rather than an earlier trace call.
DrainErrors();
const GLint savedReadFramebuffer = GetInteger(GL_READ_FRAMEBUFFER_BINDING);
const GLint savedPackBuffer = GetInteger(GL_PIXEL_PACK_BUFFER_BINDING);
const GLint savedPackAlignment = GetInteger(GL_PACK_ALIGNMENT);
const GLint savedPackRowLength = GetInteger(GL_PACK_ROW_LENGTH);
const GLint savedPackSkipPixels = GetInteger(GL_PACK_SKIP_PIXELS);
const GLint savedPackSkipRows = GetInteger(GL_PACK_SKIP_ROWS);
const GLint savedPackImageHeight = GetInteger(GL_PACK_IMAGE_HEIGHT);
const GLint savedPackSkipImages = GetInteger(GL_PACK_SKIP_IMAGES);
const GLint savedPackSwapBytes = GetInteger(GL_PACK_SWAP_BYTES);
DrainErrors();
if (savedPackBuffer != 0) {
glBindBuffer(GL_PIXEL_PACK_BUFFER, 0);
}
glPixelStorei(GL_PACK_ALIGNMENT, 1);
glPixelStorei(GL_PACK_ROW_LENGTH, 0);
glPixelStorei(GL_PACK_SKIP_PIXELS, 0);
glPixelStorei(GL_PACK_SKIP_ROWS, 0);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, 0);
glPixelStorei(GL_PACK_SKIP_IMAGES, 0);
glPixelStorei(GL_PACK_SWAP_BYTES, GL_FALSE);
DrainErrors();
const std::string identity = "texture" + std::to_string(point.texture) +
"-level" + std::to_string(point.level);
const std::string manifestPath = point.directory + "/manifest.txt";
std::ofstream manifest(manifestPath, std::ios::trunc);
manifest << "call " << retrace::callNo << " texture " << point.texture << " level "
<< point.level << "\n";
AttachmentDesc desc;
if (glIsTexture(point.texture) == GL_FALSE ||
!DescribeTexture2D(point.texture, static_cast<GLint>(point.level), desc)) {
manifest << identity << " skipped=not-live-2d-texture\n";
} else {
std::vector<float> pixels;
const bool read = ReadTexture2DFloats(desc, pixels);
const bool wrote = read && WriteFloatPng(point.directory + "/" + identity + ".png", desc, 4, pixels);
manifest << identity << " object=texture name=" << desc.objectName << " level=" << desc.level
<< " size=" << desc.width << "x" << desc.height << " internalformat=0x" << std::hex
<< static_cast<unsigned>(desc.internalFormat) << std::dec
<< " component=" << ComponentTypeName(desc.componentType);
if (read) {
manifest << FormatStatistics(pixels, 4);
} else {
manifest << " read=failed";
}
if (!wrote) {
manifest << " png=failed";
}
manifest << "\n";
}
manifest.flush();
glBindFramebuffer(GL_READ_FRAMEBUFFER, static_cast<GLuint>(savedReadFramebuffer));
if (savedPackBuffer != 0) {
glBindBuffer(GL_PIXEL_PACK_BUFFER, static_cast<GLuint>(savedPackBuffer));
}
glPixelStorei(GL_PACK_ALIGNMENT, savedPackAlignment);
glPixelStorei(GL_PACK_ROW_LENGTH, savedPackRowLength);
glPixelStorei(GL_PACK_SKIP_PIXELS, savedPackSkipPixels);
glPixelStorei(GL_PACK_SKIP_ROWS, savedPackSkipRows);
glPixelStorei(GL_PACK_IMAGE_HEIGHT, savedPackImageHeight);
glPixelStorei(GL_PACK_SKIP_IMAGES, savedPackSkipImages);
glPixelStorei(GL_PACK_SWAP_BYTES, savedPackSwapBytes);
DrainErrors();
std::cerr << "MOBILEGL_TRACE_TEXTURE_2D_DUMP: call " << retrace::callNo << " texture "
<< point.texture << " level " << point.level << " -> " << manifestPath << "\n";
point.done = true;
}
void RunPendingDumps() { void RunPendingDumps() {
for (DumpPoint &point : gDumpPoints) { for (DumpPoint &point : gDumpPoints) {
if (!point.done && point.call == retrace::callNo) { if (!point.done && point.call == retrace::callNo) {
RunDumpPoint(point); RunDumpPoint(point);
} }
} }
for (Texture2dDumpPoint &point : gTexture2dDumpPoints) {
if (!point.done && point.call == retrace::callNo) {
RunTexture2dDumpPoint(point);
}
}
} }
class DumpingDumper final : public retrace::Dumper { class DumpingDumper final : public retrace::Dumper {
@@ -511,8 +727,12 @@ void InstallIfRequested() {
if (spec != nullptr && spec[0] != '\0') { if (spec != nullptr && spec[0] != '\0') {
ParseDumpPoints(spec); ParseDumpPoints(spec);
} }
const char *textureSpec = std::getenv(kTexture2dDumpPointsEnv);
if (textureSpec != nullptr && textureSpec[0] != '\0') {
ParseTexture2dDumpPoints(textureSpec);
}
} }
if (gDumpPoints.empty()) { if (gDumpPoints.empty() && gTexture2dDumpPoints.empty()) {
gInstalled = true; gInstalled = true;
return; return;
} }
@@ -528,6 +748,11 @@ void InstallIfRequested() {
std::cerr << "MOBILEGL_TRACE_FBO_DUMP: armed for call " << point.call << " -> " std::cerr << "MOBILEGL_TRACE_FBO_DUMP: armed for call " << point.call << " -> "
<< point.directory << "\n"; << point.directory << "\n";
} }
for (const Texture2dDumpPoint &point : gTexture2dDumpPoints) {
std::cerr << "MOBILEGL_TRACE_TEXTURE_2D_DUMP: armed for call " << point.call
<< " texture " << point.texture << " level " << point.level << " -> "
<< point.directory << "\n";
}
} }
} // namespace mobilegl_trace_dump } // namespace mobilegl_trace_dump
+4 -3
View File
@@ -2,9 +2,10 @@
namespace mobilegl_trace_dump { namespace mobilegl_trace_dump {
// Installs the framebuffer-attachment dump hook when MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS // Installs the opt-in framebuffer-attachment and named-2D-texture dump hooks. The respective
// describes at least one dump point. Safe and cheap to call on every makeCurrent: the // environments are MOBILEGL_TRACE_DUMP_FBO_ATTACHMENTS and MOBILEGL_TRACE_DUMP_TEXTURE_2D.
// environment is consulted once and the hook is installed at most once. // Safe and cheap to call on every makeCurrent: the environment is consulted once and the hook is
// installed at most once.
void InstallIfRequested(); void InstallIfRequested();
} // namespace mobilegl_trace_dump } // namespace mobilegl_trace_dump