Compare commits

..
20 Commits
Author SHA1 Message Date
swung0x48 fc4cd980f2 [Fix] (DirectVulkan): bound image mutability so Adreno keeps UBWC compression
- Every storage-capable colour texture was created MUTABLE_FORMAT, and Adreno
  gives up bandwidth compression on an image that may be viewed as any format in
  its compatibility class. MC's main render target therefore ran uncompressed;
  in a fill-bound scene that is the whole frame budget. Measured on Adreno 650,
  MC 26.2, same scene and camera, device cooled to 38-40C before each run:
  65.3 -> 80.9 fps (+23.9%), GPU busy ~93% in both.
- VK_KHR_image_format_list (enabled when present) fixes it without giving up
  mutability: VkImageFormatListCreateInfo names the exact formats a view may
  use, so the driver can keep the image compressed. The set must be exhaustive
  or the result is undefined - for sampled views it is exactly what
  ResolveSampledImageViewFormat can return over the three numeric domains.
- glBindImageTexture may name any compatible format, which cannot be enumerated
  ahead of time, so a texture bound to an image unit gets no format list. That
  is what VK_IMAGE_USAGE_STORAGE_BIT becoming on-demand is for: it makes
  "unmarked" mean "will never receive an arbitrary-format storage view", which
  is what makes the list sound. Removing STORAGE is worth nothing on its own
  (65.4 fps, measured) - only the mutability bound pays.
- MarkStorageImageTexture runs over every collected image-unit texture before
  the probe loop in PrepareStorageImageTextures, because that loop stops at the
  first texture needing work and would leave the rest unmarked. The mark makes
  NeedsStorageImagePreparation report true, which is what ends the render pass,
  so the recreate lands outside it.
- storageUsageResolved separates "not upgraded yet" from "this format can never
  carry STORAGE", so a format whose optimalTilingFeatures lack STORAGE_IMAGE
  cannot ask for a recreate that will never happen. SyncTexture's cross-draw
  early-out also has to break on a pending upgrade or the recreate never runs.
- An upgrade recreates the image and carries its contents forward through
  PreserveTextureContentsOnRecreate, which submits its own command buffer and
  waits. Whatever the frame already recorded into the old image is still
  unsubmitted, so that copy would read pre-frame content and this frame's
  rendering into the texture would be lost - exactly the render-target-then-
  image-unit case. PrepareStorageImageTextures now flushes first; it takes the
  FrameData rather than a command buffer because the flush retires the current
  one, and drops the sampled-descriptor-set memo that described it.
2026-07-29 07:07:50 -04:00
swung0x48 992d16267c [Fix] (DirectVulkan): rewrite implicit-LOD fragment samples to explicit LOD 0 when every bound sampler is pinned to a single mip level - Adreno 650 (driver 512.502) reads outside a full-screen colour render target's allocation on its implicit-LOD sampling path and faults the GPU, which killed MC 26.2 on its own blit shader (texture(InSampler, texCoord)) between frames 344-421 on every run; this is the same driver defect the default-framebuffer blit shader already works around with textureLod, but an application's shader cannot be edited, so ForceExplicitLod0SamplePass converts OpImageSample*ImplicitLod to the explicit form at the SPIR-V level under a new CompileOptionBit that is only requested when the rewrite provably cannot move a texel (every sampler binding on a single-level view, no anisotropy, and either a LOD clamp that already pins lambda at 0 or min and mag filters that agree - an explicit LOD 0 always takes the magnification side of the min/mag decision); a single-level view now also clamps its sampler to mipmapMode NEAREST with maxLod min(maxLod, 0.25) rather than 0, since collapsing the clamp would make every fragment magnify and quietly retire the min filter; and the program's backend hash memo grows from one slot to four so a program resolved under two compile-flag sets in the same frame stops re-hashing every stage's SPIR-V once per draw 2026-07-29 03:26:57 -04:00
swung0x48 0ea9e6de5f [Fix] (DirectVulkan): follow surface resizes instead of rebuilding the swapchain on VK_SUBOPTIMAL_KHR - a per-frame surface-capabilities comparison (ANGLE's model) is now the only thing that schedules a rebuild, so a launcher-side resolution change reaches the swapchain and the compositor scales the smaller image up to the view, while a driver that merely reports the surface as suboptimal can no longer rebuild every frame (each rebuild destroys every pipeline, resets the render-pass manager and reallocates the default framebuffer, which showed as flicker, then corruption, then a crash); the comparison runs in SURFACE space against the extent the live swapchain was created from, since comparing against the swapchain's own quarter-turn-swapped extent reports a difference on every rotated frame 2026-07-28 21:06:57 -04:00
swung0x48 241ed377b4 [Fix] (macOS): harden Cocoa context setup and isolate embedded glslang 2026-07-28 11:54:50 -04:00
swung0x48 bf312a4b67 [Fix] (DirectVulkan): explicit-LOD blit sampling and present-path hardening - the default-framebuffer blit shader now samples with textureLod 0 (a blit reads exactly the selected level; Adreno 650's implicit-LOD path reads past a single-mip UBWC render target's allocation despite maxLod=0, page-faulting the GPU on MC 26.2's second startup frame once the neighbouring startup staging memory is returned - the invalidated context then failed the next Present submit with EDEADLK/DEVICE_LOST), TransitionToPresent appends the present barrier into the frame's open recording instead of silently dropping it whenever anything was recorded (frames without a default-FBO render pass presented images stuck in their acquired layout), VK_SUBOPTIMAL_KHR acquires are treated as the success they are (image acquired, semaphore signal armed - the early return skipped the fence reset and consumed-flag clear, and callers re-acquired on the same binary semaphore; rebuilds now defer to after the signal is consumed), and validation builds report through VK_EXT_debug_report when VK_EXT_debug_utils is absent instead of aborting instance creation 2026-07-28 06:00:20 -04:00
swung0x48 56b31a9587 [Fix] (FastSTL): bump submodule for the erase(iterator) double-advance fix and add erase-while-iterating regression tests - the old semantics skipped one live element per erase and ran past end() when erasing the highest occupied bucket, sending the new mass pipeline-cache eviction sweeps off the bucket array (device crash on first eviction during world load: garbage handles fed to vkDestroyPipeline) 2026-07-27 23:50:25 -04:00
swung0x48 8a0a8a0274 [Fix] (DirectVulkan): harden the leak-fix round after adversarial review - pipeline memo now drops at every command-buffer boundary (a flush-loop-memoized pipeline could age out and be destroyed while its submission was in flight), mid-frame drains no longer rewind the arena or advance the cache-aging clocks in presenting apps (gated to every 8th drain since the last Present, so readback/fence-heavy frames neither churn conversions nor shrink the 1024-boundary retire window), render-pass eviction notifies the pipeline cache once per sweep batch instead of once per dying pass, descriptor pools use FREE_DESCRIPTOR_SET_BIT so a destroyed layout's cached sets are freed back and credited instead of abandoning pool slots (the live-layout age sweep that could orphan slots is removed - layout destruction is the sole purge path), and renderbuffer respecify parks the old backing for aged destruction instead of destroying it while possibly in flight 2026-07-27 22:51:26 -04:00
swung0x48 d076c29146 [Fix] (DirectVulkan): bound the vertex-input and sampler caches and sweep undeleted GL syncs - both caches age out entries idle >1024 frame boundaries (animated LOD bias no longer mints a VkSampler per float value, buffer/VAO churn no longer grows the vertex-input map for the whole session), and library teardown drains the live-sync registry exactly as glDeleteSync would since GL requires syncs to die with their context 2026-07-27 22:16:16 -04:00
swung0x48 930a607bdf [Fix] (DirectVulkan): make texture/renderbuffer GC reach every dead resource - name-deleted textures register via weak_from_this so first-sync-after-delete can no longer orphan a TextureResource, an orphan sweep makes GC authoritative over the resource map, dead-texture pruning moves to a frame-boundary gate (64 frames) so churn through clears/readbacks reclaims without draws, and dead renderbuffers age past frames-in-flight before their VkImage/view is destroyed instead of leaking until shutdown (or being freed while in flight) 2026-07-27 22:16:15 -04:00
swung0x48 34685b4bb0 [Fix] (DirectVulkan): age-based eviction for the content-addressed cache family - ProgramFactory entries (shader modules/layouts), PipelineFactory graphics pipelines, compute pipelines and per-layout descriptor-set tracking now retire after ~1024 idle frame boundaries (render-pass-manager sweep precedent), render-pass eviction purges pipelines hashed on the dying handle (closes a handle-recycling stale-pipeline hazard), and the program reflection cache is lifetime-id-keyed and cleared at EGL teardown - shader/program churn no longer grows Vulkan objects without bound 2026-07-27 22:05:25 -04:00
swung0x48 c540fb88ee [Fix] (DirectVulkan): drain frame transients on present-less paths - readback waits, suspended presentation, blocking sync waits and flush completion polls now run Present's per-frame drains (deferred buffer/texture releases, transient arena rewind, descriptor cursors, retired command buffers, conversion caches) whenever every submission is provably complete, so offscreen/minimized workloads stay bounded; never blocks, frames-in-flight overlap untouched 2026-07-27 21:45:21 -04:00
swung0x48 6ae3245a0d [Test] (CTS): raise the no-output abort threshold - consecutive instant-crash cases are real progress once device liveness is confirmed 2026-07-26 19:23:05 -04:00
swung0x48 7e048fc2bf [Fix] (DirectVulkan): map RGB10_A2(UI) to A2B10G10R10 - GL 2_10_10_10_REV puts R in bits 0-9 so the A2R10G10B10 mapping silently swapped R/B on upload; also decode both 1010102 variants in readback 2026-07-26 19:13:46 -04:00
swung0x48 83cdfd6bdd [Fix] (DirectVulkan): GetTexImage reads all 3D slices/array layers with PACK_IMAGE_HEIGHT/SKIP_IMAGES semantics, and sRGB readback returns raw sRGB-encoded bytes instead of linearizing 2026-07-26 18:30:43 -04:00
swung0x48 1c76f886cf [Fix] (DirectVulkan): back legacy low-bit formats (RGB565/RGB5A1/RGBA4/R3G3B2/RGB4/RGBA2/RGB10/12) with their UNorm8/16 canonical shadow layouts and add capability fallbacks - they mapped to VK_FORMAT_UNDEFINED and crashed or wedged the GPU on upload; also admit 2DMSArray/CubeMap/3D color attachment targets in the render pass 2026-07-26 18:30:42 -04:00
swung0x48 a2e109beff [Fix] (DirectVulkan): general (format,type) readback conversion - hoist the CTS-verified StoreWideRowsToClient into shared ReadbackImpl and decode any color VkFormat to wide RGBA rows; readback previously supported only RGB/BGR/RGBA/BGRA x UNSIGNED_BYTE/FLOAT and silently returned zeros for everything else 2026-07-26 18:30:41 -04:00
swung0x48 63f0756644 [Fix] (DirectVulkan): support UBO instance arrays as arrayed descriptors - uniform Block{...}b[N] reflected as one binding with descriptorCount=N, per-element GL block mapping, per-element buffer infos and dynamic offsets; non-UBO descriptor arrays now fail program creation cleanly instead of continuing corrupt 2026-07-26 18:30:41 -04:00
swung0x48 450215d12c [Fix] (DirectVulkan): implement color renderbuffer attachments - render pass/pipeline/blit/copy/readback/clear paths treated color renderbuffers as absent (writes masked to VK_ATTACHMENT_UNUSED, glClear dropped, readback zeros) 2026-07-26 18:30:40 -04:00
swung0x48 3a9e520170 [Test] (CTS): isolate the DirectVulkan renderbuffer-FBO readback defect so the rest of KHR-GL33 can be measured 2026-07-26 18:30:39 -04:00
swung0x48 d2996ba1cf [Test] (CTS): run VK-GL-CTS KHR-GL33 against MobileGL on Android via a standalone glcts binary 2026-07-26 18:30:39 -04:00
59 changed files with 1988 additions and 1430 deletions
+14 -3
View File
@@ -194,9 +194,6 @@ set(SOURCE_FILES
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/StripUboMemberRelaxedPrecisionPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/StripUboMemberRelaxedPrecisionPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/StripNoPerspectivePass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/StripNoPerspectivePass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EmulateNoPerspectivePass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EmulateNoPerspectivePass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FoldConstOffsetFor1DFetchPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/LowerClipDistanceForEsslPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DefeatConstStructArrayLutPass.cpp
MobileGL/MG_Util/BackendLoaders/OpenGL/Loader.cpp MobileGL/MG_Util/BackendLoaders/OpenGL/Loader.cpp
MobileGL/MG_Util/BackendLoaders/Vulkan/Loader.cpp MobileGL/MG_Util/BackendLoaders/Vulkan/Loader.cpp
@@ -458,8 +455,21 @@ if (ANDROID)
endif() endif()
if (APPLE AND NOT MOBILEGL_IOS) if (APPLE AND NOT MOBILEGL_IOS)
# MobileGL statically embeds glslang, SPIRV-Tools, and SPIRV-Cross. When
# this dylib is injected with DYLD_INSERT_LIBRARIES, exporting those C++
# symbols interposes incompatible copies embedded by host libraries such
# as shaderc. Keep only the public GL/EGL/CGL loader surface globally
# visible; GetProcAddress can still return pointers to hidden internals.
set(MOBILEGL_MACOS_EXPORTED_SYMBOLS
"${CMAKE_CURRENT_SOURCE_DIR}/MobileGL/MG_Impl/DyldInterpose/ExportedSymbols.txt")
target_link_options(${CMAKE_PROJECT_NAME} PRIVATE
"LINKER:-exported_symbols_list,${MOBILEGL_MACOS_EXPORTED_SYMBOLS}")
set_property(TARGET ${CMAKE_PROJECT_NAME} APPEND PROPERTY
LINK_DEPENDS "${MOBILEGL_MACOS_EXPORTED_SYMBOLS}")
target_link_libraries(${CMAKE_PROJECT_NAME} PUBLIC target_link_libraries(${CMAKE_PROJECT_NAME} PUBLIC
"-framework Cocoa" "-framework Cocoa"
"-framework CoreVideo"
"-framework QuartzCore" "-framework QuartzCore"
"-framework Foundation" "-framework Foundation"
"-framework OpenGL" "-framework OpenGL"
@@ -467,6 +477,7 @@ if (APPLE AND NOT MOBILEGL_IOS)
if(TARGET ${CMAKE_PROJECT_NAME}_s) if(TARGET ${CMAKE_PROJECT_NAME}_s)
target_link_libraries(${CMAKE_PROJECT_NAME}_s PUBLIC target_link_libraries(${CMAKE_PROJECT_NAME}_s PUBLIC
"-framework Cocoa" "-framework Cocoa"
"-framework CoreVideo"
"-framework QuartzCore" "-framework QuartzCore"
"-framework Foundation" "-framework Foundation"
"-framework OpenGL" "-framework OpenGL"
-5
View File
@@ -80,11 +80,6 @@ namespace MobileGL::MG_Config {
// rewrites the recognized workgroup prefix-scan template on Qualcomm devices with // rewrites the recognized workgroup prefix-scan template on Qualcomm devices with
// subgroups wider than 32 lanes (see ShaderSourceProcessor's quirk registry). // subgroups wider than 32 lanes (see ShaderSourceProcessor's quirk registry).
QuirkOverride SubgroupPrefixScanQuirk = QuirkOverride::Auto; QuirkOverride SubgroupPrefixScanQuirk = QuirkOverride::Auto;
// MOBILEGL_QUIRK_CLIP_DISTANCE: overrides the DirectGLES quirk that lowers
// gl_ClipDistance for Adreno's ESSL compiler (shadow Private arrays with
// constant-index builtin flushes, dynamic-index gl_in copy loop, redeclaration
// strip, and const struct-array LUT splitting). Auto detects Qualcomm.
QuirkOverride ClipDistanceQuirk = QuirkOverride::Auto;
// MOBILEGL_MAGMA_DISABLE_BLENDED_DEPTH_WRITE: overrides the DirectVulkan quirk that // MOBILEGL_MAGMA_DISABLE_BLENDED_DEPTH_WRITE: overrides the DirectVulkan quirk that
// strips depth writes from accumulation-blended pipelines (MIN/MAX or additive // strips depth writes from accumulation-blended pipelines (MIN/MAX or additive
// ONE+ONE - the multi-pass depth-equality signature) on drivers without // ONE+ONE - the multi-pass depth-equality signature) on drivers without
-1
View File
@@ -135,7 +135,6 @@ namespace MobileGL::MG_ConfigLoader {
features.DisableUboRing = QueryEnvFlag("MOBILEGL_DISABLE_UBO_RING"); features.DisableUboRing = QueryEnvFlag("MOBILEGL_DISABLE_UBO_RING");
features.RelaxedSemantics = QueryEnvFlag("MOBILEGL_RELAXED_SEMANTICS"); features.RelaxedSemantics = QueryEnvFlag("MOBILEGL_RELAXED_SEMANTICS");
features.SubgroupPrefixScanQuirk = QueryEnvQuirkOverride("MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN"); features.SubgroupPrefixScanQuirk = QueryEnvQuirkOverride("MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN");
features.ClipDistanceQuirk = QueryEnvQuirkOverride("MOBILEGL_QUIRK_CLIP_DISTANCE");
features.MagmaDisableBlendedDepthWriteQuirk = features.MagmaDisableBlendedDepthWriteQuirk =
QueryEnvQuirkOverride("MOBILEGL_MAGMA_DISABLE_BLENDED_DEPTH_WRITE"); QueryEnvQuirkOverride("MOBILEGL_MAGMA_DISABLE_BLENDED_DEPTH_WRITE");
features.DisableRobustBufferAccess = QueryEnvFlag("MOBILEGL_DISABLE_ROBUST_BUFFER_ACCESS"); features.DisableRobustBufferAccess = QueryEnvFlag("MOBILEGL_DISABLE_ROBUST_BUFFER_ACCESS");
+13 -4
View File
@@ -14,6 +14,7 @@
#include <MG_State/EGLState/Core.h> #include <MG_State/EGLState/Core.h>
#include <MG_Impl/GLImpl/Texture/ProxyTexture.h> #include <MG_Impl/GLImpl/Texture/ProxyTexture.h>
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h> #include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <atomic> #include <atomic>
#include <mutex> #include <mutex>
@@ -37,6 +38,12 @@ namespace MobileGL {
MGLOG_I("MobileGL closing..."); MGLOG_I("MobileGL closing...");
} }
glslang::FinalizeProcess(); glslang::FinalizeProcess();
// GL syncs die with their contexts, and every context is gone by the
// time full teardown runs: drain the live-sync registry while the
// backend function table can still release the backend handles (and
// before a re-initialized library could pair them with the wrong
// backend's DeleteSync).
MG_Impl::GLImpl::DestroyAllSyncObjects();
MG_Backend::pActiveBackendObject.reset(); MG_Backend::pActiveBackendObject.reset();
MG_State::pGLContext.reset(); MG_State::pGLContext.reset();
MG_State::pEGLContext.reset(); MG_State::pEGLContext.reset();
@@ -100,9 +107,11 @@ namespace MobileGL {
// (EGL/WGL/CGL): initialization happens lazily on the first entry point // (EGL/WGL/CGL): initialization happens lazily on the first entry point
// via EnsureInitialized(), and full teardown happens deterministically // via EnsureInitialized(), and full teardown happens deterministically
// when the last EGL display is terminated with nothing current (EGLImpl // when the last EGL display is terminated with nothing current (EGLImpl
// calls Destroy()). There is intentionally no static constructor, no // calls Destroy()). There is intentionally no backend-initializing static
// static destructor, and no DllMain: the global singletons use // constructor, no static destructor, and no DllMain: the global singletons
// leak-at-exit storage (see GlobalObjects.cpp), so a process that exits // use leak-at-exit storage (see GlobalObjects.cpp), so a process that exits
// without eglTerminate simply leaks them to the OS instead of running // without eglTerminate simply leaks them to the OS instead of running
// backend destructors during static teardown. // backend destructors during static teardown. macOS has a lightweight
// dyld constructor that installs NSOpenGL dispatch hooks only; full backend
// initialization still enters here from the first hooked CGL context.
} // namespace MobileGL } // namespace MobileGL
+4 -3
View File
@@ -13,9 +13,10 @@ namespace MobileGL {
void Initialize(); void Initialize();
// Thread-safe, idempotent, and re-entrant wrapper around Initialize(). // Thread-safe, idempotent, and re-entrant wrapper around Initialize().
// Host layers (EGL/WGL/CGL entry points) call this lazily on first use so // Host layers (EGL/WGL/CGL entry points) call this lazily on first use so
// MobileGL's lifecycle never depends on ELF/DLL static constructors, and // full backend initialization never depends on ELF/DLL static constructors,
// so a fresh init can follow a full Destroy() (e.g. after the last // and so a fresh init can follow a full Destroy() (e.g. after the last
// eglTerminate). // eglTerminate). The macOS dyld bootstrap installs only lightweight
// NSOpenGL method hooks.
void EnsureInitialized(); void EnsureInitialized();
void Destroy(); void Destroy();
@@ -449,20 +449,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
} }
} }
// Textures attached only to the READ framebuffer (blit / ReadPixels sources) need
// their content synced too, or the backend reads stale texel data.
const auto& readFBO =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Read).GetBoundObject();
if (readFBO && readFBO != currentFBO) {
for (const auto& attachment : readFBO->GetAllAttachmentObjects()) {
if (!attachment.IsTexture()) continue;
auto& textureObject = attachment.GetTexture();
if (textureObject) {
SyncTextureObjectToBackend(textureObject);
}
}
}
} }
static Bool SupportsLayeredImageBinding(TextureTarget target) { static Bool SupportsLayeredImageBinding(TextureTarget target) {
+2 -104
View File
@@ -1262,31 +1262,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
const auto& allAttributes = stateVAOObject->GetAllAttributes(); const auto& allAttributes = stateVAOObject->GetAllAttributes();
for (Uint attribIndex = 0; attribIndex < allAttributes.size(); ++attribIndex) { for (Uint attribIndex = 0; attribIndex < allAttributes.size(); ++attribIndex) {
const auto& attrib = allAttributes[attribIndex]; const auto& attrib = allAttributes[attribIndex];
const Uint32 attribBit = 1u << attribIndex;
// An enabled attrib with neither a buffer object nor a client pointer has no
// source; GL tolerates the state (only draws consuming it are undefined), but
// Adreno's ES driver memcpys the "client array" from address 0 at draw time
// (SIGSEGV). Keep such attribs disabled on the backend VAO and re-enable them
// the moment they gain a source - the mask-vs-current compare below triggers
// the enable even when only the Buffer/Format versions changed.
const Bool unsourceable = attrib.Enabled && !attrib.Buffer && attrib.Offset == 0;
const Bool wasForceDisabled = (m_forceDisabledAttribsMask & attribBit) != 0;
Bool needsSyncSwitch = allAttributeVersions[attribIndex].SwitchVersion != Bool needsSyncSwitch = allAttributeVersions[attribIndex].SwitchVersion !=
m_syncedAttributeVersions[attribIndex].SwitchVersion; m_syncedAttributeVersions[attribIndex].SwitchVersion;
if (needsSyncSwitch || unsourceable != wasForceDisabled) { if (needsSyncSwitch) {
if (attrib.Enabled && !unsourceable) { if (attrib.Enabled) {
g_GLESFuncs.glEnableVertexAttribArray(attribIndex); g_GLESFuncs.glEnableVertexAttribArray(attribIndex);
} else { } else {
g_GLESFuncs.glDisableVertexAttribArray(attribIndex); g_GLESFuncs.glDisableVertexAttribArray(attribIndex);
} }
} }
if (unsourceable) {
m_forceDisabledAttribsMask |= attribBit;
} else {
m_forceDisabledAttribsMask &= ~attribBit;
}
Bool needsSyncFormat = allAttributeVersions[attribIndex].FormatVersion != Bool needsSyncFormat = allAttributeVersions[attribIndex].FormatVersion !=
m_syncedAttributeVersions[attribIndex].FormatVersion; m_syncedAttributeVersions[attribIndex].FormatVersion;
@@ -1294,17 +1278,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_syncedAttributeVersions[attribIndex].BufferVersion; m_syncedAttributeVersions[attribIndex].BufferVersion;
if (!needsSyncFormat && !needsSyncBuffer) continue; if (!needsSyncFormat && !needsSyncBuffer) continue;
if (unsourceable) continue;
// Client-side array with a non-null pointer: the pointer is uploaded and applied
// per draw by SyncClientSideAttributesForDrawArrays.
if (!attrib.Buffer) continue;
if (!BindAttributeBuffer(attrib)) { if (!BindAttributeBuffer(attrib)) {
if (attrib.Enabled) {
g_GLESFuncs.glDisableVertexAttribArray(attribIndex);
m_forceDisabledAttribsMask |= attribBit;
}
continue; continue;
} }
@@ -3291,19 +3265,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
auto& shaderSpirvs = stateProgramObject->GetGeneratedSpirv(); auto& shaderSpirvs = stateProgramObject->GetGeneratedSpirv();
// Adreno's ESSL compiler mishandles gl_ClipDistance (rejects redeclarations,
// miscompiles non-constant-index writes and constant-index gl_in element reads,
// crashes on whole-array gl_in reads) and cannot dynamically index the global
// const struct[] LUTs SPIRV-Cross likes to emit. Gate the workarounds to
// Qualcomm; MOBILEGL_QUIRK_CLIP_DISTANCE overrides the device detection.
const MG_Config::QuirkOverride clipDistanceQuirkOverride =
MG_Config::Features.ClipDistanceQuirk;
const Bool applyClipDistanceQuirk =
clipDistanceQuirkOverride == MG_Config::QuirkOverride::ForceOn ||
(clipDistanceQuirkOverride == MG_Config::QuirkOverride::Auto &&
pActiveBackendObject &&
pActiveBackendObject->GetDynamicParameters().GpuVendor == GpuVendorKind::Qualcomm);
for (int index = 0; index < attachedShaders.size(); ++index) { for (int index = 0; index < attachedShaders.size(); ++index) {
auto& shader = attachedShaders[index]; auto& shader = attachedShaders[index];
GLenum glShaderType = MG_Util::ConvertShaderStageToGLEnum(shader->GetShaderStage()); GLenum glShaderType = MG_Util::ConvertShaderStageToGLEnum(shader->GetShaderStage());
@@ -3326,63 +3287,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
effectiveSpirv = &loweredSpirv; effectiveSpirv = &loweredSpirv;
} }
// GL 3.3 only promises undefined *values* for out-of-bounds array indexing, but
// Adreno's ESSL compiler constant-folds a provably out-of-bounds local-array
// index into poison that corrupts the whole shader's output. Clamp every
// access-chain index to its declared bounds before transpiling.
Vector<unsigned int> clampedSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::ClampAccessChainIndicesForEssl(*effectiveSpirv,
clampedSpirv) &&
!clampedSpirv.empty()) {
effectiveSpirv = &clampedSpirv;
} else {
MGLOG_W("ClampAccessChainIndicesForEssl failed, continuing with unclamped SPIR-V.");
}
// SPIRV-Cross emulates 1D samplers as 2D for ES: it widens texelFetch coordinates
// to ivec2 but keeps the ConstOffset operand scalar, which is not a valid ESSL
// texelFetchOffset overload (Adreno rejects it). Fold the constant offset into the
// coordinate instead (texelFetchOffset(t,P,l,o) == texelFetch(t,P+o,l)).
Vector<unsigned int> foldedOffsetSpirv;
if (MG_Util::ShaderTranspiler::ShaderCompiler::FoldConstOffsetFor1DFetchForEssl(
*effectiveSpirv, foldedOffsetSpirv) &&
!foldedOffsetSpirv.empty()) {
effectiveSpirv = &foldedOffsetSpirv;
} else {
MGLOG_W("FoldConstOffsetFor1DFetchForEssl failed, continuing with unfolded SPIR-V.");
}
// Adreno quirk: shadow gl_ClipDistance in Private arrays so the transpiled
// ESSL only writes the builtin with literal constant indices (flushed before
// EmitVertex/return) and only reads gl_in clip distances through dynamic loop
// indices - the shapes this driver compiles correctly. Must run after the
// access-chain clamp above so the flush indices stay literal constants.
Vector<unsigned int> clipDistanceSpirv;
if (applyClipDistanceQuirk &&
(glShaderType == GL_VERTEX_SHADER || glShaderType == GL_GEOMETRY_SHADER)) {
if (MG_Util::ShaderTranspiler::ShaderCompiler::LowerClipDistanceForEssl(
*effectiveSpirv, clipDistanceSpirv) &&
!clipDistanceSpirv.empty()) {
effectiveSpirv = &clipDistanceSpirv;
} else {
MGLOG_W("LowerClipDistanceForEssl failed, continuing with unlowered SPIR-V.");
}
}
// Adreno quirk: split single constant-composite stores of struct arrays so
// SPIRV-Cross does not promote them to global const struct[] LUTs, which this
// driver cannot dynamically index ("Cannot offset into the structure").
Vector<unsigned int> structLutSpirv;
if (applyClipDistanceQuirk) {
if (MG_Util::ShaderTranspiler::ShaderCompiler::DefeatConstStructArrayLutForEssl(
*effectiveSpirv, structLutSpirv) &&
!structLutSpirv.empty()) {
effectiveSpirv = &structLutSpirv;
} else {
MGLOG_W("DefeatConstStructArrayLutForEssl failed, continuing with unsplit SPIR-V.");
}
}
// ESSL stage-matches uniform blocks by member precision, but SPIRV-Cross prints // ESSL stage-matches uniform blocks by member precision, but SPIRV-Cross prints
// a RelaxedPrecision member as explicit "mediump" in the vertex stage and as // a RelaxedPrecision member as explicit "mediump" in the vertex stage and as
// UNQUALIFIED (mediump-by-default) in the fragment stage; after // UNQUALIFIED (mediump-by-default) in the fragment stage; after
@@ -3441,12 +3345,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
source = RebindImageUniformsToFrontendUnits(std::move(source), stateProgramObject); source = RebindImageUniformsToFrontendUnits(std::move(source), stateProgramObject);
source = RemoveLayoutBinding(source); source = RemoveLayoutBinding(source);
if (applyClipDistanceQuirk) {
// Adreno rejects the gl_ClipDistance redeclaration SPIRV-Cross still emits
// ("reserved built-in name") but accepts plain usage with
// GL_EXT_clip_cull_distance required; drop the line, keep the #extension.
source = RemoveClipDistanceRedeclaration(source);
}
source = ProcessOutColorLocations(source); source = ProcessOutColorLocations(source);
source = ForceFlatIntegerVaryings(source, glShaderType); source = ForceFlatIntegerVaryings(source, glShaderType);
source = EmulateBaseInstanceInVertexShader(std::move(source), glShaderType); source = EmulateBaseInstanceInVertexShader(std::move(source), glShaderType);
@@ -258,11 +258,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
private: private:
Uint m_backendVAOId = 0; Uint m_backendVAOId = 0;
Array<Uint, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> m_clientAttributeBufferIds; Array<Uint, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> m_clientAttributeBufferIds;
// Attribs the frontend has Enabled but that have no source at all (no buffer object
// and NULL client pointer). GL keeps such attribs latently enabled, but Adreno's ES
// driver treats them as client arrays and memcpys from address 0 at draw time
// (SIGSEGV), so they are kept disabled on the backend VAO until they gain a source.
Uint32 m_forceDisabledAttribsMask = 0;
Bool m_isInitialized = false; Bool m_isInitialized = false;
Uint16 m_syncedIndexBufferVersion = 0; Uint16 m_syncedIndexBufferVersion = 0;
Array<MG_State::GLState::VertexAttributeVersion, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> Array<MG_State::GLState::VertexAttributeVersion, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS>
-35
View File
@@ -342,41 +342,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
return result; return result;
} }
String RemoveClipDistanceRedeclaration(const String& glslCode) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
// Adreno rejects any redeclaration of gl_ClipDistance/gl_CullDistance ("reserved
// built-in name") even with GL_EXT_clip_cull_distance required, but accepts plain
// usage of the builtin. Drop the desktop-style redeclaration line SPIRV-Cross
// prints; the "#extension GL_EXT_clip_cull_distance : require" line stays.
static const std::regex redeclarationRegex(
R"(^\s*(?:out|in)\s+(?:(?:high|medium|low)p\s+)?float\s+gl_(?:Clip|Cull)Distance\[[0-9]+\];\s*$)");
String result;
result.reserve(glslCode.size());
SizeT lineStart = 0;
Bool firstLine = true;
while (lineStart <= glslCode.size()) {
SizeT lineEnd = glslCode.find('\n', lineStart);
const Bool lastLine = lineEnd == String::npos;
String line = glslCode.substr(lineStart, lastLine ? String::npos : lineEnd - lineStart);
if (!std::regex_match(line, redeclarationRegex)) {
if (!firstLine) {
result += '\n';
}
result += line;
firstLine = false;
}
if (lastLine) {
break;
}
lineStart = lineEnd + 1;
}
return result;
}
} // namespace PrgramImpl } // namespace PrgramImpl
namespace Utils { namespace Utils {
-1
View File
@@ -105,7 +105,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
Uint32 unormOutputMask); Uint32 unormOutputMask);
String ForceFlatIntegerVaryings(const String& glslCode, GLenum shaderType); String ForceFlatIntegerVaryings(const String& glslCode, GLenum shaderType);
String RemoveLayoutBinding(const String& glslCode); String RemoveLayoutBinding(const String& glslCode);
String RemoveClipDistanceRedeclaration(const String& glslCode);
} // namespace PrgramImpl } // namespace PrgramImpl
namespace Utils { namespace Utils {
@@ -469,6 +469,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// treat them as signaled/available with zero results from here on. // treat them as signaled/available with zero results from here on.
BumpRendererGeneration(); BumpRendererGeneration();
pVulkanRenderer.reset(); pVulkanRenderer.reset();
// The reflection cache is file-scope, not renderer-owned; without this the
// deleted programs' reflection strings survive full context teardown.
ClearProgramResourceCaches();
BackendObject::ReleaseEGLResources(); BackendObject::ReleaseEGLResources();
} }
@@ -478,6 +481,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// treat them as signaled/available with zero results from here on. // treat them as signaled/available with zero results from here on.
BumpRendererGeneration(); BumpRendererGeneration();
pVulkanRenderer.reset(); pVulkanRenderer.reset();
// The reflection cache is file-scope, not renderer-owned; without this the
// deleted programs' reflection strings survive full context teardown.
ClearProgramResourceCaches();
} }
const RendererInfo& BackendObject_DirectVulkan::GetRendererInfo() const { const RendererInfo& BackendObject_DirectVulkan::GetRendererInfo() const {
@@ -61,6 +61,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}; };
struct ProgramResourceCache { struct ProgramResourceCache {
// Lifetime id of the program the cached reflection belongs to. GL names are
// recycled (IndexGenerator hands freed indices straight back), and a
// recreated program's backendStateVersion restarts at the same small values,
// so the version alone can collide; the never-reused lifetime id makes the
// slot's ownership unambiguous.
Uint64 programLifetimeId = 0;
Uint32 backendStateVersion = 0; Uint32 backendStateVersion = 0;
Vector<StorageBlockResource> storageBlocks; Vector<StorageBlockResource> storageBlocks;
Vector<BufferVariableResource> bufferVariables; Vector<BufferVariableResource> bufferVariables;
@@ -82,6 +88,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 baseInstance = 0; Uint32 baseInstance = 0;
}; };
// Keyed by GL program name so the freed-name reuse in IndexGenerator bounds the
// map at the peak-simultaneous-program high-water mark; each slot's ownership is
// checked against the program's lifetime id before it is served (see
// GetProgramResourceCache). Cleared wholesale at EGL teardown via
// ClearProgramResourceCaches.
UnorderedMap<GLuint, ProgramResourceCache> g_programResourceCaches; UnorderedMap<GLuint, ProgramResourceCache> g_programResourceCaches;
void ClearReadPixelsOutput(GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels) { void ClearReadPixelsOutput(GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels) {
@@ -142,13 +153,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ProgramResourceCache& GetProgramResourceCache(const MG_State::GLState::ProgramObject& program) { ProgramResourceCache& GetProgramResourceCache(const MG_State::GLState::ProgramObject& program) {
auto& cache = g_programResourceCaches[program.GetExternalIndex()]; auto& cache = g_programResourceCaches[program.GetExternalIndex()];
const Uint64 programLifetimeId = program.GetLifetimeId();
const Uint32 backendStateVersion = program.GetBackendStateVersion(); const Uint32 backendStateVersion = program.GetBackendStateVersion();
if (cache.backendStateVersion == backendStateVersion && // The lifetime id must match too: a new program that reuses a deleted
// program's name and happens to land on the same backendStateVersion (both
// count from zero) would otherwise be served the dead program's reflection.
if (cache.programLifetimeId == programLifetimeId &&
cache.backendStateVersion == backendStateVersion &&
(!cache.storageBlocks.empty() || !cache.bufferVariables.empty())) { (!cache.storageBlocks.empty() || !cache.bufferVariables.empty())) {
return cache; return cache;
} }
cache = {}; cache = {};
cache.programLifetimeId = programLifetimeId;
cache.backendStateVersion = backendStateVersion; cache.backendStateVersion = backendStateVersion;
Vector<SpvReflectShaderModule> modules; Vector<SpvReflectShaderModule> modules;
@@ -366,6 +383,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
} // namespace } // namespace
void ClearProgramResourceCaches() {
// Called from EGL teardown while the backend's m_eglStateMutex is held; GL
// calls are serialized in this codebase (contexts migrate threads but never
// run concurrently), so no other thread can be inside the unsynchronized map.
// Live programs in another context self-heal: their entry rebuilds from the
// retained generated SPIR-V on the next resource query.
g_programResourceCaches.clear();
}
GLuint GetShaderStorageBlockIndex(const MG_State::GLState::ProgramObject& program, const String& name) { GLuint GetShaderStorageBlockIndex(const MG_State::GLState::ProgramObject& program, const String& name) {
auto& cache = GetProgramResourceCache(program); auto& cache = GetProgramResourceCache(program);
const auto it = std::find_if(cache.storageBlocks.begin(), cache.storageBlocks.end(), const auto it = std::find_if(cache.storageBlocks.begin(), cache.storageBlocks.end(),
@@ -23,6 +23,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 GetRendererGeneration(); Uint64 GetRendererGeneration();
void BumpRendererGeneration(); void BumpRendererGeneration();
// Drops every cached program-resource reflection entry (CPU-side strings/vectors
// only, no Vulkan handles). Called at EGL teardown next to the renderer reset;
// safe because GL calls are serialized in this codebase, and any still-live
// program rebuilds its entry from the retained generated SPIR-V on demand.
void ClearProgramResourceCaches();
void ClearBufferfi(GLenum buffer, GLint drawbuffer, GLfloat depth, GLint stencil); void ClearBufferfi(GLenum buffer, GLint drawbuffer, GLfloat depth, GLint stencil);
void ClearBufferfv(GLenum buffer, GLint drawbuffer, const GLfloat* value); void ClearBufferfv(GLenum buffer, GLint drawbuffer, const GLfloat* value);
void ClearBufferuiv(GLenum buffer, GLint drawbuffer, const GLuint* value); void ClearBufferuiv(GLenum buffer, GLint drawbuffer, const GLuint* value);
@@ -150,12 +150,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool FrameContext::TransitionToPresent(VkImage image, VkImageLayout oldLayout, VkImageLayout presentLayout) { Bool FrameContext::TransitionToPresent(VkImage image, VkImageLayout oldLayout, VkImageLayout presentLayout) {
auto& frame = GetCurrent(); auto& frame = GetCurrent();
if (frame.hasCommandBufferRecorded || frame.isCommandRecording || oldLayout == presentLayout || if (oldLayout == presentLayout || oldLayout == VK_IMAGE_LAYOUT_SHARED_PRESENT_KHR) {
oldLayout == VK_IMAGE_LAYOUT_SHARED_PRESENT_KHR) {
return false; return false;
} }
auto& commandBuffer = BeginCommandRecording(); // The barrier belongs in the frame's own recording. Bailing out because
// something was already recorded (the previous behaviour) dropped the
// transition entirely for every frame that never ran a default-framebuffer
// render pass - the only other thing that carries the image to
// PRESENT_SRC_KHR, via that pass's finalLayout - so the swapchain image was
// handed to the WSI still in the layout it was acquired in.
// A closed-but-unsubmitted buffer can only come from a submit that already
// failed (SubmitPendingCommandBuffer leaves the flag set on error), and
// appending to it is illegal while reopening would reset the frame's own
// commands away. The device is gone on that path anyway - stay silent-safe
// rather than trade a lost device for a barrier into a closed buffer.
if (frame.hasCommandBufferRecorded) {
MGLOG_E("TransitionToPresent: command buffer already closed; skipping the present barrier");
return false;
}
// Reopening a recording here would vkResetCommandBuffer this frame's own
// commands away, so append to the open one and let the caller close it.
const Bool openedRecording = !frame.isCommandRecording;
VkCommandBuffer commandBuffer = openedRecording ? BeginCommandRecording() : frame.commandBuffer;
VkImageMemoryBarrier presentBarrier{}; VkImageMemoryBarrier presentBarrier{};
presentBarrier.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER; presentBarrier.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER;
@@ -174,7 +192,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, 0, vkCmdPipelineBarrier(commandBuffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, 0, 0,
nullptr, 0, nullptr, 1, &presentBarrier); nullptr, 0, nullptr, 1, &presentBarrier);
EndCommandRecording(); if (openedRecording) {
EndCommandRecording();
}
return true; return true;
} }
@@ -227,12 +247,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
result = vkAcquireNextImageKHR(device, swapchain, timeout, frame.imageAvailableSemaphore, acquireFence, result = vkAcquireNextImageKHR(device, swapchain, timeout, frame.imageAvailableSemaphore, acquireFence,
&outImageIndex); &outImageIndex);
if (result != VK_SUCCESS) { // VK_SUBOPTIMAL_KHR is a success code: an image *was* acquired and
// imageAvailableSemaphore *will* be signaled. Bailing out on it skipped both
// the consumed-flag reset (leaving a stale "already consumed", so the next
// submit never waited on the pending signal) and the fence reset (leaving
// the slot's fence signaled for the next submit to reuse). Only a genuine
// failure - VK_ERROR_OUT_OF_DATE_KHR and friends, where nothing is acquired
// and nothing is signaled - skips the bookkeeping.
if (result != VK_SUCCESS && result != VK_SUBOPTIMAL_KHR) {
return result; return result;
} }
frame.imageAvailableSemaphoreConsumed = false; frame.imageAvailableSemaphoreConsumed = false;
return vkResetFences(device, 1, &frame.imageInFlightFence); const VkResult resetResult = vkResetFences(device, 1, &frame.imageInFlightFence);
// Hand the acquire's own code back so the caller can schedule a rebuild.
return resetResult == VK_SUCCESS ? result : resetResult;
} }
Uint32 FrameContext::GetCurrentFrameIndex() const { Uint32 FrameContext::GetCurrentFrameIndex() const {
@@ -264,7 +293,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (result != VK_SUCCESS) { if (result != VK_SUCCESS) {
return result; return result;
} }
frame.retiredCommandBuffers.push_back(frame.commandBuffer); // lastSubmitIndex was just written by the renderer for the submission
// that carried this command buffer.
frame.retiredCommandBuffers.push_back({frame.commandBuffer, frame.lastSubmitIndex});
frame.commandBuffer = replacement; frame.commandBuffer = replacement;
return VK_SUCCESS; return VK_SUCCESS;
} }
@@ -274,12 +305,40 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return; return;
} }
if (m_device != VK_NULL_HANDLE && m_commandPool != VK_NULL_HANDLE) { if (m_device != VK_NULL_HANDLE && m_commandPool != VK_NULL_HANDLE) {
vkFreeCommandBuffers(m_device, m_commandPool, static_cast<Uint32>(frame.retiredCommandBuffers.size()), for (const auto& retired : frame.retiredCommandBuffers) {
frame.retiredCommandBuffers.data()); vkFreeCommandBuffers(m_device, m_commandPool, 1, &retired.commandBuffer);
}
} }
frame.retiredCommandBuffers.clear(); frame.retiredCommandBuffers.clear();
} }
void FrameContext::FreeRetiredCommandBuffersCompletedUpTo(Uint64 completedSubmitIndex) {
if (m_device == VK_NULL_HANDLE || m_commandPool == VK_NULL_HANDLE) {
return;
}
for (auto& frame : m_frames) {
// Retired buffers are appended in submit order, so the completed
// ones form a prefix.
SizeT completedCount = 0;
while (completedCount < frame.retiredCommandBuffers.size() &&
frame.retiredCommandBuffers[completedCount].submitIndex <= completedSubmitIndex) {
vkFreeCommandBuffers(m_device, m_commandPool, 1,
&frame.retiredCommandBuffers[completedCount].commandBuffer);
++completedCount;
}
if (completedCount > 0) {
frame.retiredCommandBuffers.erase(frame.retiredCommandBuffers.begin(),
frame.retiredCommandBuffers.begin() + completedCount);
}
}
}
void FrameContext::FreeAllRetiredCommandBuffers() {
for (auto& frame : m_frames) {
FreeRetiredCommandBuffers(frame);
}
}
void FrameContext::AssertValidFrameIndex(Uint32 frameIndex) const { void FrameContext::AssertValidFrameIndex(Uint32 frameIndex) const {
MOBILEGL_ASSERT(frameIndex < m_frames.size(), "FrameContext index out of range"); MOBILEGL_ASSERT(frameIndex < m_frames.size(), "FrameContext index out of range");
} }
@@ -40,6 +40,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkPresentInfoKHR presentInfo{VK_STRUCTURE_TYPE_PRESENT_INFO_KHR}; VkPresentInfoKHR presentInfo{VK_STRUCTURE_TYPE_PRESENT_INFO_KHR};
}; };
// A command buffer submitted mid-frame (FlushPendingCommands), tagged
// with the submit-tracker index it was submitted under so it can be
// freed as soon as that submission is observed complete - without
// waiting for the slot's fence to be waited again (present-less flush
// loops never wait it).
struct RetiredCommandBuffer {
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
Uint64 submitIndex = 0;
};
struct FrameData { struct FrameData {
VkCommandBuffer commandBuffer = VK_NULL_HANDLE; VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
VkSemaphore imageAvailableSemaphore = VK_NULL_HANDLE; VkSemaphore imageAvailableSemaphore = VK_NULL_HANDLE;
@@ -47,10 +57,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool isCommandRecording = false; Bool isCommandRecording = false;
Bool hasCommandBufferRecorded = false; Bool hasCommandBufferRecorded = false;
Bool imageAvailableSemaphoreConsumed = false; Bool imageAvailableSemaphoreConsumed = false;
// Command buffers submitted mid-frame (FlushPendingCommands) whose // Command buffers submitted mid-frame (FlushPendingCommands),
// execution is only known complete once this slot's fence has been // appended in submit order; freed once their submission is known
// waited again; freed at that point. // complete (fence wait or completion poll).
Vector<VkCommandBuffer> retiredCommandBuffers; Vector<RetiredCommandBuffer> retiredCommandBuffers;
// Submit-tracker index of this slot's most recent queue submission // Submit-tracker index of this slot's most recent queue submission
// (written by the renderer at submit time). // (written by the renderer at submit time).
Uint64 lastSubmitIndex = 0; Uint64 lastSubmitIndex = 0;
@@ -79,9 +89,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Parks the current (already ended and submitted) command buffer on the // Parks the current (already ended and submitted) command buffer on the
// slot's retired list and installs a freshly allocated one, so recording // slot's retired list and installs a freshly allocated one, so recording
// can restart while the submitted buffer is still executing. Retired // can restart while the submitted buffer is still executing. Retired
// buffers are freed after the slot's fence is next waited. // buffers are freed after the slot's fence is next waited, or as soon
// as their submission is observed complete.
VkResult RetireCurrentCommandBuffer(); VkResult RetireCurrentCommandBuffer();
// Frees every retired command buffer whose tagged submission index is
// known complete. Driven by the renderer's submit tracker on completion
// events (fence waits and non-blocking polls), so present-less flush
// loops reclaim their buffers without any extra wait.
void FreeRetiredCommandBuffersCompletedUpTo(Uint64 completedSubmitIndex);
// Frees every slot's retired command buffers. Only valid when the
// caller has proven every queue submission complete.
void FreeAllRetiredCommandBuffers();
Uint32 GetCurrentFrameIndex() const; Uint32 GetCurrentFrameIndex() const;
Uint32 GetFrameCount() const; Uint32 GetFrameCount() const;
@@ -8,6 +8,7 @@
#include "PipelineFactory.h" #include "PipelineFactory.h"
#include <algorithm>
namespace MobileGL::MG_Backend::DirectVulkan { namespace MobileGL::MG_Backend::DirectVulkan {
static const char* PrimitiveTopologyToString(VkPrimitiveTopology topology) { static const char* PrimitiveTopologyToString(VkPrimitiveTopology topology) {
@@ -243,23 +244,108 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const HashType hash = ComputeHash(payload); const HashType hash = ComputeHash(payload);
auto it = m_cache.find(hash); auto it = m_cache.find(hash);
if (it != m_cache.end()) { if (it != m_cache.end()) {
return it->second; it->second.lastUsedFrame = m_frameCounter;
return it->second.pipeline;
} }
VkPipeline pipeline = CreatePipeline(payload); VkPipeline pipeline = CreatePipeline(payload);
m_cache.emplace(hash, pipeline); m_cache.emplace(hash, PipelineCacheEntry{pipeline, payload.programHash, payload.renderPass,
m_frameCounter});
return pipeline; return pipeline;
} }
void PipelineFactory::DestroyAll() { void PipelineFactory::DestroyAll() {
for (auto& pair : m_cache) { for (auto& pair : m_cache) {
if (pair.second != VK_NULL_HANDLE) { if (pair.second.pipeline != VK_NULL_HANDLE) {
vkDestroyPipeline(m_device, pair.second, nullptr); vkDestroyPipeline(m_device, pair.second.pipeline, nullptr);
} }
} }
m_cache.clear(); m_cache.clear();
} }
Uint32 PipelineFactory::OnFrameBoundary() {
++m_frameCounter;
// Sweep cadence and retire age mirror VkRenderPassManager::OnPresent: an entry
// idle for more than kRetireAgeFrames frame boundaries cannot be referenced by
// any in-flight command buffer (frames-in-flight <= MOBILEGL_MAGMA_FRAMESINFLIGHT),
// so immediate vkDestroyPipeline is safe. The caller must drop its "last
// pipeline" memo when this returns non-zero: the memo can return a cached
// handle without touching this cache, so an evicted pipeline may still be
// memoized (present-less flush loops never reset the memo per frame).
constexpr Uint64 kSweepInterval = 256;
constexpr Uint64 kRetireAgeFrames = 1024;
if ((m_frameCounter % kSweepInterval) != 0) {
return 0;
}
Uint32 evicted = 0;
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (m_frameCounter - it->second.lastUsedFrame > kRetireAgeFrames) {
if (it->second.pipeline != VK_NULL_HANDLE) {
vkDestroyPipeline(m_device, it->second.pipeline, nullptr);
}
it = m_cache.erase(it);
++evicted;
} else {
++it;
}
}
if (evicted > 0) {
MGLOG_D("PipelineFactory::OnFrameBoundary: evicted %u idle pipelines (%zu remain)", evicted,
m_cache.size());
}
return evicted;
}
Uint32 PipelineFactory::EvictByRenderPasses(const Vector<VkRenderPass>& renderPasses) {
if (renderPasses.empty() || m_cache.empty()) {
return 0;
}
// Sorted-batch membership test keeps a mass eviction (shader-pack switch,
// dimension exit) at one O(cache * log batch) scan instead of one full scan
// per dying pass.
Vector<VkRenderPass> sortedPasses = renderPasses;
std::sort(sortedPasses.begin(), sortedPasses.end());
Uint32 evicted = 0;
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (std::binary_search(sortedPasses.begin(), sortedPasses.end(), it->second.renderPass)) {
if (it->second.pipeline != VK_NULL_HANDLE) {
vkDestroyPipeline(m_device, it->second.pipeline, nullptr);
}
it = m_cache.erase(it);
++evicted;
} else {
++it;
}
}
if (evicted > 0) {
MGLOG_D("PipelineFactory::EvictByRenderPasses: evicted %u pipelines for %zu destroyed render passes",
evicted, sortedPasses.size());
}
return evicted;
}
Uint32 PipelineFactory::EvictByProgramHash(HashType programHash) {
Uint32 evicted = 0;
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (it->second.programHash == programHash) {
if (it->second.pipeline != VK_NULL_HANDLE) {
vkDestroyPipeline(m_device, it->second.pipeline, nullptr);
}
it = m_cache.erase(it);
++evicted;
} else {
++it;
}
}
if (evicted > 0) {
MGLOG_D("PipelineFactory::EvictByProgramHash: evicted %u pipelines for program hash 0x%llx",
evicted, static_cast<unsigned long long>(programHash));
}
return evicted;
}
VkPipeline PipelineFactory::CreatePipeline(const PipelineCreatePayload& payload) const { VkPipeline PipelineFactory::CreatePipeline(const PipelineCreatePayload& payload) const {
MOBILEGL_ASSERT(payload.stages != nullptr && !payload.stages->empty(), "PipelineFactory: stages are empty"); MOBILEGL_ASSERT(payload.stages != nullptr && !payload.stages->empty(), "PipelineFactory: stages are empty");
MOBILEGL_ASSERT(payload.vertexInputState != nullptr, "PipelineFactory: vertexInputState is null"); MOBILEGL_ASSERT(payload.vertexInputState != nullptr, "PipelineFactory: vertexInputState is null");
@@ -65,6 +65,26 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkPipeline GetOrCreatePipeline(const PipelineCreatePayload& payload); VkPipeline GetOrCreatePipeline(const PipelineCreatePayload& payload);
void DestroyAll(); void DestroyAll();
// Frame boundary hook: ages the pipeline cache and destroys long-unused entries
// (their command buffers retired many frames ago), mirroring
// VkRenderPassManager::OnPresent's sweep. Returns the number of pipelines
// destroyed so the caller can drop any memoized VkPipeline handle.
Uint32 OnFrameBoundary();
// Destroys every cached pipeline hashed on one of `renderPasses`. Only safe
// when the caller guarantees GPU idleness for them - the render-pass manager
// calls this (via the renderer) for passes its own >1024-boundary-idle sweep
// just evicted, and a pipeline hashed on those handles is only ever bound by
// draws that also hit the render-pass entries. Also closes the handle-recycling
// hazard: a recycled VkRenderPass value must never serve a stale pipeline.
// Batched: one cache scan regardless of how many passes died in the sweep.
// Returns the number destroyed (callers invalidate memos when non-zero).
Uint32 EvictByRenderPasses(const Vector<VkRenderPass>& renderPasses);
// Destroys every cached pipeline built from the program with content hash
// `programHash`. Called from the ProgramFactory eviction path, which proves the
// same >1024-boundary idleness (the program's pipelines are only bound by draws
// that stamp its factory entry). Returns the number destroyed.
Uint32 EvictByProgramHash(HashType programHash);
// Driver quirk: suppress depth writes on accumulation-blended pipelines. Multi-pass // Driver quirk: suppress depth writes on accumulation-blended pipelines. Multi-pass
// depth-equality rendering (a blended prepass writes depth that later passes re-test // depth-equality rendering (a blended prepass writes depth that later passes re-test
// with an equality-inclusive compare on the re-rasterized geometry) requires // with an equality-inclusive compare on the re-rasterized geometry) requires
@@ -87,12 +107,26 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static Bool ShouldSuppressDepthWrite(const PipelineCreatePayload& payload); static Bool ShouldSuppressDepthWrite(const PipelineCreatePayload& payload);
private: private:
struct PipelineCacheEntry {
VkPipeline pipeline = VK_NULL_HANDLE;
// The hashed inputs the eviction paths key on: programHash ties the entry to
// its ProgramFactory entry, renderPass records the exact handle the hash
// folded in (the hash is one-way, so targeted eviction needs them verbatim).
HashType programHash = 0;
VkRenderPass renderPass = VK_NULL_HANDLE;
// Frame-boundary counter value of the last GetOrCreatePipeline hit; drives
// cache eviction (see OnFrameBoundary).
Uint64 lastUsedFrame = 0;
};
VkPipeline CreatePipeline(const PipelineCreatePayload& payload) const; VkPipeline CreatePipeline(const PipelineCreatePayload& payload) const;
VkDevice m_device = VK_NULL_HANDLE; VkDevice m_device = VK_NULL_HANDLE;
const VulkanRendererConfig& m_config; const VulkanRendererConfig& m_config;
VkPipelineCache m_pipelineCache = VK_NULL_HANDLE; VkPipelineCache m_pipelineCache = VK_NULL_HANDLE;
UnorderedMap<HashType, VkPipeline> m_cache; UnorderedMap<HashType, PipelineCacheEntry> m_cache;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameCounter = 0;
static inline XXH64_state_t* m_hashState = XXH64_createState(); static inline XXH64_state_t* m_hashState = XXH64_createState();
static inline Bool s_suppressBlendedDepthWrite = false; static inline Bool s_suppressBlendedDepthWrite = false;
}; };
@@ -923,6 +923,206 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ProgramFactory::CompileOptionFlags m_transformFlags; ProgramFactory::CompileOptionFlags m_transformFlags;
}; };
// Adreno 650 (driver 512.502) faults the GPU on an implicit-LOD sample of a full-screen
// colour render target: the texture unit's derivative path reads outside the image's
// allocation even though the sampler clamps LOD to 0 and the mapping is 1:1. MobileGL's
// own default-framebuffer blit shader works around it with textureLod, but an
// application's shader (Minecraft's blit.fsh is `texture(InSampler, texCoord)`) cannot be
// edited - so rewrite the sample at the SPIR-V level instead.
//
// The rewrite is only requested for draws whose every sampler binding is clamped to one
// mip level, where explicit LOD 0 is exactly what the implicit form must already produce:
// lambda' = clamp(lambda + bias, minLod, maxLod) with minLod = maxLod = 0. Bias and MinLod
// operands are therefore dropped rather than translated.
class ForceExplicitLod0SamplePass final : public spvtools::opt::Pass {
public:
const char* name() const override { return "force-explicit-lod0-sample"; }
Status Process() override {
Bool isFragment = false;
for (auto& entryPoint : get_module()->entry_points()) {
if (entryPoint.opcode() != spv::Op::OpEntryPoint) continue;
if (static_cast<spv::ExecutionModel>(entryPoint.GetSingleWordInOperand(0)) ==
spv::ExecutionModel::Fragment) {
isFragment = true;
break;
}
}
if (!isFragment) return Status::SuccessWithoutChange;
// Plan first, mutate second. Materializing the LOD constant is itself a module
// change, so it must not happen unless at least one rewrite is going to follow -
// otherwise the pass would grow the binary while reporting SuccessWithoutChange.
Vector<RewritePlan> plans;
for (auto& function : *get_module()) {
for (auto& block : function) {
for (auto& inst : block) {
RewritePlan plan{};
if (PlanRewrite(&inst, plan)) plans.push_back(Move(plan));
}
}
}
if (plans.empty()) return Status::SuccessWithoutChange;
const Uint32 zeroId = GetFloatZeroId();
if (zeroId == 0) return Status::SuccessWithoutChange;
for (auto& plan : plans) {
plan.operands.push_back({SPV_OPERAND_TYPE_ID, {zeroId}});
for (auto& operand : plan.trailingOperands) {
plan.operands.push_back(operand);
}
plan.instruction->SetOpcode(plan.opcode);
plan.instruction->SetInOperands(Move(plan.operands));
}
// Opcodes and operand lists changed underneath every cached analysis.
context()->InvalidateAnalysesExceptFor(spvtools::opt::IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
private:
struct RewritePlan {
spvtools::opt::Instruction* instruction = nullptr;
spv::Op opcode = spv::Op::OpNop;
// Everything up to and including the Image Operands mask; the Lod id and the
// trailing operand values are appended once the constant exists.
Vector<spvtools::opt::Operand> operands;
Vector<spvtools::opt::Operand> trailingOperands;
};
// Image Operands bits that may accompany an implicit-LOD sample, in the canonical
// ascending order SPIR-V requires the operand values to appear in.
static constexpr Uint32 kBias = 0x1;
static constexpr Uint32 kLod = 0x2;
static constexpr Uint32 kGrad = 0x4;
static constexpr Uint32 kConstOffset = 0x8;
static constexpr Uint32 kOffset = 0x10;
static constexpr Uint32 kConstOffsets = 0x20;
static constexpr Uint32 kSample = 0x40;
static constexpr Uint32 kMinLod = 0x80;
static constexpr Uint32 kKnownMask = 0xFF;
Uint32 GetFloatZeroId() {
// Reuse a 32-bit float type already in the module; a shader that samples always has
// one, and looking it up avoids depending on type-creation API details.
Uint32 floatTypeId = 0;
for (auto& inst : get_module()->types_values()) {
if (inst.opcode() == spv::Op::OpTypeFloat && inst.NumInOperands() >= 1 &&
inst.GetSingleWordInOperand(0) == 32) {
floatTypeId = inst.result_id();
break;
}
}
if (floatTypeId == 0) return 0;
const auto* floatType = context()->get_type_mgr()->GetType(floatTypeId);
if (floatType == nullptr) return 0;
const auto zeroBits = std::bit_cast<Uint32>(0.0f);
const auto* zeroConst = context()->get_constant_mgr()->GetConstant(floatType, {zeroBits});
if (zeroConst == nullptr) return 0;
auto* zeroInst = context()->get_constant_mgr()->GetDefiningInstruction(zeroConst);
return zeroInst != nullptr ? zeroInst->result_id() : 0;
}
static Bool MapOpcode(spv::Op op, spv::Op& outOpcode, Uint32& outFixedOperandCount) {
switch (op) {
case spv::Op::OpImageSampleImplicitLod:
outOpcode = spv::Op::OpImageSampleExplicitLod;
outFixedOperandCount = 2; // sampled image, coordinate
return true;
case spv::Op::OpImageSampleProjImplicitLod:
outOpcode = spv::Op::OpImageSampleProjExplicitLod;
outFixedOperandCount = 2;
return true;
case spv::Op::OpImageSampleDrefImplicitLod:
outOpcode = spv::Op::OpImageSampleDrefExplicitLod;
outFixedOperandCount = 3; // sampled image, coordinate, Dref
return true;
case spv::Op::OpImageSampleProjDrefImplicitLod:
outOpcode = spv::Op::OpImageSampleProjDrefExplicitLod;
outFixedOperandCount = 3;
return true;
default:
return false;
}
}
static Bool PlanRewrite(spvtools::opt::Instruction* inst, RewritePlan& outPlan) {
spv::Op newOpcode = spv::Op::OpNop;
Uint32 fixedCount = 0;
if (!MapOpcode(inst->opcode(), newOpcode, fixedCount)) return false;
if (inst->NumInOperands() < fixedCount) return false;
Uint32 mask = 0;
Uint32 next = fixedCount;
if (inst->NumInOperands() > fixedCount) {
mask = inst->GetSingleWordInOperand(fixedCount);
next = fixedCount + 1;
}
// An operand this pass does not model would be silently reordered or dropped, and
// Grad cannot legally accompany an implicit-LOD sample: leave such an instruction be.
if ((mask & ~kKnownMask) != 0 || (mask & kGrad) != 0) return false;
Vector<spvtools::opt::Operand> fixedOperands;
fixedOperands.reserve(fixedCount + 1);
for (Uint32 i = 0; i < fixedCount; ++i) {
fixedOperands.push_back(inst->GetInOperand(i));
}
// Collect the surviving operand values in the same ascending-bit order they were
// encoded in, so the rebuilt list stays canonical.
Uint32 keptMask = kLod;
Vector<spvtools::opt::Operand> keptOperands;
static constexpr Uint32 kOrderedBits[] = {kBias, kLod, kGrad, kConstOffset,
kOffset, kConstOffsets, kSample, kMinLod};
for (const Uint32 bit : kOrderedBits) {
if ((mask & bit) == 0) continue;
if (next >= inst->NumInOperands()) return false;
const spvtools::opt::Operand value = inst->GetInOperand(next++);
// Bias and MinLod only shift a lambda that is already clamped to 0, and any
// original Lod is replaced by the constant the caller appends.
if (bit == kBias || bit == kMinLod || bit == kLod) continue;
keptMask |= bit;
keptOperands.push_back(value);
}
fixedOperands.push_back({SPV_OPERAND_TYPE_IMAGE, {keptMask}});
outPlan.instruction = inst;
outPlan.opcode = newOpcode;
outPlan.operands = Move(fixedOperands);
outPlan.trailingOperands = Move(keptOperands);
return true;
}
};
spvtools::Optimizer::PassToken CreateForceExplicitLod0SamplePass() {
return spvtools::Optimizer::PassToken(MakeUnique<ForceExplicitLod0SamplePass>());
}
Bool TransformSpirvForExplicitLod0Sampling(const Vector<Uint>& input, Vector<Uint>& output) {
if (input.empty()) {
output.clear();
return true;
}
spvtools::Optimizer optimizer(SPV_ENV_VULKAN_1_3);
spvtools::OptimizerOptions options;
// Matches the position-fix pass: this build of spirv-tools asserts rather than
// reporting, so validation stays off in the shipping path.
options.set_run_validator(false);
optimizer.SetMessageConsumer([](spv_message_level_t, const char*, const spv_position_t&,
const char* message) {
MGLOG_E("Vulkan: explicit-LOD0 pass: %s", message != nullptr ? message : "");
});
optimizer.RegisterPass(CreateForceExplicitLod0SamplePass());
const Bool success = optimizer.Run(input.data(), input.size(), &output, options);
if (!success) {
MGLOG_E("Vulkan: explicit-LOD0 sampling pass failed; keeping the original module");
output = input;
}
return success;
}
spvtools::Optimizer::PassToken CreateGlToVulkanPositionFixPass( spvtools::Optimizer::PassToken CreateGlToVulkanPositionFixPass(
ProgramFactory::CompileOptionFlags transformFlags) { ProgramFactory::CompileOptionFlags transformFlags) {
return spvtools::Optimizer::PassToken(MakeUnique<GlToVulkanPositionFixPass>(transformFlags)); return spvtools::Optimizer::PassToken(MakeUnique<GlToVulkanPositionFixPass>(transformFlags));
@@ -1950,11 +2150,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
auto it = m_cache.find(hash); auto it = m_cache.find(hash);
if (it != m_cache.end()) { if (it != m_cache.end()) {
// Every draw/dispatch funnels through this lookup (the renderer memos only
// skip re-hashing, never the factory lookup), so an actively-used entry is
// stamped at least once per frame boundary and can never be aged out while
// any in-flight command buffer still references it.
it->second.lastUsedFrame = m_frameCounter;
return it->second; return it->second;
} }
auto& entry = m_cache[hash]; auto& entry = m_cache[hash];
entry.hash = hash; entry.hash = hash;
entry.lastUsedFrame = m_frameCounter;
auto& shaders = program.GetAttachedShaders(); auto& shaders = program.GetAttachedShaders();
auto& spirv = program.GetGeneratedSpirv(); auto& spirv = program.GetGeneratedSpirv();
Vector<Vector<Uint>> moduleSpirvs(spirv.size()); Vector<Vector<Uint>> moduleSpirvs(spirv.size());
@@ -1972,6 +2178,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
moduleSpirvs[i] = spv; moduleSpirvs[i] = spv;
} }
if ((flags & ProgramFactory::CompileOptionBit::ExplicitLod0Sampling) && shaders[i] &&
shaders[i]->GetShaderStage() == ShaderStage::Fragment) {
Vector<Uint> explicitLodSpirv;
if (TransformSpirvForExplicitLod0Sampling(moduleSpirvs[i], explicitLodSpirv)) {
moduleSpirvs[i] = Move(explicitLodSpirv);
}
}
// GL apps depend on cross-program position invariance for multi-pass equality // GL apps depend on cross-program position invariance for multi-pass equality
// depth tests (MC 26.3's OIT re-draws the cloud geometry with GEQUAL against the // depth tests (MC 26.3's OIT re-draws the cloud geometry with GEQUAL against the
// depth its own first pass wrote); decorate Position outputs Invariant so // depth its own first pass wrote); decorate Position outputs Invariant so
@@ -2068,4 +2282,42 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return entry; return entry;
} }
void ProgramFactory::OnFrameBoundary() {
++m_frameCounter;
// Sweep cadence and retire age mirror VkRenderPassManager::OnPresent: an entry
// idle for more than kRetireAgeFrames frame boundaries cannot be referenced by
// any in-flight command buffer (frames-in-flight <= MOBILEGL_MAGMA_FRAMESINFLIGHT),
// so its shader modules and layouts are destroyed immediately - no deferred-
// destroy machinery needed. Eviction is content-based, never tied to
// glDeleteProgram: the cache is content-hash-shared across GL programs, so a
// delete-driven erase could free an entry another live program still resolves.
// An evicted entry self-heals - the frontend program keeps its generated
// SPIR-V, so the next GetOrCreateProgram rebuilds it (this also covers the
// renderer's internal blit/depth-mipmap programs).
constexpr Uint64 kSweepInterval = 256;
constexpr Uint64 kRetireAgeFrames = 1024;
if ((m_frameCounter % kSweepInterval) != 0) {
return;
}
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (m_frameCounter - it->second.lastUsedFrame > kRetireAgeFrames) {
const HashType hash = it->first;
const VkDescriptorSetLayout descriptorSetLayout = it->second.descriptorSetLayout;
MGLOG_D("ProgramFactory::OnFrameBoundary: evicting idle program entry hash=0x%llx",
static_cast<unsigned long long>(hash));
// erase runs ~VkProgramObject (modules/layouts destroyed); notify after
// so an observer never observes a half-destroyed entry through a lookup.
// Observers only need the handle values to purge their keyed caches.
it = m_cache.erase(it);
if (m_evictionObserver != nullptr) {
m_evictionObserver->OnProgramEvicted(hash, descriptorSetLayout);
}
} else {
++it;
}
}
}
} // namespace MobileGL::MG_Backend::DirectVulkan } // namespace MobileGL::MG_Backend::DirectVulkan
@@ -42,6 +42,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
SurfaceRotate90 = 1 << 2, SurfaceRotate90 = 1 << 2,
SurfaceRotate180 = 1 << 3, SurfaceRotate180 = 1 << 3,
SurfaceRotate270 = 1 << 4, SurfaceRotate270 = 1 << 4,
// Rewrites the fragment stage's implicit-LOD image samples to explicit LOD 0.
// Only ever set for a draw whose every sampler binding is clamped to a single mip
// level, which makes the two forms produce identical texels (the implicit lambda is
// clamped into [minLod, maxLod] = [0, 0] regardless of derivatives or bias).
ExplicitLod0Sampling = 1 << 5,
}; };
using CompileOptionFlags = Flags<CompileOptionBit>; using CompileOptionFlags = Flags<CompileOptionBit>;
using HashType = Uint64; using HashType = Uint64;
@@ -88,6 +93,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// gl_FragDepth); shader-computed depth is immune to the cross-pipeline // gl_FragDepth); shader-computed depth is immune to the cross-pipeline
// position-invariance quirk (see PipelineFactory::ShouldSuppressDepthWrite). // position-invariance quirk (see PipelineFactory::ShouldSuppressDepthWrite).
Bool fragmentReplacesDepth = false; Bool fragmentReplacesDepth = false;
// Frame-boundary counter value of the last GetOrCreateProgram hit; drives
// cache eviction (see OnFrameBoundary).
Uint64 lastUsedFrame = 0;
static inline VkDevice s_device = VK_NULL_HANDLE; static inline VkDevice s_device = VK_NULL_HANDLE;
@@ -124,6 +132,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
producerOutputComponentCount = other.producerOutputComponentCount; producerOutputComponentCount = other.producerOutputComponentCount;
fragmentInputComponentCount = other.fragmentInputComponentCount; fragmentInputComponentCount = other.fragmentInputComponentCount;
fragmentReplacesDepth = other.fragmentReplacesDepth; fragmentReplacesDepth = other.fragmentReplacesDepth;
lastUsedFrame = other.lastUsedFrame;
other.hash = 0; other.hash = 0;
other.descriptorSetLayout = VK_NULL_HANDLE; other.descriptorSetLayout = VK_NULL_HANDLE;
other.pipelineLayout = VK_NULL_HANDLE; other.pipelineLayout = VK_NULL_HANDLE;
@@ -135,6 +144,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
other.producerOutputComponentCount = 0; other.producerOutputComponentCount = 0;
other.fragmentInputComponentCount = 0; other.fragmentInputComponentCount = 0;
other.fragmentReplacesDepth = false; other.fragmentReplacesDepth = false;
other.lastUsedFrame = 0;
} }
VkProgramObject& operator=(VkProgramObject&& other) noexcept { VkProgramObject& operator=(VkProgramObject&& other) noexcept {
if (this == &other) { if (this == &other) {
@@ -170,6 +180,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
producerOutputComponentCount = other.producerOutputComponentCount; producerOutputComponentCount = other.producerOutputComponentCount;
fragmentInputComponentCount = other.fragmentInputComponentCount; fragmentInputComponentCount = other.fragmentInputComponentCount;
fragmentReplacesDepth = other.fragmentReplacesDepth; fragmentReplacesDepth = other.fragmentReplacesDepth;
lastUsedFrame = other.lastUsedFrame;
other.hash = 0; other.hash = 0;
other.descriptorSetLayout = VK_NULL_HANDLE; other.descriptorSetLayout = VK_NULL_HANDLE;
other.pipelineLayout = VK_NULL_HANDLE; other.pipelineLayout = VK_NULL_HANDLE;
@@ -181,6 +192,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
other.producerOutputComponentCount = 0; other.producerOutputComponentCount = 0;
other.fragmentInputComponentCount = 0; other.fragmentInputComponentCount = 0;
other.fragmentReplacesDepth = false; other.fragmentReplacesDepth = false;
other.lastUsedFrame = 0;
return *this; return *this;
} }
@@ -210,6 +222,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
}; };
// Notified when the OnFrameBoundary sweep destroys an aged-out cache entry,
// carrying the entry's content hash and the VkDescriptorSetLayout it owned.
// Dependent caches (compute pipelines, PipelineFactory entries, UniformManager's
// per-layout descriptor sets) must purge in the same step: after vkDestroy the
// layout handle value may be recycled for an unrelated layout, and the program
// hash may be re-inserted by a later rebuild of the same content.
class IEvictionObserver {
public:
virtual ~IEvictionObserver() = default;
virtual void OnProgramEvicted(HashType programHash, VkDescriptorSetLayout descriptorSetLayout) = 0;
};
explicit ProgramFactory(VkDevice device, const VulkanRendererConfig& config, Uint32 maxBindings = 16, explicit ProgramFactory(VkDevice device, const VulkanRendererConfig& config, Uint32 maxBindings = 16,
Bool shaderDrawParametersEnabled = false, Bool shaderDrawParametersEnabled = false,
Bool unformattedFloatStorageImagesEnabled = false) Bool unformattedFloatStorageImagesEnabled = false)
@@ -225,6 +249,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VkProgramObject& GetOrCreateProgram( const VkProgramObject& GetOrCreateProgram(
const MG_State::GLState::ProgramObject& program, CompileOptionFlags flags); const MG_State::GLState::ProgramObject& program, CompileOptionFlags flags);
// Observer may be null (no notifications). Not owned.
void SetEvictionObserver(IEvictionObserver* observer) { m_evictionObserver = observer; }
// Frame boundary hook: ages the program cache and evicts long-unused entries
// (their command buffers retired many frames ago), mirroring
// VkRenderPassManager::OnPresent's sweep.
void OnFrameBoundary();
static VkShaderStageFlagBits ToVkStage(ShaderStage stage); static VkShaderStageFlagBits ToVkStage(ShaderStage stage);
static VkFormat ConvertSpirvImageFormatToVkFormat(SpvImageFormat format); static VkFormat ConvertSpirvImageFormatToVkFormat(SpvImageFormat format);
static SamplerNumericDomain UniformTypeToSamplerNumericDomain(GLenum glType); static SamplerNumericDomain UniformTypeToSamplerNumericDomain(GLenum glType);
@@ -266,6 +297,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// shaderStorageImageReadWithoutFormat and shaderStorageImageWriteWithoutFormat. // shaderStorageImageReadWithoutFormat and shaderStorageImageWriteWithoutFormat.
Bool m_unformattedFloatStorageImagesEnabled = false; Bool m_unformattedFloatStorageImagesEnabled = false;
mutable ProgramLookupCache m_lastLookup; mutable ProgramLookupCache m_lastLookup;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameCounter = 0;
IEvictionObserver* m_evictionObserver = nullptr;
static inline XXH64_state_t* m_hashState = XXH64_createState(); static inline XXH64_state_t* m_hashState = XXH64_createState();
}; };
} // namespace MobileGL::MG_Backend::DirectVulkan } // namespace MobileGL::MG_Backend::DirectVulkan
@@ -247,6 +247,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_surfaceFormat = {createInfo.imageFormat, createInfo.imageColorSpace}; m_surfaceFormat = {createInfo.imageFormat, createInfo.imageColorSpace};
m_extent = createInfo.imageExtent; m_extent = createInfo.imageExtent;
// The surface-space extent this swapchain was built from, i.e. before the
// quarter-turn swap above. Out-of-date checks must compare in THIS space: comparing a
// freshly queried currentExtent against the swapped m_extent flips axes every rotation
// and makes the comparison alternate forever.
m_surfaceExtent = defaultFramebufferExtent;
m_preTransform = createInfo.preTransform; m_preTransform = createInfo.preTransform;
VK_VERIFY(vkCreateSwapchainKHR(device, &createInfo, nullptr, &m_swapchain)); VK_VERIFY(vkCreateSwapchainKHR(device, &createInfo, nullptr, &m_swapchain));
@@ -35,6 +35,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkSwapchainKHR GetHandle() const { return m_swapchain; } VkSwapchainKHR GetHandle() const { return m_swapchain; }
const VkSurfaceFormatKHR& GetSurfaceFormat() const { return m_surfaceFormat; } const VkSurfaceFormatKHR& GetSurfaceFormat() const { return m_surfaceFormat; }
VkExtent2D GetExtent() const { return m_extent; } VkExtent2D GetExtent() const { return m_extent; }
// Surface-space extent (before the pre-rotation quarter-turn swap) this swapchain was
// created from - the value to compare a freshly queried currentExtent against.
VkExtent2D GetSurfaceExtent() const { return m_surfaceExtent; }
VkSurfaceTransformFlagBitsKHR GetPreTransform() const { return m_preTransform; } VkSurfaceTransformFlagBitsKHR GetPreTransform() const { return m_preTransform; }
const Vector<VkImage>& GetImages() const { return m_images; } const Vector<VkImage>& GetImages() const { return m_images; }
const Vector<VkImageView>& GetImageViews() const { return m_imageViews; } const Vector<VkImageView>& GetImageViews() const { return m_imageViews; }
@@ -63,6 +66,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkSwapchainKHR m_swapchain = VK_NULL_HANDLE; VkSwapchainKHR m_swapchain = VK_NULL_HANDLE;
VkSurfaceFormatKHR m_surfaceFormat{}; VkSurfaceFormatKHR m_surfaceFormat{};
VkExtent2D m_extent{}; VkExtent2D m_extent{};
VkExtent2D m_surfaceExtent{};
VkSurfaceTransformFlagBitsKHR m_preTransform = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR; VkSurfaceTransformFlagBitsKHR m_preTransform = VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR;
Vector<VkImage> m_images; Vector<VkImage> m_images;
Vector<VkImageView> m_imageViews; Vector<VkImageView> m_imageViews;
@@ -211,6 +211,40 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
} }
void UniformManager::OnDescriptorSetLayoutDestroyed(VkDescriptorSetLayout descriptorSetLayout) {
SizeT purgedSets = 0;
for (auto& frame : m_frames) {
const auto it = frame.descriptorSetCacheByLayout.find(descriptorSetLayout);
if (it == frame.descriptorSetCacheByLayout.end()) {
continue;
}
// Free the sets back to their pools and credit the bucket accounting, so
// program churn recycles pool capacity instead of abandoning the slots.
// GPU-safe: the layout only dies after >1024 idle frame boundaries, so no
// in-flight command buffer references these sets.
for (const auto& cached : it->second.sets) {
if (cached.set == VK_NULL_HANDLE) {
continue;
}
vkFreeDescriptorSets(m_device, cached.pool, 1, &cached.set);
const auto bucket = std::find_if(
frame.descriptorPools.begin(), frame.descriptorPools.end(),
[&cached](const DescriptorPoolBucket& candidate) { return candidate.handle == cached.pool; });
if (bucket != frame.descriptorPools.end() && bucket->allocatedSets > 0) {
--bucket->allocatedSets;
}
}
purgedSets += it->second.sets.size();
frame.descriptorSetCacheByLayout.erase(it);
}
if (purgedSets > 0) {
// The per-draw reuse memo folds the layout handle into its signature; drop
// it so a recycled handle value cannot revive a purged set mid-frame.
m_hasLastDescriptor = false;
MGLOG_D("UniformDescriptorBinder: freed %zu descriptor sets for destroyed layout", purgedSets);
}
}
Bool UniformManager::ResolveSamplerDescriptor(VkCommandBuffer commandBuffer, Bool UniformManager::ResolveSamplerDescriptor(VkCommandBuffer commandBuffer,
const MG_State::GLState::ProgramObject& program, const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, const ProgramFactory::VkProgramObject& programObj,
@@ -350,24 +384,28 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const Uint16 samplerVersion = samplerToUse->GetVersion(); const Uint16 samplerVersion = samplerToUse->GetVersion();
const Uint64 textureLifetimeId = texture->GetLifetimeId(); const Uint64 textureLifetimeId = texture->GetLifetimeId();
const Uint16 textureParamsVersion = texture->GetTextureParamsVersion(); const Uint16 textureParamsVersion = texture->GetTextureParamsVersion();
// The sampler's LOD clamp depends on how many levels the sampled view exposes, and that
// follows uploads as well as GL parameters - so it belongs in the memo key too.
const Uint32 viewLevelCount = resource->sampledLevelCount;
if (memo.valid && memo.samplerLifetimeId == samplerLifetimeId && memo.samplerVersion == samplerVersion && if (memo.valid && memo.samplerLifetimeId == samplerLifetimeId && memo.samplerVersion == samplerVersion &&
memo.textureLifetimeId == textureLifetimeId && memo.textureParamsVersion == textureParamsVersion && memo.textureLifetimeId == textureLifetimeId && memo.textureParamsVersion == textureParamsVersion &&
memo.forceNearestFiltering == forceNearestFiltering) { memo.forceNearestFiltering == forceNearestFiltering && memo.viewLevelCount == viewLevelCount) {
resolvedSampler = memo.sampler; resolvedSampler = memo.sampler;
} else { } else {
resolvedSampler = resolvedSampler = m_samplerManager->GetOrCreateSampler(*samplerToUse, *texture,
m_samplerManager->GetOrCreateSampler(*samplerToUse, *texture, forceNearestFiltering); forceNearestFiltering, viewLevelCount);
memo.samplerLifetimeId = samplerLifetimeId; memo.samplerLifetimeId = samplerLifetimeId;
memo.samplerVersion = samplerVersion; memo.samplerVersion = samplerVersion;
memo.textureLifetimeId = textureLifetimeId; memo.textureLifetimeId = textureLifetimeId;
memo.textureParamsVersion = textureParamsVersion; memo.textureParamsVersion = textureParamsVersion;
memo.forceNearestFiltering = forceNearestFiltering; memo.forceNearestFiltering = forceNearestFiltering;
memo.viewLevelCount = viewLevelCount;
memo.sampler = resolvedSampler; memo.sampler = resolvedSampler;
memo.valid = true; memo.valid = true;
} }
} else { } else {
resolvedSampler = resolvedSampler = m_samplerManager->GetOrCreateSampler(*samplerToUse, *texture, forceNearestFiltering,
m_samplerManager->GetOrCreateSampler(*samplerToUse, *texture, forceNearestFiltering); resource->sampledLevelCount);
} }
outImageInfo = { outImageInfo = {
.sampler = resolvedSampler, .sampler = resolvedSampler,
@@ -408,6 +446,48 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return outImageInfo.sampler != VK_NULL_HANDLE; return outImageInfo.sampler != VK_NULL_HANDLE;
} }
Bool UniformManager::ProgramSamplesOnlySingleLevelTextures(
const MG_State::GLState::ProgramObject& program, const ProgramFactory::VkProgramObject& programObj) {
Bool sawSampler = false;
for (Uint32 binding = 0; binding < programObj.bindingKinds.size(); ++binding) {
if (programObj.bindingKinds[binding] != ProgramFactory::DescriptorBindingKind::CombinedImageSampler) {
continue;
}
const auto* texture = ResolveSamplerTextureRaw(program, programObj, binding);
if (texture == nullptr) return false;
const auto& levelRange = texture->GetLevelRange();
if (levelRange.x() != levelRange.y()) return false;
// An explicit-LOD sample is a single filtered tap, so it also gives up anisotropic
// filtering - which a single-level view can still have. Resolve the sampler exactly
// the way ResolveSamplerDescriptor does and bail if anisotropy would apply.
const Int location = programObj.samplerUniformLocationByBinding[binding];
const Int unit = ResolveSamplerUnitIndex(program, location, binding);
const auto& samplerOverride = MG_State::pGLContext->GetTextureUnitObject(unit).GetSamplerObject();
const auto* effectiveSampler =
samplerOverride ? samplerOverride.get() : texture->GetSamplerObject().get();
if (effectiveSampler == nullptr) return false;
if (effectiveSampler->GetMaxAnisotropy() > 1.0f &&
effectiveSampler->GetMinFilter() == SamplerFilterMode::Linear &&
effectiveSampler->GetMagFilter() == SamplerFilterMode::Linear) {
return false;
}
// An explicit LOD 0 makes lambda exactly 0, which is the magnification side of the
// min/mag decision. That only matches the implicit form when lambda could not have been
// positive anyway (the LOD clamp already pins it at or below 0), or when the two
// filters are the same and the choice cannot be observed.
const Float effectiveMaxLod = effectiveSampler->GetMipmapMode() == SamplerMipmapMode::None
? 0.0f
: effectiveSampler->GetMaxLod();
if (effectiveMaxLod > 0.0f && effectiveSampler->GetMinFilter() != effectiveSampler->GetMagFilter()) {
return false;
}
sawSampler = true;
}
return sawSampler;
}
Bool UniformManager::ResolveSamplerTexture(const MG_State::GLState::ProgramObject& program, Bool UniformManager::ResolveSamplerTexture(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding, const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
SharedPtr<MG_State::GLState::ITextureObject>& outTexture) { SharedPtr<MG_State::GLState::ITextureObject>& outTexture) {
@@ -911,6 +991,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkDescriptorPoolCreateInfo poolInfo{}; VkDescriptorPoolCreateInfo poolInfo{};
poolInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO; poolInfo.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO;
// FREE_DESCRIPTOR_SET_BIT lets a destroyed layout's cached sets be freed back
// (OnDescriptorSetLayoutDestroyed) so program churn recycles pool capacity.
// The cost is on set allocation only, which happens when a layout's per-frame
// cache grows - never on the per-draw reuse path.
poolInfo.flags = VK_DESCRIPTOR_POOL_CREATE_FREE_DESCRIPTOR_SET_BIT;
poolInfo.maxSets = maxSets; poolInfo.maxSets = maxSets;
poolInfo.poolSizeCount = static_cast<Uint32>(std::size(poolSizes)); poolInfo.poolSizeCount = static_cast<Uint32>(std::size(poolSizes));
poolInfo.pPoolSizes = poolSizes; poolInfo.pPoolSizes = poolSizes;
@@ -990,7 +1075,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto& frame = m_frames[frameIndex]; auto& frame = m_frames[frameIndex];
auto& cache = frame.descriptorSetCacheByLayout[programObj.descriptorSetLayout]; auto& cache = frame.descriptorSetCacheByLayout[programObj.descriptorSetLayout];
if (cache.cursor < cache.sets.size()) { if (cache.cursor < cache.sets.size()) {
outDescriptorSet = cache.sets[cache.cursor++]; outDescriptorSet = cache.sets[cache.cursor++].set;
} else { } else {
VkResult allocResult = AllocateDescriptorSetsFromActivePool(frameIndex, programObj, outDescriptorSet); VkResult allocResult = AllocateDescriptorSetsFromActivePool(frameIndex, programObj, outDescriptorSet);
if (allocResult == VK_ERROR_OUT_OF_POOL_MEMORY || allocResult == VK_ERROR_FRAGMENTED_POOL) { if (allocResult == VK_ERROR_OUT_OF_POOL_MEMORY || allocResult == VK_ERROR_FRAGMENTED_POOL) {
@@ -1004,7 +1089,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return allocResult; return allocResult;
} }
cache.sets.push_back(outDescriptorSet); // The successful allocation came from the bucket the alloc helper left
// active; record it so a layout-destroyed purge can free the set back.
cache.sets.push_back({outDescriptorSet, frame.descriptorPools[frame.activeDescriptorPoolIndex].handle});
++cache.cursor; ++cache.cursor;
MGLOG_D("UniformDescriptorBinder: cached descriptor set count for frame=%u grew to %zu", frameIndex, MGLOG_D("UniformDescriptorBinder: cached descriptor set count for frame=%u grew to %zu", frameIndex,
cache.sets.size()); cache.sets.size());
@@ -39,6 +39,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void Shutdown(); void Shutdown();
void BeginFrame(Uint32 frameIndex); void BeginFrame(Uint32 frameIndex);
// A ProgramFactory eviction just destroyed this layout: purge every frame
// slot's cached descriptor sets for it, so a recycled handle value can never
// stale-hit sets written for the dead layout's bindings. The sets are
// vkFreeDescriptorSets'd back to their pools (created with
// FREE_DESCRIPTOR_SET_BIT) and the pool accounting is credited, so program
// churn recycles pool capacity instead of abandoning it. GPU-safe: the layout
// only dies after >1024 idle frame boundaries, so no in-flight command buffer
// references its sets. This is the only eviction path for the per-layout
// caches - a live layout's entry must never be purged (its sets would be
// unreachable pool slots), so there is deliberately no age-based sweep here.
void OnDescriptorSetLayoutDestroyed(VkDescriptorSetLayout descriptorSetLayout);
Bool CollectSampledTextures(const MG_State::GLState::ProgramObject& program, Bool CollectSampledTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, const ProgramFactory::VkProgramObject& programObj,
Vector<MG_State::GLState::ITextureObject*>& outTextures); Vector<MG_State::GLState::ITextureObject*>& outTextures);
@@ -58,6 +69,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static VkFormat ResolveStorageImageViewFormat(VkFormat reflectedFormat, GLenum bindingFormat, static VkFormat ResolveStorageImageViewFormat(VkFormat reflectedFormat, GLenum bindingFormat,
VkFormat resourceFormat, Bool useBindingFormat); VkFormat resourceFormat, Bool useBindingFormat);
// True when the program reads at least one sampler and every one of them is bound to a
// texture whose GL level range is a single level. Such a sampler resolves to
// minLod = maxLod = 0 (see VkSamplerManager::GetOrCreateSampler), so an implicit-LOD sample
// and an explicit LOD 0 sample must read the same texel - which is what makes the
// ExplicitLod0Sampling SPIR-V rewrite safe to request. Deliberately conservative: it reads
// only GL state, so a texture that ends up single-level for another reason (one uploaded
// level under a wide level range) merely misses the rewrite.
static Bool ProgramSamplesOnlySingleLevelTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj);
private: private:
struct DescriptorPoolBucket { struct DescriptorPoolBucket {
VkDescriptorPool handle = VK_NULL_HANDLE; VkDescriptorPool handle = VK_NULL_HANDLE;
@@ -65,8 +86,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 allocatedSets = 0; Uint32 allocatedSets = 0;
}; };
// A cached descriptor set together with the pool it was allocated from, so a
// layout-destroyed purge can vkFreeDescriptorSets it back and credit the
// owning bucket's accounting.
struct CachedDescriptorSet {
VkDescriptorSet set = VK_NULL_HANDLE;
VkDescriptorPool pool = VK_NULL_HANDLE;
};
struct DescriptorSetCacheEntry { struct DescriptorSetCacheEntry {
Vector<VkDescriptorSet> sets; Vector<CachedDescriptorSet> sets;
Uint32 cursor = 0; Uint32 cursor = 0;
}; };
@@ -172,6 +201,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 samplerLifetimeId = 0; Uint64 samplerLifetimeId = 0;
Uint64 textureLifetimeId = 0; Uint64 textureLifetimeId = 0;
VkSampler sampler = VK_NULL_HANDLE; VkSampler sampler = VK_NULL_HANDLE;
Uint32 viewLevelCount = 0;
Uint16 samplerVersion = 0; Uint16 samplerVersion = 0;
Uint16 textureParamsVersion = 0; Uint16 textureParamsVersion = 0;
Bool forceNearestFiltering = false; Bool forceNearestFiltering = false;
@@ -32,6 +32,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(m_hashState, &attr.IsBgra, sizeof(attr.IsBgra))); XXHASH_VERIFY(XXH64_update(m_hashState, &attr.IsBgra, sizeof(attr.IsBgra)));
XXHASH_VERIFY(XXH64_update(m_hashState, &attr.Divisor, sizeof(attr.Divisor))); XXHASH_VERIFY(XXH64_update(m_hashState, &attr.Divisor, sizeof(attr.Divisor)));
// The buffer's heap address is an identity component of the key: a freed
// buffer's reused address can alias an old cache entry, but only under a
// byte-identical attribute layout - and the entry payload is a pure function
// of the hashed inputs, with the draw path re-resolving bindingBufferKeys
// against the live VAO attribute pointers, so an aliased hit returns exactly
// what a rebuild would. Address drift only grows the map; the OnFrameBoundary
// aging sweep bounds that.
const SizeT bufferKey = reinterpret_cast<SizeT>(attr.Buffer.get()); const SizeT bufferKey = reinterpret_cast<SizeT>(attr.Buffer.get());
XXHASH_VERIFY(XXH64_update(m_hashState, &bufferKey, sizeof(bufferKey))); XXHASH_VERIFY(XXH64_update(m_hashState, &bufferKey, sizeof(bufferKey)));
} }
@@ -58,6 +65,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const MG_State::GLState::VertexArrayObject& vao, HashType hash) { const MG_State::GLState::VertexArrayObject& vao, HashType hash) {
auto it = m_cache.find(hash); auto it = m_cache.find(hash);
if (it != m_cache.end()) { if (it != m_cache.end()) {
it->second.lastUsedFrameBoundary = m_frameBoundaryCounter;
return it->second; return it->second;
} }
@@ -166,6 +174,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
auto& entry = m_cache[hash]; auto& entry = m_cache[hash];
entry.hash = hash; entry.hash = hash;
entry.lastUsedFrameBoundary = m_frameBoundaryCounter;
entry.bindings = builder.GetBindings(); entry.bindings = builder.GetBindings();
entry.attributes = builder.GetAttributes(); entry.attributes = builder.GetAttributes();
entry.bindingBufferKeys = std::move(bindingBufferKeys); entry.bindingBufferKeys = std::move(bindingBufferKeys);
@@ -180,6 +189,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return entry; return entry;
} }
void VertexInputStateFactory::OnFrameBoundary() {
++m_frameBoundaryCounter;
// Sweep occasionally; evict entries whose last hit is far in the past.
// Erasure happens only here, never mid-frame: the draw path holds a
// reference into the current entry across its setup, and unordered_map
// erase would invalidate it. Entries are CPU-side only, so no GPU-idle
// proof is needed; an evicted entry that is used again is simply rebuilt
// from the VAO state (same hash, same content).
constexpr Uint64 kSweepInterval = 256;
constexpr Uint64 kRetireAgeBoundaries = 1024;
if ((m_frameBoundaryCounter % kSweepInterval) != 0) {
return;
}
for (auto it = m_cache.begin(); it != m_cache.end();) {
if (m_frameBoundaryCounter - it->second.lastUsedFrameBoundary > kRetireAgeBoundaries) {
it = m_cache.erase(it);
} else {
++it;
}
}
}
VkFormat VertexInputStateFactory::ToVkVertexFormat(DataType type, Int size, Bool normalized, Bool isInteger, VkFormat VertexInputStateFactory::ToVkVertexFormat(DataType type, Int size, Bool normalized, Bool isInteger,
Bool isBgra) { Bool isBgra) {
if (isBgra) { if (isBgra) {
@@ -27,6 +27,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct BackendVertexInputState { struct BackendVertexInputState {
HashType hash = 0; HashType hash = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age are evicted (CPU heap only).
Uint64 lastUsedFrameBoundary = 0;
Vector<VkVertexInputBindingDescription> bindings; Vector<VkVertexInputBindingDescription> bindings;
Vector<VkVertexInputAttributeDescription> attributes; Vector<VkVertexInputAttributeDescription> attributes;
Vector<SizeT> bindingBufferKeys; Vector<SizeT> bindingBufferKeys;
@@ -55,6 +58,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const BackendVertexInputState& GetOrCreateVertexInputState( const BackendVertexInputState& GetOrCreateVertexInputState(
const MG_State::GLState::VertexArrayObject& vao, HashType hash); const MG_State::GLState::VertexArrayObject& vao, HashType hash);
const BackendVertexInputState& GetOrCreateVertexInputState(const MG_State::GLState::VertexArrayObject& vao); const BackendVertexInputState& GetOrCreateVertexInputState(const MG_State::GLState::VertexArrayObject& vao);
// Frame boundary hook: ages the cache and evicts entries not hit for many
// frames. The key mixes buffer heap addresses, so buffer/VAO churn keeps
// minting fresh keys; without eviction the map grows for the whole session.
// Entries hold no Vulkan handles (pipeline creation copies the descriptions)
// and the draw path's entry reference never spans a frame boundary, so
// eviction here needs no GPU-idle proof. Self-gated: one counter bump and
// compare except on sweep boundaries.
void OnFrameBoundary();
static SizeT GetComponentSize(DataType type); static SizeT GetComponentSize(DataType type);
// Tightly-packed byte size of one vertex element for this attribute: componentSize * size for // Tightly-packed byte size of one vertex element for this attribute: componentSize * size for
// normal types, and 4 (one packed word) for the 2_10_10_10 types and GL_BGRA. Returns 0 for // normal types, and 4 (one packed word) for the 2_10_10_10 types and GL_BGRA. Returns 0 for
@@ -70,6 +81,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VulkanRendererConfig& m_config; const VulkanRendererConfig& m_config;
VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE; VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE;
UnorderedMap<HashType, BackendVertexInputState> m_cache; UnorderedMap<HashType, BackendVertexInputState> m_cache;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameBoundaryCounter = 0;
static inline XXH64_state_t* m_hashState = XXH64_createState(); static inline XXH64_state_t* m_hashState = XXH64_createState();
}; };
} // namespace MobileGL::MG_Backend::DirectVulkan } // namespace MobileGL::MG_Backend::DirectVulkan
@@ -141,6 +141,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_transientUploadArena.BeginFrame(frameIndex); m_transientUploadArena.BeginFrame(frameIndex);
} }
void VkBufferManager::CollectAllDeferredReleases() {
for (Uint32 frameIndex = 0; frameIndex < m_deferredBufferReleases.size(); ++frameIndex) {
CollectDeferredReleases(frameIndex);
}
for (Uint32 frameIndex = 0; frameIndex < m_transientUploadArena.GetFrameCount(); ++frameIndex) {
m_transientUploadArena.CollectDeferredReleases(frameIndex);
}
}
void VkBufferManager::NotifyDeviceIdle() { void VkBufferManager::NotifyDeviceIdle() {
// Everything submitted so far has completed. Work recorded for the // Everything submitted so far has completed. Work recorded for the
// current frame has not been submitted yet, so the current serial // current frame has not been submitted yet, so the current serial
@@ -77,6 +77,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Recreate all per-frame transient arenas // Recreate all per-frame transient arenas
Bool RecreateTransientArenas(Uint32 frameCount); Bool RecreateTransientArenas(Uint32 frameCount);
void BeginFrame(Uint32 frameIndex); void BeginFrame(Uint32 frameIndex);
// Drains every frame slot's deferred buffer/resource releases (and the
// transient arena's parked superseded blocks). Only valid when the
// caller has proven every queue submission complete; used by the
// present-less frame-boundary drain.
void CollectAllDeferredReleases();
// All previously submitted GPU work has completed (vkDeviceWaitIdle). // All previously submitted GPU work has completed (vkDeviceWaitIdle).
void NotifyDeviceIdle(); void NotifyDeviceIdle();
// A frame slot's submission fence has been waited: every serial up to // A frame slot's submission fence has been waited: every serial up to
@@ -180,6 +180,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
sampleCount = VK_SAMPLE_COUNT_1_BIT; sampleCount = VK_SAMPLE_COUNT_1_BIT;
internalFormat = TextureInternalFormat::Unknown; internalFormat = TextureInternalFormat::Unknown;
samples = 0; samples = 0;
deadSinceFrame = kNeverObservedDead;
} }
VkRenderPassManager::VkRenderPassManager(VkDevice device, VkRenderPassManager::VkRenderPassManager(VkDevice device,
@@ -206,6 +207,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.Destroy(m_device, m_allocator); resource.Destroy(m_device, m_allocator);
} }
m_renderbufferResources.clear(); m_renderbufferResources.clear();
CollectDeferredRenderbufferReleases(/*destroyAll=*/true); // caller guarantees device idle
m_pendingRenderbufferClears.clear(); m_pendingRenderbufferClears.clear();
RenderPassEntry::s_textureResourcesScratch.clear(); RenderPassEntry::s_textureResourcesScratch.clear();
s_activeRenderPass = {}; s_activeRenderPass = {};
@@ -213,22 +215,75 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_rpFastValid = false; m_rpFastValid = false;
} }
void VkRenderPassManager::CollectRenderbufferGarbage() { Uint64 VkRenderPassManager::RetireAgeFrames() const {
Vector<MG_State::GLState::RenderbufferObject*> deadRenderbuffers; // MaxFramesInFlight + 2 covers the frame ring plus one boundary for the
deadRenderbuffers.reserve(m_renderbufferResources.size()); // recording-to-submit gap and one because OnPresent runs ahead of Present's
for (auto& [renderbuffer, resource] : m_renderbufferResources) { // fence wait; the floor of 8 keeps a margin over the default ring of 3 while
const auto liveRenderbuffer = resource.renderbuffer.lock(); // still releasing multi-MB attachment memory promptly (the render-pass cache's
if (!liveRenderbuffer || liveRenderbuffer.get() != renderbuffer) { // 1024-frame retirement would pin it for no additional safety).
deadRenderbuffers.emplace_back(renderbuffer); return std::max<Uint64>(8, static_cast<Uint64>(m_config.MaxFramesInFlight) + 2);
} }
void VkRenderPassManager::DeferRenderbufferBackingRelease(RenderbufferResource& resource) {
// The superseded backing may still be referenced by in-flight command buffers
// (glRenderbufferStorage can respecify a renderbuffer drawn this very frame),
// so it is parked and destroyed only after RetireAgeFrames() boundaries.
if (resource.image == VK_NULL_HANDLE && resource.view == VK_NULL_HANDLE) {
return;
} }
for (auto* renderbuffer : deadRenderbuffers) { m_deferredRenderbufferReleases.push_back({resource.image, resource.allocation, resource.view, m_frameCounter});
auto resourceIt = m_renderbufferResources.find(renderbuffer); resource.image = VK_NULL_HANDLE;
if (resourceIt != m_renderbufferResources.end()) { resource.allocation = nullptr;
resourceIt->second.Destroy(m_device, m_allocator); resource.view = VK_NULL_HANDLE;
m_renderbufferResources.erase(resourceIt); }
void VkRenderPassManager::CollectDeferredRenderbufferReleases(Bool destroyAll) {
if (m_deferredRenderbufferReleases.empty()) {
return;
}
const Uint64 retireAgeFrames = RetireAgeFrames();
std::erase_if(m_deferredRenderbufferReleases, [&](DeferredRenderbufferRelease& release) {
if (!destroyAll && m_frameCounter - release.deferredAtFrame < retireAgeFrames) {
return false;
} }
m_pendingRenderbufferClears.erase(renderbuffer); if (release.view != VK_NULL_HANDLE) {
vkDestroyImageView(m_device, release.view, nullptr);
}
if (release.image != VK_NULL_HANDLE) {
vmaDestroyImage(m_allocator, release.image, release.allocation);
}
return true;
});
}
void VkRenderPassManager::CollectRenderbufferGarbage() {
// Two-phase reclamation: a dead renderbuffer's VkImage may still be referenced by
// command buffers submitted up to frames-in-flight frames ago (it was legally
// attached and drawn right up to its deletion), so the first observation of an
// expired weak reference only stamps the current frame counter; Destroy runs once
// enough frame boundaries have passed that the stamping frame's submission fence
// has provably been waited (see RetireAgeFrames).
const Uint64 retireAgeFrames = RetireAgeFrames();
for (auto it = m_renderbufferResources.begin(); it != m_renderbufferResources.end();) {
auto& resource = it->second;
const auto liveRenderbuffer = resource.renderbuffer.lock();
if (liveRenderbuffer && liveRenderbuffer.get() == it->first) {
resource.deadSinceFrame = RenderbufferResource::kNeverObservedDead;
++it;
continue;
}
if (resource.deadSinceFrame == RenderbufferResource::kNeverObservedDead) {
resource.deadSinceFrame = m_frameCounter;
++it;
continue;
}
if (m_frameCounter - resource.deadSinceFrame < retireAgeFrames) {
++it;
continue;
}
m_pendingRenderbufferClears.erase(it->first);
resource.Destroy(m_device, m_allocator);
it = m_renderbufferResources.erase(it);
} }
} }
@@ -270,9 +325,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.samples != renderbuffer->GetSamples(); resource.samples != renderbuffer->GetSamples();
if (!needsCreate) { if (!needsCreate) {
resource.renderbuffer = renderbuffer; resource.renderbuffer = renderbuffer;
// A new renderbuffer at a recycled address may adopt a compatible entry that
// was already stamped dead; it is alive again, so cancel the aging.
resource.deadSinceFrame = RenderbufferResource::kNeverObservedDead;
return &resource; return &resource;
} }
// Respecify: park the old backing for aged destruction instead of destroying
// inline - it may still be referenced by in-flight command buffers.
DeferRenderbufferBackingRelease(resource);
resource.Destroy(m_device, m_allocator); resource.Destroy(m_device, m_allocator);
resource.renderbuffer = renderbuffer; resource.renderbuffer = renderbuffer;
@@ -1178,6 +1239,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void VkRenderPassManager::OnPresent() { void VkRenderPassManager::OnPresent() {
++m_frameCounter; ++m_frameCounter;
// Runs every frame boundary, ahead of the render-pass sweep gate below: the walk
// is O(#renderbuffer resources) — single digits in practice — and per-frame
// invocation keeps dead-resource reclaim latency at the aging bound instead of
// coupling it to renderbuffer *use* (the GetOrCreateRenderbufferResource call
// site never runs again once an app stops using renderbuffers).
CollectRenderbufferGarbage();
CollectDeferredRenderbufferReleases(/*destroyAll=*/false);
// Sweep occasionally; evict entries whose last use is far past every // Sweep occasionally; evict entries whose last use is far past every
// in-flight frame so their VkRenderPass/VkFramebuffer can be destroyed // in-flight frame so their VkRenderPass/VkFramebuffer can be destroyed
// safely (RenderPassEntry's destructor releases the handles). // safely (RenderPassEntry's destructor releases the handles).
@@ -1187,6 +1256,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return; return;
} }
// Collect the dying handles and notify once after the loop: pipelines hashed
// on them share the entries' >kRetireAgeFrames idleness (they are only bound
// by draws that hit those entries), so the observer may destroy them
// immediately - and a single batched notification costs one pipeline-cache
// scan instead of one per evicted pass.
Vector<VkRenderPass> destroyedRenderPasses;
const Uint64 activeHash = s_hasActiveRenderPass ? s_activeRenderPass.hash : 0; const Uint64 activeHash = s_hasActiveRenderPass ? s_activeRenderPass.hash : 0;
for (auto it = m_renderPasses.begin(); it != m_renderPasses.end();) { for (auto it = m_renderPasses.begin(); it != m_renderPasses.end();) {
const Bool isActive = s_hasActiveRenderPass && it->first == activeHash; const Bool isActive = s_hasActiveRenderPass && it->first == activeHash;
@@ -1194,11 +1269,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (m_rpFastValid && m_rpFastRenderPassHash == it->first) { if (m_rpFastValid && m_rpFastRenderPassHash == it->first) {
m_rpFastValid = false; m_rpFastValid = false;
} }
destroyedRenderPasses.push_back(it->second.renderPass);
it = m_renderPasses.erase(it); it = m_renderPasses.erase(it);
} else { } else {
++it; ++it;
} }
} }
if (!destroyedRenderPasses.empty() && m_evictionObserver != nullptr) {
m_evictionObserver->OnRenderPassesDestroyed(destroyedRenderPasses);
}
} }
Bool VkRenderPassManager::BeginRenderPass(VkCommandBuffer commandBuffer, RenderPassEntry& renderPassEntry) { Bool VkRenderPassManager::BeginRenderPass(VkCommandBuffer commandBuffer, RenderPassEntry& renderPassEntry) {
@@ -157,11 +157,31 @@ namespace MobileGL::MG_Backend::DirectVulkan {
class VkRenderPassManager { class VkRenderPassManager {
public: public:
using HashType = Uint64; using HashType = Uint64;
// Notified once per OnPresent sweep with every aged-out entry's VkRenderPass
// value: pipelines are hashed on the raw handle, and once destroyed the value
// may be recycled for an incompatible pass, so dependent caches must purge
// everything keyed on them before any new pass can be created (the sweep and
// the notification run back-to-back with no creation in between; observers
// compare the values, never dereference them). Batched so a mass-idle cohort
// (shader-pack switch, dimension exit) costs the observer one pipeline-cache
// scan, not one per dying pass. The wholesale paths
// (Shutdown/RecreateSwapchain) do not notify - their callers already drop
// every pipeline outright.
class IEvictionObserver {
public:
virtual ~IEvictionObserver() = default;
virtual void OnRenderPassesDestroyed(const Vector<VkRenderPass>& renderPasses) = 0;
};
VkRenderPassManager(VkDevice device, VkRenderPassManager(VkDevice device,
VkPhysicalDevice physicalDevice, VmaAllocator allocator, const VulkanRendererConfig& config, VkPhysicalDevice physicalDevice, VmaAllocator allocator, const VulkanRendererConfig& config,
VkClearManager& clearManager, VkTextureManager& textureManager, SwapchainObject& swapchainObject); VkClearManager& clearManager, VkTextureManager& textureManager, SwapchainObject& swapchainObject);
~VkRenderPassManager(); ~VkRenderPassManager();
// Observer may be null (no notifications). Not owned.
void SetEvictionObserver(IEvictionObserver* observer) { m_evictionObserver = observer; }
Bool Initialize(); Bool Initialize();
void Shutdown(); void Shutdown();
@@ -192,6 +212,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
UnorderedMap<Uint64, RenderPassEntry> m_renderPasses; UnorderedMap<Uint64, RenderPassEntry> m_renderPasses;
// Monotonic frame counter (bumped in OnPresent) for render-pass cache aging. // Monotonic frame counter (bumped in OnPresent) for render-pass cache aging.
Uint64 m_frameCounter = 0; Uint64 m_frameCounter = 0;
IEvictionObserver* m_evictionObserver = nullptr;
// Bumped whenever a renderbuffer VkImage is (re)created; together with the texture // Bumped whenever a renderbuffer VkImage is (re)created; together with the texture
// manager's image epoch this invalidates the render-pass fast path on any attachment // manager's image epoch this invalidates the render-pass fast path on any attachment
@@ -213,6 +234,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
public: public:
struct RenderbufferResource { struct RenderbufferResource {
// deadSinceFrame sentinel: the owning weak reference has not been observed
// expired. Dead resources age past every in-flight frame before Destroy
// (see CollectRenderbufferGarbage); the GPU may still reference the image
// for frames-in-flight frames after the GL object dies.
static constexpr Uint64 kNeverObservedDead = UINT64_MAX;
WeakPtr<MG_State::GLState::RenderbufferObject> renderbuffer; WeakPtr<MG_State::GLState::RenderbufferObject> renderbuffer;
VkImage image = VK_NULL_HANDLE; VkImage image = VK_NULL_HANDLE;
VmaAllocation allocation = nullptr; VmaAllocation allocation = nullptr;
@@ -224,6 +251,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkSampleCountFlagBits sampleCount = VK_SAMPLE_COUNT_1_BIT; VkSampleCountFlagBits sampleCount = VK_SAMPLE_COUNT_1_BIT;
TextureInternalFormat internalFormat = TextureInternalFormat::Unknown; TextureInternalFormat internalFormat = TextureInternalFormat::Unknown;
Int samples = 0; Int samples = 0;
// m_frameCounter value at which the weak reference was first seen expired.
Uint64 deadSinceFrame = kNeverObservedDead;
void Destroy(VkDevice device, VmaAllocator allocator); void Destroy(VkDevice device, VmaAllocator allocator);
}; };
@@ -241,12 +270,28 @@ namespace MobileGL::MG_Backend::DirectVulkan {
ClearAttachmentPayload payload{}; ClearAttachmentPayload payload{};
}; };
// A superseded renderbuffer backing (glRenderbufferStorage respecify) parked
// until enough frame boundaries have passed that no in-flight command buffer
// can still reference it; destroyed in OnPresent (see RetireAgeFrames).
struct DeferredRenderbufferRelease {
VkImage image = VK_NULL_HANDLE;
VmaAllocation allocation = nullptr;
VkImageView view = VK_NULL_HANDLE;
Uint64 deferredAtFrame = 0;
};
UnorderedMap<MG_State::GLState::RenderbufferObject*, RenderbufferResource> m_renderbufferResources; UnorderedMap<MG_State::GLState::RenderbufferObject*, RenderbufferResource> m_renderbufferResources;
UnorderedMap<MG_State::GLState::RenderbufferObject*, PendingRenderbufferClear> m_pendingRenderbufferClears; UnorderedMap<MG_State::GLState::RenderbufferObject*, PendingRenderbufferClear> m_pendingRenderbufferClears;
Vector<DeferredRenderbufferRelease> m_deferredRenderbufferReleases;
Bool HasPendingRenderbufferClear( Bool HasPendingRenderbufferClear(
const MG_State::GLState::FramebufferAttachmentObject& attachment) const; const MG_State::GLState::FramebufferAttachmentObject& attachment) const;
void CollectRenderbufferGarbage(); void CollectRenderbufferGarbage();
// Frame-boundary margin after which a resource last referenced by a retired
// GL object (or superseded backing) is provably past every in-flight frame.
Uint64 RetireAgeFrames() const;
void DeferRenderbufferBackingRelease(RenderbufferResource& resource);
void CollectDeferredRenderbufferReleases(Bool destroyAll);
static inline XXH64_state_t* m_hashState = XXH64_createState(); static inline XXH64_state_t* m_hashState = XXH64_createState();
static inline ActiveRenderPassInfo s_activeRenderPass{}; static inline ActiveRenderPassInfo s_activeRenderPass{};
@@ -51,6 +51,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Float ResolveEffectiveMinLod(const MG_State::GLState::SamplerObject& sampler, Float effectiveMaxLod) { Float ResolveEffectiveMinLod(const MG_State::GLState::SamplerObject& sampler, Float effectiveMaxLod) {
return std::min(sampler.GetMinLod(), effectiveMaxLod); return std::min(sampler.GetMinLod(), effectiveMaxLod);
} }
// A single-level view can only ever deliver the base level, but the LOD clamp must not be
// collapsed to exactly 0: both GL and Vulkan pick magFilter over minFilter from the
// *clamped* lambda, so maxLod = 0 would make every fragment magnify and quietly retire the
// min filter. 0.25 is the value VkSamplerCreateInfo's own note prescribes for emulating
// GL's non-mipmapped minification - large enough for lambda to stay positive, small enough
// that a NEAREST mip mode still rounds down to level 0. Clamped rather than assigned, so a
// texture whose GL_TEXTURE_MAX_LOD really is 0 keeps magnifying as GL says it must.
Float ResolveSingleLevelMaxLod(const MG_State::GLState::SamplerObject& sampler, Bool singleLevelView) {
const Float maxLod = ResolveEffectiveMaxLod(sampler);
return singleLevelView ? std::min(maxLod, 0.25f) : maxLod;
}
} // namespace } // namespace
Bool VkSamplerManager::Initialize(const InitInfo& initInfo) { Bool VkSamplerManager::Initialize(const InitInfo& initInfo) {
@@ -89,15 +101,43 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_device = VK_NULL_HANDLE; m_device = VK_NULL_HANDLE;
m_config = nullptr; m_config = nullptr;
m_frameBoundaryCounter = 0;
}
void VkSamplerManager::OnFrameBoundary() {
++m_frameBoundaryCounter;
// Sweep occasionally; destroy samplers whose last use is far past every
// in-flight frame. Destroy and erase must stay atomic, or Shutdown would
// double-free the handle; an evicted key that recurs simply re-creates
// its sampler on the next miss.
constexpr Uint64 kSweepInterval = 256;
constexpr Uint64 kRetireAgeBoundaries = 1024;
if ((m_frameBoundaryCounter % kSweepInterval) != 0) {
return;
}
for (auto it = m_samplers.begin(); it != m_samplers.end();) {
auto& entry = it->second;
if (m_frameBoundaryCounter - entry.lastUsedFrameBoundary > kRetireAgeBoundaries) {
if (m_device != VK_NULL_HANDLE && entry.handle != VK_NULL_HANDLE) {
vkDestroySampler(m_device, entry.handle, nullptr);
}
it = m_samplers.erase(it);
} else {
++it;
}
}
} }
Uint64 VkSamplerManager::BuildSamplerKey(const MG_State::GLState::SamplerObject& sampler, Uint64 VkSamplerManager::BuildSamplerKey(const MG_State::GLState::SamplerObject& sampler,
const MG_State::GLState::ITextureObject& texture, const MG_State::GLState::ITextureObject& texture,
Bool forceNearestFiltering) const { Bool forceNearestFiltering, Bool singleLevelView) const {
MOBILEGL_ASSERT(m_config != nullptr, "VkSamplerManager::BuildSamplerKey: m_config is null"); MOBILEGL_ASSERT(m_config != nullptr, "VkSamplerManager::BuildSamplerKey: m_config is null");
XXHASH_VERIFY(XXH64_reset(m_hashState, m_config->CacheVersion)); XXHASH_VERIFY(XXH64_reset(m_hashState, m_config->CacheVersion));
XXHASH_VERIFY(XXH64_update(m_hashState, &forceNearestFiltering, sizeof(forceNearestFiltering))); XXHASH_VERIFY(XXH64_update(m_hashState, &forceNearestFiltering, sizeof(forceNearestFiltering)));
XXHASH_VERIFY(XXH64_update(m_hashState, &singleLevelView, sizeof(singleLevelView)));
const auto minFilter = sampler.GetMinFilter(); const auto minFilter = sampler.GetMinFilter();
XXHASH_VERIFY(XXH64_update(m_hashState, &minFilter, sizeof(minFilter))); XXHASH_VERIFY(XXH64_update(m_hashState, &minFilter, sizeof(minFilter)));
@@ -111,7 +151,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(m_hashState, &wrapT, sizeof(wrapT))); XXHASH_VERIFY(XXH64_update(m_hashState, &wrapT, sizeof(wrapT)));
const auto wrapR = sampler.GetWrapR(); const auto wrapR = sampler.GetWrapR();
XXHASH_VERIFY(XXH64_update(m_hashState, &wrapR, sizeof(wrapR))); XXHASH_VERIFY(XXH64_update(m_hashState, &wrapR, sizeof(wrapR)));
const auto maxLod = ResolveEffectiveMaxLod(sampler); const auto maxLod = ResolveSingleLevelMaxLod(sampler, singleLevelView);
const auto minLod = ResolveEffectiveMinLod(sampler, maxLod); const auto minLod = ResolveEffectiveMinLod(sampler, maxLod);
XXHASH_VERIFY(XXH64_update(m_hashState, &minLod, sizeof(minLod))); XXHASH_VERIFY(XXH64_update(m_hashState, &minLod, sizeof(minLod)));
XXHASH_VERIFY(XXH64_update(m_hashState, &maxLod, sizeof(maxLod))); XXHASH_VERIFY(XXH64_update(m_hashState, &maxLod, sizeof(maxLod)));
@@ -133,10 +173,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkSampler VkSamplerManager::GetOrCreateSampler(const MG_State::GLState::SamplerObject& sampler, VkSampler VkSamplerManager::GetOrCreateSampler(const MG_State::GLState::SamplerObject& sampler,
const MG_State::GLState::ITextureObject& texture, const MG_State::GLState::ITextureObject& texture,
Bool forceNearestFiltering) { Bool forceNearestFiltering, Uint32 viewLevelCount) {
const Uint64 key = BuildSamplerKey(sampler, texture, forceNearestFiltering); // A view that exposes a single mip level has no second level to blend with, so GL's
// *_MIPMAP_* minification filters degenerate to plain filtering on the base level -
// sampling is unchanged by pinning the Vulkan sampler to NEAREST mip mode at LOD 0.
// It is not cosmetic: MobileGL backs such a view with a fully allocated mip chain whose
// tail is never written, and a LINEAR mip mode lets the texture unit issue the level+1
// fetch anyway. On Adreno that fetch lands in uninitialized UBWC pages (or past the
// allocation for a genuinely single-level image) and faults the GPU - the same failure
// the default-framebuffer blit shader had to work around with an explicit-LOD sample.
const Bool singleLevelView = viewLevelCount == 1;
const Uint64 key = BuildSamplerKey(sampler, texture, forceNearestFiltering, singleLevelView);
auto it = m_samplers.find(key); auto it = m_samplers.find(key);
if (it != m_samplers.end()) { if (it != m_samplers.end()) {
it->second.lastUsedFrameBoundary = m_frameBoundaryCounter;
return it->second.handle; return it->second.handle;
} }
@@ -144,8 +194,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerInfo.sType = VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO; samplerInfo.sType = VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO;
samplerInfo.magFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMagFilter()); samplerInfo.magFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMagFilter());
samplerInfo.minFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMinFilter()); samplerInfo.minFilter = forceNearestFiltering ? VK_FILTER_NEAREST : ToVkFilter(sampler.GetMinFilter());
samplerInfo.mipmapMode = forceNearestFiltering ? VK_SAMPLER_MIPMAP_MODE_NEAREST samplerInfo.mipmapMode = (forceNearestFiltering || singleLevelView)
: ToVkMipmapMode(sampler.GetMipmapMode()); ? VK_SAMPLER_MIPMAP_MODE_NEAREST
: ToVkMipmapMode(sampler.GetMipmapMode());
samplerInfo.addressModeU = ToVkAddressMode(sampler.GetWrapS()); samplerInfo.addressModeU = ToVkAddressMode(sampler.GetWrapS());
samplerInfo.addressModeV = ToVkAddressMode(sampler.GetWrapT()); samplerInfo.addressModeV = ToVkAddressMode(sampler.GetWrapT());
samplerInfo.addressModeW = ToVkAddressMode(sampler.GetWrapR()); samplerInfo.addressModeW = ToVkAddressMode(sampler.GetWrapR());
@@ -157,7 +208,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerInfo.maxAnisotropy = maxAnisotropy; samplerInfo.maxAnisotropy = maxAnisotropy;
samplerInfo.compareEnable = sampler.GetCompareMode() == SamplerCompareMode::CompareToTexture ? VK_TRUE : VK_FALSE; samplerInfo.compareEnable = sampler.GetCompareMode() == SamplerCompareMode::CompareToTexture ? VK_TRUE : VK_FALSE;
samplerInfo.compareOp = ToVkCompareOp(ResolveCompareFunc(sampler, texture)); samplerInfo.compareOp = ToVkCompareOp(ResolveCompareFunc(sampler, texture));
samplerInfo.maxLod = ResolveEffectiveMaxLod(sampler); // Must match BuildSamplerKey's resolution exactly.
samplerInfo.maxLod = ResolveSingleLevelMaxLod(sampler, singleLevelView);
samplerInfo.minLod = ResolveEffectiveMinLod(sampler, samplerInfo.maxLod); samplerInfo.minLod = ResolveEffectiveMinLod(sampler, samplerInfo.maxLod);
samplerInfo.borderColor = ResolveVkBorderColor(sampler, texture); samplerInfo.borderColor = ResolveVkBorderColor(sampler, texture);
samplerInfo.unnormalizedCoordinates = VK_FALSE; samplerInfo.unnormalizedCoordinates = VK_FALSE;
@@ -169,6 +221,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
entry.handle = vkSampler; entry.handle = vkSampler;
entry.externalIndex = sampler.GetExternalIndex(); entry.externalIndex = sampler.GetExternalIndex();
entry.version = sampler.GetVersion(); entry.version = sampler.GetVersion();
entry.lastUsedFrameBoundary = m_frameBoundaryCounter;
m_samplers[key] = entry; m_samplers[key] = entry;
return vkSampler; return vkSampler;
} }
@@ -33,20 +33,38 @@ public:
Bool Initialize(const InitInfo& initInfo); Bool Initialize(const InitInfo& initInfo);
void Shutdown(); void Shutdown();
// viewLevelCount is the mip-level count of the image view this sampler will be paired
// with; 0 means "unknown, do not narrow". See GetOrCreateSampler for why it matters.
VkSampler GetOrCreateSampler(const MG_State::GLState::SamplerObject& sampler, VkSampler GetOrCreateSampler(const MG_State::GLState::SamplerObject& sampler,
const MG_State::GLState::ITextureObject& texture, const MG_State::GLState::ITextureObject& texture,
Bool forceNearestFiltering = false); Bool forceNearestFiltering = false,
Uint32 viewLevelCount = 0);
// Frame boundary hook: ages the sampler cache and destroys samplers not used
// for many frames. The key hashes continuous float state (lodBias, LOD clamps,
// anisotropy), so an app animating those would otherwise mint an unbounded
// stream of never-destroyed VkSamplers and eventually exhaust the device's
// maxSamplerAllocationCount. A sampler idle for over a thousand frame
// boundaries cannot be referenced by any in-flight command buffer (frames in
// flight are single digits), and every descriptor set the GPU consumes is
// written that same frame with live handles (the per-binding resolve memo and
// descriptor-set reuse are both frame-reset), so destruction here needs no
// fence wait. Self-gated: one counter bump and compare except on sweep
// boundaries.
void OnFrameBoundary();
private: private:
struct SamplerCacheEntry { struct SamplerCacheEntry {
VkSampler handle = VK_NULL_HANDLE; VkSampler handle = VK_NULL_HANDLE;
Uint externalIndex = 0; Uint externalIndex = 0;
Uint16 version = 0; Uint16 version = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age have their VkSampler destroyed.
Uint64 lastUsedFrameBoundary = 0;
}; };
Uint64 BuildSamplerKey(const MG_State::GLState::SamplerObject& sampler, Uint64 BuildSamplerKey(const MG_State::GLState::SamplerObject& sampler,
const MG_State::GLState::ITextureObject& texture, const MG_State::GLState::ITextureObject& texture,
Bool forceNearestFiltering) const; Bool forceNearestFiltering, Bool singleLevelView) const;
static VkFilter ToVkFilter(SamplerFilterMode mode); static VkFilter ToVkFilter(SamplerFilterMode mode);
static VkSamplerMipmapMode ToVkMipmapMode(SamplerMipmapMode mode); static VkSamplerMipmapMode ToVkMipmapMode(SamplerMipmapMode mode);
static VkSamplerAddressMode ToVkAddressMode(SamplerWrapMode mode); static VkSamplerAddressMode ToVkAddressMode(SamplerWrapMode mode);
@@ -67,6 +85,8 @@ private:
Bool m_samplerAnisotropySupported = false; Bool m_samplerAnisotropySupported = false;
Float m_maxSamplerAnisotropy = 1.0f; Float m_maxSamplerAnisotropy = 1.0f;
UnorderedMap<Uint64, SamplerCacheEntry> m_samplers; UnorderedMap<Uint64, SamplerCacheEntry> m_samplers;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameBoundaryCounter = 0;
static inline XXH64_state_t* m_hashState = XXH64_createState(); static inline XXH64_state_t* m_hashState = XXH64_createState();
}; };
} // namespace MobileGL::MG_Backend::DirectVulkan } // namespace MobileGL::MG_Backend::DirectVulkan
@@ -587,6 +587,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_allocator = initInfo.allocator; m_allocator = initInfo.allocator;
m_commandPool = initInfo.commandPool; m_commandPool = initInfo.commandPool;
m_graphicsQueue = initInfo.graphicsQueue; m_graphicsQueue = initInfo.graphicsQueue;
m_imageFormatListSupported = initInfo.imageFormatListSupported;
m_currentFrameIndex = 0; m_currentFrameIndex = 0;
m_deferredReleases.clear(); m_deferredReleases.clear();
m_deferredReleases.resize(initInfo.frameCount); m_deferredReleases.resize(initInfo.frameCount);
@@ -609,6 +610,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
DestroyDeferredReleases(); DestroyDeferredReleases();
m_textureResources.clear(); m_textureResources.clear();
m_aliveObjects.clear(); m_aliveObjects.clear();
m_storageImageTextures.clear();
m_device = VK_NULL_HANDLE; m_device = VK_NULL_HANDLE;
m_physicalDevice = VK_NULL_HANDLE; m_physicalDevice = VK_NULL_HANDLE;
@@ -627,6 +629,26 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frameIndex, m_deferredViewReleases.size()); frameIndex, m_deferredViewReleases.size());
m_currentFrameIndex = frameIndex; m_currentFrameIndex = frameIndex;
CollectDeferredReleases(frameIndex); CollectDeferredReleases(frameIndex);
// Frame-boundary GC: every 64 frame boundaries (~1 s at 60 fps) bounds the reclaim
// latency for dead textures regardless of draw traffic — workloads that churn
// textures through clears/readbacks alone never reach the draw-gated
// CollectGarbage. Must run after CollectDeferredReleases above: the prune defers
// its releases into this frame's slot, which was just drained, so they are
// destroyed only after the slot's fence has been waited again one full frame-ring
// cycle from now (never while an in-flight frame may still reference them).
constexpr Uint32 kGcFrameInterval = 64;
++m_gcFrameCounter;
if (m_gcFrameCounter % kGcFrameInterval == 0) {
PruneDeadTextures();
}
}
void VkTextureManager::CollectAllDeferredReleases() {
const SizeT frameCount = std::min(m_deferredReleases.size(), m_deferredViewReleases.size());
for (SizeT frameIndex = 0; frameIndex < frameCount; ++frameIndex) {
CollectDeferredReleases(static_cast<Uint32>(frameIndex));
}
} }
void VkTextureManager::EraseTrackedTexture(const TextureIdentity& identity) { void VkTextureManager::EraseTrackedTexture(const TextureIdentity& identity) {
@@ -636,6 +658,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
m_textureResources.erase(resourceIt); m_textureResources.erase(resourceIt);
} }
m_aliveObjects.erase(identity); m_aliveObjects.erase(identity);
m_storageImageTextures.erase(identity);
} }
void VkTextureManager::PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture) { void VkTextureManager::PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture) {
@@ -702,9 +725,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// construction introduces a new identity. Doing this unconditionally made every // construction introduces a new identity. Doing this unconditionally made every
// sampled-texture sync scan the entire alive-texture map per draw. // sampled-texture sync scan the entire alive-texture map per draw.
if (aliveIt == m_aliveObjects.end()) { if (aliveIt == m_aliveObjects.end()) {
WeakPtr<MG_State::GLState::ITextureObject> aliveTexture;
const auto& liveTexture = MG_State::pGLContext->GetTextureObject(texture.GetExternalIndex()); const auto& liveTexture = MG_State::pGLContext->GetTextureObject(texture.GetExternalIndex());
if (liveTexture && liveTexture.get() == &texture) { if (liveTexture && liveTexture.get() == &texture) {
m_aliveObjects[identity] = WeakPtr<MG_State::GLState::ITextureObject>(liveTexture); aliveTexture = liveTexture;
} else {
// The name lookup legally fails while the object is alive: the name was
// deleted with the texture still attached to an FBO (the attachment's
// SharedPtr keeps it alive), or the name was reused by a new texture, or
// this is a default texture object (name 0 lives outside the name map).
// Register through the object's own control block so the resource created
// below still participates in weak-expiry GC instead of becoming an
// orphan no reclamation path can reach until Shutdown.
aliveTexture = texture.weak_from_this();
}
if (!aliveTexture.expired()) {
m_aliveObjects[identity] = Move(aliveTexture);
PruneStaleTextureAliases(&texture); PruneStaleTextureAliases(&texture);
} }
} }
@@ -1156,8 +1192,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return ok; return ok;
} }
void VkTextureManager::MarkStorageImageTexture(MG_State::GLState::ITextureObject& texture) {
m_storageImageTextures.insert(MakeTextureIdentity(&texture));
}
Bool VkTextureManager::NeedsStorageUsageUpgrade(MG_State::GLState::ITextureObject& texture) const {
const TextureIdentity identity = MakeTextureIdentity(&texture);
if (m_storageImageTextures.find(identity) == m_storageImageTextures.end()) {
return false;
}
const auto it = m_textureResources.find(identity);
// No image yet: the first sync creates it with STORAGE straight away, so there is nothing
// to preserve and nothing to order against.
return it != m_textureResources.end() && it->second.image != VK_NULL_HANDLE &&
!it->second.storageUsageResolved;
}
Bool VkTextureManager::NeedsStorageImagePreparation(MG_State::GLState::ITextureObject& texture) const { Bool VkTextureManager::NeedsStorageImagePreparation(MG_State::GLState::ITextureObject& texture) const {
const auto it = m_textureResources.find(MakeTextureIdentity(&texture)); const TextureIdentity identity = MakeTextureIdentity(&texture);
const auto it = m_textureResources.find(identity);
if (it == m_textureResources.end()) { if (it == m_textureResources.end()) {
return true; return true;
} }
@@ -1165,6 +1218,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (resource.image == VK_NULL_HANDLE || resource.layout != VK_IMAGE_LAYOUT_GENERAL) { if (resource.image == VK_NULL_HANDLE || resource.layout != VK_IMAGE_LAYOUT_GENERAL) {
return true; return true;
} }
// The image predates this texture's first image-unit binding, so it was created without
// STORAGE usage and has to be recreated - which is illegal inside a render pass.
if (!resource.storageUsageResolved &&
m_storageImageTextures.find(identity) != m_storageImageTextures.end()) {
return true;
}
// Mirror SyncTexture's cross-draw skip condition: any version drift means the sync // Mirror SyncTexture's cross-draw skip condition: any version drift means the sync
// path may upload or rebuild, both of which need the render pass ended first. // path may upload or rebuild, both of which need the render pass ended first.
const auto* mipTexture = MG_State::GLState::AsMipmapTexture(&texture); const auto* mipTexture = MG_State::GLState::AsMipmapTexture(&texture);
@@ -1212,10 +1271,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
SizeT VkTextureManager::CollectGarbage() { SizeT VkTextureManager::CollectGarbage() {
// Draw-gated stagger (1 in 256 calls): keeps the per-draw cost at one counter
// bump. The guaranteed reclaim path is the frame-boundary prune in BeginFrame;
// this remains as a cheap assist so draw-heavy workloads reclaim sooner.
m_gcCounter++; m_gcCounter++;
if (m_gcCounter != 0) { if (m_gcCounter != 0) {
return 0; return 0;
} }
return PruneDeadTextures();
}
SizeT VkTextureManager::PruneDeadTextures() {
// Erasing entries would dangle the raw TextureResource pointers memoized for the
// current draw; every call path (BeginFrame, and CollectGarbage at the top of a
// freshly opened draw-sync scope) runs before any memo entry is recorded.
MOBILEGL_ASSERT(m_drawSyncedThisDraw.empty(),
"PruneDeadTextures: draw-sync memo holds raw resource pointers an erase would dangle");
Vector<MG_State::GLState::ITextureObject*> expiredTextures; Vector<MG_State::GLState::ITextureObject*> expiredTextures;
expiredTextures.reserve(m_aliveObjects.size()); expiredTextures.reserve(m_aliveObjects.size());
@@ -1227,7 +1298,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (auto* texture : expiredTextures) { for (auto* texture : expiredTextures) {
PruneStaleTextureAliases(texture); PruneStaleTextureAliases(texture);
} }
return expiredTextures.size(); SizeT prunedCount = expiredTextures.size();
// Orphan sweep: after the pass above, m_aliveObjects holds only live entries.
// Registration in SyncTextureAndGetDescriptor cannot fail for a SharedPtr-owned
// texture (weak_from_this fallback), so a resource whose identity has no alive
// entry has no trackable owner: its GL-side object is gone, or was never
// shared-owned, in which case recreation on a later sync is the safe fallback.
// Destruction goes through the per-frame deferred queues, never immediate.
Vector<TextureIdentity> orphanIdentities;
for (auto it = m_textureResources.begin(); it != m_textureResources.end(); ++it) {
if (m_aliveObjects.find(it->first) == m_aliveObjects.end()) {
orphanIdentities.emplace_back(it->first);
}
}
for (const auto& identity : orphanIdentities) {
EraseTrackedTexture(identity);
}
prunedCount += orphanIdentities.size();
return prunedCount;
} }
Bool VkTextureManager::SyncTexture(MG_State::GLState::ITextureObject &texture, Bool VkTextureManager::SyncTexture(MG_State::GLState::ITextureObject &texture,
@@ -1241,7 +1330,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const auto* syncingMipTexture = MG_State::GLState::AsMipmapTexture(&texture); const auto* syncingMipTexture = MG_State::GLState::AsMipmapTexture(&texture);
const Uint32 syncingMipLevelCount = const Uint32 syncingMipLevelCount =
syncingMipTexture != nullptr ? syncingMipTexture->GetMipmapLevelCount() : 0u; syncingMipTexture != nullptr ? syncingMipTexture->GetMipmapLevelCount() : 0u;
if (outResource.image != VK_NULL_HANDLE && // A pending storage-usage upgrade also has to bust the skip: nothing about the texture's
// content or params changed, but the image itself must be recreated with STORAGE usage
// before it can back an image-unit descriptor.
const Bool storageUpgradePending =
!outResource.storageUsageResolved &&
m_storageImageTextures.find(MakeTextureIdentity(&texture)) != m_storageImageTextures.end();
if (outResource.image != VK_NULL_HANDLE && !storageUpgradePending &&
outResource.syncedContentVersion == syncingContentVersion && outResource.syncedContentVersion == syncingContentVersion &&
outResource.syncedTextureParamsVersion == texture.GetTextureParamsVersion() && outResource.syncedTextureParamsVersion == texture.GetTextureParamsVersion() &&
outResource.syncedMipLevelCount == syncingMipLevelCount) { outResource.syncedMipLevelCount == syncingMipLevelCount) {
@@ -1352,16 +1447,44 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VkImageAspectFlags aspect = GetAspectMaskForFormat(format); const VkImageAspectFlags aspect = GetAspectMaskForFormat(format);
VkFormatProperties formatProperties{}; VkFormatProperties formatProperties{};
vkGetPhysicalDeviceFormatProperties(m_physicalDevice, format, &formatProperties); vkGetPhysicalDeviceFormatProperties(m_physicalDevice, format, &formatProperties);
const Bool supportsStorageImage = // Only textures that have actually been bound to a GL image unit get STORAGE usage (and
// the MUTABLE_FORMAT it drags in for format-reinterpreting image views). Requesting it
// for every storage-capable colour texture costs real bandwidth: Adreno cannot keep UBWC
// compression on an image that may be written through a storage descriptor, so the whole
// render target - MC's included - runs uncompressed. MarkStorageImageTexture upgrades a
// texture before its first image-unit draw, and the usage below feeds the compatibility
// check so the upgrade recreates the image.
const Bool markedAsStorageImage =
m_storageImageTextures.find(MakeTextureIdentity(
const_cast<MG_State::GLState::ITextureObject*>(&texture))) != m_storageImageTextures.end();
// Storage-image CAPABILITY (does the format allow it at all) is deliberately separate from
// whether this texture actually needs the usage. MUTABLE_FORMAT keys off capability, as
// before: format-reinterpreting views are not a storage-only concern - the SAMPLED path
// needs them too (GetOrCreateSampledImageView bails out without it, see ~line 892), so
// tying MUTABLE_FORMAT to the image-unit mark would break sampled format reinterpretation
// for every texture that never becomes a storage image.
const Bool storageImageCapable =
!isMultisampleTexture && !isMultisampleTexture &&
(aspect & VK_IMAGE_ASPECT_COLOR_BIT) != 0 && (aspect & VK_IMAGE_ASPECT_COLOR_BIT) != 0 &&
(formatProperties.optimalTilingFeatures & VK_FORMAT_FEATURE_STORAGE_IMAGE_BIT) != 0; (formatProperties.optimalTilingFeatures & VK_FORMAT_FEATURE_STORAGE_IMAGE_BIT) != 0;
const Bool supportsStorageImage = storageImageCapable && markedAsStorageImage;
VkImageCreateFlags imageCreateFlags = shapeInfo.imageFlags; VkImageCreateFlags imageCreateFlags = shapeInfo.imageFlags;
if (supportsStorageImage && IsMutableStorageImageFormat(format) && if (storageImageCapable && IsMutableStorageImageFormat(format) &&
m_mutableFormatUnsupported.find(format) == m_mutableFormatUnsupported.end()) { m_mutableFormatUnsupported.find(format) == m_mutableFormatUnsupported.end()) {
imageCreateFlags |= VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT; imageCreateFlags |= VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT;
} }
VkImageUsageFlags desiredUsage =
VK_IMAGE_USAGE_SAMPLED_BIT |
(supportsStorageImage ? VK_IMAGE_USAGE_STORAGE_BIT : 0) |
((aspect & VK_IMAGE_ASPECT_COLOR_BIT) ? VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT : 0) |
(((aspect & VK_IMAGE_ASPECT_DEPTH_BIT) || (aspect & VK_IMAGE_ASPECT_STENCIL_BIT)) ?
VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT :
0);
if (!isMultisampleTexture) {
desiredUsage |= VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT;
}
const Bool compatible = resource.image != VK_NULL_HANDLE && resource.format == format && const Bool compatible = resource.image != VK_NULL_HANDLE && resource.format == format &&
resource.extent.width == static_cast<Uint32>(texelSize.x()) && resource.extent.width == static_cast<Uint32>(texelSize.x()) &&
resource.extent.height == static_cast<Uint32>(texelSize.y()) && resource.extent.height == static_cast<Uint32>(texelSize.y()) &&
@@ -1370,6 +1493,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.viewType == shapeInfo.viewType && resource.viewType == shapeInfo.viewType &&
resource.sampleCount == resolvedSampleCount && resource.sampleCount == resolvedSampleCount &&
resource.imageCreateFlags == imageCreateFlags && resource.imageCreateFlags == imageCreateFlags &&
resource.usageFlags == desiredUsage &&
resource.mipLevels == backingMipLevels; resource.mipLevels == backingMipLevels;
if (compatible) { if (compatible) {
if (resource.perMipViews.size() != backingMipLevels) { if (resource.perMipViews.size() != backingMipLevels) {
@@ -1378,6 +1502,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (resource.perMipSampledViews.size() != backingMipLevels) { if (resource.perMipSampledViews.size() != backingMipLevels) {
resource.perMipSampledViews.resize(backingMipLevels, VK_NULL_HANDLE); resource.perMipSampledViews.resize(backingMipLevels, VK_NULL_HANDLE);
} }
// Keeping the image is itself the answer to the mark: either it already carries
// STORAGE, or this format can never carry it. Either way there is nothing left to
// recreate, so stop reporting the texture as needing preparation.
resource.storageUsageResolved = markedAsStorageImage;
return true; return true;
} }
@@ -1392,7 +1520,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.sampleCount == resolvedSampleCount && resource.sampleCount == resolvedSampleCount &&
resource.imageCreateFlags == imageCreateFlags && resource.imageCreateFlags == imageCreateFlags &&
resolvedSampleCount == VK_SAMPLE_COUNT_1_BIT && resolvedSampleCount == VK_SAMPLE_COUNT_1_BIT &&
resource.mipLevels < backingMipLevels && // '<=' rather than '<': a storage-usage upgrade recreates the image with an
// unchanged mip count, and its contents (a render target's pixels live only on the
// GPU) still have to survive. The vkCmdCopyImage below copies min(mipLevels).
resource.mipLevels <= backingMipLevels &&
resource.layout != VK_IMAGE_LAYOUT_UNDEFINED; resource.layout != VK_IMAGE_LAYOUT_UNDEFINED;
std::unique_ptr<TextureResource> preservedResource; std::unique_ptr<TextureResource> preservedResource;
@@ -1414,16 +1545,37 @@ namespace MobileGL::MG_Backend::DirectVulkan {
imageInfo.format = format; imageInfo.format = format;
imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL; imageInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED; imageInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
imageInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT | imageInfo.usage = desiredUsage;
(supportsStorageImage ? VK_IMAGE_USAGE_STORAGE_BIT : 0) |
((aspect & VK_IMAGE_ASPECT_COLOR_BIT) ? VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT : 0) |
(((aspect & VK_IMAGE_ASPECT_DEPTH_BIT) || (aspect & VK_IMAGE_ASPECT_STENCIL_BIT)) ?
VK_IMAGE_USAGE_DEPTH_STENCIL_ATTACHMENT_BIT :
0);
if (!isMultisampleTexture) {
imageInfo.usage |= VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_TRANSFER_SRC_BIT;
}
imageInfo.samples = resolvedSampleCount; imageInfo.samples = resolvedSampleCount;
// Bound the mutability. A blindly-mutable image has to be laid out so that ANY format in
// its compatibility class can be viewed, which costs bandwidth compression on tilers;
// naming the exact set instead lets the driver keep it. Only safe when that set really is
// exhaustive, so it is restricted to textures that are not image-unit bound: sampled views
// can only ever ask for ResolveSampledImageViewFormat's output, whereas glBindImageTexture
// may name any compatible format, which nothing here can enumerate ahead of time.
Vector<VkFormat> viewFormats;
VkImageFormatListCreateInfo formatListInfo{};
if (m_imageFormatListSupported && !supportsStorageImage &&
(imageInfo.flags & VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT) != 0) {
viewFormats.push_back(format);
for (const SamplerNumericDomain domain : {SamplerNumericDomain::Float,
SamplerNumericDomain::SignedInteger,
SamplerNumericDomain::UnsignedInteger}) {
const VkFormat viewFormat = ResolveSampledImageViewFormat(format, domain);
if (viewFormat == VK_FORMAT_UNDEFINED) {
continue;
}
if (std::find(viewFormats.begin(), viewFormats.end(), viewFormat) == viewFormats.end()) {
viewFormats.push_back(viewFormat);
}
}
formatListInfo.sType = VK_STRUCTURE_TYPE_IMAGE_FORMAT_LIST_CREATE_INFO;
formatListInfo.viewFormatCount = static_cast<Uint32>(viewFormats.size());
formatListInfo.pViewFormats = viewFormats.data();
imageInfo.pNext = &formatListInfo;
}
if (isMultisampleTexture || (imageInfo.flags & VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT) != 0) { if (isMultisampleTexture || (imageInfo.flags & VK_IMAGE_CREATE_MUTABLE_FORMAT_BIT) != 0) {
VkImageFormatProperties imageFormatProperties{}; VkImageFormatProperties imageFormatProperties{};
VkResult imageFormatResult = vkGetPhysicalDeviceImageFormatProperties( VkResult imageFormatResult = vkGetPhysicalDeviceImageFormatProperties(
@@ -1480,6 +1632,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.viewType = shapeInfo.viewType; resource.viewType = shapeInfo.viewType;
resource.sampleCount = resolvedSampleCount; resource.sampleCount = resolvedSampleCount;
resource.imageCreateFlags = imageCreateFlags; resource.imageCreateFlags = imageCreateFlags;
resource.usageFlags = imageInfo.usage;
resource.storageUsageResolved = markedAsStorageImage;
resource.syncedTextureParamsVersion = 0; resource.syncedTextureParamsVersion = 0;
if (preservedResource) { if (preservedResource) {
@@ -53,6 +53,9 @@ public:
VkCommandPool commandPool = VK_NULL_HANDLE; VkCommandPool commandPool = VK_NULL_HANDLE;
VkQueue graphicsQueue = VK_NULL_HANDLE; VkQueue graphicsQueue = VK_NULL_HANDLE;
Uint32 frameCount = 0; Uint32 frameCount = 0;
// VK_KHR_image_format_list is enabled: MUTABLE_FORMAT images can name the exact set of
// formats they will be viewed as, which is what lets a tiler keep them compressed.
Bool imageFormatListSupported = false;
}; };
struct TextureResource { struct TextureResource {
@@ -157,6 +160,17 @@ public:
VkImageViewType viewType = VK_IMAGE_VIEW_TYPE_2D; VkImageViewType viewType = VK_IMAGE_VIEW_TYPE_2D;
VkSampleCountFlagBits sampleCount = VK_SAMPLE_COUNT_1_BIT; VkSampleCountFlagBits sampleCount = VK_SAMPLE_COUNT_1_BIT;
VkImageCreateFlags imageCreateFlags = 0; VkImageCreateFlags imageCreateFlags = 0;
// Usage the live image was created with. STORAGE is only requested for textures that
// have actually been bound to a GL image unit, because on Adreno a storage-capable
// image loses UBWC bandwidth compression; a later image binding upgrades the usage
// and recreates the image, so the resolved usage has to be part of the compatibility
// check that decides whether the existing image can be kept.
VkImageUsageFlags usageFlags = 0;
// True once this image was (re)resolved while the texture was already marked as an
// image-unit texture. Distinguishes "not upgraded yet" from "cannot be upgraded"
// (a format whose optimalTilingFeatures lack STORAGE_IMAGE never gains the bit), so
// NeedsStorageImagePreparation cannot ask for a recreate that will never happen.
Bool storageUsageResolved = false;
Uint16 syncedTextureParamsVersion = 0; Uint16 syncedTextureParamsVersion = 0;
// Snapshot of ITextureObject::GetContentVersion() at the last successful sync; // Snapshot of ITextureObject::GetContentVersion() at the last successful sync;
// lets SyncTexture skip the whole re-check/re-upload when content is unchanged. // lets SyncTexture skip the whole re-check/re-upload when content is unchanged.
@@ -190,6 +204,8 @@ public:
std::swap(this->viewType, that.viewType); std::swap(this->viewType, that.viewType);
std::swap(this->sampleCount, that.sampleCount); std::swap(this->sampleCount, that.sampleCount);
std::swap(this->imageCreateFlags, that.imageCreateFlags); std::swap(this->imageCreateFlags, that.imageCreateFlags);
std::swap(this->usageFlags, that.usageFlags);
std::swap(this->storageUsageResolved, that.storageUsageResolved);
std::swap(this->syncedTextureParamsVersion, that.syncedTextureParamsVersion); std::swap(this->syncedTextureParamsVersion, that.syncedTextureParamsVersion);
std::swap(this->syncedContentVersion, that.syncedContentVersion); std::swap(this->syncedContentVersion, that.syncedContentVersion);
std::swap(this->syncedMipLevelCount, that.syncedMipLevelCount); std::swap(this->syncedMipLevelCount, that.syncedMipLevelCount);
@@ -251,6 +267,8 @@ public:
viewType = VK_IMAGE_VIEW_TYPE_2D; viewType = VK_IMAGE_VIEW_TYPE_2D;
sampleCount = VK_SAMPLE_COUNT_1_BIT; sampleCount = VK_SAMPLE_COUNT_1_BIT;
imageCreateFlags = 0; imageCreateFlags = 0;
usageFlags = 0;
storageUsageResolved = false;
syncedTextureParamsVersion = 0; syncedTextureParamsVersion = 0;
syncedContentVersion = 0; syncedContentVersion = 0;
syncedMipLevelCount = 0; syncedMipLevelCount = 0;
@@ -267,6 +285,10 @@ public:
Bool Initialize(const InitInfo& initInfo); Bool Initialize(const InitInfo& initInfo);
void Shutdown(); void Shutdown();
void BeginFrame(Uint32 frameIndex); void BeginFrame(Uint32 frameIndex);
// Drains every frame slot's deferred image/view releases. Only valid when
// the caller has proven every queue submission complete; used by the
// present-less frame-boundary drain.
void CollectAllDeferredReleases();
TextureResource* SyncTextureAndGetDescriptor( TextureResource* SyncTextureAndGetDescriptor(
MG_State::GLState::ITextureObject& texture); MG_State::GLState::ITextureObject& texture);
@@ -285,6 +307,17 @@ public:
VkImageLayout newLayout); VkImageLayout newLayout);
Bool TransitionTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture); Bool TransitionTextureForSampling(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
Bool TransitionTextureForStorageImage(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture); Bool TransitionTextureForStorageImage(VkCommandBuffer commandBuffer, MG_State::GLState::ITextureObject& texture);
// Records that this texture is bound to a GL image unit, so its image must carry
// VK_IMAGE_USAGE_STORAGE_BIT. Must be called before NeedsStorageImagePreparation, and
// therefore before the render pass is committed: an image that has to be upgraded is
// recreated, which is illegal inside a render pass. Sticky for the texture's lifetime -
// GL lets an image binding come and go, and re-creating the image every time it does
// would cost far more than the compression it wins back.
void MarkStorageImageTexture(MG_State::GLState::ITextureObject& texture);
// True when this texture is marked but its live image predates the mark, i.e. the next sync
// will recreate it with STORAGE usage and copy the old contents forward. Callers use this to
// submit their pending recording first, so that copy cannot read pre-flush content.
Bool NeedsStorageUsageUpgrade(MG_State::GLState::ITextureObject& texture) const;
// Non-mutating probe for the per-draw storage-image fast path: true when preparing this // Non-mutating probe for the per-draw storage-image fast path: true when preparing this
// texture as a storage image may need work that is illegal inside a render pass (resource // texture as a storage image may need work that is illegal inside a render pass (resource
// creation, dirty-content upload, or a layout transition to GENERAL). Unknown state reports // creation, dirty-content upload, or a layout transition to GENERAL). Unknown state reports
@@ -364,15 +397,20 @@ private:
static TextureIdentity MakeTextureIdentity(MG_State::GLState::ITextureObject* texture); static TextureIdentity MakeTextureIdentity(MG_State::GLState::ITextureObject* texture);
void EraseTrackedTexture(const TextureIdentity& identity); void EraseTrackedTexture(const TextureIdentity& identity);
void PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture); void PruneStaleTextureAliases(MG_State::GLState::ITextureObject* texture);
SizeT PruneDeadTextures();
VkDevice m_device = VK_NULL_HANDLE; VkDevice m_device = VK_NULL_HANDLE;
VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE; VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE;
VmaAllocator m_allocator = nullptr; VmaAllocator m_allocator = nullptr;
VkCommandPool m_commandPool = VK_NULL_HANDLE; VkCommandPool m_commandPool = VK_NULL_HANDLE;
VkQueue m_graphicsQueue = VK_NULL_HANDLE; VkQueue m_graphicsQueue = VK_NULL_HANDLE;
Bool m_imageFormatListSupported = false;
Uint32 m_currentFrameIndex = 0; Uint32 m_currentFrameIndex = 0;
Uint8 m_gcCounter = 0; Uint8 m_gcCounter = 0;
// Frame-boundary GC gate: counts BeginFrame calls, not draws, so texture churn
// through non-draw paths (FBO clears, readbacks) still reaches the prune.
Uint32 m_gcFrameCounter = 0;
// Active only between BeginDrawSyncScope/EndDrawSyncScope; identities of // Active only between BeginDrawSyncScope/EndDrawSyncScope; identities of
// textures already fully synced in the current draw (small N -> flat scan). // textures already fully synced in the current draw (small N -> flat scan).
Bool m_drawSyncScopeActive = false; Bool m_drawSyncScopeActive = false;
@@ -390,6 +428,8 @@ private:
std::unordered_set<VkFormat> m_mutableFormatUnsupported; std::unordered_set<VkFormat> m_mutableFormatUnsupported;
std::unordered_map<TextureIdentity, WeakPtr<MG_State::GLState::ITextureObject>, TextureIdentityHash> m_aliveObjects; std::unordered_map<TextureIdentity, WeakPtr<MG_State::GLState::ITextureObject>, TextureIdentityHash> m_aliveObjects;
std::unordered_map<TextureIdentity, TextureResource, TextureIdentityHash> m_textureResources; std::unordered_map<TextureIdentity, TextureResource, TextureIdentityHash> m_textureResources;
// Textures that have been bound to a GL image unit (see MarkStorageImageTexture).
std::unordered_set<TextureIdentity, TextureIdentityHash> m_storageImageTextures;
Vector<Vector<TextureResource>> m_deferredReleases; Vector<Vector<TextureResource>> m_deferredReleases;
Vector<Vector<VkImageView>> m_deferredViewReleases; Vector<Vector<VkImageView>> m_deferredViewReleases;
}; };
@@ -1016,7 +1016,18 @@ layout(location = 0) in vec2 vTexCoord;
layout(location = 0) out vec4 outColor; layout(location = 0) out vec4 outColor;
void main() { void main() {
outColor = texture(uSource, vTexCoord); // Explicit LOD, not texture(): a blit reads exactly the selected level, so
// derivative-based mip selection has no business here. It is also load-bearing:
// on Adreno 650 (driver 512.502) an implicit-LOD sample of this single-mip
// UBWC render target through the pre-rotation (ROTATE_90) mapping reads past
// the image's allocation - despite the sampler's maxLod=0 and a nominal 1:1
// texel mapping whose LOD is 0, so the driver's implicit-LOD path itself is at
// fault - and page-faults the GPU once the neighbouring memory is returned to
// the kernel (frame 2 of Minecraft 26.2's resource reload; the kernel then
// invalidates the context and the next submit dies with EDEADLK ->
// VK_ERROR_DEVICE_LOST at Present). Verified on device: texture() faults on
// the second frame every run, textureLod survives with identical state.
outColor = textureLod(uSource, vTexCoord, 0.0);
} }
)"; )";
@@ -1910,8 +1921,10 @@ void main() {
case VK_FORMAT_R16G16_UNORM: out = {ReadbackSourceClass::Float, 2, 16}; return true; case VK_FORMAT_R16G16_UNORM: out = {ReadbackSourceClass::Float, 2, 16}; return true;
case VK_FORMAT_R16G16B16A16_UNORM: out = {ReadbackSourceClass::Float, 4, 16}; return true; case VK_FORMAT_R16G16B16A16_UNORM: out = {ReadbackSourceClass::Float, 4, 16}; return true;
// --- SRGB (decode to linear like GL readback of sRGB textures) --- // --- SRGB (decode to linear like GL readback of sRGB textures) ---
case VK_FORMAT_R8G8B8A8_SRGB: out = {ReadbackSourceClass::Float, 4, 8, false, true}; return true; // GL GetTexImage/ReadPixels of sRGB textures return the raw sRGB-encoded
case VK_FORMAT_B8G8R8A8_SRGB: out = {ReadbackSourceClass::Float, 4, 8, false, true, true}; return true; // bytes (GL 3.3 has no FRAMEBUFFER_SRGB read decode) - do NOT linearize.
case VK_FORMAT_R8G8B8A8_SRGB: out = {ReadbackSourceClass::Float, 4, 8}; return true;
case VK_FORMAT_B8G8R8A8_SRGB: out = {ReadbackSourceClass::Float, 4, 8, false, false, true}; return true;
// --- SNORM --- // --- SNORM ---
case VK_FORMAT_R8_SNORM: out = {ReadbackSourceClass::Float, 1, 8, true}; return true; case VK_FORMAT_R8_SNORM: out = {ReadbackSourceClass::Float, 1, 8, true}; return true;
case VK_FORMAT_R8G8_SNORM: out = {ReadbackSourceClass::Float, 2, 8, true}; return true; case VK_FORMAT_R8G8_SNORM: out = {ReadbackSourceClass::Float, 2, 8, true}; return true;
@@ -1949,6 +1962,8 @@ void main() {
// --- packed / special --- // --- packed / special ---
case VK_FORMAT_A2B10G10R10_UNORM_PACK32: case VK_FORMAT_A2B10G10R10_UNORM_PACK32:
case VK_FORMAT_A2B10G10R10_UINT_PACK32: case VK_FORMAT_A2B10G10R10_UINT_PACK32:
case VK_FORMAT_A2R10G10B10_UNORM_PACK32:
case VK_FORMAT_A2R10G10B10_UINT_PACK32:
case VK_FORMAT_B10G11R11_UFLOAT_PACK32: case VK_FORMAT_B10G11R11_UFLOAT_PACK32:
case VK_FORMAT_E5B9G9R9_UFLOAT_PACK32: case VK_FORMAT_E5B9G9R9_UFLOAT_PACK32:
case VK_FORMAT_R5G6B5_UNORM_PACK16: case VK_FORMAT_R5G6B5_UNORM_PACK16:
@@ -1958,7 +1973,8 @@ void main() {
case VK_FORMAT_B5G5R5A1_UNORM_PACK16: case VK_FORMAT_B5G5R5A1_UNORM_PACK16:
case VK_FORMAT_R4G4B4A4_UNORM_PACK16: case VK_FORMAT_R4G4B4A4_UNORM_PACK16:
case VK_FORMAT_B4G4R4A4_UNORM_PACK16: case VK_FORMAT_B4G4R4A4_UNORM_PACK16:
out.sourceClass = format == VK_FORMAT_A2B10G10R10_UINT_PACK32 ? out.sourceClass = (format == VK_FORMAT_A2B10G10R10_UINT_PACK32 ||
format == VK_FORMAT_A2R10G10B10_UINT_PACK32) ?
ReadbackSourceClass::UnsignedInt : ReadbackSourceClass::Float; ReadbackSourceClass::UnsignedInt : ReadbackSourceClass::Float;
out.special = format; out.special = format;
return true; return true;
@@ -2034,6 +2050,15 @@ void main() {
rgba[3] = static_cast<Float>((word >> 30) & 0x3u) / 3.0f; rgba[3] = static_cast<Float>((word >> 30) & 0x3u) / 3.0f;
return; return;
} }
case VK_FORMAT_A2R10G10B10_UNORM_PACK32: {
Uint32 word = 0;
Memcpy(&word, source, sizeof(word));
rgba[2] = static_cast<Float>(word & 0x3FFu) / 1023.0f;
rgba[1] = static_cast<Float>((word >> 10) & 0x3FFu) / 1023.0f;
rgba[0] = static_cast<Float>((word >> 20) & 0x3FFu) / 1023.0f;
rgba[3] = static_cast<Float>((word >> 30) & 0x3u) / 3.0f;
return;
}
case VK_FORMAT_B10G11R11_UFLOAT_PACK32: { case VK_FORMAT_B10G11R11_UFLOAT_PACK32: {
Uint32 word = 0; Uint32 word = 0;
Memcpy(&word, source, sizeof(word)); Memcpy(&word, source, sizeof(word));
@@ -2185,6 +2210,13 @@ void main() {
rgba[1] = (word >> 10) & 0x3FFu; rgba[1] = (word >> 10) & 0x3FFu;
rgba[2] = (word >> 20) & 0x3FFu; rgba[2] = (word >> 20) & 0x3FFu;
rgba[3] = (word >> 30) & 0x3u; rgba[3] = (word >> 30) & 0x3u;
} else if (srcFormat == VK_FORMAT_A2R10G10B10_UINT_PACK32) {
Uint32 word = 0;
Memcpy(&word, source, sizeof(word));
rgba[2] = word & 0x3FFu;
rgba[1] = (word >> 10) & 0x3FFu;
rgba[0] = (word >> 20) & 0x3FFu;
rgba[3] = (word >> 30) & 0x3u;
} else { } else {
for (Int c = 0; c < desc.channels; ++c) { for (Int c = 0; c < desc.channels; ++c) {
if (desc.componentBits == 8) { if (desc.componentBits == 8) {
@@ -2216,8 +2248,9 @@ void main() {
} }
static Bool PackReadbackToClientOrPbo(const Uint8* srcPixels, VkFormat srcFormat, GLsizei width, static Bool PackReadbackToClientOrPbo(const Uint8* srcPixels, VkFormat srcFormat, GLsizei width,
GLsizei height, GLenum format, GLenum type, void* pixels) { GLsizei sliceHeight, GLsizei sliceCount, GLenum format, GLenum type,
if (width <= 0 || height <= 0) { void* pixels, Bool applyPackImageParams) {
if (width <= 0 || sliceHeight <= 0 || sliceCount <= 0) {
return true; return true;
} }
@@ -2230,7 +2263,8 @@ void main() {
Vector<Uint8> wide; Vector<Uint8> wide;
GLenum wideType = GL_FLOAT; GLenum wideType = GL_FLOAT;
if (!DecodeReadbackRowsToWide(srcPixels, srcFormat, width, height, wide, wideType)) { if (!DecodeReadbackRowsToWide(srcPixels, srcFormat, width,
sliceHeight * sliceCount, wide, wideType)) {
MGLOG_E("DirectVulkan readback skipped: unsupported source format=%d", MGLOG_E("DirectVulkan readback skipped: unsupported source format=%d",
static_cast<Int>(srcFormat)); static_cast<Int>(srcFormat));
return false; return false;
@@ -2243,9 +2277,9 @@ void main() {
return false; return false;
} }
return DirectGLES::ReadbackImpl::StoreWideRowsToClient(wide.data(), wideType, width, height, return DirectGLES::ReadbackImpl::StoreWideRowsToClient(wide.data(), wideType, width, sliceHeight,
/*sliceCount=*/1, mapping, type, pixels, sliceCount, mapping, type, pixels,
/*applyPackImageParams=*/false); applyPackImageParams);
} }
} // namespace } // namespace
@@ -2457,7 +2491,7 @@ void main() {
MOBILEGL_ASSERT(m_textureManager != nullptr, "VkTextureManager creation failed."); MOBILEGL_ASSERT(m_textureManager != nullptr, "VkTextureManager creation failed.");
succeeded = m_textureManager->Initialize( succeeded = m_textureManager->Initialize(
{m_device, m_physicalDevice.handle, m_allocator, m_commandPool, m_graphicsQueue, {m_device, m_physicalDevice.handle, m_allocator, m_commandPool, m_graphicsQueue,
m_frameContext.GetFrameCount()}); m_frameContext.GetFrameCount(), m_imageFormatListExtensionEnabled});
MOBILEGL_ASSERT(succeeded, "VkTextureManager initialization failed."); MOBILEGL_ASSERT(succeeded, "VkTextureManager initialization failed.");
m_clearManager = MakeUnique<VkClearManager>(); m_clearManager = MakeUnique<VkClearManager>();
MOBILEGL_ASSERT(m_clearManager != nullptr, "VkClearManager creation failed."); MOBILEGL_ASSERT(m_clearManager != nullptr, "VkClearManager creation failed.");
@@ -2506,6 +2540,11 @@ void main() {
m_shaderDrawParametersFeatureEnabled, m_shaderDrawParametersFeatureEnabled,
m_unformattedFloatStorageImagesEnabled); m_unformattedFloatStorageImagesEnabled);
MOBILEGL_ASSERT(m_programFactory != nullptr, "ProgramFactory creation failed."); MOBILEGL_ASSERT(m_programFactory != nullptr, "ProgramFactory creation failed.");
// Aging evictions (render passes and program entries) must purge the dependent
// pipeline / compute-pipeline / descriptor-set caches in the same step; both
// sweeps only run from the frame-boundary seams, long after initialization.
m_renderPassManager->SetEvictionObserver(this);
m_programFactory->SetEvictionObserver(this);
m_samplerManager = MakeUnique<VkSamplerManager>(); m_samplerManager = MakeUnique<VkSamplerManager>();
MOBILEGL_ASSERT(m_samplerManager != nullptr, "VkSamplerManager creation failed."); MOBILEGL_ASSERT(m_samplerManager != nullptr, "VkSamplerManager creation failed.");
@@ -2535,11 +2574,20 @@ void main() {
if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE) { if (m_swapchainObject.GetHandle() != VK_NULL_HANDLE) {
VkResult acquireResult = VkResult acquireResult =
m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired); m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
if (acquireResult == VK_ERROR_OUT_OF_DATE_KHR || acquireResult == VK_SUBOPTIMAL_KHR) { if (acquireResult == VK_ERROR_OUT_OF_DATE_KHR) {
// Nothing was acquired and no semaphore signal was armed, so
// rebuilding and re-acquiring on the same semaphore is safe.
MGLOG_D("Initialize, vkAcquireNextImageKHR got %d, recreating swapchain", acquireResult); MGLOG_D("Initialize, vkAcquireNextImageKHR got %d, recreating swapchain", acquireResult);
RecreateSwapchain(); RecreateSwapchain();
acquireResult = acquireResult =
m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired); m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
} else if (acquireResult == VK_SUBOPTIMAL_KHR) {
// The image is usable, and its acquire signal is already armed on
// imageAvailableSemaphore. Re-acquiring here would arm a second signal on a
// binary semaphore whose first one nobody has waited on yet; keep the image.
// Only a real surface change schedules a rebuild.
m_swapchainResizeRequested = m_swapchainResizeRequested || SwapchainIsOutOfDate();
acquireResult = VK_SUCCESS;
} }
VK_VERIFY(acquireResult, "Initialize, WaitAndAcquireNextImage"); VK_VERIFY(acquireResult, "Initialize, WaitAndAcquireNextImage");
} else { } else {
@@ -2566,6 +2614,15 @@ void main() {
DestroyDeferredDepthMipmapCleanup(); DestroyDeferredDepthMipmapCleanup();
DestroyComputePipelines(); DestroyComputePipelines();
// No sweep runs during teardown, but the observers point at this renderer
// and the factories die at different times below; disconnect them first.
if (m_renderPassManager) {
m_renderPassManager->SetEvictionObserver(nullptr);
}
if (m_programFactory) {
m_programFactory->SetEvictionObserver(nullptr);
}
m_pipelineFactory.reset(); m_pipelineFactory.reset();
ShutdownBlitResources(); ShutdownBlitResources();
ShutdownDepthMipmapResources(); ShutdownDepthMipmapResources();
@@ -2656,6 +2713,7 @@ void main() {
DestroyDebugMessenger(); DestroyDebugMessenger();
m_debugMessenger = VK_NULL_HANDLE; m_debugMessenger = VK_NULL_HANDLE;
} }
DestroyDebugReportCallback();
if (m_instance != VK_NULL_HANDLE) { if (m_instance != VK_NULL_HANDLE) {
vkDestroyInstance(m_instance, nullptr); vkDestroyInstance(m_instance, nullptr);
@@ -4135,7 +4193,7 @@ void main() {
} }
Bool VulkanRenderer::PrepareStorageImageTextures( Bool VulkanRenderer::PrepareStorageImageTextures(
VkCommandBuffer commandBuffer, FrameContext::FrameData& frame,
const MG_State::GLState::ProgramObject& program, const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj) { const ProgramFactory::VkProgramObject& programObj) {
if (!programObj.hasStorageImages) { if (!programObj.hasStorageImages) {
@@ -4156,9 +4214,18 @@ void main() {
// keep the render pass alive instead of splitting it on every storage-image draw (on // keep the render pass alive instead of splitting it on every storage-image draw (on
// tiled GPUs each split is a full tile load/store). GL makes cross-draw image-store // tiled GPUs each split is a full tile load/store). GL makes cross-draw image-store
// coherence the app's job (glMemoryBarrier), so no implicit barrier is owed here. // coherence the app's job (glMemoryBarrier), so no implicit barrier is owed here.
Bool anyNeedsPreparation = false; // Record every image-unit binding before probing anything: a texture whose image was
// created without STORAGE usage (the default - it costs UBWC compression on Adreno)
// needs a recreate, and the probe below is what ends the render pass so that recreate
// lands here rather than mid-pass. This cannot be folded into the probe loop, which
// stops at the first texture that needs work and would leave the rest unmarked.
for (auto* texture : storageTextures) { for (auto* texture : storageTextures) {
MOBILEGL_ASSERT(texture != nullptr, "%s: collected a null storage texture", __func__); MOBILEGL_ASSERT(texture != nullptr, "%s: collected a null storage texture", __func__);
m_textureManager->MarkStorageImageTexture(*texture);
}
Bool anyNeedsPreparation = false;
for (auto* texture : storageTextures) {
if (m_textureManager->NeedsStorageImagePreparation(*texture) || if (m_textureManager->NeedsStorageImagePreparation(*texture) ||
m_clearManager->HasPendingClear(texture)) { m_clearManager->HasPendingClear(texture)) {
anyNeedsPreparation = true; anyNeedsPreparation = true;
@@ -4169,21 +4236,51 @@ void main() {
return true; return true;
} }
// A first-time storage-usage upgrade recreates the image and carries the old contents
// forward with an out-of-band, immediately-submitted copy (PreserveTextureContentsOnRecreate).
// Whatever this frame already recorded into the old image is still sitting unsubmitted in
// this command buffer, so that copy would read pre-frame content and this frame's rendering
// into the texture would be lost - precisely the render-target-then-image-unit case this
// whole path exists for. Submit what is recorded first; the copy then queues behind it.
Bool anyNeedsStorageUpgrade = false;
for (auto* texture : storageTextures) {
if (m_textureManager->NeedsStorageUsageUpgrade(*texture)) {
anyNeedsStorageUpgrade = true;
break;
}
}
if (anyNeedsStorageUpgrade && HasPendingRecordedWork()) {
if (FlushPendingCommands()) {
// Fresh command buffer: the sampled-descriptor-set memo describes bindings that
// only existed in the retired one. FlushPendingCommands drops the pipeline memo
// itself; this is the other command-buffer-scoped cache.
m_lastSampledSetValid = false;
} else {
// Best effort: the upgrade still produces a correct image, only its preserved
// contents may predate this frame's writes. Dropping the draw would be worse.
MGLOG_E("%s: flush before a storage-usage image upgrade failed; preserved contents "
"may be stale for one frame", __func__);
}
}
if (!frame.isCommandRecording) {
m_frameContext.BeginCommandRecording();
}
// Image uploads, deferred-clear materialization, and layout barriers are illegal inside // Image uploads, deferred-clear materialization, and layout barriers are illegal inside
// a classic render pass. Do this before sampler preparation as well: a texture used by // a classic render pass. Do this before sampler preparation as well: a texture used by
// both a sampler and an image must stay in GENERAL, and both descriptors must name that // both a sampler and an image must stay in GENERAL, and both descriptors must name that
// same layout independent of SPIR-V reflection/binding order. // same layout independent of SPIR-V reflection/binding order.
if (VkRenderPassManager::GetActiveRenderPass() != nullptr) { if (VkRenderPassManager::GetActiveRenderPass() != nullptr) {
VkRenderPassManager::EndRenderPass(commandBuffer); VkRenderPassManager::EndRenderPass(frame.commandBuffer);
} }
for (auto* texture : storageTextures) { for (auto* texture : storageTextures) {
if (!MaterializePendingClearForTexture(commandBuffer, *texture)) { if (!MaterializePendingClearForTexture(frame.commandBuffer, *texture)) {
MGLOG_E("%s: failed to materialize pending clear for storage textureId=%d", MGLOG_E("%s: failed to materialize pending clear for storage textureId=%d",
__func__, texture->GetExternalIndex()); __func__, texture->GetExternalIndex());
return false; return false;
} }
if (!m_textureManager->TransitionTextureForStorageImage(commandBuffer, *texture)) { if (!m_textureManager->TransitionTextureForStorageImage(frame.commandBuffer, *texture)) {
MGLOG_E("%s: failed to prepare storage textureId=%d", MGLOG_E("%s: failed to prepare storage textureId=%d",
__func__, texture->GetExternalIndex()); __func__, texture->GetExternalIndex());
return false; return false;
@@ -4209,7 +4306,16 @@ void main() {
const auto& vao = *MG_State::pGLContext->GetBoundVertexArray(); const auto& vao = *MG_State::pGLContext->GetBoundVertexArray();
const auto& program = *MG_State::pGLContext->GetCurrentProgram(); const auto& program = *MG_State::pGLContext->GetCurrentProgram();
ProgramFactory::CompileOptionFlags transformFlags = GetShaderTransformFlags(m_swapchainObject.GetPreTransform()); ProgramFactory::CompileOptionFlags transformFlags = GetShaderTransformFlags(m_swapchainObject.GetPreTransform());
const auto& programObj = m_programFactory->GetOrCreateProgram(program, transformFlags); const auto* programObjPtr = &m_programFactory->GetOrCreateProgram(program, transformFlags);
// Sampling a colour render target through the driver's implicit-LOD path faults the GPU on
// Adreno 650 (see ForceExplicitLod0SamplePass); ask for the explicit-LOD variant when doing
// so cannot change a texel, i.e. when every sampler this program reads is pinned to a
// single mip level.
if (UniformManager::ProgramSamplesOnlySingleLevelTextures(program, *programObjPtr)) {
transformFlags |= ProgramFactory::CompileOptionBit::ExplicitLod0Sampling;
programObjPtr = &m_programFactory->GetOrCreateProgram(program, transformFlags);
}
const auto& programObj = *programObjPtr;
// Begin command recording if not yet // Begin command recording if not yet
if (!frame.isCommandRecording) { if (!frame.isCommandRecording) {
@@ -4219,7 +4325,7 @@ void main() {
m_lastSampledSetValid = false; m_lastSampledSetValid = false;
} }
if (!PrepareStorageImageTextures(frame.commandBuffer, program, programObj)) { if (!PrepareStorageImageTextures(frame, program, programObj)) {
MGLOG_E("SetupDraw skipped: storage image preparation failed"); MGLOG_E("SetupDraw skipped: storage image preparation failed");
return false; return false;
} }
@@ -4444,7 +4550,7 @@ void main() {
VkRenderPassManager::EndRenderPass(frame.commandBuffer); VkRenderPassManager::EndRenderPass(frame.commandBuffer);
} }
if (!PrepareStorageImageTextures(frame.commandBuffer, program, programObj)) { if (!PrepareStorageImageTextures(frame, program, programObj)) {
MGLOG_E("DispatchCompute skipped: storage image preparation failed"); MGLOG_E("DispatchCompute skipped: storage image preparation failed");
return; return;
} }
@@ -4484,7 +4590,7 @@ void main() {
VkRenderPassManager::EndRenderPass(frame.commandBuffer); VkRenderPassManager::EndRenderPass(frame.commandBuffer);
} }
if (!PrepareStorageImageTextures(frame.commandBuffer, program, programObj)) { if (!PrepareStorageImageTextures(frame, program, programObj)) {
MGLOG_E("DispatchComputeIndirect skipped: storage image preparation failed"); MGLOG_E("DispatchComputeIndirect skipped: storage image preparation failed");
return; return;
} }
@@ -6181,12 +6287,12 @@ void main() {
frame.hasCommandBufferRecorded = false; frame.hasCommandBufferRecorded = false;
frame.isCommandRecording = false; frame.isCommandRecording = false;
// The wait proved every descriptor set this slot has in flight idle; // The wait proved every submission complete, so the full frame-boundary
// rewind the reuse cursors so present-less readback loops stay // drain applies: descriptor cursors, transient arenas, deferred
// bounded (Present is the only other rewind point). // texture/buffer releases, retired command buffers and the converted
if (m_uniformManager) { // vertex-stream cache all rewind here, keeping present-less readback
m_uniformManager->BeginFrame(m_frameContext.GetCurrentFrameIndex()); // loops bounded (Present is the only other drain point).
} TryDrainFrameTransients();
return true; return true;
} }
@@ -6333,7 +6439,8 @@ void main() {
if (RemapDefaultFboReadbackToGLOrientation(mapped, swapchainExtent, preTransform, if (RemapDefaultFboReadbackToGLOrientation(mapped, swapchainExtent, preTransform,
sourceTexelSize, sourceTexelSize,
remapped.data())) { remapped.data())) {
PackReadbackToClientOrPbo(remapped.data(), srcFormat, width, height, format, type, pixels); PackReadbackToClientOrPbo(remapped.data(), srcFormat, width, height, 1, format, type, pixels,
/*applyPackImageParams=*/false);
return; return;
} }
} }
@@ -6342,7 +6449,8 @@ void main() {
width, height, swapchainExtent.width, swapchainExtent.height, width, height, swapchainExtent.width, swapchainExtent.height,
static_cast<Int>(preTransform)); static_cast<Int>(preTransform));
} }
PackReadbackToClientOrPbo(mapped, srcFormat, width, height, format, type, pixels); PackReadbackToClientOrPbo(mapped, srcFormat, width, height, 1, format, type, pixels,
/*applyPackImageParams=*/false);
} }
void VulkanRenderer::GetTexImage(GLenum target, GLint level, GLenum format, GLenum type, GLvoid* pixels) { void VulkanRenderer::GetTexImage(GLenum target, GLint level, GLenum format, GLenum type, GLvoid* pixels) {
@@ -6395,6 +6503,17 @@ void main() {
if (width <= 0 || height <= 0) { if (width <= 0 || height <= 0) {
return; return;
} }
// GetTexImage returns every slice of a 3D level and every layer of an array
// level; GL_PACK_IMAGE_HEIGHT / GL_PACK_SKIP_IMAGES apply to the 3D/array
// destination layout (GL 3.3 section 6.1.4).
const auto imageTextureTarget = textureObject->GetTarget();
const Bool is3dImage = imageTextureTarget == TextureTarget::Texture3D;
const Bool isArrayImage = imageTextureTarget == TextureTarget::Texture1DArray ||
imageTextureTarget == TextureTarget::Texture2DArray ||
imageTextureTarget == TextureTarget::TextureCubeMapArray;
const GLsizei depthSlices = is3dImage ? std::max<GLsizei>(texelSize.z(), 1) : 1;
const GLsizei arrayLayers = isArrayImage ? static_cast<GLsizei>(resource->arrayLayers) : 1;
const GLsizei sliceCount = std::max<GLsizei>(depthSlices * arrayLayers, 1);
if (bufSize >= 0) { if (bufSize >= 0) {
const Int dstChannels = GetReadbackChannelCount(format); const Int dstChannels = GetReadbackChannelCount(format);
if ((type == GL_UNSIGNED_BYTE || type == GL_FLOAT) && dstChannels > 0) { if ((type == GL_UNSIGNED_BYTE || type == GL_FLOAT) && dstChannels > 0) {
@@ -6415,7 +6534,8 @@ void main() {
return; return;
} }
const VkDeviceSize readbackSize = static_cast<VkDeviceSize>(width) * const VkDeviceSize readbackSize = static_cast<VkDeviceSize>(width) *
static_cast<VkDeviceSize>(height) * sourceTexelSize; static_cast<VkDeviceSize>(height) *
static_cast<VkDeviceSize>(sliceCount) * sourceTexelSize;
VkBufferObject readback; VkBufferObject readback;
if (!readback.Create({ if (!readback.Create({
.allocator = m_allocator, .allocator = m_allocator,
@@ -6443,8 +6563,9 @@ void main() {
copyRegion.imageSubresource.aspectMask = resource->aspect; copyRegion.imageSubresource.aspectMask = resource->aspect;
copyRegion.imageSubresource.mipLevel = static_cast<Uint32>(level); copyRegion.imageSubresource.mipLevel = static_cast<Uint32>(level);
copyRegion.imageSubresource.baseArrayLayer = 0; copyRegion.imageSubresource.baseArrayLayer = 0;
copyRegion.imageSubresource.layerCount = 1; copyRegion.imageSubresource.layerCount = static_cast<Uint32>(arrayLayers);
copyRegion.imageExtent = {static_cast<Uint32>(width), static_cast<Uint32>(height), 1}; copyRegion.imageExtent = {static_cast<Uint32>(width), static_cast<Uint32>(height),
static_cast<Uint32>(depthSlices)};
vkCmdCopyImageToBuffer(frame.commandBuffer, resource->image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, vkCmdCopyImageToBuffer(frame.commandBuffer, resource->image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
readback.GetHandle(), 1, &copyRegion); readback.GetHandle(), 1, &copyRegion);
@@ -6470,7 +6591,8 @@ void main() {
MGLOG_E("DirectVulkan::GetTextureImage skipped: failed to invalidate readback buffer"); MGLOG_E("DirectVulkan::GetTextureImage skipped: failed to invalidate readback buffer");
return; return;
} }
PackReadbackToClientOrPbo(mapped, resource->format, width, height, format, type, pixels); PackReadbackToClientOrPbo(mapped, resource->format, width, height, sliceCount, format, type, pixels,
/*applyPackImageParams=*/is3dImage || isArrayImage);
} }
void VulkanRenderer::GenerateMipmap(GLenum target) { void VulkanRenderer::GenerateMipmap(GLenum target) {
@@ -7084,6 +7206,9 @@ void main() {
} }
m_bufferManager.NotifyDeviceIdle(); m_bufferManager.NotifyDeviceIdle();
OnSubmitsCompletedUpTo(m_submitCounter); OnSubmitsCompletedUpTo(m_submitCounter);
// The queue was just drained; take the free frame-boundary drain when
// nothing is recorded (present-less timer-query loops). No-op otherwise.
TryDrainFrameTransients();
return true; return true;
} }
@@ -7152,6 +7277,85 @@ void main() {
vkDestroyFence(m_device, record.fence, nullptr); vkDestroyFence(m_device, record.fence, nullptr);
} }
} }
// Mid-frame-flushed command buffers whose submission just completed can
// be freed now; present-less flush loops have no other reclaim point.
m_frameContext.FreeRetiredCommandBuffersCompletedUpTo(m_completedSubmitCounter);
}
Bool VulkanRenderer::TryDrainFrameTransients() {
if (m_device == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return false;
}
if (m_completedSubmitCounter != m_submitCounter) {
RefreshCompletedSubmits();
if (m_completedSubmitCounter != m_submitCounter) {
return false;
}
}
if (HasPendingRecordedWork()) {
return false;
}
// Every submission is complete and nothing recorded references the
// per-frame transients. Pure-reclaim work runs on every drain: it only
// releases memory that is provably dead, never invalidates anything a
// later draw would have to rebuild. Raise the buffer manager's
// completed floor first so busy-tracking reflects the proven idleness.
m_bufferManager.NotifyDeviceIdle();
const Uint32 frameIndex = m_frameContext.GetCurrentFrameIndex();
m_frameContext.FreeAllRetiredCommandBuffers();
for (Uint32 slot = 0; slot < m_deferredDepthMipmapCleanup.size(); ++slot) {
CollectDeferredDepthMipmapCleanup(slot);
}
if (m_textureManager) {
m_textureManager->CollectAllDeferredReleases();
}
m_bufferManager.CollectAllDeferredReleases();
// Descriptor cursors rewind on every drain (the pre-drain readback path
// already did exactly this), keeping fence/readback loops' set usage bounded.
if (m_uniformManager) {
m_uniformManager->BeginFrame(frameIndex);
}
// Frame-boundary-equivalent work - transient arena rewind (which invalidates
// the conversion cache) and the cache-aging clocks - is gated to every 8th
// drain since the last Present: a presenting app's mid-frame readbacks/waits
// must neither force re-conversion/re-upload churn for the rest of the frame
// nor multiply the aging rate (which would shrink the 1024-boundary retire
// window and thrash periodically-used pipelines/programs), while present-less
// loops still rewind the arena and age their caches every 8 iterations -
// bounded by 8 iterations' transient usage.
++m_drainsSinceLastPresent;
if ((m_drainsSinceLastPresent % 8) != 0) {
return true;
}
if (m_textureManager) {
m_textureManager->BeginFrame(frameIndex);
}
m_bufferManager.BeginFrame(frameIndex);
// The cached conversion slices point into the transient arena the
// BeginFrame above just rewound; drop them together.
m_convertedVertexStreams.clear();
if (m_renderPassManager) {
m_renderPassManager->OnPresent();
}
// The pipeline memo can survive across these boundaries (no per-frame reset
// on this path), so it must drop whenever the sweep destroys anything.
if (m_programFactory) {
m_programFactory->OnFrameBoundary();
}
if (m_pipelineFactory && m_pipelineFactory->OnFrameBoundary() > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
}
if (m_vertexInputStateFactory) {
m_vertexInputStateFactory->OnFrameBoundary();
}
if (m_samplerManager) {
m_samplerManager->OnFrameBoundary();
}
return true;
} }
VkFence VulkanRenderer::AcquirePooledSubmitFence() { VkFence VulkanRenderer::AcquirePooledSubmitFence() {
@@ -7214,6 +7418,10 @@ void main() {
if (m_device == VK_NULL_HANDLE || m_graphicsQueue == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) { if (m_device == VK_NULL_HANDLE || m_graphicsQueue == VK_NULL_HANDLE || m_frameContext.GetFrameCount() == 0) {
return false; return false;
} }
// Non-blocking completion poll: gives flush-only workloads (no sync
// objects, no present) a point where finished submissions retire their
// pooled fences and mid-frame command buffers.
RefreshCompletedSubmits();
auto& frame = m_frameContext.GetCurrent(); auto& frame = m_frameContext.GetCurrent();
if (!frame.isCommandRecording && !frame.hasCommandBufferRecorded) { if (!frame.isCommandRecording && !frame.hasCommandBufferRecorded) {
return false; return false;
@@ -7238,6 +7446,14 @@ void main() {
return false; return false;
} }
// Command-buffer boundary: the pipeline memo must not survive it, or a
// pipeline bound only through memo hits is never re-stamped in the factory
// cache and the aging sweep could destroy it while the flushed submission
// still references it. Mirrors the drops at the readback and Present
// boundaries; costs one full pipeline lookup on the next draw.
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
// The submitted command buffer may still be executing; recording must // The submitted command buffer may still be executing; recording must
// restart on a fresh one. If none can be allocated, fall back to // restart on a fresh one. If none can be allocated, fall back to
// draining this submission so reusing the buffer stays legal. // draining this submission so reusing the buffer stays legal.
@@ -7289,6 +7505,10 @@ void main() {
const VkResult result = vkWaitForFences(m_device, 1, &record.fence, VK_TRUE, timeoutNs); const VkResult result = vkWaitForFences(m_device, 1, &record.fence, VK_TRUE, timeoutNs);
if (result == VK_SUCCESS) { if (result == VK_SUCCESS) {
OnSubmitsCompletedUpTo(record.submitIndex); OnSubmitsCompletedUpTo(record.submitIndex);
// The wait already stalled the pipeline; if it happens to
// have drained everything (present-less fence loops), take
// the free frame-boundary drain. No-op otherwise.
TryDrainFrameTransients();
return true; return true;
} }
if (result != VK_TIMEOUT) { if (result != VK_TIMEOUT) {
@@ -7378,34 +7598,69 @@ void main() {
suspendedFrame.isCommandRecording = false; suspendedFrame.isCommandRecording = false;
suspendedFrame.hasCommandBufferRecorded = false; suspendedFrame.hasCommandBufferRecorded = false;
m_lastPipelineValid = false; m_lastPipelineValid = false;
// The dropped recording is never submitted, so once the fence
// poll shows the pre-suspension submissions complete the frame
// transients (descriptor sets, transient arenas, deferred
// releases, conversion caches) can rewind; without this a
// minimized-window app accumulates them for the whole
// suspension.
TryDrainFrameTransients();
MGLOG_D("Present skipped: no usable swapchain (zero-area window)"); MGLOG_D("Present skipped: no usable swapchain (zero-area window)");
return; return;
} }
m_presentSuspended = false; m_presentSuspended = false;
const VkResult acquireResult = const VkResult acquireResult =
m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired); m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
if (acquireResult != VK_SUBOPTIMAL_KHR) { if (acquireResult == VK_SUBOPTIMAL_KHR) {
// Usable image with its acquire signal already armed; a rebuild is scheduled
// only if the surface genuinely no longer matches (see step 4 of Present).
m_swapchainResizeRequested = m_swapchainResizeRequested || SwapchainIsOutOfDate();
} else {
VK_VERIFY(acquireResult, "Present, deferred first WaitAndAcquireNextImage"); VK_VERIFY(acquireResult, "Present, deferred first WaitAndAcquireNextImage");
} }
} }
MOBILEGL_ASSERT(m_imageIndexAcquired < m_swapchainObject.GetImageCount(), MOBILEGL_ASSERT(m_imageIndexAcquired < m_swapchainObject.GetImageCount(),
"Present, acquired image index out of range"); "Present, acquired image index out of range");
m_renderPassManager->OnPresent(); m_renderPassManager->OnPresent();
// A real presented frame is the canonical aging cadence; mid-frame drains
// count against this and only age when presents stop coming.
m_drainsSinceLastPresent = 0;
// Age the content-addressed caches on the same frame-boundary cadence. Each
// keeps its own internal 256-sweep gate, so the per-frame cost is one counter
// increment and compare per cache; entries used by this frame's still-
// unsubmitted recording were stamped this boundary (every command-buffer
// boundary drops the pipeline memo, so the first draw of each recording
// performs a real, stamping lookup) and can never age out.
m_programFactory->OnFrameBoundary();
if (m_pipelineFactory->OnFrameBoundary() > 0) {
m_lastPipelineValid = false; // an aged-out pipeline may still be memoized
m_lastPipelineResult = VK_NULL_HANDLE;
}
m_vertexInputStateFactory->OnFrameBoundary();
m_samplerManager->OnFrameBoundary();
auto& frame = m_frameContext.GetCurrent(); auto& frame = m_frameContext.GetCurrent();
auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass(); auto* activeRenderPass = VkRenderPassManager::GetActiveRenderPass();
if (activeRenderPass) if (activeRenderPass)
VkRenderPassManager::EndRenderPass(frame.commandBuffer); VkRenderPassManager::EndRenderPass(frame.commandBuffer);
// Transition while this frame's recording is still open. A frame that
// rendered only into FBOs has no default-framebuffer render pass, and that
// pass's finalLayout is the only other thing that carries the swapchain
// image to PRESENT_SRC_KHR - so closing the buffer first, which made
// TransitionToPresent refuse to record, handed the image to
// vkQueuePresentKHR in the layout it was acquired in (UNDEFINED on a fresh
// swapchain). The SetImageLayout below then made the tracker's
// disagreement with reality permanent for that image index.
const auto acquiredImageLayout = m_swapchainObject.GetImageLayout(m_imageIndexAcquired);
m_frameContext.TransitionToPresent(m_swapchainObject.GetImage(m_imageIndexAcquired), acquiredImageLayout);
if (frame.isCommandRecording) { if (frame.isCommandRecording) {
m_frameContext.EndCommandRecording(); m_frameContext.EndCommandRecording();
frame.hasCommandBufferRecorded = true; frame.hasCommandBufferRecorded = true;
m_lastPipelineValid = false; // command-buffer boundary: drop the pipeline memo m_lastPipelineValid = false; // command-buffer boundary: drop the pipeline memo
} }
const auto acquiredImageLayout = m_swapchainObject.GetImageLayout(m_imageIndexAcquired); const Bool shouldSubmitCommandBuffer = frame.hasCommandBufferRecorded;
const Bool needsLayoutTransitionForPresent =
m_frameContext.TransitionToPresent(m_swapchainObject.GetImage(m_imageIndexAcquired), acquiredImageLayout);
const Bool shouldSubmitCommandBuffer = frame.hasCommandBufferRecorded || needsLayoutTransitionForPresent;
// 1) Submit current frame work. // 1) Submit current frame work.
auto submitPacket = m_frameContext.GetSubmitInfo(shouldSubmitCommandBuffer, m_imageIndexAcquired); auto submitPacket = m_frameContext.GetSubmitInfo(shouldSubmitCommandBuffer, m_imageIndexAcquired);
@@ -7419,7 +7674,15 @@ void main() {
// 2) Present current frame. // 2) Present current frame.
auto presentPacket = m_frameContext.GetPresentInfo(m_swapchainObject.GetHandle(), m_imageIndexAcquired); auto presentPacket = m_frameContext.GetPresentInfo(m_swapchainObject.GetHandle(), m_imageIndexAcquired);
auto result = vkQueuePresentKHR(m_presentQueue, &presentPacket.presentInfo); auto result = vkQueuePresentKHR(m_presentQueue, &presentPacket.presentInfo);
if (result == VK_ERROR_OUT_OF_DATE_KHR || result == VK_SUBOPTIMAL_KHR) { if (result == VK_SUBOPTIMAL_KHR) {
// Suboptimal is not a reason to rebuild on its own: a driver may report it for a
// surface whose size and orientation still match what we built from (Android does
// this routinely), and rebuilding on it alone destroys every pipeline and
// reallocates the default framebuffer once per frame - flicker, then garbage.
// Defer to the surface-capabilities comparison below.
result = VK_SUCCESS;
}
if (result == VK_ERROR_OUT_OF_DATE_KHR) {
MGLOG_D("Present, vkQueuePresentKHR got %d, recreating swapchain", result); MGLOG_D("Present, vkQueuePresentKHR got %d, recreating swapchain", result);
if (!RecreateSwapchain()) { if (!RecreateSwapchain()) {
// Window went zero-area (minimize) with the swapchain out of date: // Window went zero-area (minimize) with the swapchain out of date:
@@ -7433,6 +7696,13 @@ void main() {
result = VK_SUCCESS; result = VK_SUCCESS;
} }
VK_VERIFY(result, "Present, vkQueuePresentKHR"); VK_VERIFY(result, "Present, vkQueuePresentKHR");
// The authoritative check, done here - after the frame is presented, before the next
// acquire. This is what makes a launcher-side resolution change take effect: shrinking
// the window's buffer (SurfaceHolder.setFixedSize) moves currentExtent, the swapchain
// follows, and the compositor scales the smaller image up to the view for free.
if (!m_swapchainResizeRequested && SwapchainIsOutOfDate()) {
m_swapchainResizeRequested = true;
}
if (m_swapchainResizeRequested) { if (m_swapchainResizeRequested) {
MGLOG_D("Present, processing requested swapchain resize"); MGLOG_D("Present, processing requested swapchain resize");
if (!RecreateSwapchain()) { if (!RecreateSwapchain()) {
@@ -7449,7 +7719,16 @@ void main() {
// 4) Wait/reset/acquire for next frame. // 4) Wait/reset/acquire for next frame.
result = m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired); result = m_frameContext.WaitAndAcquireNextImage(m_device, m_swapchainObject.GetHandle(), m_imageIndexAcquired);
if (result == VK_ERROR_OUT_OF_DATE_KHR || result == VK_SUBOPTIMAL_KHR) { if (result == VK_SUBOPTIMAL_KHR) {
// An image WAS acquired and its signal is armed on this slot's
// imageAvailableSemaphore, so the frame proceeds normally. Whether a rebuild is
// actually needed is decided by the surface-capabilities comparison at the next
// Present - suboptimal alone must not schedule one, or a driver that reports it
// every frame would rebuild every frame.
m_swapchainResizeRequested = m_swapchainResizeRequested || SwapchainIsOutOfDate();
result = VK_SUCCESS;
} else if (result == VK_ERROR_OUT_OF_DATE_KHR) {
// Nothing acquired, nothing signaled: safe to rebuild and re-acquire.
MGLOG_D("Present, vkAcquireNextImageKHR got %d, recreating swapchain", result); MGLOG_D("Present, vkAcquireNextImageKHR got %d, recreating swapchain", result);
if (!RecreateSwapchain()) { if (!RecreateSwapchain()) {
m_presentSuspended = true; m_presentSuspended = true;
@@ -7497,6 +7776,24 @@ void main() {
m_validationLayersEnabled = m_config.EnableValidationLayers && validationLayerAvailable; m_validationLayersEnabled = m_config.EnableValidationLayers && validationLayerAvailable;
// The debug messenger is a VK_EXT_debug_utils object, but a driver can ship
// the validation layers while exposing only the older VK_EXT_debug_report
// (Adreno 650 / Vulkan 1.1.128 does exactly that). Requesting the extension
// unconditionally tripped the required-extension assert below, aborting every
// validation-enabled build in CreateInstance. Keep the layers - they still
// validate, and on Android they report to logcat on their own - and drop only
// the messenger.
const Bool debugUtilsAvailable =
m_validationLayersEnabled && IsExtensionSupported(m_extensions, VK_EXT_DEBUG_UTILS_EXTENSION_NAME);
// Without a reporting channel the layers validate but say nothing, so fall
// back to VK_EXT_debug_report when debug_utils is missing.
const Bool debugReportAvailable = m_validationLayersEnabled && !debugUtilsAvailable &&
IsExtensionSupported(m_extensions, VK_EXT_DEBUG_REPORT_EXTENSION_NAME);
if (m_validationLayersEnabled && !debugUtilsAvailable) {
MGLOG_I("%s not available; validation reports via %s instead.", VK_EXT_DEBUG_UTILS_EXTENSION_NAME,
debugReportAvailable ? VK_EXT_DEBUG_REPORT_EXTENSION_NAME : "(no channel)");
}
// ---------------- App info ------------------- // ---------------- App info -------------------
VkApplicationInfo appInfo = {}; VkApplicationInfo appInfo = {};
appInfo.sType = VK_STRUCTURE_TYPE_APPLICATION_INFO; appInfo.sType = VK_STRUCTURE_TYPE_APPLICATION_INFO;
@@ -7547,8 +7844,10 @@ void main() {
} }
#endif #endif
if (m_validationLayersEnabled) { if (debugUtilsAvailable) {
exts.push_back(VK_EXT_DEBUG_UTILS_EXTENSION_NAME); exts.push_back(VK_EXT_DEBUG_UTILS_EXTENSION_NAME);
} else if (debugReportAvailable) {
exts.push_back(VK_EXT_DEBUG_REPORT_EXTENSION_NAME);
} }
MGLOG_I("Enabling %d Vulkan instance extensions:", exts.size()); MGLOG_I("Enabling %d Vulkan instance extensions:", exts.size());
@@ -7573,7 +7872,8 @@ void main() {
MGLOG_I("Enabling validation layer..."); MGLOG_I("Enabling validation layer...");
instanceInfo.enabledLayerCount = static_cast<uint32_t>(std::size(s_validationLayerNames)); instanceInfo.enabledLayerCount = static_cast<uint32_t>(std::size(s_validationLayerNames));
instanceInfo.ppEnabledLayerNames = s_validationLayerNames; instanceInfo.ppEnabledLayerNames = s_validationLayerNames;
instanceInfo.pNext = &debugMessengerCreateInfo; // Chaining the messenger create-info is only legal with the extension on.
instanceInfo.pNext = debugUtilsAvailable ? &debugMessengerCreateInfo : nullptr;
} else { } else {
instanceInfo.enabledLayerCount = 0; instanceInfo.enabledLayerCount = 0;
instanceInfo.pNext = nullptr; instanceInfo.pNext = nullptr;
@@ -7581,7 +7881,40 @@ void main() {
VK_VERIFY(vkCreateInstance(&instanceInfo, nullptr, &m_instance), "vkCreateInstance failed"); VK_VERIFY(vkCreateInstance(&instanceInfo, nullptr, &m_instance), "vkCreateInstance failed");
if (m_validationLayersEnabled) VK_VERIFY(SetupDebugMessenger()); if (debugUtilsAvailable) {
VK_VERIFY(SetupDebugMessenger());
} else if (debugReportAvailable) {
VK_VERIFY(SetupDebugReportCallback());
}
}
static VKAPI_ATTR VkBool32 VKAPI_CALL DebugReportCallback(VkDebugReportFlagsEXT flags, VkDebugReportObjectTypeEXT,
Uint64, size_t, Int32 messageCode, const char* pLayerPrefix,
const char* pMessage, void*) {
if ((flags & (VK_DEBUG_REPORT_ERROR_BIT_EXT | VK_DEBUG_REPORT_WARNING_BIT_EXT |
VK_DEBUG_REPORT_PERFORMANCE_WARNING_BIT_EXT)) != 0) {
MGLOG_F("[Vulkan %s %d] %s", pLayerPrefix ? pLayerPrefix : "?", messageCode, pMessage ? pMessage : "");
}
return VK_FALSE;
}
VkResult VulkanRenderer::SetupDebugReportCallback() {
auto vkCreateDebugReportCallbackEXT =
(PFN_vkCreateDebugReportCallbackEXT)vkGetInstanceProcAddr(m_instance, "vkCreateDebugReportCallbackEXT");
if (!vkCreateDebugReportCallbackEXT) return VK_ERROR_EXTENSION_NOT_PRESENT;
VkDebugReportCallbackCreateInfoEXT createInfo{VK_STRUCTURE_TYPE_DEBUG_REPORT_CALLBACK_CREATE_INFO_EXT};
createInfo.flags = VK_DEBUG_REPORT_ERROR_BIT_EXT | VK_DEBUG_REPORT_WARNING_BIT_EXT |
VK_DEBUG_REPORT_PERFORMANCE_WARNING_BIT_EXT;
createInfo.pfnCallback = &DebugReportCallback;
return vkCreateDebugReportCallbackEXT(m_instance, &createInfo, nullptr, &m_debugReportCallback);
}
void VulkanRenderer::DestroyDebugReportCallback() {
if (m_debugReportCallback == VK_NULL_HANDLE) return;
auto func = (PFN_vkDestroyDebugReportCallbackEXT)vkGetInstanceProcAddr(m_instance,
"vkDestroyDebugReportCallbackEXT");
if (func != nullptr) func(m_instance, m_debugReportCallback, nullptr);
m_debugReportCallback = VK_NULL_HANDLE;
} }
VkResult VulkanRenderer::SetupDebugMessenger() { VkResult VulkanRenderer::SetupDebugMessenger() {
@@ -7863,6 +8196,18 @@ void main() {
const Vector<VkExtensionProperties> availableExtensions = EnumerateDeviceExtensions(m_physicalDevice.handle); const Vector<VkExtensionProperties> availableExtensions = EnumerateDeviceExtensions(m_physicalDevice.handle);
ResolveOptionalDeviceExtensions(availableExtensions, enabledDeviceExtensions); ResolveOptionalDeviceExtensions(availableExtensions, enabledDeviceExtensions);
// VK_KHR_image_format_list lets a MUTABLE_FORMAT image declare exactly which formats it
// may be viewed as. Adreno drops UBWC bandwidth compression on a blindly-mutable image
// (measured: 65 -> 80 fps in MC 26.2 once mutability is not requested); an explicit,
// compression-compatible format list is the portable way to keep both.
m_imageFormatListExtensionEnabled =
IsExtensionSupported(availableExtensions, VK_KHR_IMAGE_FORMAT_LIST_EXTENSION_NAME);
if (m_imageFormatListExtensionEnabled) {
enabledDeviceExtensions.push_back(VK_KHR_IMAGE_FORMAT_LIST_EXTENSION_NAME);
}
MGLOG_I("VK_KHR_image_format_list enabled: %s",
m_imageFormatListExtensionEnabled ? "true" : "false");
MGLOG_I("VK_KHR_draw_indirect_count enabled: %s", m_drawIndirectCountExtensionEnabled ? "true" : "false"); MGLOG_I("VK_KHR_draw_indirect_count enabled: %s", m_drawIndirectCountExtensionEnabled ? "true" : "false");
m_indexTypeUint8ExtensionEnabled = false; m_indexTypeUint8ExtensionEnabled = false;
@@ -8327,6 +8672,38 @@ void main() {
return m_physicalDevice; return m_physicalDevice;
} }
Bool VulkanRenderer::SwapchainIsOutOfDate() {
if (m_surface == VK_NULL_HANDLE || m_swapchainObject.GetHandle() == VK_NULL_HANDLE) {
return false;
}
VkSurfaceCapabilitiesKHR surfaceCaps{};
if (vkGetPhysicalDeviceSurfaceCapabilitiesKHR(m_physicalDevice.handle, m_surface, &surfaceCaps) !=
VK_SUCCESS) {
return false;
}
// A driver-defined currentExtent (UINT32_MAX) means the surface takes its size from the
// swapchain, so there is nothing to compare against - the app's requested size wins and
// only an explicit RequestSwapchainResize can change it.
if (surfaceCaps.currentExtent.width == UINT32_MAX || surfaceCaps.currentExtent.height == UINT32_MAX) {
return false;
}
// Compare in SURFACE space against the extent the live swapchain was created from. Using
// the swapchain's own (quarter-turn swapped) extent here would report a difference on
// every rotated frame and rebuild forever.
const VkExtent2D builtFrom = m_swapchainObject.GetSurfaceExtent();
const Bool extentChanged = surfaceCaps.currentExtent.width != builtFrom.width ||
surfaceCaps.currentExtent.height != builtFrom.height;
const Bool transformChanged = surfaceCaps.currentTransform != m_swapchainObject.GetPreTransform();
if (!extentChanged && !transformChanged) {
return false;
}
MGLOG_I("Swapchain out of date: surface %ux%u transform %u -> %ux%u transform %u",
builtFrom.width, builtFrom.height, static_cast<Uint32>(m_swapchainObject.GetPreTransform()),
surfaceCaps.currentExtent.width, surfaceCaps.currentExtent.height,
static_cast<Uint32>(surfaceCaps.currentTransform));
return true;
}
void VulkanRenderer::RequestSwapchainResize(Uint32 width, Uint32 height) { void VulkanRenderer::RequestSwapchainResize(Uint32 width, Uint32 height) {
width = std::max<Uint32>(width, 1); width = std::max<Uint32>(width, 1);
height = std::max<Uint32>(height, 1); height = std::max<Uint32>(height, 1);
@@ -8423,6 +8800,43 @@ void main() {
m_computePipelines.clear(); m_computePipelines.clear();
} }
void VulkanRenderer::OnRenderPassesDestroyed(const Vector<VkRenderPass>& renderPasses) {
if (m_pipelineFactory == nullptr) {
return;
}
// The render-pass sweep's >1024-boundary idle guarantee covers these pipelines
// too (they are only bound by draws that hit the dying entries), so the factory
// destroys them immediately. The memo must drop as well: it can hand out a
// cached handle without touching the factory.
if (m_pipelineFactory->EvictByRenderPasses(renderPasses) > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
}
}
void VulkanRenderer::OnProgramEvicted(ProgramFactory::HashType programHash,
VkDescriptorSetLayout descriptorSetLayout) {
// Same >1024-boundary idleness as the program entry: its compute pipeline is
// only dispatched, and its graphics pipelines only bound, through paths that
// stamp the entry, so immediate destruction is GPU-safe. (The graphics memo
// never holds compute pipelines; it only needs invalidating for the factory
// eviction below.)
const auto computeIt = m_computePipelines.find(programHash);
if (computeIt != m_computePipelines.end()) {
if (computeIt->second != VK_NULL_HANDLE && m_device != VK_NULL_HANDLE) {
vkDestroyPipeline(m_device, computeIt->second, nullptr);
}
m_computePipelines.erase(computeIt);
}
if (m_pipelineFactory != nullptr && m_pipelineFactory->EvictByProgramHash(programHash) > 0) {
m_lastPipelineValid = false;
m_lastPipelineResult = VK_NULL_HANDLE;
}
if (m_uniformManager != nullptr) {
m_uniformManager->OnDescriptorSetLayoutDestroyed(descriptorSetLayout);
}
}
VkPipeline VulkanRenderer::GetOrCreateComputePipeline(const ProgramFactory::VkProgramObject& programObj) { VkPipeline VulkanRenderer::GetOrCreateComputePipeline(const ProgramFactory::VkProgramObject& programObj) {
const auto it = m_computePipelines.find(programObj.hash); const auto it = m_computePipelines.find(programObj.hash);
if (it != m_computePipelines.end()) { if (it != m_computePipelines.end()) {
@@ -114,7 +114,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} }
}; };
class VulkanRenderer : public IBufferCopyCommandProvider, public FrameContext::IRecordingObserver { class VulkanRenderer : public IBufferCopyCommandProvider,
public FrameContext::IRecordingObserver,
public VkRenderPassManager::IEvictionObserver,
public ProgramFactory::IEvictionObserver {
public: public:
VulkanRenderer(NativeWindowType window, const VulkanRendererConfig& cfg = {}); VulkanRenderer(NativeWindowType window, const VulkanRendererConfig& cfg = {});
~VulkanRenderer(); ~VulkanRenderer();
@@ -131,6 +134,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// recording, before any render pass. // recording, before any render pass.
void OnFrameCommandRecordingBegan(VkCommandBuffer commandBuffer) override; void OnFrameCommandRecordingBegan(VkCommandBuffer commandBuffer) override;
// VkRenderPassManager::IEvictionObserver: the render-pass aging sweep just
// destroyed these VkRenderPasses; evict every graphics pipeline hashed on a
// dying handle (they share its >1024-boundary idleness, so immediate
// destruction is safe) and drop the last-pipeline memo if any went.
void OnRenderPassesDestroyed(const Vector<VkRenderPass>& renderPasses) override;
// ProgramFactory::IEvictionObserver: an aged-out program entry was
// destroyed; evict its compute pipeline and graphics pipelines (same
// idleness guarantee - they are only bound through draws/dispatches that
// stamp the program entry) and purge the descriptor-set cache entries
// keyed by its now-recyclable VkDescriptorSetLayout handle.
void OnProgramEvicted(ProgramFactory::HashType programHash,
VkDescriptorSetLayout descriptorSetLayout) override;
Bool SetupDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects, Bool SetupDraw(FrameContext::FrameData& frame, GLenum mode, Flags<DrawSetupAspect> aspects,
const DrawCmdParam& drawParams, const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView = nullptr); const IndexBufferView* pIndexBufferView = nullptr);
@@ -257,6 +274,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 GetTimerQueryTimestampNs(const VkTimerQueryManager::TimestampRecord& record) const; Uint64 GetTimerQueryTimestampNs(const VkTimerQueryManager::TimestampRecord& record) const;
void RequestSwapchainResize(Uint32 width, Uint32 height); void RequestSwapchainResize(Uint32 width, Uint32 height);
// Re-query the surface and report whether the live swapchain no longer matches it
// (size or orientation). This - not a VK_SUBOPTIMAL_KHR result - is what decides a
// rebuild, so a surface the driver merely considers suboptimal cannot thrash.
Bool SwapchainIsOutOfDate();
// Returns false when the surface is zero-area (minimized/hidden window): // Returns false when the surface is zero-area (minimized/hidden window):
// no new swapchain is installed and presentation must stay suspended. // no new swapchain is installed and presentation must stay suspended.
Bool RecreateSwapchain(); Bool RecreateSwapchain();
@@ -345,11 +366,26 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkFence AcquirePooledSubmitFence(); VkFence AcquirePooledSubmitFence();
void DestroySubmitFencePool(); void DestroySubmitFencePool();
Bool HasPendingRecordedWork() const; Bool HasPendingRecordedWork() const;
// Frame-boundary housekeeping for paths that never reach Present's
// tail (present-less readback loops, suspended presentation, blocking
// sync waits): runs the same per-frame drains Present performs, but
// only when every queue submission has been observed complete AND no
// recorded-but-unsubmitted commands exist - i.e. when CPU-GPU overlap
// is provably already zero. Never blocks (non-blocking fence poll
// only), so the presenting path's frames-in-flight pipelining is
// untouched. Returns true when the drain ran.
Bool TryDrainFrameTransients();
Vector<SubmitRecord> m_inFlightSubmits; Vector<SubmitRecord> m_inFlightSubmits;
Vector<VkFence> m_freeSubmitFences; Vector<VkFence> m_freeSubmitFences;
Uint64 m_submitCounter = 0; Uint64 m_submitCounter = 0;
Uint64 m_completedSubmitCounter = 0; Uint64 m_completedSubmitCounter = 0;
// Drains since the last Present, gating the drain's frame-boundary-equivalent
// work (arena rewind + cache aging): a presenting app's mid-frame
// readbacks/waits must neither churn the transient caches nor accelerate the
// aging clocks, while present-less loops still cross a boundary every few
// iterations. Reset in Present.
Uint32 m_drainsSinceLastPresent = 0;
NativeWindowType m_window = 0; NativeWindowType m_window = 0;
void* m_platformDisplay = nullptr; void* m_platformDisplay = nullptr;
@@ -367,6 +403,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<VkExtensionProperties> m_extensions; Vector<VkExtensionProperties> m_extensions;
VkInstance m_instance = VK_NULL_HANDLE; VkInstance m_instance = VK_NULL_HANDLE;
VkDebugUtilsMessengerEXT m_debugMessenger = VK_NULL_HANDLE; VkDebugUtilsMessengerEXT m_debugMessenger = VK_NULL_HANDLE;
// Fallback reporting channel for drivers that ship the validation layers but
// only expose the older VK_EXT_debug_report (Adreno 650 / Vulkan 1.1.128).
VkDebugReportCallbackEXT m_debugReportCallback = VK_NULL_HANDLE;
PhysicalDevice m_physicalDevice; PhysicalDevice m_physicalDevice;
VkDevice m_device = VK_NULL_HANDLE; VkDevice m_device = VK_NULL_HANDLE;
VmaAllocator m_allocator = nullptr; VmaAllocator m_allocator = nullptr;
@@ -517,6 +556,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void CreateInstance(); void CreateInstance();
VkResult SetupDebugMessenger(); VkResult SetupDebugMessenger();
VkResult DestroyDebugMessenger(); VkResult DestroyDebugMessenger();
VkResult SetupDebugReportCallback();
void DestroyDebugReportCallback();
VkDebugUtilsMessengerCreateInfoEXT PopulateDebugMessengerCreateInfo(); VkDebugUtilsMessengerCreateInfoEXT PopulateDebugMessengerCreateInfo();
void CreateSurface(); void CreateSurface();
void PickPhysicalDevice(); void PickPhysicalDevice();
@@ -535,8 +576,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const RenderPassEntry& renderPassEntry); const RenderPassEntry& renderPassEntry);
VkPipeline GetOrCreateComputePipeline(const ProgramFactory::VkProgramObject& programObj); VkPipeline GetOrCreateComputePipeline(const ProgramFactory::VkProgramObject& programObj);
void DestroyComputePipelines(); void DestroyComputePipelines();
// Takes the frame rather than a command buffer: a first-time storage-usage upgrade has to
// flush the pending recording (see the body), which retires the current command buffer.
Bool PrepareStorageImageTextures( Bool PrepareStorageImageTextures(
VkCommandBuffer commandBuffer, FrameContext::FrameData& frame,
const MG_State::GLState::ProgramObject& program, const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj); const ProgramFactory::VkProgramObject& programObj);
@@ -598,6 +641,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const PhysicalDevice& compareWithDevice, const PhysicalDevice& compareWithDevice,
PhysicalDevice& outBetterDevice); PhysicalDevice& outBetterDevice);
static constexpr const char* s_validationLayerNames[] = {"VK_LAYER_KHRONOS_validation"}; static constexpr const char* s_validationLayerNames[] = {"VK_LAYER_KHRONOS_validation"};
// VK_KHR_image_format_list: lets MUTABLE_FORMAT images declare their exact view-format
// set so the driver can keep bandwidth compression (see CreateLogicalDeviceAndQueues).
Bool m_imageFormatListExtensionEnabled = false;
static constexpr const char* s_deviceExtensionNames[] = {VK_KHR_SWAPCHAIN_EXTENSION_NAME}; static constexpr const char* s_deviceExtensionNames[] = {VK_KHR_SWAPCHAIN_EXTENSION_NAME};
static Bool CheckValidationLayerSupport(); static Bool CheckValidationLayerSupport();
+33
View File
@@ -24,6 +24,7 @@ namespace MobileGL::MG_Impl::CGLImpl {
GLint Samples = 0; GLint Samples = 0;
GLint Profile = kCGLOGLPVersion_3_2_Core; GLint Profile = kCGLOGLPVersion_3_2_Core;
GLint RendererId = 0x4d474c; GLint RendererId = 0x4d474c;
GLint DisplayMask = 0;
}; };
struct ContextObject { struct ContextObject {
@@ -134,6 +135,9 @@ namespace MobileGL::MG_Impl::CGLImpl {
case kCGLPFARendererID: case kCGLPFARendererID:
pixelFormat.RendererId = value; pixelFormat.RendererId = value;
break; break;
case kCGLPFADisplayMask:
pixelFormat.DisplayMask = value;
break;
default: default:
break; break;
} }
@@ -343,6 +347,9 @@ namespace MobileGL::MG_Impl::CGLImpl {
case kCGLPFARendererID: case kCGLPFARendererID:
*value = pixelFormat->RendererId; *value = pixelFormat->RendererId;
return kCGLNoError; return kCGLNoError;
case kCGLPFADisplayMask:
*value = pixelFormat->DisplayMask;
return kCGLNoError;
case kCGLPFAOpenGLProfile: case kCGLPFAOpenGLProfile:
*value = pixelFormat->Profile; *value = pixelFormat->Profile;
return kCGLNoError; return kCGLNoError;
@@ -481,6 +488,32 @@ namespace MobileGL::MG_Impl::CGLImpl {
return it == currentContexts.end() ? nullptr : it->second; return it == currentContexts.end() ? nullptr : it->second;
} }
CGLError SetVirtualScreen(CGLContextObj ctx, GLint screen) {
const std::lock_guard<std::recursive_mutex> lock(RegistryMutex());
auto* object = TryGetContext(ctx);
if (!object) {
return kCGLBadContext;
}
if (screen != 0) {
return kCGLBadValue;
}
object->VirtualScreen = screen;
return kCGLNoError;
}
CGLError GetVirtualScreen(CGLContextObj ctx, GLint* screen) {
const std::lock_guard<std::recursive_mutex> lock(RegistryMutex());
auto* object = TryGetContext(ctx);
if (!object) {
return kCGLBadContext;
}
if (!screen) {
return kCGLBadAddress;
}
*screen = object->VirtualScreen;
return kCGLNoError;
}
CGLError SetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params) { CGLError SetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params) {
const std::lock_guard<std::recursive_mutex> lock(RegistryMutex()); const std::lock_guard<std::recursive_mutex> lock(RegistryMutex());
auto* object = TryGetContext(ctx); auto* object = TryGetContext(ctx);
+2
View File
@@ -32,6 +32,8 @@ namespace MobileGL::MG_Impl::CGLImpl {
CGLError SetCurrentContext(CGLContextObj ctx); CGLError SetCurrentContext(CGLContextObj ctx);
CGLContextObj GetCurrentContext(); CGLContextObj GetCurrentContext();
CGLError SetVirtualScreen(CGLContextObj ctx, GLint screen);
CGLError GetVirtualScreen(CGLContextObj ctx, GLint* screen);
CGLError SetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params); CGLError SetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params);
CGLError GetParameter(CGLContextObj ctx, CGLContextParameter pname, GLint* params); CGLError GetParameter(CGLContextObj ctx, CGLContextParameter pname, GLint* params);
CGLError UpdateContext(CGLContextObj ctx); CGLError UpdateContext(CGLContextObj ctx);
@@ -71,6 +71,14 @@ MOBILEGL_CGL_API CGLContextObj CGLGetCurrentContext(void) {
return MobileGL::MG_Impl::CGLImpl::GetCurrentContext(); return MobileGL::MG_Impl::CGLImpl::GetCurrentContext();
} }
MOBILEGL_CGL_API CGLError CGLSetVirtualScreen(CGLContextObj ctx, GLint screen) {
return MobileGL::MG_Impl::CGLImpl::SetVirtualScreen(ctx, screen);
}
MOBILEGL_CGL_API CGLError CGLGetVirtualScreen(CGLContextObj ctx, GLint* screen) {
return MobileGL::MG_Impl::CGLImpl::GetVirtualScreen(ctx, screen);
}
MOBILEGL_CGL_API CGLError CGLSetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params) { MOBILEGL_CGL_API CGLError CGLSetParameter(CGLContextObj ctx, CGLContextParameter pname, const GLint* params) {
return MobileGL::MG_Impl::CGLImpl::SetParameter(ctx, pname, params); return MobileGL::MG_Impl::CGLImpl::SetParameter(ctx, pname, params);
} }
@@ -10,8 +10,12 @@
#if defined(__APPLE__) #if defined(__APPLE__)
#include "MG_Impl/CGLImpl/CGLImpl.h"
#include "MG_Impl/GetProcAddress.h" #include "MG_Impl/GetProcAddress.h"
#include <CoreGraphics/CoreGraphics.h>
#include <CoreVideo/CVDisplayLink.h>
#include <cstdint>
#include <dlfcn.h> #include <dlfcn.h>
namespace { namespace {
@@ -47,10 +51,52 @@ namespace {
return dlsym(handle, symbol); return dlsym(handle, symbol);
} }
CGDirectDisplayID DisplayForMask(GLint displayMask) {
constexpr std::uint32_t MaxDisplays = sizeof(CGOpenGLDisplayMask) * 8;
CGDirectDisplayID displays[MaxDisplays] = {};
std::uint32_t displayCount = 0;
if (displayMask != 0 &&
CGGetActiveDisplayList(MaxDisplays, displays, &displayCount) == kCGErrorSuccess) {
const auto mask = static_cast<CGOpenGLDisplayMask>(displayMask);
for (std::uint32_t i = 0; i < displayCount; ++i) {
if ((CGDisplayIDToOpenGLDisplayMask(displays[i]) & mask) != 0) {
return displays[i];
}
}
}
return CGMainDisplayID();
}
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
CVReturn MobileGLCVDisplayLinkSetCurrentCGDisplayFromOpenGLContext(
CVDisplayLinkRef displayLink,
CGLContextObj context,
CGLPixelFormatObj pixelFormat) {
GLint virtualScreen = 0;
if (MobileGL::MG_Impl::CGLImpl::GetVirtualScreen(context, &virtualScreen) == kCGLNoError) {
GLint displayMask = 0;
if (!displayLink ||
MobileGL::MG_Impl::CGLImpl::DescribePixelFormat(
pixelFormat, virtualScreen, kCGLPFADisplayMask, &displayMask) != kCGLNoError) {
return kCVReturnInvalidArgument;
}
return CVDisplayLinkSetCurrentCGDisplay(displayLink, DisplayForMask(displayMask));
}
using OriginalFunction = CVReturn (*)(CVDisplayLinkRef, CGLContextObj, CGLPixelFormatObj);
static const auto original = reinterpret_cast<OriginalFunction>(
dlsym(RTLD_NEXT, "CVDisplayLinkSetCurrentCGDisplayFromOpenGLContext"));
return original ? original(displayLink, context, pixelFormat) : kCVReturnError;
}
__attribute__((used)) static const DyldInterposeEntry kMobileGLDyldInterpose[] __attribute__((used)) static const DyldInterposeEntry kMobileGLDyldInterpose[]
__attribute__((section("__DATA,__interpose"))) = { __attribute__((section("__DATA,__interpose"))) = {
{reinterpret_cast<const void*>(MobileGLDlsym), reinterpret_cast<const void*>(dlsym)}, {reinterpret_cast<const void*>(MobileGLDlsym), reinterpret_cast<const void*>(dlsym)},
{reinterpret_cast<const void*>(MobileGLCVDisplayLinkSetCurrentCGDisplayFromOpenGLContext),
reinterpret_cast<const void*>(CVDisplayLinkSetCurrentCGDisplayFromOpenGLContext)},
}; };
#pragma clang diagnostic pop
} // namespace } // namespace
#endif #endif
@@ -0,0 +1,10 @@
# Public CGL entry points.
_CGL*
# Public EGL entry points.
_egl*
# Public OpenGL and GLX entry points. OpenGL function names always use an
# uppercase letter or digit after the "gl" prefix; excluding lowercase here
# deliberately prevents glslang_* from matching this pattern.
_gl[A-Z0-9]*
+29
View File
@@ -133,4 +133,33 @@ namespace MobileGL::MG_Impl::GLImpl {
values[0] = value; values[0] = value;
} }
} }
void DestroyAllSyncObjects() {
// Detach the registry under the lock, release outside it. Entries the app
// already deleted were erased by DeleteSync, so nothing here double-frees;
// a DeleteSync racing this sweep finds an empty registry and returns. A
// thread still blocked inside ClientWaitSync/GetSynciv during teardown
// holds a raw SyncObject* these deletes invalidate - the same undefined
// race an app-driven DeleteSync already has.
UnorderedMap<GLsync, SyncObject*> orphans;
{
const std::lock_guard<std::mutex> lock(g_syncObjectsMutex);
orphans.swap(g_liveSyncObjects);
}
if (orphans.empty()) {
return;
}
// Both backends' DeleteSync only free the heap wrapper once their GL
// context/renderer is gone (generation/current-thread guards), so this is
// safe after the backend has released its EGL resources - but not after
// the function table itself is cleared.
const auto backendDeleteSync = MG_Backend::gBackendFunctionsTable.GL.DeleteSync;
for (const auto& [_, syncObject] : orphans) {
if (backendDeleteSync && syncObject->backendHandle) {
backendDeleteSync(syncObject->backendHandle);
}
delete syncObject;
}
MGLOG_D("DestroyAllSyncObjects: reclaimed %zu sync object(s) the app left undeleted", orphans.size());
}
} // namespace MobileGL::MG_Impl::GLImpl } // namespace MobileGL::MG_Impl::GLImpl
+8
View File
@@ -16,4 +16,12 @@ namespace MobileGL::MG_Impl::GLImpl {
void WaitSync(GLsync sync, GLbitfield flags, GLuint64 timeout); void WaitSync(GLsync sync, GLbitfield flags, GLuint64 timeout);
void DeleteSync(GLsync sync); void DeleteSync(GLsync sync);
void GetSynciv(GLsync sync, GLenum pname, GLsizei bufSize, GLsizei* length, GLint* values); void GetSynciv(GLsync sync, GLenum pname, GLsizei bufSize, GLsizei* length, GLint* values);
// Destroys every still-registered sync object exactly as DeleteSync would.
// GL requires syncs to die with their context; called only from full library
// teardown (DestroyImpl), where no context survives on any thread, so the
// process-global registry can be drained wholesale. Must run while the
// backend function table is still populated: each backend handle has to be
// released by the backend that created it, never by a later re-initialized
// one.
void DestroyAllSyncObjects();
} // namespace MobileGL::MG_Impl::GLImpl } // namespace MobileGL::MG_Impl::GLImpl
+2
View File
@@ -85,6 +85,8 @@ namespace MobileGL::MG_Impl {
GETPROC(CGLGetPixelFormat, name); GETPROC(CGLGetPixelFormat, name);
GETPROC(CGLSetCurrentContext, name); GETPROC(CGLSetCurrentContext, name);
GETPROC(CGLGetCurrentContext, name); GETPROC(CGLGetCurrentContext, name);
GETPROC(CGLSetVirtualScreen, name);
GETPROC(CGLGetVirtualScreen, name);
GETPROC(CGLSetParameter, name); GETPROC(CGLSetParameter, name);
GETPROC(CGLGetParameter, name); GETPROC(CGLGetParameter, name);
GETPROC(CGLUpdateContext, name); GETPROC(CGLUpdateContext, name);
+36 -4
View File
@@ -29,10 +29,19 @@ namespace MobileGL::MG_Impl::NSOpenGLImpl {
char kContextViewKey; char kContextViewKey;
char kContextLayerKey; char kContextLayerKey;
std::once_flag g_installOnce;
IMP g_pixelFormatDealloc = nullptr; IMP g_pixelFormatDealloc = nullptr;
IMP g_contextDealloc = nullptr; IMP g_contextDealloc = nullptr;
std::mutex& HookInstallMutex() {
static auto* mutex = new std::mutex();
return *mutex;
}
Bool& HooksInstalled() {
static auto* installed = new Bool(false);
return *installed;
}
template <typename Fn> template <typename Fn>
Fn ObjcMsgSend() { Fn ObjcMsgSend() {
return reinterpret_cast<Fn>(objc_msgSend); return reinterpret_cast<Fn>(objc_msgSend);
@@ -431,12 +440,12 @@ namespace MobileGL::MG_Impl::NSOpenGLImpl {
method_setImplementation(method, replacement); method_setImplementation(method, replacement);
} }
void InstallHooksOnce() { Bool InstallHooksOnce() {
Class pixelFormatClass = objc_getClass("NSOpenGLPixelFormat"); Class pixelFormatClass = objc_getClass("NSOpenGLPixelFormat");
Class contextClass = objc_getClass("NSOpenGLContext"); Class contextClass = objc_getClass("NSOpenGLContext");
if (!pixelFormatClass || !contextClass) { if (!pixelFormatClass || !contextClass) {
MGLOG_W("NSOpenGLImpl: NSOpenGL classes are not loaded; hooks not installed"); MGLOG_W("NSOpenGLImpl: NSOpenGL classes are not loaded; hooks not installed");
return; return false;
} }
ReplaceInstanceMethod(pixelFormatClass, "initWithAttributes:", ReplaceInstanceMethod(pixelFormatClass, "initWithAttributes:",
@@ -471,11 +480,34 @@ namespace MobileGL::MG_Impl::NSOpenGLImpl {
ReplaceInstanceMethod(contextClass, "dealloc", reinterpret_cast<IMP>(ContextDealloc), &g_contextDealloc); ReplaceInstanceMethod(contextClass, "dealloc", reinterpret_cast<IMP>(ContextDealloc), &g_contextDealloc);
MGLOG_I("NSOpenGLImpl hooks installed"); MGLOG_I("NSOpenGLImpl hooks installed");
return true;
} }
} // namespace } // namespace
void InstallHooks() { void InstallHooks() {
std::call_once(g_installOnce, InstallHooksOnce); const std::lock_guard<std::mutex> lock(HookInstallMutex());
if (!HooksInstalled()) {
// Do not permanently consume the install attempt when the OpenGL
// framework has not registered its Objective-C classes yet. The
// dyld bootstrap normally runs after framework dependencies, but
// an explicitly loaded/static-linked MobileGL can arrive earlier.
HooksInstalled() = InstallHooksOnce();
}
} }
} // namespace MobileGL::MG_Impl::NSOpenGLImpl } // namespace MobileGL::MG_Impl::NSOpenGLImpl
namespace {
// SDL's Cocoa backend creates NSOpenGLPixelFormat/NSOpenGLContext before
// its first dlsym("glGetString") or other MobileGL host-API call. Install
// only the lightweight Objective-C dispatch hooks while the injected dylib
// is loading so those first Cocoa objects are routed through CGLImpl. The
// hooked context constructor reaches EGLImpl::GetDisplay(), which performs
// the full, thread-safe MobileGL initialization outside this bootstrap.
//
// There is intentionally no matching destructor: backend teardown remains
// owned by the EGL lifecycle and process-exit globals remain leak-at-exit.
__attribute__((constructor)) void BootstrapNSOpenGLHooks() {
MobileGL::MG_Impl::NSOpenGLImpl::InstallHooks();
}
} // namespace
#endif #endif
@@ -334,16 +334,32 @@ namespace MobileGL::MG_State::GLState {
// draw. The memo is keyed by (backendStateVersion, flags); ResetLinkArtifacts and // draw. The memo is keyed by (backendStateVersion, flags); ResetLinkArtifacts and
// the binding setters below invalidate it by bumping m_backendStateVersion. // the binding setters below invalidate it by bumping m_backendStateVersion.
Bool GetBackendHashMemo(Uint flags, Uint64& outHash) const { Bool GetBackendHashMemo(Uint flags, Uint64& outHash) const {
if (m_backendHashMemoVersion != m_backendStateVersion || m_backendHashMemoFlags != flags) { if (m_backendHashMemoVersion != m_backendStateVersion) return false;
return false; for (const auto& slot : m_backendHashMemoSlots) {
if (slot.valid && slot.flags == flags) {
outHash = slot.hash;
return true;
}
} }
outHash = m_backendHashMemo; return false;
return true;
} }
void SetBackendHashMemo(Uint flags, Uint64 hash) const { void SetBackendHashMemo(Uint flags, Uint64 hash) const {
m_backendHashMemo = hash; if (m_backendHashMemoVersion != m_backendStateVersion) {
m_backendHashMemoVersion = m_backendStateVersion; for (auto& slot : m_backendHashMemoSlots) slot.valid = false;
m_backendHashMemoFlags = flags; m_backendHashMemoVersion = m_backendStateVersion;
m_backendHashMemoNextSlot = 0;
}
for (auto& slot : m_backendHashMemoSlots) {
if (slot.valid && slot.flags == flags) {
slot.hash = hash;
return;
}
}
auto& slot = m_backendHashMemoSlots[m_backendHashMemoNextSlot];
slot.flags = flags;
slot.hash = hash;
slot.valid = true;
m_backendHashMemoNextSlot = (m_backendHashMemoNextSlot + 1) % kBackendHashMemoSlotCount;
} }
void SetUniformSamplerOrImageUnitIndex(Uint location, Int unit) { void SetUniformSamplerOrImageUnitIndex(Uint location, Int unit) {
@@ -527,10 +543,19 @@ namespace MobileGL::MG_State::GLState {
Uint32 m_backendStateVersion = 0; Uint32 m_backendStateVersion = 0;
// Backend-owned content-hash memo (see GetBackendHashMemo): valid only while // Backend-owned content-hash memo (see GetBackendHashMemo): valid only while
// m_backendStateVersion and the compile flags match the recorded values. // m_backendStateVersion matches. Several slots, not one: a backend may resolve the same
mutable Uint64 m_backendHashMemo = 0; // program under more than one compile-flag set within a frame (surface rotation, and the
// explicit-LOD sampling variant), and a single slot would then miss on every lookup and
// re-hash the program's whole SPIR-V once per draw.
static constexpr SizeT kBackendHashMemoSlotCount = 4;
struct BackendHashMemoSlot {
Uint64 hash = 0;
Uint flags = 0;
Bool valid = false;
};
mutable Array<BackendHashMemoSlot, kBackendHashMemoSlotCount> m_backendHashMemoSlots{};
mutable SizeT m_backendHashMemoNextSlot = 0;
mutable Uint32 m_backendHashMemoVersion = ~0u; mutable Uint32 m_backendHashMemoVersion = ~0u;
mutable Uint m_backendHashMemoFlags = 0;
Uint32 m_uboContentVersion = 0; Uint32 m_uboContentVersion = 0;
Uint32 m_linkVersion = 0; Uint32 m_linkVersion = 0;
}; };
@@ -15,7 +15,11 @@
#include <MG_Util/Math/VectorTypes.h> #include <MG_Util/Math/VectorTypes.h>
namespace MobileGL::MG_State::GLState { namespace MobileGL::MG_State::GLState {
class ITextureObject { // Texture objects are always SharedPtr-owned (TextureState creates every instance via
// MakeShared, including the per-target default objects). enable_shared_from_this lets
// backends that only receive a reference (e.g. syncing a name-deleted texture kept
// alive by an FBO attachment) still register a weak liveness reference for GC.
class ITextureObject : public std::enable_shared_from_this<ITextureObject> {
public: public:
using TargetEnum = TextureTarget; using TargetEnum = TextureTarget;
virtual ~ITextureObject() = default; virtual ~ITextureObject() = default;
+59
View File
@@ -33,6 +33,7 @@
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h> #include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h> #include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h>
#include <MG_Util/Debug/Log.h> #include <MG_Util/Debug/Log.h>
#include <FastSTL/UnorderedMap.h>
namespace { namespace {
class DynamicParameterBackend final : public MobileGL::MG_Backend::BackendObject { class DynamicParameterBackend final : public MobileGL::MG_Backend::BackendObject {
@@ -1736,3 +1737,61 @@ TEST(DirectGLESStateGuards, DefaultFramebufferBindGoesThroughShadow) {
FramebufferImpl::BindFramebufferId(GL_DRAW_FRAMEBUFFER, 7); // must reach the driver again FramebufferImpl::BindFramebufferId(GL_DRAW_FRAMEBUFFER, 7); // must reach the driver again
EXPECT_EQ(mocks.log.Count("BindFramebuffer:"), 3u); EXPECT_EQ(mocks.log.Count("BindFramebuffer:"), 3u);
} }
// FastSTL::unordered_map::erase(iterator) regression coverage. The open-addressing
// iterator constructor snaps forward from a tombstoned slot to the successor, so
// erase must NOT advance the rebuilt iterator again: the old double-advance skipped
// one live element per erase, and erasing the element in the highest occupied
// bucket pushed the returned index past bucket_count where it never compared equal
// to end() again - erase-while-iterating sweeps (pipeline/program cache eviction)
// then ran off the bucket array and fed garbage handles to vkDestroyPipeline
// (device crash on first mass eviction during world load).
TEST(FastSTLSanity, EraseWhileIteratingVisitsEveryElementExactlyOnce) {
FastSTL::unordered_map<MobileGL::Uint64, MobileGL::Uint64> map;
constexpr MobileGL::Uint64 kCount = 1000;
for (MobileGL::Uint64 key = 0; key < kCount; ++key) {
map.emplace(key * 0x9e3779b97f4a7c15ull, key);
}
ASSERT_EQ(map.size(), kCount);
MobileGL::SizeT visited = 0;
for (auto it = map.begin(); it != map.end();) {
it = map.erase(it);
++visited;
ASSERT_LE(visited, kCount); // old code: runaway past end / skipped entries
}
EXPECT_EQ(visited, kCount);
EXPECT_EQ(map.size(), 0u);
}
TEST(FastSTLSanity, EraseReturnsTheSuccessorElement) {
FastSTL::unordered_map<MobileGL::Uint32, MobileGL::Uint32> map;
for (MobileGL::Uint32 key = 1; key <= 64; ++key) {
map.emplace(key, key);
}
// Erasing every other visited element must still visit all 64 exactly once:
// the iterator returned by erase names the very next element, not one past it.
MobileGL::SizeT visited = 0;
MobileGL::SizeT erased = 0;
for (auto it = map.begin(); it != map.end();) {
++visited;
if ((visited & 1) != 0) {
it = map.erase(it);
++erased;
} else {
++it;
}
ASSERT_LE(visited, 64u);
}
EXPECT_EQ(visited, 64u);
EXPECT_EQ(map.size(), 64u - erased);
}
TEST(FastSTLSanity, ErasingTheOnlyElementReturnsEnd) {
FastSTL::unordered_map<MobileGL::Uint32, MobileGL::Uint32> map;
map.emplace(42u, 1u);
auto next = map.erase(map.begin());
EXPECT_EQ(next, map.end());
EXPECT_TRUE(map.empty());
}
@@ -133,9 +133,11 @@ namespace MobileGL {
case TextureInternalFormat::RGBA8Snorm: case TextureInternalFormat::RGBA8Snorm:
return VK_FORMAT_R8G8B8A8_SNORM; return VK_FORMAT_R8G8B8A8_SNORM;
case TextureInternalFormat::RGB10A2: case TextureInternalFormat::RGB10A2:
return VK_FORMAT_A2R10G10B10_UNORM_PACK32; // GL_UNSIGNED_INT_2_10_10_10_REV puts R in bits 0-9, which is Vulkan's
// A2B10G10R10 layout - A2R10G10B10 silently swaps R and B on upload.
return VK_FORMAT_A2B10G10R10_UNORM_PACK32;
case TextureInternalFormat::RGB10A2UI: case TextureInternalFormat::RGB10A2UI:
return VK_FORMAT_A2R10G10B10_UINT_PACK32; return VK_FORMAT_A2B10G10R10_UINT_PACK32;
case TextureInternalFormat::RGBA16: case TextureInternalFormat::RGBA16:
return VK_FORMAT_R16G16B16A16_UNORM; return VK_FORMAT_R16G16B16A16_UNORM;
case TextureInternalFormat::RGBA16Snorm: case TextureInternalFormat::RGBA16Snorm:
+3 -7
View File
@@ -9,14 +9,10 @@
#pragma once #pragma once
#include <Includes.h> #include <Includes.h>
// Severity-ordered: a build compiled at level X keeps X and everything MORE
// severe. INFO builds must keep WARN/ERROR/FATAL — the old ordering
// (WARN=1/ERROR=2 below INFO=3) compiled every warning and error out of
// release builds and hid real backend failures.
#define MOBILEGL_LOG_LEVEL_DEBUG 0 #define MOBILEGL_LOG_LEVEL_DEBUG 0
#define MOBILEGL_LOG_LEVEL_INFO 1 #define MOBILEGL_LOG_LEVEL_WARN 1
#define MOBILEGL_LOG_LEVEL_WARN 2 #define MOBILEGL_LOG_LEVEL_ERROR 2
#define MOBILEGL_LOG_LEVEL_ERROR 3 #define MOBILEGL_LOG_LEVEL_INFO 3
#define MOBILEGL_LOG_LEVEL_FATAL 4 #define MOBILEGL_LOG_LEVEL_FATAL 4
#define MOBILEGL_LOG_INTERNAL(levelTag, androidLogLevel, fmt, ...) \ #define MOBILEGL_LOG_INTERNAL(levelTag, androidLogLevel, fmt, ...) \
@@ -22,9 +22,6 @@
#include "SpirvPasses/StripUboMemberRelaxedPrecisionPass.h" #include "SpirvPasses/StripUboMemberRelaxedPrecisionPass.h"
#include "SpirvPasses/StripNoPerspectivePass.h" #include "SpirvPasses/StripNoPerspectivePass.h"
#include "SpirvPasses/EmulateNoPerspectivePass.h" #include "SpirvPasses/EmulateNoPerspectivePass.h"
#include "SpirvPasses/FoldConstOffsetFor1DFetchPass.h"
#include "SpirvPasses/LowerClipDistanceForEsslPass.h"
#include "SpirvPasses/DefeatConstStructArrayLutPass.h"
#include "spirv-tools/libspirv.h" #include "spirv-tools/libspirv.h"
#include "spirv-tools/optimizer.hpp" #include "spirv-tools/optimizer.hpp"
@@ -326,55 +323,6 @@ namespace MobileGL {
return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options); return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options);
} }
bool ShaderCompiler::ClampAccessChainIndicesForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary) {
using namespace spvtools;
OptimizerOptions options;
options.set_run_validator(false);
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(CreateGraphicsRobustAccessPass());
return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options);
}
bool ShaderCompiler::FoldConstOffsetFor1DFetchForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary) {
using namespace spvtools;
OptimizerOptions options;
options.set_run_validator(false);
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(FoldConstOffsetFor1DFetchPass::CreateFoldConstOffsetFor1DFetchPass());
return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options);
}
bool ShaderCompiler::LowerClipDistanceForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary) {
using namespace spvtools;
OptimizerOptions options;
options.set_run_validator(false);
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(LowerClipDistanceForEsslPass::CreateLowerClipDistanceForEsslPass());
return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options);
}
bool ShaderCompiler::DefeatConstStructArrayLutForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary) {
using namespace spvtools;
OptimizerOptions options;
options.set_run_validator(false);
Optimizer optimizer(SPV_ENV_VULKAN_1_1);
optimizer.RegisterPass(
DefeatConstStructArrayLutPass::CreateDefeatConstStructArrayLutPass());
return optimizer.Run(inputBinary.data(), inputBinary.size(), &outputBinary, options);
}
bool ShaderCompiler::StripUboMemberRelaxedPrecisionForEssl(const Vector<Uint32>& inputBinary, bool ShaderCompiler::StripUboMemberRelaxedPrecisionForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary) { Vector<uint32_t>& outputBinary) {
using namespace spvtools; using namespace spvtools;
@@ -27,36 +27,6 @@ namespace MobileGL {
// Only for backends without native draw-parameter support (DirectGLES). // Only for backends without native draw-parameter support (DirectGLES).
static bool LowerDrawParametersForEssl(const Vector<Uint32>& inputBinary, static bool LowerDrawParametersForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary); Vector<uint32_t>& outputBinary);
// Clamps every access-chain index to its declared bounds (spirv-tools
// GraphicsRobustAccessPass). GL 3.3 only promises undefined *values* for
// out-of-bounds indexing, but Adreno's ESSL compiler constant-folds a provably
// out-of-bounds local-array index into poison that corrupts the whole shader's
// output; clamping restores the "some value from the array" contract. Only for
// the DirectGLES transpile path.
static bool ClampAccessChainIndicesForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary);
// Folds the ConstOffset image operand of Dim1D OpImageFetch into the integer
// coordinate (texelFetchOffset(t,P,l,o) == texelFetch(t,P+o,l)). SPIRV-Cross
// emulates 1D samplers as 2D for ES: it widens the coordinate to ivec2 but keeps
// the scalar offset, and ESSL has no texelFetchOffset(sampler2D, ivec2, int,
// scalar) overload, so Adreno rejects the shader. Only for the DirectGLES
// transpile path.
static bool FoldConstOffsetFor1DFetchForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary);
// Shadows gl_ClipDistance in Private mg_ClipDistance/mg_ClipDistanceIn arrays so
// the decompiled ESSL only writes the builtin with literal constant indices
// (flush before EmitVertex/return) and only reads gl_in clip distances with
// dynamic loop indices (copy loop): the other shapes miscompile or crash
// Adreno's ESSL compiler. Vertex/geometry stages; DirectGLES transpile path on
// Qualcomm only (quirk-gated). See LowerClipDistanceForEsslPass.
static bool LowerClipDistanceForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary);
// Splits a Function-storage array-of-structs variable's single constant-composite
// store into per-element stores so SPIRV-Cross does not hoist it into a global
// const struct[] LUT, which Adreno cannot dynamically index. DirectGLES
// transpile path on Qualcomm only (quirk-gated). See DefeatConstStructArrayLutPass.
static bool DefeatConstStructArrayLutForEssl(const Vector<Uint32>& inputBinary,
Vector<uint32_t>& outputBinary);
// Drops RelaxedPrecision member decorations from uniform-block structs so // Drops RelaxedPrecision member decorations from uniform-block structs so
// SPIRV-Cross prints the same (highp) member precision in every stage; ES // SPIRV-Cross prints the same (highp) member precision in every stage; ES
// drivers reject cross-stage uniform blocks whose member precisions differ. // drivers reject cross-stage uniform blocks whose member precisions differ.
@@ -1,175 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DefeatConstStructArrayLutPass.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "DefeatConstStructArrayLutPass.h"
#include "spirv.hpp"
#include "source/opt/constants.h"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/opt/type_manager.h"
#include "source/opt/types.h"
#include "source/util/make_unique.h"
#include <vector>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
namespace {
using spvtools::opt::BasicBlock;
using spvtools::opt::Function;
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
using spvtools::opt::Operand;
namespace analysis = spvtools::opt::analysis;
uint32_t PointerTypeTo(IRContext* ctx, uint32_t pointeeId, spv::StorageClass sc) {
analysis::Type* pointee = ctx->get_type_mgr()->GetType(pointeeId);
analysis::Pointer ptr(pointee, sc);
return ctx->get_type_mgr()->GetTypeInstruction(&ptr);
}
uint32_t SignedIntConstant(IRContext* ctx, uint32_t value) {
analysis::Integer i(32, true);
analysis::Type* reg = ctx->get_type_mgr()->GetRegisteredType(&i);
const analysis::Constant* c = ctx->get_constant_mgr()->GetConstant(reg, {value});
return ctx->get_constant_mgr()->GetDefiningInstruction(c)->result_id();
}
// True when |var| (a Function-storage OpVariable) points to an array of structs.
// Reports the struct type id on success.
bool IsArrayOfStructsVariable(IRContext* ctx, Instruction* var, uint32_t& structTypeId) {
auto* defUse = ctx->get_def_use_mgr();
Instruction* ptrType = defUse->GetDef(var->type_id());
if (ptrType == nullptr || ptrType->opcode() != spv::Op::OpTypePointer) return false;
Instruction* pointee = defUse->GetDef(ptrType->GetSingleWordInOperand(1));
if (pointee == nullptr || pointee->opcode() != spv::Op::OpTypeArray) return false;
Instruction* element = defUse->GetDef(pointee->GetSingleWordInOperand(0));
if (element == nullptr || element->opcode() != spv::Op::OpTypeStruct) return false;
structTypeId = element->result_id();
return true;
}
} // namespace
spvtools::opt::Pass::Status DefeatConstStructArrayLutPass::Process() {
auto* ctx = context();
auto* defUse = ctx->get_def_use_mgr();
bool modified = false;
for (Function& function : *get_module()) {
if (function.begin() == function.end()) continue;
BasicBlock* entryBlock = &*function.begin();
// Candidate variables: Function-storage arrays of structs declared in this
// function's entry block (where OpVariables must live).
struct Candidate {
Instruction* var;
uint32_t structTypeId;
};
std::vector<Candidate> candidates;
for (Instruction& inst : *entryBlock) {
if (inst.opcode() != spv::Op::OpVariable) break;
// Variables with initializers keep SPIRV-Cross's initializer path; the
// glslang pattern under attack is initializer-free with one OpStore.
if (inst.NumInOperands() > 1) continue;
uint32_t structTypeId = 0;
if (IsArrayOfStructsVariable(ctx, &inst, structTypeId)) {
candidates.push_back({&inst, structTypeId});
}
}
for (const Candidate& candidate : candidates) {
Instruction* var = candidate.var;
// The variable qualifies only when its single write is one direct
// OpStore of an OpConstantComposite; any other write shape already
// defeats SPIRV-Cross's LUT promotion, so it is left untouched.
Instruction* singleStore = nullptr;
bool disqualified = false;
defUse->ForEachUser(var, [&](Instruction* user) {
if (user->opcode() == spv::Op::OpStore &&
user->GetSingleWordInOperand(0) == var->result_id()) {
if (singleStore != nullptr) {
disqualified = true;
} else {
singleStore = user;
}
} else if (user->opcode() == spv::Op::OpCopyMemory) {
disqualified = true;
} else if (user->opcode() == spv::Op::OpAccessChain ||
user->opcode() == spv::Op::OpInBoundsAccessChain) {
defUse->ForEachUser(user, [&](Instruction* chainUser) {
if (chainUser->opcode() == spv::Op::OpStore ||
chainUser->opcode() == spv::Op::OpCopyMemory) {
disqualified = true;
}
});
}
});
if (disqualified || singleStore == nullptr) continue;
Instruction* composite = defUse->GetDef(singleStore->GetSingleWordInOperand(1));
if (composite == nullptr ||
composite->opcode() != spv::Op::OpConstantComposite) {
continue;
}
// The store must sit in the entry block: that is the only placement
// SPIRV-Cross treats as a LUT initializer.
bool storeInEntryBlock = false;
for (Instruction& inst : *entryBlock) {
if (&inst == singleStore) {
storeInEntryBlock = true;
break;
}
}
if (!storeInEntryBlock) continue;
// Split the composite store into one constant-index store per element.
const uint32_t ptrFnStruct =
PointerTypeTo(ctx, candidate.structTypeId, spv::StorageClass::Function);
for (uint32_t element = 0; element < composite->NumInOperands(); ++element) {
const uint32_t elementConstId = composite->GetSingleWordInOperand(element);
const uint32_t chainId = ctx->TakeNextId();
Instruction* chain =
singleStore->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpAccessChain, ptrFnStruct, chainId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {var->result_id()}},
{SPV_OPERAND_TYPE_ID, {SignedIntConstant(ctx, element)}}}));
ctx->AnalyzeDefUse(chain);
Instruction* store =
singleStore->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpStore, 0, 0,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {chainId}},
{SPV_OPERAND_TYPE_ID, {elementConstId}}}));
ctx->AnalyzeDefUse(store);
}
ctx->KillInst(singleStore);
modified = true;
}
}
if (!modified) {
return Status::SuccessWithoutChange;
}
ctx->InvalidateAnalysesExceptFor(spvtools::opt::IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
spvtools::Optimizer::PassToken
DefeatConstStructArrayLutPass::CreateDefeatConstStructArrayLutPass() {
return spvtools::Optimizer::PassToken(MakeUnique<DefeatConstStructArrayLutPass>());
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -1,36 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DefeatConstStructArrayLutPass.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// SPIRV-Cross hoists a Function-storage array variable whose only write is a single
// constant-composite store into a global `const struct[]` LUT (variable_is_lut).
// Adreno's ESSL compiler cannot dynamically index such a global const struct array
// ("Cannot offset into the structure" - device-verified on Adreno 750). Splitting
// the one composite store into per-element constant-index stores makes
// variable_is_lut fail, so SPIRV-Cross keeps the array as an ordinary local that
// Adreno indexes fine. Scalar/vector const arrays are unaffected on Adreno and are
// left alone - only arrays OF STRUCTS are rewritten. Only meant for the DirectGLES
// transpile path on Qualcomm devices.
class DefeatConstStructArrayLutPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "defeat-const-struct-array-lut"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateDefeatConstStructArrayLutPass();
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -1,131 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FoldConstOffsetFor1DFetchPass.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "FoldConstOffsetFor1DFetchPass.h"
#include "spirv.hpp"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_builder.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/util/make_unique.h"
#include <vector>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
namespace {
using spvtools::opt::Instruction;
using spvtools::opt::InstructionBuilder;
using spvtools::opt::IRContext;
using spvtools::opt::Operand;
// Number of ImageOperands ids that precede the ConstOffset id: one per
// lower-order bit set in the mask, except Grad which carries two ids.
uint32_t CountIdsBeforeConstOffset(uint32_t mask) {
uint32_t count = 0;
if (mask & static_cast<uint32_t>(spv::ImageOperandsMask::Bias)) count += 1;
if (mask & static_cast<uint32_t>(spv::ImageOperandsMask::Lod)) count += 1;
if (mask & static_cast<uint32_t>(spv::ImageOperandsMask::Grad)) count += 2;
return count;
}
} // namespace
spvtools::opt::Pass::Status FoldConstOffsetFor1DFetchPass::Process() {
auto* irContext = context();
auto* defUseMgr = irContext->get_def_use_mgr();
Bool modified = false;
constexpr uint32_t kConstOffsetBit =
static_cast<uint32_t>(spv::ImageOperandsMask::ConstOffset);
for (auto& function : *get_module()) {
for (auto& block : function) {
for (auto& inst : block) {
if (inst.opcode() != spv::Op::OpImageFetch) continue;
// In-operands: image, coordinate, [ImageOperands mask, ids...].
if (inst.NumInOperands() < 3) continue;
const uint32_t operandsMask = inst.GetSingleWordInOperand(2);
if ((operandsMask & kConstOffsetBit) == 0) continue;
Instruction* imageInst = defUseMgr->GetDef(inst.GetSingleWordInOperand(0));
if (imageInst == nullptr) continue;
Instruction* imageType = defUseMgr->GetDef(imageInst->type_id());
if (imageType == nullptr || imageType->opcode() != spv::Op::OpTypeImage ||
static_cast<spv::Dim>(imageType->GetSingleWordInOperand(1)) != spv::Dim::Dim1D) {
continue;
}
const uint32_t offsetOperandIndex = 3 + CountIdsBeforeConstOffset(operandsMask);
const uint32_t offsetId = inst.GetSingleWordInOperand(offsetOperandIndex);
const uint32_t coordId = inst.GetSingleWordInOperand(1);
Instruction* coordType = defUseMgr->GetDef(defUseMgr->GetDef(coordId)->type_id());
InstructionBuilder builder(
irContext, &inst,
IRContext::kAnalysisDefUse | IRContext::kAnalysisInstrToBlockMapping);
uint32_t newCoordId = 0;
if (coordType->opcode() == spv::Op::OpTypeVector) {
// Arrayed 1D fetch: component 0 is the texel coordinate,
// component 1 the layer - only component 0 takes the offset.
const uint32_t componentTypeId = coordType->GetSingleWordInOperand(0);
Instruction* extracted = builder.AddCompositeExtract(componentTypeId, coordId, {0});
Instruction* sum =
builder.AddIAdd(componentTypeId, extracted->result_id(), offsetId);
Instruction* inserted = builder.AddInstruction(spvtools::MakeUnique<Instruction>(
irContext, spv::Op::OpCompositeInsert, coordType->result_id(),
irContext->TakeNextId(),
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {sum->result_id()}},
{SPV_OPERAND_TYPE_ID, {coordId}},
{SPV_OPERAND_TYPE_LITERAL_INTEGER, {0}}}));
newCoordId = inserted->result_id();
} else {
Instruction* sum = builder.AddIAdd(coordType->result_id(), coordId, offsetId);
newCoordId = sum->result_id();
}
const uint32_t newMask = operandsMask & ~kConstOffsetBit;
// 3 fixed operands + the offset id: anything beyond that is another
// image-operand id that must keep the mask word alive.
const Bool otherOperandIdsRemain = inst.NumInOperands() > 4;
irContext->ForgetUses(&inst);
std::vector<Operand> newOperands;
newOperands.push_back(inst.GetInOperand(0));
newOperands.push_back({SPV_OPERAND_TYPE_ID, {newCoordId}});
if (newMask != 0 || otherOperandIdsRemain) {
Operand maskOperand = inst.GetInOperand(2);
maskOperand.words[0] = newMask;
newOperands.push_back(maskOperand);
for (uint32_t i = 3; i < inst.NumInOperands(); ++i) {
if (i == offsetOperandIndex) continue;
newOperands.push_back(inst.GetInOperand(i));
}
}
inst.SetInOperands(std::move(newOperands));
irContext->AnalyzeUses(&inst);
modified = true;
}
}
}
return modified ? Status::SuccessWithChange : Status::SuccessWithoutChange;
}
spvtools::Optimizer::PassToken FoldConstOffsetFor1DFetchPass::CreateFoldConstOffsetFor1DFetchPass() {
return spvtools::Optimizer::PassToken(MakeUnique<FoldConstOffsetFor1DFetchPass>());
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -1,36 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FoldConstOffsetFor1DFetchPass.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// SPIRV-Cross emulates 1D textures as 2D for ES targets: it widens the texelFetch
// coordinate to ivec2 but keeps the ConstOffset image operand scalar, and ESSL has
// no texelFetchOffset(sampler2D, ivec2, int, scalar-offset) overload, so drivers
// (Adreno) reject the transpiled shader. This pass folds the constant offset into
// the integer coordinate before the fetch - texelFetchOffset(t, P, l, o) ==
// texelFetch(t, P + o, l) per the GLSL spec - and drops the ConstOffset operand,
// so SPIRV-Cross emits a plain texelFetch. For arrayed 1D fetches only coordinate
// component 0 is offset (component 1 is the layer). Only meant for the DirectGLES
// transpile path.
class FoldConstOffsetFor1DFetchPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "fold-const-offset-for-1d-fetch"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateFoldConstOffsetFor1DFetchPass();
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -1,613 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/LowerClipDistanceForEsslPass.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "LowerClipDistanceForEsslPass.h"
#include "spirv.hpp"
#include "source/opt/basic_block.h"
#include "source/opt/constants.h"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/opt/type_manager.h"
#include "source/opt/types.h"
#include "source/util/make_unique.h"
#include <memory>
#include <vector>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
namespace {
using spvtools::opt::BasicBlock;
using spvtools::opt::Function;
using spvtools::opt::Instruction;
using spvtools::opt::IRContext;
using spvtools::opt::Operand;
namespace analysis = spvtools::opt::analysis;
spv::ExecutionModel EntryExecutionModel(IRContext* ctx) {
for (Instruction& ep : ctx->module()->entry_points()) {
return static_cast<spv::ExecutionModel>(ep.GetSingleWordInOperand(0));
}
return spv::ExecutionModel::Max;
}
uint32_t EntryFunctionId(IRContext* ctx) {
for (Instruction& ep : ctx->module()->entry_points()) {
// OpEntryPoint <model> <function> "name" <interface...>
return ep.GetSingleWordInOperand(1);
}
return 0;
}
uint32_t VariablePointeeType(IRContext* ctx, Instruction* var) {
Instruction* ptrType = ctx->get_def_use_mgr()->GetDef(var->type_id());
// OpTypePointer <storage-class> <pointee>
return ptrType->GetSingleWordInOperand(1);
}
uint32_t PointerTypeTo(IRContext* ctx, uint32_t pointeeId, spv::StorageClass sc) {
analysis::Type* pointee = ctx->get_type_mgr()->GetType(pointeeId);
analysis::Pointer ptr(pointee, sc);
return ctx->get_type_mgr()->GetTypeInstruction(&ptr);
}
uint32_t IntConstant(IRContext* ctx, bool isSigned, uint32_t value) {
analysis::Integer i(32, isSigned);
analysis::Type* reg = ctx->get_type_mgr()->GetRegisteredType(&i);
const analysis::Constant* c = ctx->get_constant_mgr()->GetConstant(reg, {value});
return ctx->get_constant_mgr()->GetDefiningInstruction(c)->result_id();
}
uint32_t UintType(IRContext* ctx) {
analysis::Integer i(32, false);
return ctx->get_type_mgr()->GetTypeInstruction(&i);
}
uint32_t BoolType(IRContext* ctx) {
analysis::Bool b;
return ctx->get_type_mgr()->GetTypeInstruction(&b);
}
// Constant length of OpTypeArray |arrayTypeId| (0 when not a sized constant).
uint32_t ArrayLength(IRContext* ctx, uint32_t arrayTypeId) {
Instruction* arrayType = ctx->get_def_use_mgr()->GetDef(arrayTypeId);
if (arrayType == nullptr || arrayType->opcode() != spv::Op::OpTypeArray) {
return 0;
}
Instruction* length = ctx->get_def_use_mgr()->GetDef(arrayType->GetSingleWordInOperand(1));
if (length == nullptr || length->opcode() != spv::Op::OpConstant) {
return 0;
}
return length->GetSingleWordInOperand(0);
}
bool IsConstantWithValue(IRContext* ctx, uint32_t id, uint32_t value) {
Instruction* def = ctx->get_def_use_mgr()->GetDef(id);
return def != nullptr && def->opcode() == spv::Op::OpConstant &&
def->GetSingleWordInOperand(0) == value;
}
bool IsAccessChain(const Instruction* inst) {
return inst->opcode() == spv::Op::OpAccessChain ||
inst->opcode() == spv::Op::OpInBoundsAccessChain;
}
Instruction* AddPrivateVariable(IRContext* ctx, uint32_t pointeeTypeId, const char* name) {
const uint32_t ptrType = PointerTypeTo(ctx, pointeeTypeId, spv::StorageClass::Private);
const uint32_t varId = ctx->TakeNextId();
ctx->AddGlobalValue(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpVariable, ptrType, varId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_STORAGE_CLASS,
{static_cast<uint32_t>(spv::StorageClass::Private)}}}));
ctx->AddDebug2Inst(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpName, 0, 0,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_ID, {varId}},
{SPV_OPERAND_TYPE_LITERAL_STRING, spvtools::utils::MakeVector(name)}}));
return ctx->get_def_use_mgr()->GetDef(varId);
}
// Retargets |chain| onto |newBaseId|, dropping the first |dropIndexCount| index
// operands and switching the result pointer's storage class to Private.
void RetargetChainToPrivate(IRContext* ctx, Instruction* chain, uint32_t newBaseId,
uint32_t dropIndexCount) {
Instruction* chainPtrType = ctx->get_def_use_mgr()->GetDef(chain->type_id());
const uint32_t pointeeId = chainPtrType->GetSingleWordInOperand(1);
const uint32_t newPtrType = PointerTypeTo(ctx, pointeeId, spv::StorageClass::Private);
ctx->ForgetUses(chain);
std::vector<Operand> newOperands;
newOperands.push_back({SPV_OPERAND_TYPE_ID, {newBaseId}});
for (uint32_t i = 1 + dropIndexCount; i < chain->NumInOperands(); ++i) {
newOperands.push_back(chain->GetInOperand(i));
}
chain->SetResultType(newPtrType);
chain->SetInOperands(std::move(newOperands));
ctx->AnalyzeUses(chain);
}
// ---- Output side --------------------------------------------------------------
struct OutputTarget {
Instruction* var = nullptr; // Output gl_PerVertex block or standalone builtin
bool isBlockMember = false;
uint32_t memberIndex = 0; // valid when isBlockMember
uint32_t arrayTypeId = 0; // float[N]
uint32_t elemTypeId = 0; // float
uint32_t arrayLen = 0; // N
};
// Inserts "gl_ClipDistance[k] = mg_ClipDistance[k]" for every literal k before
// |before|. Constant-index writes are the only write shape Adreno links correctly.
void InsertFlushBefore(IRContext* ctx, Instruction* before, const OutputTarget& target,
uint32_t mgVarId) {
const uint32_t ptrPrivElem =
PointerTypeTo(ctx, target.elemTypeId, spv::StorageClass::Private);
const uint32_t ptrOutElem =
PointerTypeTo(ctx, target.elemTypeId, spv::StorageClass::Output);
for (uint32_t k = 0; k < target.arrayLen; ++k) {
const uint32_t kConst = IntConstant(ctx, true, k);
const uint32_t srcChainId = ctx->TakeNextId();
before->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpAccessChain, ptrPrivElem, srcChainId,
std::initializer_list<Operand>{{SPV_OPERAND_TYPE_ID, {mgVarId}},
{SPV_OPERAND_TYPE_ID, {kConst}}}));
const uint32_t valId = ctx->TakeNextId();
before->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpLoad, target.elemTypeId, valId,
std::initializer_list<Operand>{{SPV_OPERAND_TYPE_ID, {srcChainId}}}));
const uint32_t dstChainId = ctx->TakeNextId();
std::vector<Operand> dstOperands;
dstOperands.push_back({SPV_OPERAND_TYPE_ID, {target.var->result_id()}});
if (target.isBlockMember) {
dstOperands.push_back(
{SPV_OPERAND_TYPE_ID, {IntConstant(ctx, true, target.memberIndex)}});
}
dstOperands.push_back({SPV_OPERAND_TYPE_ID, {kConst}});
before->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpAccessChain, ptrOutElem, dstChainId, dstOperands));
before->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpStore, 0, 0,
std::initializer_list<Operand>{{SPV_OPERAND_TYPE_ID, {dstChainId}},
{SPV_OPERAND_TYPE_ID, {valId}}}));
}
}
bool LowerOutputClipDistance(IRContext* ctx, bool isGeometry) {
auto* defUse = ctx->get_def_use_mgr();
// Collect (struct type, member) pairs decorated BuiltIn ClipDistance and
// standalone variables decorated BuiltIn ClipDistance.
std::vector<std::pair<uint32_t, uint32_t>> memberTargets; // (structId, member)
std::vector<uint32_t> plainTargets; // variable ids
for (Instruction& ann : ctx->annotations()) {
if (ann.opcode() == spv::Op::OpMemberDecorate && ann.NumInOperands() >= 4 &&
static_cast<spv::Decoration>(ann.GetSingleWordInOperand(2)) ==
spv::Decoration::BuiltIn &&
static_cast<spv::BuiltIn>(ann.GetSingleWordInOperand(3)) ==
spv::BuiltIn::ClipDistance) {
memberTargets.emplace_back(ann.GetSingleWordInOperand(0),
ann.GetSingleWordInOperand(1));
} else if (ann.opcode() == spv::Op::OpDecorate && ann.NumInOperands() >= 3 &&
static_cast<spv::Decoration>(ann.GetSingleWordInOperand(1)) ==
spv::Decoration::BuiltIn &&
static_cast<spv::BuiltIn>(ann.GetSingleWordInOperand(2)) ==
spv::BuiltIn::ClipDistance) {
plainTargets.push_back(ann.GetSingleWordInOperand(0));
}
}
std::vector<OutputTarget> targets;
for (Instruction& inst : ctx->module()->types_values()) {
if (inst.opcode() != spv::Op::OpVariable ||
static_cast<spv::StorageClass>(inst.GetSingleWordInOperand(0)) !=
spv::StorageClass::Output) {
continue;
}
const uint32_t pointee = VariablePointeeType(ctx, &inst);
for (const auto& [structId, member] : memberTargets) {
if (pointee != structId) continue;
Instruction* structType = defUse->GetDef(structId);
if (structType == nullptr || member >= structType->NumInOperands()) continue;
OutputTarget target;
target.var = &inst;
target.isBlockMember = true;
target.memberIndex = member;
target.arrayTypeId = structType->GetSingleWordInOperand(member);
target.arrayLen = ArrayLength(ctx, target.arrayTypeId);
targets.push_back(target);
}
for (const uint32_t varId : plainTargets) {
if (inst.result_id() != varId) continue;
OutputTarget target;
target.var = &inst;
target.isBlockMember = false;
target.arrayTypeId = pointee;
target.arrayLen = ArrayLength(ctx, target.arrayTypeId);
targets.push_back(target);
}
}
bool changed = false;
for (OutputTarget& target : targets) {
if (target.arrayLen == 0) continue;
Instruction* arrayType = defUse->GetDef(target.arrayTypeId);
target.elemTypeId = arrayType->GetSingleWordInOperand(0);
// Collect the accesses to redirect. For the block form only chains whose
// leading index selects the ClipDistance member count; for the standalone
// form every chain plus whole-variable loads/stores.
std::vector<Instruction*> chains;
std::vector<Instruction*> directAccesses;
bool unsupportedUse = false;
defUse->ForEachUser(target.var, [&](Instruction* user) {
if (IsAccessChain(user) &&
user->GetSingleWordInOperand(0) == target.var->result_id()) {
if (target.isBlockMember) {
if (user->NumInOperands() >= 2 &&
IsConstantWithValue(ctx, user->GetSingleWordInOperand(1),
target.memberIndex)) {
chains.push_back(user);
}
} else {
chains.push_back(user);
}
} else if (!target.isBlockMember) {
if (user->opcode() == spv::Op::OpLoad ||
(user->opcode() == spv::Op::OpStore &&
user->GetSingleWordInOperand(0) == target.var->result_id())) {
directAccesses.push_back(user);
} else if (user->opcode() == spv::Op::OpCopyMemory) {
unsupportedUse = true;
}
}
});
if (unsupportedUse || (chains.empty() && directAccesses.empty())) {
continue;
}
Instruction* mgVar = AddPrivateVariable(ctx, target.arrayTypeId, "mg_ClipDistance");
const uint32_t mgVarId = mgVar->result_id();
for (Instruction* chain : chains) {
const uint32_t dropCount = target.isBlockMember ? 1u : 0u;
if (chain->NumInOperands() == 1 + dropCount) {
// Pointer to the whole float[N]: reuse the private variable itself.
ctx->ReplaceAllUsesWith(chain->result_id(), mgVarId);
ctx->KillInst(chain);
} else {
RetargetChainToPrivate(ctx, chain, mgVarId, dropCount);
}
}
for (Instruction* access : directAccesses) {
ctx->ForgetUses(access);
access->SetInOperand(0, {mgVarId});
ctx->AnalyzeUses(access);
}
// Flush the shadow into the real builtin: geometry right before every
// EmitVertex, vertex before every return of the entry point. The flush is
// also what keeps the builtin statically used for cross-stage IO matching.
std::vector<Instruction*> flushSites;
if (isGeometry) {
for (Function& function : *ctx->module()) {
function.ForEachInst([&](Instruction* inst) {
if (inst->opcode() == spv::Op::OpEmitVertex) {
flushSites.push_back(inst);
}
});
}
} else {
const uint32_t entryFuncId = EntryFunctionId(ctx);
for (Function& function : *ctx->module()) {
if (function.result_id() != entryFuncId) continue;
function.ForEachInst([&](Instruction* inst) {
if (inst->opcode() == spv::Op::OpReturn ||
inst->opcode() == spv::Op::OpReturnValue) {
flushSites.push_back(inst);
}
});
}
}
for (Instruction* site : flushSites) {
InsertFlushBefore(ctx, site, target, mgVarId);
}
changed = true;
}
return changed;
}
// ---- Input side (geometry gl_in) ----------------------------------------------
bool LowerInputClipDistance(IRContext* ctx) {
auto* defUse = ctx->get_def_use_mgr();
auto* typeMgr = ctx->get_type_mgr();
// Locate the gl_in block member decorated ClipDistance.
Instruction* glInVar = nullptr;
uint32_t memberIndex = 0;
uint32_t arrayTypeId = 0; // float[N]
for (Instruction& ann : ctx->annotations()) {
if (ann.opcode() != spv::Op::OpMemberDecorate || ann.NumInOperands() < 4 ||
static_cast<spv::Decoration>(ann.GetSingleWordInOperand(2)) !=
spv::Decoration::BuiltIn ||
static_cast<spv::BuiltIn>(ann.GetSingleWordInOperand(3)) !=
spv::BuiltIn::ClipDistance) {
continue;
}
const uint32_t structId = ann.GetSingleWordInOperand(0);
const uint32_t member = ann.GetSingleWordInOperand(1);
for (Instruction& inst : ctx->module()->types_values()) {
if (inst.opcode() != spv::Op::OpVariable ||
static_cast<spv::StorageClass>(inst.GetSingleWordInOperand(0)) !=
spv::StorageClass::Input) {
continue;
}
const uint32_t pointee = VariablePointeeType(ctx, &inst);
Instruction* pointeeType = defUse->GetDef(pointee);
if (pointeeType == nullptr || pointeeType->opcode() != spv::Op::OpTypeArray ||
pointeeType->GetSingleWordInOperand(0) != structId) {
continue;
}
Instruction* structType = defUse->GetDef(structId);
if (structType == nullptr || member >= structType->NumInOperands()) continue;
glInVar = &inst;
memberIndex = member;
arrayTypeId = structType->GetSingleWordInOperand(member);
break;
}
if (glInVar != nullptr) break;
}
if (glInVar == nullptr) {
return false;
}
const uint32_t clipCount = ArrayLength(ctx, arrayTypeId);
const uint32_t vertexCount = ArrayLength(ctx, VariablePointeeType(ctx, glInVar));
if (clipCount == 0 || vertexCount == 0) {
return false;
}
// Every gl_in chain that selects the ClipDistance member:
// (vertex, member) yields a whole float[N], (vertex, member, k) an element.
std::vector<Instruction*> chains;
defUse->ForEachUser(glInVar, [&](Instruction* user) {
if (IsAccessChain(user) && user->GetSingleWordInOperand(0) == glInVar->result_id() &&
user->NumInOperands() >= 3 &&
IsConstantWithValue(ctx, user->GetSingleWordInOperand(2), memberIndex)) {
chains.push_back(user);
}
});
if (chains.empty()) {
return false;
}
Instruction* arrayTypeInst = defUse->GetDef(arrayTypeId);
const uint32_t elemTypeId = arrayTypeInst->GetSingleWordInOperand(0);
// Private mg_ClipDistanceIn = float[vertexCount][clipCount].
const uint32_t vertexCountConst = IntConstant(ctx, false, vertexCount);
analysis::Type* innerType = typeMgr->GetType(arrayTypeId);
analysis::Array outerArray(
innerType, analysis::Array::LengthInfo{
vertexCountConst,
{analysis::Array::LengthInfo::kConstant, vertexCount}});
const uint32_t outerArrayTypeId = typeMgr->GetTypeInstruction(&outerArray);
Instruction* mgInVar = AddPrivateVariable(ctx, outerArrayTypeId, "mg_ClipDistanceIn");
const uint32_t mgInVarId = mgInVar->result_id();
// Copy loop at the top of the entry point:
// for (uint t = 0; t < vertexCount * clipCount; ++t)
// mg_ClipDistanceIn[t / clipCount][t % clipCount] =
// gl_in[t / clipCount].gl_ClipDistance[t % clipCount];
// Both gl_in indices are loop-derived (dynamic): constant-index element reads
// miscompile and whole-array reads crash the Adreno compiler.
const uint32_t entryFuncId = EntryFunctionId(ctx);
Function* entryFn = nullptr;
for (Function& function : *ctx->module()) {
if (function.result_id() == entryFuncId) {
entryFn = &function;
break;
}
}
if (entryFn == nullptr || entryFn->begin() == entryFn->end()) {
return false;
}
const uint32_t uintTypeId = UintType(ctx);
const uint32_t boolTypeId = BoolType(ctx);
const uint32_t ptrFnUint = PointerTypeTo(ctx, uintTypeId, spv::StorageClass::Function);
const uint32_t ptrInElem = PointerTypeTo(ctx, elemTypeId, spv::StorageClass::Input);
const uint32_t ptrPrivElem = PointerTypeTo(ctx, elemTypeId, spv::StorageClass::Private);
const uint32_t uint0 = IntConstant(ctx, false, 0);
const uint32_t uint1 = IntConstant(ctx, false, 1);
const uint32_t uintN = IntConstant(ctx, false, clipCount);
const uint32_t uintTotal = IntConstant(ctx, false, vertexCount * clipCount);
const uint32_t memberConst = IntConstant(ctx, true, memberIndex);
BasicBlock* entryBlock = &*entryFn->begin();
auto splitPoint = entryBlock->begin();
while (splitPoint != entryBlock->end() &&
splitPoint->opcode() == spv::Op::OpVariable) {
++splitPoint;
}
// Loop counter lives with the other function-local variables.
const uint32_t counterVarId = ctx->TakeNextId();
splitPoint->InsertBefore(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpVariable, ptrFnUint, counterVarId,
std::initializer_list<Operand>{
{SPV_OPERAND_TYPE_STORAGE_CLASS,
{static_cast<uint32_t>(spv::StorageClass::Function)}}}));
const uint32_t restLabelId = ctx->TakeNextId();
BasicBlock* restBlock = entryBlock->SplitBasicBlock(ctx, restLabelId, splitPoint);
const uint32_t headerLabelId = ctx->TakeNextId();
const uint32_t checkLabelId = ctx->TakeNextId();
const uint32_t bodyLabelId = ctx->TakeNextId();
const uint32_t continueLabelId = ctx->TakeNextId();
auto makeBlock = [&](uint32_t labelId) {
return spvtools::MakeUnique<BasicBlock>(spvtools::MakeUnique<Instruction>(
ctx, spv::Op::OpLabel, 0, labelId, std::initializer_list<Operand>{}));
};
auto addInst = [&](BasicBlock* block, spv::Op opcode, uint32_t typeId,
uint32_t resultId, std::vector<Operand> operands) {
block->AddInstruction(spvtools::MakeUnique<Instruction>(
ctx, opcode, typeId, resultId, std::move(operands)));
};
// entry: t = 0; branch header
addInst(entryBlock, spv::Op::OpStore, 0, 0,
{{SPV_OPERAND_TYPE_ID, {counterVarId}}, {SPV_OPERAND_TYPE_ID, {uint0}}});
addInst(entryBlock, spv::Op::OpBranch, 0, 0, {{SPV_OPERAND_TYPE_ID, {headerLabelId}}});
// header: structured loop header
auto headerBlock = makeBlock(headerLabelId);
addInst(headerBlock.get(), spv::Op::OpLoopMerge, 0, 0,
{{SPV_OPERAND_TYPE_ID, {restLabelId}},
{SPV_OPERAND_TYPE_ID, {continueLabelId}},
{SPV_OPERAND_TYPE_LOOP_CONTROL,
{static_cast<uint32_t>(spv::LoopControlMask::MaskNone)}}});
addInst(headerBlock.get(), spv::Op::OpBranch, 0, 0,
{{SPV_OPERAND_TYPE_ID, {checkLabelId}}});
// check: t < vertexCount * clipCount ?
auto checkBlock = makeBlock(checkLabelId);
const uint32_t tCheckId = ctx->TakeNextId();
addInst(checkBlock.get(), spv::Op::OpLoad, uintTypeId, tCheckId,
{{SPV_OPERAND_TYPE_ID, {counterVarId}}});
const uint32_t condId = ctx->TakeNextId();
addInst(checkBlock.get(), spv::Op::OpULessThan, boolTypeId, condId,
{{SPV_OPERAND_TYPE_ID, {tCheckId}}, {SPV_OPERAND_TYPE_ID, {uintTotal}}});
addInst(checkBlock.get(), spv::Op::OpBranchConditional, 0, 0,
{{SPV_OPERAND_TYPE_ID, {condId}},
{SPV_OPERAND_TYPE_ID, {bodyLabelId}},
{SPV_OPERAND_TYPE_ID, {restLabelId}}});
// body: mg_ClipDistanceIn[t / N][t % N] = gl_in[t / N].gl_ClipDistance[t % N]
auto bodyBlock = makeBlock(bodyLabelId);
const uint32_t tBodyId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpLoad, uintTypeId, tBodyId,
{{SPV_OPERAND_TYPE_ID, {counterVarId}}});
const uint32_t vertexIdxId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpUDiv, uintTypeId, vertexIdxId,
{{SPV_OPERAND_TYPE_ID, {tBodyId}}, {SPV_OPERAND_TYPE_ID, {uintN}}});
const uint32_t clipIdxId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpUMod, uintTypeId, clipIdxId,
{{SPV_OPERAND_TYPE_ID, {tBodyId}}, {SPV_OPERAND_TYPE_ID, {uintN}}});
const uint32_t srcChainId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpAccessChain, ptrInElem, srcChainId,
{{SPV_OPERAND_TYPE_ID, {glInVar->result_id()}},
{SPV_OPERAND_TYPE_ID, {vertexIdxId}},
{SPV_OPERAND_TYPE_ID, {memberConst}},
{SPV_OPERAND_TYPE_ID, {clipIdxId}}});
const uint32_t valId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpLoad, elemTypeId, valId,
{{SPV_OPERAND_TYPE_ID, {srcChainId}}});
const uint32_t dstChainId = ctx->TakeNextId();
addInst(bodyBlock.get(), spv::Op::OpAccessChain, ptrPrivElem, dstChainId,
{{SPV_OPERAND_TYPE_ID, {mgInVarId}},
{SPV_OPERAND_TYPE_ID, {vertexIdxId}},
{SPV_OPERAND_TYPE_ID, {clipIdxId}}});
addInst(bodyBlock.get(), spv::Op::OpStore, 0, 0,
{{SPV_OPERAND_TYPE_ID, {dstChainId}}, {SPV_OPERAND_TYPE_ID, {valId}}});
addInst(bodyBlock.get(), spv::Op::OpBranch, 0, 0,
{{SPV_OPERAND_TYPE_ID, {continueLabelId}}});
// continue: ++t
auto continueBlock = makeBlock(continueLabelId);
const uint32_t tContinueId = ctx->TakeNextId();
addInst(continueBlock.get(), spv::Op::OpLoad, uintTypeId, tContinueId,
{{SPV_OPERAND_TYPE_ID, {counterVarId}}});
const uint32_t tIncId = ctx->TakeNextId();
addInst(continueBlock.get(), spv::Op::OpIAdd, uintTypeId, tIncId,
{{SPV_OPERAND_TYPE_ID, {tContinueId}}, {SPV_OPERAND_TYPE_ID, {uint1}}});
addInst(continueBlock.get(), spv::Op::OpStore, 0, 0,
{{SPV_OPERAND_TYPE_ID, {counterVarId}}, {SPV_OPERAND_TYPE_ID, {tIncId}}});
addInst(continueBlock.get(), spv::Op::OpBranch, 0, 0,
{{SPV_OPERAND_TYPE_ID, {headerLabelId}}});
BasicBlock* headerPtr = entryFn->InsertBasicBlockBefore(std::move(headerBlock), restBlock);
BasicBlock* checkPtr = entryFn->InsertBasicBlockAfter(std::move(checkBlock), headerPtr);
BasicBlock* bodyPtr = entryFn->InsertBasicBlockAfter(std::move(bodyBlock), checkPtr);
entryFn->InsertBasicBlockAfter(std::move(continueBlock), bodyPtr);
// Redirect the pre-existing accesses to the shadow copy.
for (Instruction* chain : chains) {
if (chain->NumInOperands() == 3) {
// (vertex, member): whole float[N] of one vertex.
Instruction* chainPtrType = defUse->GetDef(chain->type_id());
const uint32_t pointeeId = chainPtrType->GetSingleWordInOperand(1);
const uint32_t newPtrType =
PointerTypeTo(ctx, pointeeId, spv::StorageClass::Private);
ctx->ForgetUses(chain);
std::vector<Operand> newOperands;
newOperands.push_back({SPV_OPERAND_TYPE_ID, {mgInVarId}});
newOperands.push_back(chain->GetInOperand(1));
chain->SetResultType(newPtrType);
chain->SetInOperands(std::move(newOperands));
ctx->AnalyzeUses(chain);
} else {
// (vertex, member, k, ...): drop the member index.
Instruction* chainPtrType = defUse->GetDef(chain->type_id());
const uint32_t pointeeId = chainPtrType->GetSingleWordInOperand(1);
const uint32_t newPtrType =
PointerTypeTo(ctx, pointeeId, spv::StorageClass::Private);
ctx->ForgetUses(chain);
std::vector<Operand> newOperands;
newOperands.push_back({SPV_OPERAND_TYPE_ID, {mgInVarId}});
newOperands.push_back(chain->GetInOperand(1));
for (uint32_t i = 3; i < chain->NumInOperands(); ++i) {
newOperands.push_back(chain->GetInOperand(i));
}
chain->SetResultType(newPtrType);
chain->SetInOperands(std::move(newOperands));
ctx->AnalyzeUses(chain);
}
}
return true;
}
} // namespace
spvtools::opt::Pass::Status LowerClipDistanceForEsslPass::Process() {
auto* ctx = context();
const spv::ExecutionModel model = EntryExecutionModel(ctx);
const bool isVertex = model == spv::ExecutionModel::Vertex;
const bool isGeometry = model == spv::ExecutionModel::Geometry;
if (!isVertex && !isGeometry) {
return Status::SuccessWithoutChange;
}
bool changed = LowerOutputClipDistance(ctx, isGeometry);
if (isGeometry) {
changed |= LowerInputClipDistance(ctx);
}
if (!changed) {
return Status::SuccessWithoutChange;
}
ctx->InvalidateAnalysesExceptFor(spvtools::opt::IRContext::kAnalysisNone);
return Status::SuccessWithChange;
}
spvtools::Optimizer::PassToken
LowerClipDistanceForEsslPass::CreateLowerClipDistanceForEsslPass() {
return spvtools::Optimizer::PassToken(MakeUnique<LowerClipDistanceForEsslPass>());
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -1,44 +0,0 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/LowerClipDistanceForEsslPass.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Adreno's ESSL compiler mishandles gl_ClipDistance (device-verified on Adreno 750):
// - writes through non-constant indices silently fail to link,
// - reads of gl_in[i].gl_ClipDistance[k] with a CONSTANT k >= 1 fail to compile
// ("array indexing out of boundary") while dynamic-index reads work,
// - compiling a whole-array read of gl_in[i].gl_ClipDistance segfaults the
// compiler backend (libllvm-qgl.so).
// This pass shadows the builtin so the decompiled ESSL only ever touches it in the
// shapes Adreno accepts. Output side (vertex + geometry): all accesses to the
// Output ClipDistance (gl_PerVertex member or standalone variable) are redirected
// to a Private mg_ClipDistance array, and a flush writing the real builtin with
// literal constant indices is inserted before every OpEmitVertex (geometry) or
// every return of the entry point (vertex). Input side (geometry): accesses to
// gl_in[...].gl_ClipDistance are redirected to a Private mg_ClipDistanceIn
// array-of-arrays filled once at the top of the entry point by a structured loop
// whose gl_in reads use dynamic (loop-variable) indices. The builtin members stay
// statically referenced by the flush/copy so cross-stage IO matching is intact.
// Only meant for the DirectGLES transpile path on Qualcomm devices.
class LowerClipDistanceForEsslPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "lower-clip-distance-for-essl"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateLowerClipDistanceForEsslPass();
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
+1 -1
View File
@@ -87,7 +87,7 @@ def main():
ap.add_argument("--device-dir", default="/data/local/tmp/mgcts") ap.add_argument("--device-dir", default="/data/local/tmp/mgcts")
ap.add_argument("--surface", default="fbo", help="--deqp-surface-type value") ap.add_argument("--surface", default="fbo", help="--deqp-surface-type value")
ap.add_argument("--max-rounds", type=int, default=4000) ap.add_argument("--max-rounds", type=int, default=4000)
ap.add_argument("--max-empty-streak", type=int, default=8, ap.add_argument("--max-empty-streak", type=int, default=64,
help="abort after this many consecutive chunks that produce no log at all") help="abort after this many consecutive chunks that produce no log at all")
ap.add_argument("--min-mem-kb", type=int, default=400000, ap.add_argument("--min-mem-kb", type=int, default=400000,
help="pause when the device drops below this much available memory") help="pause when the device drops below this much available memory")