Compare commits

...
8 Commits
Author SHA1 Message Date
BZLZHH c93e5fa409 [Refactor] (MG_State, MG_Util): join-by-construction link/compile artifacts (P1 stage 2)
Still fully synchronous - EnsureLinkJoined()/EnsureCompileJoined() are empty
inline no-ops (verified to fold away at every one of the ~1200 call sites;
this project builds without LTO) - but every read of link- or compile-produced
state now goes through a private accessor the compiler enforces, so when
stage 4 moves the bodies onto pool workers, 'which reads must join' is a
type-system fact instead of a 400-line audit.

- ProgramObject: the 31 fields ResetLinkArtifacts clears plus the 5 link
  outputs it forgot (infoLog, linkedFragData{Location,Index}, the geometry
  strip-capture pair) move into a nested LinkArtifacts behind Artifacts().
  ResetLinkArtifacts is now a worker-safe pure clear; the link-observable
  version bumps (backendState/link/uboContent) move to a GL-thread-only
  BumpLinkObservableVersions() called once from Link()'s prologue and from
  glProgramBinary's mandated failure - the link body never writes them, so
  a stage-4 worker cannot lose an invalidation against the draw path.
- ShaderObject: compile artifacts (TShader, preprocessed source, side-channel
  maps, status/log, consume-once flag) behind Compiled(); the P0b layer-1
  memo trio deliberately stays outside as the future non-joining
  COMPLETION_STATUS_KHR fast path.
- CompileEnv (new): a GL-thread snapshot of everything the compile pipeline
  used to read live from the backend mid-parse - compute limits (the
  GetIntegeri_v reach-back is gone from the worker path), advertised
  extensions, device quirks, TBuiltInResource inputs. Captured lazily per
  backend activation; the consume-once re-parse now runs against the same
  env as the original parse.
- The GL-thread prologue / worker-body boundary is marked in Link() where
  the stage sort ends; everything below is a pure function of the snapshot.

Public getter signatures unchanged - MG_Impl and both backends compile
untouched. Unit 476/476, Program suites 117/117, DirectGLES retrace 38/39 on
llvmpipe (the one failure is the known pre-existing non-CI iterationrp case;
the NVIDIA userspace driver was updated out from under the running kernel
module mid-session, so GLX there is down until a reboot).
2026-08-08 07:12:37 -04:00
BZLZHH 8191075133 [Feat] (MG_Util): the async-compile pool skeleton behind a default-off flag (P1 stage 1)
Standalone Asio (submodule, asio-1-38-2 @ 8806a680, ASIO_STANDALONE +
ASIO_NO_DEPRECATED, header-only - no linked artifact) and the job machinery
the async shader pipeline will run on: JobNode (state machine with deferred
errors, continuations firing exactly once, dependency counters, cancel
semantics split into request vs outcome) and ShaderCompilePool
(asio::thread_pool behind a pimpl so no header leaks asio; big-core count
via cpufreq at >=85% of peak clamped to [1,4]; lazily constructed, so with
the flag off no worker thread ever exists; StopAndDrain leads DestroyImpl).

MOBILEGL_ASYNC_SHADER_COMPILE / _THREADS config knobs, default OFF. Nothing
in the GL pipeline references the pool yet - grep-verified; the full
DirectGLES retrace and compile benches are byte- and time-identical. 25
threaded unit tests, clean across 20x gtest_repeat.
2026-08-08 05:28:51 -04:00
BZLZHH d6caed7822 [Fix] (MG_Util, MG_State): five latent frontend bugs the async work made load-bearing
- SpvcSession's move constructor and move assignment dropped the parsed
  metadata, so a moved-to session silently reported empty reflection.
- ParseComputeLocalSize used std::stoull, whose std::out_of_range escaped
  glCompileShader on an oversized local_size literal; now std::from_chars
  saturating to UINT_MAX, pinned by a regression test that reproduced the
  escaping exception.
- The compute local_size std::regex was rebuilt on every compile; hoisted.
- LinkProgram dumped every shader's full source through MGLOG_D per link.
- glslang::FinalizeProcess ran before the GL context tore down, leaving the
  context's live TShaders pointing at freed builtin symbol tables.
2026-08-08 05:28:28 -04:00
BZLZHH 9152e88734 [Perf] (MG_State): dedupe shader compiles by source hash
Iris-style packs hand MobileGL the same source text repeatedly: probed across
three shaderpack traces, 28-32% of all glCompileShader work was redundant -
~9% same-object recompiles with byte-identical source, ~21% distinct shader
objects sharing identical source (the same common GLSL chunk glued into many
program stages). Two layers, both keyed by XXH64 + length with a full byte
compare on every hit (correctness never rides on the hash):

- Per-object: a successful (or failed) compile remembers its source hash;
  glShaderSource with byte-identical text keeps the compiled state and
  glCompileShader on unchanged source returns immediately. Deterministic
  (stage, source) pipeline makes the memo observationally identical to
  recompiling; the consume-once TakeShaderForLink re-parse path is untouched.

- Cross-object: a per-context bounded cache (ProgramState-owned, declared to
  outlive every shader object) shares the preprocessed source, both explicit
  side-channel maps, and the validation verdicts between objects with equal
  source; only the glslang parse stays per-object. Single-GL-thread today;
  flagged for a mutex when compiles go async (P1).

Interleaved A/B on the iterationrp trace (the recompile-heavy pack):
5.65s -> 5.46s median total replay, every round faster; BSL/complementary
stay flat (their duplicate sources are the small common shaders, so calls
drop but wall time is parse-bound on unique sources). Full DirectGLES
retrace, 445-test unit suite, and dedupe-semantics tests (no-op recompile,
invalidation on new source, failed-compile memo, cache bounds) all green.
2026-08-08 04:18:32 -04:00
BZLZHH 2406e2d219 [Perf, Fix] (MG_Util): preprocessing cleanups - dead scanners, quote-mask bug, one version inspection per compile
Three scoped changes to ShaderSourceProcessor, none altering any transform's
output (pinned by a byte-stability test across the legacy-shader anchor path):

- Delete BlankBlockComments and RemoveDefineForIdentifier - dead since their
  callers left; the former's newline-terminated quote handling moves into
  MaskCommentsAndQuotedText (below) together with its rationale comment.

- Fix MaskCommentsAndQuotedText treating a quote as running past end-of-line.
  GLSL has no multi-line literals, but a stray apostrophe in a directive or
  comment tail ("#pragma message can't") blanked the REST OF THE FILE for
  every masked consumer - the tokenizer, the version inspection, and the P0a
  explicit-location/binding extractors silently lost everything after it.

- Inspect the shader language once per PreprocessShaderSource run instead of
  up to five times: NormalizeVersionDirective now takes the already-computed
  ShaderLanguageInfo, and the two after-version injections share one
  AfterVersionAnchor instead of re-running a full masked sweep each
  (FindAfterVersionDirective -> InspectShaderLanguage) to find the same spot.

Compile-phase timings hold (BSL 1.848s, complementary-reimagined ~5.7s);
retraces and the 435-test unit suite unchanged.
2026-08-08 03:43:56 -04:00
BZLZHH b228f813c0 [Perf] (MG_Util): replace the builtin-shadowing string scans with one tokenize and a SPIR-V OpName pass
RenameBuiltinShadowingFunction probed the whole source ten times per compile
(5 names x mask + scan, each a full-text pass) and still had two blind spots:
a 5-name list and single-line-definition-only detection. On Complementary-scale
packs (4.5MB of sources) that was ~68% of the compile phase.

The rename is now split by FAILURE LAYER, both halves sharing one name table
header so they cannot drift:

- A SPIR-V OpName pass in SanitizeAndOptimizeBinary covers the full ESSL 3.20
  builtin set (~146 names). Renaming a function id is safe by construction:
  builtin calls are GLSL.std.450 instructions and can never resolve to a user
  OpFunction, overloads are distinct ids (a helper overload delegating to the
  real builtin keeps working), dead preprocessor branches never reach SPIR-V,
  and macro-expanded definitions are covered. ESSL 3.x is the only consumer
  that forbids the redefinitions, and this pass runs before its transpile.

- A lexical pass covers only the 5 names whose exact-signature redefinitions
  glslang's relaxed parse rejects outright (never producing SPIR-V for the
  backstop): the historical fma/max3/min3/round/tanh. One TokenizeCode pass;
  definition detection requires brace depth 0, a type-identifier previous
  token that is neither a statement keyword nor a directive tail, and skips
  files whose token-level braces do not balance (preprocessor-asymmetric
  arms) - over-detection is unrecoverable, so every ambiguity falls through
  to the backstop.

Measured on the compile phase (prefix-diff, 3-run medians, Espryt/NVIDIA):
complementary-reimagined 20.0s -> 5.5s, BSL 2.14s -> 1.85s. bliss (the pack
that ships from-scratch fma/tanh helpers) stays at SSIM 0.999962.

Tests: end-to-end ESSL assertions for the multiline-definition and
new-overload shapes, the three adversarial-review reproductions (statement-
keyword call under asymmetric braces, dead-#if compat shim, overload
delegating to the shadowed builtin), and a source-level assertion pinning
the lexical half specifically.
2026-08-08 03:11:38 -04:00
BZLZHH 0d0527192a [Perf] (MG_State, MG_Util): compile shaders with a single relaxed parse
glCompileShader used to parse every source twice: once under the GL client
(reflection only) and once under the relaxed Vulkan client (SPIR-V + the
plain-uniform global UBO), with GenerateBinary re-preprocessing, re-parsing
and re-linking every attached shader on every glLinkProgram. The GL-client
pass is gone: Compile() performs the one link-compatible relaxed parse and
the linked TProgram serves reflection and codegen both. Measured on the BSL
shaderpack compile phase: Espryt 2.80s -> 2.14s, Magma 3.78s -> 3.07s.

What the relaxed parse cannot provide is restored explicitly:
- explicit layout(location/binding) qualifiers on default-block uniforms and
  samplers are extracted lexically at Compile() (the relaxed parse strips
  them) and merged per link with cross-stage conflict checks;
- uniforms the relaxed parse sweeps into MGL_GLOBAL_UBO but no stage reads
  are filtered from the GL reflection surface through GL<->TProgram index
  translation maps (dead uniforms stay inactive, the synthesized block stays
  hidden, builtins reflect under their GL spellings);
- SPIR-V is generated BEFORE buildReflection touches the program (its
  live-variable analysis perturbs GlslangToSpv output - generated modules
  stay bit-identical to the old pipeline's), while the glUniform*-to-scratch
  routing tables are built strictly AFTER reflection, whose results size and
  key them;
- a TShader feeds exactly one link (mapIO mutates the intermediate); relinks
  and multi-program attachments re-parse the stored preprocessed source.

Validated: DirectGLES retrace suite green (two pre-existing local-driver
failures unchanged old vs new), KHR-GL30 877/878 on Espryt/NVIDIA (the one
failure pre-exists this change), unit tests green, per-module SPIR-V hashes
identical across a full DirectVulkan replay.
2026-08-08 01:25:54 -04:00
BZLZHH 81bcbd6c14 [Fix] (MG_State): allocate program and shader names from one shared name space
GL 3.3 core 2.11 puts program and shader names in one name space: a shader
name passed where a program is expected must fail with INVALID_OPERATION,
and vice versa. Two independent IndexGenerators handed out colliding names
(shader 2 and program 2 could coexist), so CheckProgramNameValidity resolved
a shader handle to an unrelated linked program and the error checks in
KHR-GL30.get_uniform_tests.get_uniform were silently swallowed - the case
only ever passed because the collided program happened to reject the queried
location. One shared generator keeps the names disjoint; the per-kind object
tables are unchanged.
2026-08-08 01:25:37 -04:00
36 changed files with 4809 additions and 839 deletions
+3
View File
@@ -31,3 +31,6 @@
[submodule "3rdparty/apitrace"] [submodule "3rdparty/apitrace"]
path = 3rdparty/apitrace path = 3rdparty/apitrace
url = https://github.com/MobileGL-Dev/apitrace.git url = https://github.com/MobileGL-Dev/apitrace.git
[submodule "3rdparty/asio"]
path = 3rdparty/asio
url = https://github.com/chriskohlhoff/asio.git
Vendored Submodule
+1
Submodule 3rdparty/asio added at 8806a6803c
+20
View File
@@ -154,6 +154,9 @@ set(SOURCE_FILES
MobileGL/MG_Util/Debug/Log.cpp MobileGL/MG_Util/Debug/Log.cpp
MobileGL/MG_Util/Async/JobNode.cpp
MobileGL/MG_Util/Async/ShaderCompilePool.cpp
MobileGL/MG_Util/Math/VectorTypes.cpp MobileGL/MG_Util/Math/VectorTypes.cpp
MobileGL/MG_Util/Metrics/TextureMetrics.cpp MobileGL/MG_Util/Metrics/TextureMetrics.cpp
@@ -187,6 +190,7 @@ set(SOURCE_FILES
MobileGL/MG_Util/Classifiers/TextureEnumClassifier.cpp MobileGL/MG_Util/Classifiers/TextureEnumClassifier.cpp
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp
MobileGL/MG_Util/ShaderTranspiler/SpvcSession.cpp MobileGL/MG_Util/ShaderTranspiler/SpvcSession.cpp
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp
@@ -194,6 +198,7 @@ set(SOURCE_FILES
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FlattenInterfaceStructPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/FlattenInterfaceStructPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EliminateFloatEqualsZeroPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/EliminateFloatEqualsZeroPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RenameSamplerFunctionParameterPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RenameSamplerFunctionParameterPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RenameBuiltinShadowingFunctionsPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DecomposeWorkgroupVec3Pass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DecomposeWorkgroupVec3Pass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DecoratePositionInvariantPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/DecoratePositionInvariantPass.cpp
MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/LowerDrawParametersPass.cpp MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/LowerDrawParametersPass.cpp
@@ -290,6 +295,7 @@ set(SOURCE_FILES
MobileGL/MG_State/GLState/TextureState/TextureState.cpp MobileGL/MG_State/GLState/TextureState/TextureState.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramObject.cpp MobileGL/MG_State/GLState/ProgramState/ProgramObject.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramState.cpp MobileGL/MG_State/GLState/ProgramState/ProgramState.cpp
MobileGL/MG_State/GLState/RenderState/RenderState.cpp MobileGL/MG_State/GLState/RenderState/RenderState.cpp
MobileGL/MG_State/GLState/FramebufferState/FramebufferObject.cpp MobileGL/MG_State/GLState/FramebufferState/FramebufferObject.cpp
@@ -323,6 +329,11 @@ if (WIN32)
) )
endif() endif()
# The shader-compile pool runs standalone Asio on real threads. This host's glibc (>= 2.34)
# merged pthread into libc, so it links without asking, but the NDK and musl are not
# guaranteed to be as forgiving - ask for it explicitly rather than rely on the accident.
find_package(Threads REQUIRED)
set(MOBILEGL_LINK_LIBRARIES set(MOBILEGL_LINK_LIBRARIES
glslang::glslang glslang::glslang
spirv-cross-c spirv-cross-c
@@ -332,12 +343,17 @@ set(MOBILEGL_LINK_LIBRARIES
GPUOpen::VulkanMemoryAllocator GPUOpen::VulkanMemoryAllocator
Vulkan::UtilityHeaders Vulkan::UtilityHeaders
spirv-reflect-static spirv-reflect-static
Threads::Threads
) )
set(MOBILEGL_COMPILE_DEF set(MOBILEGL_COMPILE_DEF
-DVMA_STATIC_VULKAN_FUNCTIONS=0 -DVMA_STATIC_VULKAN_FUNCTIONS=0
-DVMA_DYNAMIC_VULKAN_FUNCTIONS=1 -DVMA_DYNAMIC_VULKAN_FUNCTIONS=1
-DVMA_VULKAN_VERSION=1001000 -DVMA_VULKAN_VERSION=1001000
# Header-only Asio, no Boost, no deprecated interfaces. Set on the definition list
# rather than per-target so the shared library and the _s static target agree.
-DASIO_STANDALONE
-DASIO_NO_DEPRECATED
) )
message(STATUS "MOBILEGL_COMPILE_DEF=${MOBILEGL_COMPILE_DEF}") message(STATUS "MOBILEGL_COMPILE_DEF=${MOBILEGL_COMPILE_DEF}")
@@ -349,6 +365,10 @@ set(MOBILEGL_INCLUDE_DIR
${spirv-tools_SOURCE_DIR}/include ${spirv-tools_SOURCE_DIR}/include
${spirv-tools_BINARY_DIR} ${spirv-tools_BINARY_DIR}
${SPIRV-Headers_SOURCE_DIR}/include ${SPIRV-Headers_SOURCE_DIR}/include
# Header-only submodule: no add_subdirectory, no link target. Only
# MG_Util/Async/ShaderCompilePool.cpp includes it, and it stays behind that file's
# pimpl so no consumer target needs this path.
${CMAKE_SOURCE_DIR}/3rdparty/asio/asio/include
) )
add_library(${CMAKE_PROJECT_NAME} SHARED add_library(${CMAKE_PROJECT_NAME} SHARED
+9
View File
@@ -127,6 +127,15 @@ namespace MobileGL::MG_Config {
// "compute", see GLESMultiDrawMode). Clamped to driver support; unset picks the best // "compute", see GLESMultiDrawMode). Clamped to driver support; unset picks the best
// supported tier, which never includes "compute" - see the note on its resolution. // supported tier, which never includes "compute" - see the note on its resolution.
GLESMultiDrawMode EsprytMultiDrawMode = GLESMultiDrawMode::Auto; GLESMultiDrawMode EsprytMultiDrawMode = GLESMultiDrawMode::Auto;
// MOBILEGL_ASYNC_SHADER_COMPILE: overrides asynchronous shader compilation. Unset
// keeps the built-in default (MG_Util::Async::kAsyncShaderCompileDefault); falsy
// forces every glCompileShader/glLinkProgram to run synchronously on the calling
// thread AND withdraws GL_KHR_parallel_shader_compile, so the single switch reverts
// both the threading and the application-visible behaviour change.
QuirkOverride AsyncShaderCompile = QuirkOverride::Auto;
// MOBILEGL_ASYNC_SHADER_COMPILE_THREADS: shader-compile worker count. 0 (unset) means
// auto, which is min(4, big cores); an explicit value is honoured as given.
Uint32 AsyncShaderCompileThreads = 0;
}; };
extern FeaturesTable Features; extern FeaturesTable Features;
} // namespace MobileGL::MG_Config } // namespace MobileGL::MG_Config
+2
View File
@@ -181,6 +181,8 @@ namespace MobileGL::MG_ConfigLoader {
features.DisableRobustBufferAccess = QueryEnvFlag("MOBILEGL_DISABLE_ROBUST_BUFFER_ACCESS"); features.DisableRobustBufferAccess = QueryEnvFlag("MOBILEGL_DISABLE_ROBUST_BUFFER_ACCESS");
features.MagmaMultiDrawMode = QueryEnvMultiDrawMode("MOBILEGL_MAGMA_MULTIDRAW_MODE"); features.MagmaMultiDrawMode = QueryEnvMultiDrawMode("MOBILEGL_MAGMA_MULTIDRAW_MODE");
features.EsprytMultiDrawMode = QueryEnvGLESMultiDrawMode("MOBILEGL_ESPRYT_MULTIDRAW_MODE"); features.EsprytMultiDrawMode = QueryEnvGLESMultiDrawMode("MOBILEGL_ESPRYT_MULTIDRAW_MODE");
features.AsyncShaderCompile = QueryEnvQuirkOverride("MOBILEGL_ASYNC_SHADER_COMPILE");
features.AsyncShaderCompileThreads = QueryEnvUint32("MOBILEGL_ASYNC_SHADER_COMPILE_THREADS", 0, 0, 64);
} }
inline void InitBackendType() { inline void InitBackendType() {
+13 -1
View File
@@ -15,6 +15,7 @@
#include <MG_Impl/GLImpl/Texture/ProxyTexture.h> #include <MG_Impl/GLImpl/Texture/ProxyTexture.h>
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h> #include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Sync/GL_Sync.h> #include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <atomic> #include <atomic>
#include <mutex> #include <mutex>
@@ -37,7 +38,12 @@ namespace MobileGL {
if (logLifecycle) { if (logLifecycle) {
MGLOG_I("MobileGL closing..."); MGLOG_I("MobileGL closing...");
} }
glslang::FinalizeProcess(); // First, before anything else is torn down. In-flight compile/link jobs own
// their own inputs and are safe against everything below EXCEPT glslang's
// process globals and the TShader/TProgram objects hanging off pGLContext,
// both of which this function is about to destroy. This is the one
// cancellation path in the whole design that waits.
MG_Util::Async::ShaderCompilePool::Get().StopAndDrain();
// GL syncs die with their contexts, and every context is gone by the // GL syncs die with their contexts, and every context is gone by the
// time full teardown runs: drain the live-sync registry while the // time full teardown runs: drain the live-sync registry while the
// backend function table can still release the backend handles (and // backend function table can still release the backend handles (and
@@ -49,6 +55,12 @@ namespace MobileGL {
MG_State::pEGLContext.reset(); MG_State::pEGLContext.reset();
MG_Impl::GLImpl::TextureImpl::pProxyTextureManager.reset(); MG_Impl::GLImpl::TextureImpl::pProxyTextureManager.reset();
MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo.reset(); MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo.reset();
// Must run AFTER pGLContext.reset(). FinalizeProcess -> ShFinalize deletes
// glslang's process-wide pool allocator and every cached built-in symbol table,
// while the TShader/TProgram objects owned by the shader and program objects
// still reference levels adopted from those tables. Finalizing first left live
// glslang objects pointing at freed memory for the rest of the teardown.
glslang::FinalizeProcess();
MG_Backend::gBackendFunctionsTable = {}; MG_Backend::gBackendFunctionsTable = {};
g_isInitialized = false; g_isInitialized = false;
if (logLifecycle) { if (logLifecycle) {
+14
View File
@@ -9,6 +9,8 @@
#include "Core.h" #include "Core.h"
#include "MG_State/GLState/RenderbufferState/RenderbufferObject.h" #include "MG_State/GLState/RenderbufferState/RenderbufferObject.h"
#include "MG_State/EGLState/Core.h" #include "MG_State/EGLState/Core.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <Config.h> #include <Config.h>
namespace MobileGL::MG_State { namespace MobileGL::MG_State {
@@ -24,6 +26,18 @@ namespace MobileGL::MG_State {
} }
namespace GLState { namespace GLState {
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv>& GLContext::GetCompileEnv() {
const void* backend = static_cast<const void*>(MG_Backend::pActiveBackendObject.get());
if (!m_compileEnv || m_compileEnvBackend != backend) {
// First use, or the backend was swapped underneath us. Re-capturing rolls the
// fingerprint, so every P0b preprocess memo computed against the old backend's
// limits becomes structurally unreachable instead of silently reusable.
m_compileEnv = MG_Util::ShaderTranspiler::CaptureCompileEnv();
m_compileEnvBackend = backend;
}
return m_compileEnv;
}
// Error // Error
void GLContext::RecordError(ErrorCode code, UniquePtr<ErrorInfo> info) { void GLContext::RecordError(ErrorCode code, UniquePtr<ErrorInfo> info) {
m_errorState.RecordError(code, Move(info)); m_errorState.RecordError(code, Move(info));
+18
View File
@@ -21,6 +21,10 @@
#include "VertexArrayState/VertexArrayState.h" #include "VertexArrayState/VertexArrayState.h"
#include "RenderbufferState/RenderbufferState.h" #include "RenderbufferState/RenderbufferState.h"
namespace MobileGL::MG_Util::ShaderTranspiler {
struct CompileEnv;
}
namespace MobileGL { namespace MobileGL {
namespace MG_State { namespace MG_State {
void Init(); void Init();
@@ -380,6 +384,15 @@ namespace MobileGL {
Bool ValidateRenderbufferName(Uint index) const; Bool ValidateRenderbufferName(Uint index) const;
Bool ValidateRenderbufferObject(Uint index) const; Bool ValidateRenderbufferObject(Uint index) const;
// P1: the shader compile/link pipeline's snapshot of everything it reads from
// outside its own (stage, source) inputs. Captured lazily here because it
// cannot be captured in MG_State::Init() - that runs BEFORE MG_Backend::Init(),
// so there is no backend to query yet. Re-captured whenever the active backend
// object changes, which also rolls the fingerprint and therefore invalidates
// every P0b preprocess memo keyed against the old one.
// GL thread only.
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv>& GetCompileEnv();
private: private:
// State Components // State Components
ErrorState m_errorState; ErrorState m_errorState;
@@ -437,6 +450,11 @@ namespace MobileGL {
FramebufferState m_framebufferState; FramebufferState m_framebufferState;
SamplerState m_samplerState; SamplerState m_samplerState;
RenderbufferState m_renderbufferState; RenderbufferState m_renderbufferState;
mutable SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> m_compileEnv;
// Identity of the backend object m_compileEnv was captured against; a plain
// pointer compare, never dereferenced.
const void* m_compileEnvBackend = nullptr;
}; };
} // namespace GLState } // namespace GLState
File diff suppressed because it is too large Load Diff
@@ -44,28 +44,28 @@ namespace MobileGL::MG_State::GLState {
Vector<SharedPtr<ShaderObject>>& GetAttachedShaders(); Vector<SharedPtr<ShaderObject>>& GetAttachedShaders();
const Vector<SharedPtr<ShaderObject>>& GetAttachedShaders() const; const Vector<SharedPtr<ShaderObject>>& GetAttachedShaders() const;
const String& GetInfoLog() const { return m_infoLog; } const String& GetInfoLog() const { return Artifacts().infoLog; }
// glCreateShaderProgramv folds the shader's compile log into the program's log, which // glCreateShaderProgramv folds the shader's compile log into the program's log, which
// is the only place a caller can read it from once the shader name is gone. // is the only place a caller can read it from once the shader name is gone.
void AppendInfoLog(const String& text) { void AppendInfoLog(const String& text) {
if (text.empty()) return; if (text.empty()) return;
if (!m_infoLog.empty() && m_infoLog.back() != '\n') m_infoLog += '\n'; if (!Artifacts().infoLog.empty() && Artifacts().infoLog.back() != '\n') Artifacts().infoLog += '\n';
m_infoLog += text; Artifacts().infoLog += text;
} }
Int GetUniformMaxLength() const { return m_uniformNameMaxLength; } Int GetUniformMaxLength() const { return Artifacts().uniformNameMaxLength; }
Uint GetUniformCount() const { return m_activeUniformCount; } Uint GetUniformCount() const { return Artifacts().activeUniformCount; }
Uint GetMaxUniformLocation() const { return m_maxUniformLocation; } Uint GetMaxUniformLocation() const { return Artifacts().maxUniformLocation; }
Int GetUniformLocation(const String& name) const { Int GetUniformLocation(const String& name) const {
const auto it = m_uniformLocations.find(name); const auto it = Artifacts().uniformLocations.find(name);
if (it != m_uniformLocations.end()) return (Int)it->second; if (it != Artifacts().uniformLocations.end()) return (Int)it->second;
// Reflection stores GL-style names: an array uniform is keyed "arr[0]" (its base // Reflection stores GL-style names: an array uniform is keyed "arr[0]" (its base
// location). A bare "arr" query resolves to that entry; an "arr[k]" query resolves // location). A bare "arr" query resolves to that entry; an "arr[k]" query resolves
// to base + k because DoReflection reserves one location per array element. // to base + k because DoReflection reserves one location per array element.
if (name.empty()) return -1; if (name.empty()) return -1;
if (name.back() != ']') { if (name.back() != ']') {
const auto suffixedIt = m_uniformLocations.find(name + "[0]"); const auto suffixedIt = Artifacts().uniformLocations.find(name + "[0]");
if (suffixedIt != m_uniformLocations.end()) return (Int)suffixedIt->second; if (suffixedIt != Artifacts().uniformLocations.end()) return (Int)suffixedIt->second;
return -1; return -1;
} }
if (name.length() < 4) return -1; if (name.length() < 4) return -1;
@@ -78,21 +78,21 @@ namespace MobileGL::MG_State::GLState {
element = element * 10 + static_cast<Uint>(name[i] - '0'); element = element * 10 + static_cast<Uint>(name[i] - '0');
if (element > 0x0FFFFFFFu) return -1; if (element > 0x0FFFFFFFu) return -1;
} }
auto baseIt = m_uniformLocations.find(name.substr(0, bracket) + "[0]"); auto baseIt = Artifacts().uniformLocations.find(name.substr(0, bracket) + "[0]");
if (baseIt == m_uniformLocations.end()) { if (baseIt == Artifacts().uniformLocations.end()) {
// Legacy key without the "[0]" suffix (defensive; reflection normally // Legacy key without the "[0]" suffix (defensive; reflection normally
// stores the suffixed form for arrays). // stores the suffixed form for arrays).
baseIt = m_uniformLocations.find(name.substr(0, bracket)); baseIt = Artifacts().uniformLocations.find(name.substr(0, bracket));
if (baseIt == m_uniformLocations.end()) return -1; if (baseIt == Artifacts().uniformLocations.end()) return -1;
} }
const Int base = (Int)baseIt->second; const Int base = (Int)baseIt->second;
if (!IsValidUniformLocation(base)) return -1; if (!IsValidUniformLocation(base)) return -1;
const Int index = m_uniformIndexInTProgram[base]; const Int index = Artifacts().uniformIndexInTProgram[base];
// "[k]" only addresses arrays ("scalar[0]" is not a uniform name), and only // "[k]" only addresses arrays ("scalar[0]" is not a uniform name), and only
// in-range elements. // in-range elements.
const glslang::TType* type = m_program->getUniform(index).getType(); const glslang::TType* type = Artifacts().program->getUniform(index).getType();
if (type == nullptr || !type->isArray()) return -1; if (type == nullptr || !type->isArray()) return -1;
if (static_cast<GLint>(element) >= GetActiveUniformArraySize(index)) return -1; if (static_cast<GLint>(element) >= GetUniformArraySizeByTIndex(index)) return -1;
const Int location = base + (Int)element; const Int location = base + (Int)element;
if (!UniformLocationsAliasSameUniform(base, location)) return -1; if (!UniformLocationsAliasSameUniform(base, location)) return -1;
return location; return location;
@@ -101,14 +101,33 @@ namespace MobileGL::MG_State::GLState {
// True when both locations are element slots of the same uniform variable. // True when both locations are element slots of the same uniform variable.
Bool UniformLocationsAliasSameUniform(Int a, Int b) const { Bool UniformLocationsAliasSameUniform(Int a, Int b) const {
if (!IsValidUniformLocation(a) || !IsValidUniformLocation(b)) return false; if (!IsValidUniformLocation(a) || !IsValidUniformLocation(b)) return false;
return m_uniformIndexInTProgram[a] == m_uniformIndexInTProgram[b]; return Artifacts().uniformIndexInTProgram[a] == Artifacts().uniformIndexInTProgram[b];
}
// ---- GL index <-> glslang TProgram index translation ----
// The single relaxed parse enumerates artifacts GL must not see: every declared
// default-block uniform (even dead ones) as a member of the synthesized
// MGL_GLOBAL_UBO, and that block itself. DoReflection builds filtered GL-facing
// index spaces; every public "index"-taking getter translates through them, so
// GL and backend consumers keep seeing exactly the pre-P0a surface.
Int TProgramUniformIndex(Uint glIndex) const {
return Artifacts().glUniformIndexToTProgram[glIndex];
}
Int GlUniformIndexFromTProgram(Int tIndex) const {
if (tIndex < 0 || tIndex >= static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size())) return -1;
return Artifacts().tProgramUniformIndexToGl[tIndex];
}
Int GlBlockIndexFromTProgram(Int tBlockIndex) const {
if (tBlockIndex < 0 || tBlockIndex >= static_cast<Int>(Artifacts().tProgramBlockIndexToGl.size())) return -1;
return Artifacts().tProgramBlockIndexToGl[tBlockIndex];
} }
Int GetActiveUniformIndex(const String& name) const { Int GetActiveUniformIndex(const String& name) const {
const Int uniformIndex = m_program->getUniformIndex(name.c_str()); const Int tProgramCount = static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
if (uniformIndex >= 0 && uniformIndex < m_activeUniformCount && const Int uniformIndex = Artifacts().program->getUniformIndex(name.c_str());
m_program->getUniform(uniformIndex).name == name) { if (uniformIndex >= 0 && uniformIndex < tProgramCount &&
return uniformIndex; Artifacts().program->getUniform(uniformIndex).name == name) {
return GlUniformIndexFromTProgram(uniformIndex);
} }
// Reflection stores an array uniform under "arr[0]"; accept the bare "arr" // Reflection stores an array uniform under "arr[0]"; accept the bare "arr"
@@ -116,45 +135,48 @@ namespace MobileGL::MG_State::GLState {
// robustness against non-suffixed reflection entries. // robustness against non-suffixed reflection entries.
if (!name.empty() && name.back() != ']') { if (!name.empty() && name.back() != ']') {
const String suffixedName = name + "[0]"; const String suffixedName = name + "[0]";
const Int suffixedIndex = m_program->getUniformIndex(suffixedName.c_str()); const Int suffixedIndex = Artifacts().program->getUniformIndex(suffixedName.c_str());
if (suffixedIndex >= 0 && suffixedIndex < m_activeUniformCount && if (suffixedIndex >= 0 && suffixedIndex < tProgramCount &&
m_program->getUniform(suffixedIndex).name == suffixedName) { Artifacts().program->getUniform(suffixedIndex).name == suffixedName) {
return suffixedIndex; return GlUniformIndexFromTProgram(suffixedIndex);
} }
return -1; return -1;
} }
if (name.length() <= 3 || name.compare(name.length() - 3, 3, "[0]") != 0) return -1; if (name.length() <= 3 || name.compare(name.length() - 3, 3, "[0]") != 0) return -1;
const String baseName = name.substr(0, name.length() - 3); const String baseName = name.substr(0, name.length() - 3);
const Int baseIndex = m_program->getUniformIndex(baseName.c_str()); const Int baseIndex = Artifacts().program->getUniformIndex(baseName.c_str());
if (baseIndex < 0 || baseIndex >= m_activeUniformCount) return -1; if (baseIndex < 0 || baseIndex >= tProgramCount) return -1;
return m_program->getUniform(baseIndex).name == baseName ? baseIndex : -1; return Artifacts().program->getUniform(baseIndex).name == baseName ? GlUniformIndexFromTProgram(baseIndex)
: -1;
} }
Bool IsValidUniformLocation(Int location) const { Bool IsValidUniformLocation(Int location) const {
if (location < 0 || location > static_cast<Int>(m_maxUniformLocation)) return false; if (location < 0 || location > static_cast<Int>(Artifacts().maxUniformLocation)) return false;
if (static_cast<SizeT>(location) >= m_uniformIndexInTProgram.size()) return false; if (static_cast<SizeT>(location) >= Artifacts().uniformIndexInTProgram.size()) return false;
const Int uniformIndexInProgram = m_uniformIndexInTProgram[location]; const Int uniformIndexInProgram = Artifacts().uniformIndexInTProgram[location];
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd && return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
uniformIndexInProgram >= 0 && uniformIndexInProgram < m_activeUniformCount; uniformIndexInProgram >= 0 &&
uniformIndexInProgram < static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
} }
GLenum GetUniformType(Uint location) const { GLenum GetUniformType(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]); auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.glDefineType; return uniform.glDefineType;
} }
GLenum GetActiveUniformType(Uint index) const { GLenum GetActiveUniformType(Uint index) const {
auto& uniform = m_program->getUniform(static_cast<Int>(index)); auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
return uniform.glDefineType; return uniform.glDefineType;
} }
// Number of active array elements (GL_UNIFORM_SIZE / GL_ARRAY_SIZE); 1 for a non-array. // Number of active array elements (GL_UNIFORM_SIZE / GL_ARRAY_SIZE); 1 for a non-array.
// glslang's TObjectReflection.size only carries the element count for a NON-block array; for // glslang's TObjectReflection.size only carries the element count for a NON-block array; for
// a block array member it reports 1, so take the count from the TType, which is authoritative // a block array member it reports 1, so take the count from the TType, which is authoritative
// for both. GL 3.3 core uniforms are always sized. // for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
GLint GetActiveUniformArraySize(Uint index) const { // the artifacts' uniformIndexInTProgram stores).
const auto& uniform = m_program->getUniform(static_cast<Int>(index)); GLint GetUniformArraySizeByTIndex(Int tIndex) const {
const auto& uniform = Artifacts().program->getUniform(tIndex);
const glslang::TType* type = uniform.getType(); const glslang::TType* type = uniform.getType();
if (type != nullptr && type->isSizedArray()) { if (type != nullptr && type->isSizedArray()) {
return type->getOuterArraySize(); return type->getOuterArraySize();
@@ -162,15 +184,23 @@ namespace MobileGL::MG_State::GLState {
return uniform.size < 1 ? 1 : uniform.size; return uniform.size < 1 ? 1 : uniform.size;
} }
Int GetActiveUniformBlockIndex(Uint index) const { GLint GetActiveUniformArraySize(Uint index) const {
auto& uniform = m_program->getUniform(static_cast<Int>(index)); return GetUniformArraySizeByTIndex(TProgramUniformIndex(index));
return uniform.index;
} }
// GL_UNIFORM_OFFSET: byte offset within the owning named block. glslang already reports -1 Int GetActiveUniformBlockIndex(Uint index) const {
// for a default-block uniform, which is exactly the spec value there. auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
// Members of the synthesized global UBO are default-block uniforms to GL: -1.
return GlBlockIndexFromTProgram(uniform.index);
}
// GL_UNIFORM_OFFSET: byte offset within the owning named block; -1 for a default-block
// uniform. The relaxed parse gives global-UBO members real byte offsets, but GL must keep
// seeing them as default-block uniforms, so gate on the GL-visible block index.
GLint GetActiveUniformOffset(Uint index) const { GLint GetActiveUniformOffset(Uint index) const {
return m_program->getUniform(static_cast<Int>(index)).offset; const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
return uniform.offset;
} }
// GL_UNIFORM_ARRAY_STRIDE: byte stride of an array member in a named block; 0 for a non-array // GL_UNIFORM_ARRAY_STRIDE: byte stride of an array member in a named block; 0 for a non-array
@@ -182,8 +212,8 @@ namespace MobileGL::MG_State::GLState {
// generated SPIR-V lay the array out with std140 16-byte-rounded strides. MobileGL's UBO // generated SPIR-V lay the array out with std140 16-byte-rounded strides. MobileGL's UBO
// layout is always std140, where every array element stride rounds up to a vec4. // layout is always std140, where every array element stride rounds up to a vec4.
GLint GetActiveUniformArrayStride(Uint index) const { GLint GetActiveUniformArrayStride(Uint index) const {
const auto& uniform = m_program->getUniform(static_cast<Int>(index)); const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (uniform.index < 0) return -1; if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
const glslang::TType* type = uniform.getType(); const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isArray()) return 0; if (type == nullptr || !type->isArray()) return 0;
if (type->isMatrix()) { if (type->isMatrix()) {
@@ -202,13 +232,13 @@ namespace MobileGL::MG_State::GLState {
// check suffices; the getUniformBlock() fallback is defensive for a config that instead leaves // check suffices; the getUniformBlock() fallback is defensive for a config that instead leaves
// an inheriting member's layoutMatrix == ElmNone. // an inheriting member's layoutMatrix == ElmNone.
GLint GetActiveUniformIsRowMajor(Uint index) const { GLint GetActiveUniformIsRowMajor(Uint index) const {
const auto& uniform = m_program->getUniform(static_cast<Int>(index)); const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (uniform.index < 0) return 0; if (GlBlockIndexFromTProgram(uniform.index) < 0) return 0;
const glslang::TType* type = uniform.getType(); const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isMatrix()) return 0; if (type == nullptr || !type->isMatrix()) return 0;
glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix; glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix;
if (layoutMatrix == glslang::ElmNone) { if (layoutMatrix == glslang::ElmNone) {
layoutMatrix = m_program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix; layoutMatrix = Artifacts().program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
} }
return (layoutMatrix == glslang::ElmRowMajor) ? 1 : 0; return (layoutMatrix == glslang::ElmRowMajor) ? 1 : 0;
} }
@@ -220,13 +250,13 @@ namespace MobileGL::MG_State::GLState {
// out as std140 (packed/shared are coerced), so this matches the offsets glslang reports. For // out as std140 (packed/shared are coerced), so this matches the offsets glslang reports. For
// every GL 3.3 float matrix this evaluates to 16, independent of majorness. // every GL 3.3 float matrix this evaluates to 16, independent of majorness.
GLint GetActiveUniformMatrixStride(Uint index) const { GLint GetActiveUniformMatrixStride(Uint index) const {
const auto& uniform = m_program->getUniform(static_cast<Int>(index)); const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (uniform.index < 0) return -1; if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
const glslang::TType* type = uniform.getType(); const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isMatrix()) return 0; if (type == nullptr || !type->isMatrix()) return 0;
glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix; glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix;
if (layoutMatrix == glslang::ElmNone) { if (layoutMatrix == glslang::ElmNone) {
layoutMatrix = m_program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix; layoutMatrix = Artifacts().program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
} }
const bool rowMajor = (layoutMatrix == glslang::ElmRowMajor); const bool rowMajor = (layoutMatrix == glslang::ElmRowMajor);
const int strideVectorComponents = rowMajor ? type->getMatrixCols() : type->getMatrixRows(); const int strideVectorComponents = rowMajor ? type->getMatrixCols() : type->getMatrixRows();
@@ -238,50 +268,50 @@ namespace MobileGL::MG_State::GLState {
} }
const glslang::TType* GetUniformTType(Uint location) const { const glslang::TType* GetUniformTType(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]); auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.getType(); return uniform.getType();
} }
Bool IsUniformOpaqueAtLocation(Uint location) const { return GetUniformTType(location)->isOpaque(); } Bool IsUniformOpaqueAtLocation(Uint location) const { return GetUniformTType(location)->isOpaque(); }
const String& GetUniformName(Uint location) const { const String& GetUniformName(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]); auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.name; return uniform.name;
} }
const String& GetActiveUniformName(Uint index) const { const String& GetActiveUniformName(Uint index) const {
auto& uniform = m_program->getUniform(static_cast<Int>(index)); auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
return uniform.name; return uniform.name;
} }
// Sentinel for a uniform location without global-UBO backing storage (should not // Sentinel for a uniform location without global-UBO backing storage (should not
// survive linking: GenerateBinary falls back to tail-allocated scratch storage). // survive linking: GenerateBinary falls back to tail-allocated scratch storage).
static constexpr Uint kInvalidUniformOffset = ~0u; static constexpr Uint kInvalidUniformOffset = ~0u;
Uint GetUniformOffset(Uint location) const { return m_uniformOffsets[location]; } Uint GetUniformOffset(Uint location) const { return Artifacts().uniformOffsets[location]; }
Uint GetUniformSizesInBytes(Uint location) const { return MG_Util::GetGLTypeSize(GetUniformType(location)); } Uint GetUniformSizesInBytes(Uint location) const { return MG_Util::GetGLTypeSize(GetUniformType(location)); }
Int GetAttributeLocation(const String& name) { Int GetAttributeLocation(const String& name) {
const auto it = std::find(m_attribs.begin(), m_attribs.end(), name); const auto it = std::find(Artifacts().attribs.begin(), Artifacts().attribs.end(), name);
return (it == m_attribs.end()) ? -1 : (Int)std::distance(m_attribs.begin(), it); return (it == Artifacts().attribs.end()) ? -1 : (Int)std::distance(Artifacts().attribs.begin(), it);
} }
Uint32 GetActiveAttributeLocationMask() const { Uint32 GetActiveAttributeLocationMask() const {
Uint32 mask = 0; Uint32 mask = 0;
const SizeT count = std::min<SizeT>(m_attribs.size(), 32); const SizeT count = std::min<SizeT>(Artifacts().attribs.size(), 32);
for (SizeT index = 0; index < count; ++index) { for (SizeT index = 0; index < count; ++index) {
if (!m_attribs[index].empty()) { if (!Artifacts().attribs[index].empty()) {
mask |= (1u << index); mask |= (1u << index);
} }
} }
return mask; return mask;
} }
Uint32 GetActiveFragmentOutputLocationMask() const { Uint32 GetActiveFragmentOutputLocationMask() const {
if (!m_program) { if (!Artifacts().program) {
return 0; return 0;
} }
Uint32 mask = 0; Uint32 mask = 0;
const Int outputCount = m_program->getNumPipeOutputs(); const Int outputCount = Artifacts().program->getNumPipeOutputs();
for (Int index = 0; index < outputCount; ++index) { for (Int index = 0; index < outputCount; ++index) {
const Int location = static_cast<Int>(m_program->getPipeOutput(index).layoutLocation()); const Int location = static_cast<Int>(Artifacts().program->getPipeOutput(index).layoutLocation());
if (location >= 0 && location < 32) { if (location >= 0 && location < 32) {
mask |= (1u << location); mask |= (1u << location);
} }
@@ -289,42 +319,54 @@ namespace MobileGL::MG_State::GLState {
return mask; return mask;
} }
Int GetActiveFragmentOutputCount() const { Int GetActiveFragmentOutputCount() const {
return m_program ? m_program->getNumPipeOutputs() : 0; return Artifacts().program ? Artifacts().program->getNumPipeOutputs() : 0;
} }
const String& GetActiveFragmentOutputName(Uint index) const { const String& GetActiveFragmentOutputName(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetActiveFragmentOutputName: program is null"); MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetActiveFragmentOutputName: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()), MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetActiveFragmentOutputName: index=%u out of range", index); "ProgramObject::GetActiveFragmentOutputName: index=%u out of range", index);
return m_program->getPipeOutput(static_cast<Int>(index)).name; return Artifacts().program->getPipeOutput(static_cast<Int>(index)).name;
} }
Int GetFragmentOutputLocation(Uint index) const { Int GetFragmentOutputLocation(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetFragmentOutputLocation: program is null"); MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetFragmentOutputLocation: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()), MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetFragmentOutputLocation: index=%u out of range", "ProgramObject::GetFragmentOutputLocation: index=%u out of range",
index); index);
return static_cast<Int>(m_program->getPipeOutput(static_cast<Int>(index)).layoutLocation()); return static_cast<Int>(Artifacts().program->getPipeOutput(static_cast<Int>(index)).layoutLocation());
} }
GLint GetActiveFragmentOutputArraySize(Uint index) const { GLint GetActiveFragmentOutputArraySize(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetActiveFragmentOutputArraySize: program is null"); MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetActiveFragmentOutputArraySize: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()), MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetActiveFragmentOutputArraySize: index=%u out of range", index); "ProgramObject::GetActiveFragmentOutputArraySize: index=%u out of range", index);
return m_program->getPipeOutput(static_cast<Int>(index)).size; return Artifacts().program->getPipeOutput(static_cast<Int>(index)).size;
} }
GLenum GetFragmentOutputType(Uint index) const { GLenum GetFragmentOutputType(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetFragmentOutputType: program is null"); MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetFragmentOutputType: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()), MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetFragmentOutputType: index=%u out of range", "ProgramObject::GetFragmentOutputType: index=%u out of range",
index); index);
return m_program->getPipeOutput(static_cast<Int>(index)).glDefineType; return Artifacts().program->getPipeOutput(static_cast<Int>(index)).glDefineType;
} }
GLenum GetAttribType(Uint index) const { return m_attribTypes[index]; } GLenum GetAttribType(Uint index) const { return Artifacts().attribTypes[index]; }
const String& GetAttribName(Uint index) const { return m_attribs[index]; } const String& GetAttribName(Uint index) const { return Artifacts().attribs[index]; }
GLenum GetActiveAttribType(Uint index) const { return m_program->getPipeInput(static_cast<Int>(index)).glDefineType; } GLenum GetActiveAttribType(Uint index) const { return Artifacts().program->getPipeInput(static_cast<Int>(index)).glDefineType; }
GLint GetActiveAttribArraySize(Uint index) const { return m_program->getPipeInput(static_cast<Int>(index)).size; } GLint GetActiveAttribArraySize(Uint index) const { return Artifacts().program->getPipeInput(static_cast<Int>(index)).size; }
const String& GetActiveAttribName(Uint index) const { return m_program->getPipeInput(static_cast<Int>(index)).name; } // The Vulkan-semantics parse reflects the vertex builtins under their SPIR-V names;
void* MapUBO() { return m_globalUboScratch.data(); } // GL must keep reporting the GL spellings (glGetActiveAttrib and the program-input
const void* GetUBOData() const { return m_globalUboScratch.data(); } // resource queries enumerate builtins).
Uint GetUBOSize() const { return static_cast<Uint>(m_globalUboScratch.size()); } static const String& NormalizeBuiltinPipeInputName(const String& name) {
static const String kGlVertexId = "gl_VertexID";
static const String kGlInstanceId = "gl_InstanceID";
if (name == "gl_VertexIndex") return kGlVertexId;
if (name == "gl_InstanceIndex") return kGlInstanceId;
return name;
}
const String& GetActiveAttribName(Uint index) const {
return NormalizeBuiltinPipeInputName(Artifacts().program->getPipeInput(static_cast<Int>(index)).name);
}
void* MapUBO() { return Artifacts().globalUboScratch.data(); }
const void* GetUBOData() const { return Artifacts().globalUboScratch.data(); }
Uint GetUBOSize() const { return static_cast<Uint>(Artifacts().globalUboScratch.size()); }
// Content version of the CPU-side global-UBO shadow: writers bump it so backends // Content version of the CPU-side global-UBO shadow: writers bump it so backends
// can skip re-uploading an unchanged UBO on every draw. ~0u is reserved as the // can skip re-uploading an unchanged UBO on every draw. ~0u is reserved as the
// backends' "never uploaded" sentinel, so skip over it on wrap. // backends' "never uploaded" sentinel, so skip over it on wrap.
@@ -370,20 +412,20 @@ namespace MobileGL::MG_State::GLState {
} }
void SetUniformSamplerOrImageUnitIndex(Uint location, Int unit) { void SetUniformSamplerOrImageUnitIndex(Uint location, Int unit) {
if (location >= m_uniformSamplerOrImageUnitIndex.size() || if (location >= Artifacts().uniformSamplerOrImageUnitIndex.size() ||
m_uniformSamplerOrImageUnitIndex[location] == unit) { Artifacts().uniformSamplerOrImageUnitIndex[location] == unit) {
return; return;
} }
m_uniformSamplerOrImageUnitIndex[location] = unit; Artifacts().uniformSamplerOrImageUnitIndex[location] = unit;
++m_backendStateVersion; ++m_backendStateVersion;
} }
Int GetUniformSamplerOrImageUnitIndex(Uint location) const { Int GetUniformSamplerOrImageUnitIndex(Uint location) const {
return m_uniformSamplerOrImageUnitIndex[location]; return Artifacts().uniformSamplerOrImageUnitIndex[location];
} }
Bool GetDeleteStatus() const { return m_deleteStatus; } Bool GetDeleteStatus() const { return m_deleteStatus; }
Bool GetLinkStatus() const { return m_linkStatus; } Bool GetLinkStatus() const { return Artifacts().linkStatus; }
// GL_PROGRAM_BINARY_RETRIEVABLE_HINT. MobileGL exposes no program binary format // GL_PROGRAM_BINARY_RETRIEVABLE_HINT. MobileGL exposes no program binary format
// (GL_NUM_PROGRAM_BINARY_FORMATS is 0), so the hint is pure state - which is all // (GL_NUM_PROGRAM_BINARY_FORMATS is 0), so the hint is pure state - which is all
// ARB_get_program_binary requires of it. // ARB_get_program_binary requires of it.
@@ -397,24 +439,27 @@ namespace MobileGL::MG_State::GLState {
// glProgramBinary always fails here (there is no format it could accept) and the // glProgramBinary always fails here (there is no format it could accept) and the
// spec then requires the program's LINK_STATUS to read FALSE. // spec then requires the program's LINK_STATUS to read FALSE.
void MarkLinkFailedByProgramBinary() { void MarkLinkFailedByProgramBinary() {
BumpLinkObservableVersions();
ResetLinkArtifacts(); ResetLinkArtifacts();
m_infoLog = "No program binary format is supported."; Artifacts().infoLog = "No program binary format is supported.";
} }
Bool GetValidateStatus() const { return m_validateStatus; } Bool GetValidateStatus() const { return m_validateStatus; }
Int GetActiveAtomicCounterCount() const { return m_program->getNumAtomicCounters(); } Int GetActiveAtomicCounterCount() const { return Artifacts().program->getNumAtomicCounters(); }
Int GetActiveAttributesCount() const { return m_program->getNumPipeInputs(); } Int GetActiveAttributesCount() const { return Artifacts().program->getNumPipeInputs(); }
Int GetActiveUniformBlocksCount() const { return m_program->getNumUniformBlocks(); } // GL-visible uniform blocks only: the synthesized MGL_GLOBAL_UBO the relaxed parse
GLuint GetComputeLocalSize(Uint dim) const { return m_program->getLocalSize(static_cast<Int>(dim)); } // materializes for default-block uniforms is filtered out by DoReflection.
Int GetActiveAttributesMaxLength() const { return m_attribInNameMaxLength; } Int GetActiveUniformBlocksCount() const { return static_cast<Int>(Artifacts().glBlockIndexToTProgram.size()); }
Int GetActiveUniformBlocksMaxNameLength() const { return m_uniformBlockNameMaxLength; } GLuint GetComputeLocalSize(Uint dim) const { return Artifacts().program->getLocalSize(static_cast<Int>(dim)); }
Int GetActiveAttributesMaxLength() const { return Artifacts().attribInNameMaxLength; }
Int GetActiveUniformBlocksMaxNameLength() const { return Artifacts().uniformBlockNameMaxLength; }
Uint GetUniformBlockIndex(const char* name) const { Uint GetUniformBlockIndex(const char* name) const {
auto it = m_uniformBlockIndexByName.find(name); auto it = Artifacts().uniformBlockIndexByName.find(name);
if (it != m_uniformBlockIndexByName.end()) return it->second; if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
// Instances of an arrayed block are reflected as "Block[0]".."Block[N-1]"; // Instances of an arrayed block are reflected as "Block[0]".."Block[N-1]";
// a bare "Block" query resolves to the first instance per GL semantics. // a bare "Block" query resolves to the first instance per GL semantics.
const String suffixedName = String(name) + "[0]"; const String suffixedName = String(name) + "[0]";
it = m_uniformBlockIndexByName.find(suffixedName); it = Artifacts().uniformBlockIndexByName.find(suffixedName);
if (it != m_uniformBlockIndexByName.end()) return it->second; if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
return 0xFFFFFFFFu; // GL_INVALID_INDEX return 0xFFFFFFFFu; // GL_INVALID_INDEX
} }
Bool IsActiveUniformBlock(Uint index) const { Bool IsActiveUniformBlock(Uint index) const {
@@ -427,11 +472,11 @@ namespace MobileGL::MG_State::GLState {
// (like a std140 struct) occupies a vec4-rounded size, and that is what the // (like a std140 struct) occupies a vec4-rounded size, and that is what the
// backend compiles: ES drivers reject draws whose bound UBO range is smaller // backend compiles: ES drivers reject draws whose bound UBO range is smaller
// than the block (a block ending in ivec3 reported 12 while the driver needs 16). // than the block (a block ending in ivec3 reported 12 while the driver needs 16).
return (m_program->getUniformBlock((Int)index).size + 15u) & ~15u; return (Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]).size + 15u) & ~15u;
} }
const String& GetUniformBlockName(Uint index) const { const String& GetUniformBlockName(Uint index) const {
auto& ubo = m_program->getUniformBlock((Int)index); auto& ubo = Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]);
return ubo.name; return ubo.name;
} }
@@ -443,8 +488,8 @@ namespace MobileGL::MG_State::GLState {
if (name.empty() || name.back() != ']') return index; if (name.empty() || name.back() != ']') return index;
const SizeT bracket = name.rfind('['); const SizeT bracket = name.rfind('[');
if (bracket == String::npos) return index; if (bracket == String::npos) return index;
const auto it = m_uniformBlockIndexByName.find(name.substr(0, bracket) + "[0]"); const auto it = Artifacts().uniformBlockIndexByName.find(name.substr(0, bracket) + "[0]");
if (it != m_uniformBlockIndexByName.end()) return it->second; if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
return index; return index;
} }
@@ -455,31 +500,31 @@ namespace MobileGL::MG_State::GLState {
Int GetUniformBlockActiveUniformCount(Uint index) const { Int GetUniformBlockActiveUniformCount(Uint index) const {
const Int ownerIndex = static_cast<Int>(GetUniformBlockMemberOwnerIndex(index)); const Int ownerIndex = static_cast<Int>(GetUniformBlockMemberOwnerIndex(index));
Int count = 0; Int count = 0;
for (Uint uniformIndex = 0; uniformIndex < m_activeUniformCount; ++uniformIndex) { for (Uint uniformIndex = 0; uniformIndex < Artifacts().activeUniformCount; ++uniformIndex) {
if (GetActiveUniformBlockIndex(uniformIndex) == ownerIndex) ++count; if (GetActiveUniformBlockIndex(uniformIndex) == ownerIndex) ++count;
} }
return count; return count;
} }
Bool IsUniformBlockReferencedByStage(Uint index, EShLanguage stage) const { Bool IsUniformBlockReferencedByStage(Uint index, EShLanguage stage) const {
const auto& ubo = m_program->getUniformBlock((Int)index); const auto& ubo = Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]);
const auto stageMask = static_cast<EShLanguageMask>(1 << stage); const auto stageMask = static_cast<EShLanguageMask>(1 << stage);
return (ubo.stages & stageMask) != 0; return (ubo.stages & stageMask) != 0;
} }
// Set by glUniformBlockBinding // Set by glUniformBlockBinding
void SetUniformBlockBinding(Uint index, Uint binding) { void SetUniformBlockBinding(Uint index, Uint binding) {
if (index >= m_uniformBlockBinding.size() || m_uniformBlockBinding[index] == static_cast<Int>(binding)) { if (index >= Artifacts().uniformBlockBinding.size() || Artifacts().uniformBlockBinding[index] == static_cast<Int>(binding)) {
return; return;
} }
m_uniformBlockBinding[index] = static_cast<Int>(binding); Artifacts().uniformBlockBinding[index] = static_cast<Int>(binding);
++m_backendStateVersion; ++m_backendStateVersion;
} }
Uint GetUniformBlockBinding(Uint index) const { return m_uniformBlockBinding[index]; } Uint GetUniformBlockBinding(Uint index) const { return Artifacts().uniformBlockBinding[index]; }
Vector<Vector<unsigned>>& GetGeneratedSpirv() { return m_generatedSpirv; } Vector<Vector<unsigned>>& GetGeneratedSpirv() { return Artifacts().generatedSpirv; }
const Vector<Vector<unsigned>>& GetGeneratedSpirv() const { return m_generatedSpirv; } const Vector<Vector<unsigned>>& GetGeneratedSpirv() const { return Artifacts().generatedSpirv; }
Int GetShaderIndexByStage(ShaderStage stage) const { Int GetShaderIndexByStage(ShaderStage stage) const {
auto it = std::find_if(m_shaders.begin(), m_shaders.end(), [stage](const SharedPtr<ShaderObject>& shader) { auto it = std::find_if(m_shaders.begin(), m_shaders.end(), [stage](const SharedPtr<ShaderObject>& shader) {
@@ -501,95 +546,54 @@ namespace MobileGL::MG_State::GLState {
// layout captures into; see NeedsScatteredTransformFeedbackCapture. // layout captures into; see NeedsScatteredTransformFeedbackCapture.
Uint32 packedOffsetBytes = 0; Uint32 packedOffsetBytes = 0;
}; };
void SetTransformFeedbackVaryings(Vector<String>&& names, GLenum bufferMode) {
m_requestedXfbVaryings = Move(names);
m_requestedXfbBufferMode = bufferMode;
}
GLenum GetTransformFeedbackBufferMode() const { return m_xfbBufferMode; }
SizeT GetTransformFeedbackVaryingCount() const { return m_xfbVaryings.size(); }
const XfbVarying* GetTransformFeedbackVarying(SizeT index) const {
return index < m_xfbVaryings.size() ? &m_xfbVaryings[index] : nullptr;
}
const Vector<XfbVarying>& GetTransformFeedbackVaryings() const { return m_xfbVaryings; }
// Stride of one captured vertex in the given capture buffer slot.
Uint32 GetTransformFeedbackStride(Uint32 bufferIndex) const {
return bufferIndex < m_xfbStrides.size() ? m_xfbStrides[bufferIndex] : 0;
}
SizeT GetTransformFeedbackBufferCount() const { return m_xfbStrides.size(); }
Int GetTransformFeedbackVaryingMaxLength() const { return m_xfbVaryingNameMaxLength; }
// True when the capture layout uses gl_SkipComponents / gl_NextBuffer
// (ARB_transform_feedback3), which no ES driver can express: it can only pack every
// captured varying into one record with no gaps. A backend that captures through
// such a driver has to capture into scratch storage and scatter the records into the
// application's buffers itself, using packedOffsetBytes as the source offset and
// (bufferIndex, offsetBytes, stride) as the destination.
Bool NeedsScatteredTransformFeedbackCapture() const { return m_xfbNeedsScatteredCapture; }
// Bytes one gap-free captured record occupies.
Uint32 GetTransformFeedbackPackedStride() const { return m_xfbPackedStride; }
// True when the capture stage is a triangle-strip geometry shader with a
// statically-known emit sequence: the Vulkan capture order then needs the GL
// odd-triangle vertex swap after EndTransformFeedback.
Bool HasGsTriangleStripCaptureFixup() const { return m_gsStripCaptureFixup; }
// Triangles per strip, in emission order, for ONE geometry invocation.
const Vector<Uint32>& GetGsStripTriangles() const { return m_gsStripTriangles; }
// GL_GEOMETRY_INPUT_TYPE of the linked geometry stage (GL_POINTS, GL_LINES,
// GL_LINES_ADJACENCY, GL_TRIANGLES or GL_TRIANGLES_ADJACENCY), or GL_NONE when the
// program has no geometry stage. Draws must present a compatible primitive type.
GLenum GetGeometryInputType() const { return m_gsInputPrimitive; }
Uint GetExternalIndex() const { return m_externalIndex; } // ---- P1: everything a link PRODUCES, in one movable block ----
// Globally-unique, never-reused id for this program object's lifetime. Unlike the GL //
// name (external index), which is freed to a LIFO list and immediately handed back by // The membership rule is mechanical, not editorial: this is exactly the field list
// the next glCreateProgram, this distinguishes a deleted-and-recreated program from the // ResetLinkArtifacts() clears (plus the four it forgot to - infoLog,
// original, so an identity cache can't false-hit on name recycling. // linkedFragDataLocation/Index and the geometry strip-capture pair - which are just
Uint64 GetLifetimeId() const { return m_lifetimeId; } // as much link output). Nothing else belongs here.
//
private: // Why a struct: once glLinkProgram runs on a worker (P1 stage 4) the worker writes
void ResetLinkArtifacts(); // its OWN LinkArtifacts and the GL thread publishes it with a single move, instead
void DoReflection(); // of thirty cross-thread field assignments. Until then this is a pure refactor.
// Resolves the requested transform feedback varyings against the linked //
// vertex stage; fails the link (GL semantics) on unknown or duplicate // Access rule (invariant I5): the member below is private and reachable ONLY
// names or exceeded capture limits. // through ProgramObject::Artifacts(), which calls EnsureLinkJoined() first. That is
Bool ResolveTransformFeedbackVaryings(); // what makes "every read of link output joins the pending link" a property the
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate); // compiler checks rather than a review item - a new reader cannot spell the field
void GenerateBinary(); // without going through the gate.
void WaitUntilGenerationCompleted() const; struct LinkArtifacts {
void AddDefaultFragmentShaderIfMissing(); SharedPtr<glslang::TProgram> program;
Bool ValidateFragmentOutputLocations(); Vector<Vector<unsigned>> generatedSpirv;
static Uint64 AllocateLifetimeId();
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = 0;
Vector<SharedPtr<ShaderObject>> m_shaders;
Vector<SharedPtr<ShaderObject>> m_detachedShaders; // Store detached shaders and remove on next link
SharedPtr<glslang::TProgram> m_program;
Vector<Vector<unsigned>> m_generatedSpirv;
// Attributes (Vertex in) // Attributes (Vertex in)
UnorderedMap<String, Uint> m_explicitAttribLocations; Vector<String> attribs;
Vector<String> m_attribs; Vector<GLenum> attribTypes;
Vector<GLenum> m_attribTypes;
// FragData (Frag out) // FragData (Frag out): the per-link snapshot of the explicit request maps.
UnorderedMap<String, Uint> m_explicitFragDataLocation; UnorderedMap<String, Uint> linkedFragDataLocation;
UnorderedMap<String, Uint> m_linkedFragDataLocation; UnorderedMap<String, Uint> linkedFragDataIndex;
// Dual-source blend color index per output name (glBindFragDataLocationIndexed); snapshotted
// into the linked map at link time, like the location maps above.
UnorderedMap<String, Uint> m_explicitFragDataIndex;
UnorderedMap<String, Uint> m_linkedFragDataIndex;
Int m_maxFragmentOutputColorNumber = 8;
// Uniforms // GL-facing index spaces (see the translation helpers above): GL active-uniform
UnorderedMap<String, Uint> m_uniformLocations; // index <-> glslang TProgram uniform index, GL uniform-block index <-> TProgram
// block index. -1 marks a TProgram entry GL does not expose (dead default-block
// uniforms swept into MGL_GLOBAL_UBO by the relaxed parse, and that block itself).
Vector<Int> glUniformIndexToTProgram;
Vector<Int> tProgramUniformIndexToGl;
Vector<Int> glBlockIndexToTProgram;
Vector<Int> tProgramBlockIndexToGl;
// Per-link merged snapshot of the attached shaders' lexically extracted
// layout(location = N) default-block uniform qualifiers (the relaxed parse drops
// them from reflection; the DoReflection assigner restores them from here).
UnorderedMap<String, Int> linkedExplicitUniformLocations;
UnorderedMap<String, Uint> uniformLocations;
// Ordered by location, // Ordered by location,
// aka. m_uniformIndexInTProgram[loc] == "uniform index of TProgram at location `loc`" // aka. uniformIndexInTProgram[loc] == "uniform index of TProgram at location `loc`"
Vector<Int> m_uniformIndexInTProgram; Vector<Int> uniformIndexInTProgram;
// ditto. Will be set at glUniform1i // ditto. Will be set at glUniform1i
Vector<Int> m_uniformSamplerOrImageUnitIndex; Vector<Int> uniformSamplerOrImageUnitIndex;
UnorderedMap<String, Uint> m_explicitOpaqueUniformBindings; UnorderedMap<String, Uint> explicitOpaqueUniformBindings;
// Ordered by uniform block index // Ordered by uniform block index
// index is DIFFERENT from binding!!! // index is DIFFERENT from binding!!!
@@ -599,23 +603,162 @@ namespace MobileGL::MG_State::GLState {
// `prog->getUniformBlock(i) == "BlockName"` // `prog->getUniformBlock(i) == "BlockName"`
// These stuff are present for GL semantics, not for backend inspection // These stuff are present for GL semantics, not for backend inspection
// These may change after-link (because GL spec decided to have `glUniformBlockBinding`) // These may change after-link (because GL spec decided to have `glUniformBlockBinding`)
UnorderedMap<String, Uint> m_uniformBlockIndexByName; UnorderedMap<String, Uint> uniformBlockIndexByName;
Vector<Int> m_uniformBlockBinding; Vector<Int> uniformBlockBinding;
// Need to be reflected after linking of SPIR-V binary // Need to be reflected after linking of SPIR-V binary
Vector<Uint> m_uniformOffsets; Vector<Uint> uniformOffsets;
Vector<Uint> m_uniformSizesInBytes; Vector<Uint> uniformSizesInBytes;
Vector<Uint8> m_globalUboScratch; Vector<Uint8> globalUboScratch;
Uint m_activeUniformCount = 0; Uint activeUniformCount = 0;
Uint m_maxUniformLocation = 0; Uint maxUniformLocation = 0;
Int m_uniformNameMaxLength = 0; Int uniformNameMaxLength = 0;
Int m_attribInNameMaxLength = 0; Int attribInNameMaxLength = 0;
Int m_uniformBlockNameMaxLength = 0; Int uniformBlockNameMaxLength = 0;
String infoLog;
Bool linkStatus = false;
// Transform feedback: the linked snapshot (the request lives outside, on the
// GL-thread-owned side).
Vector<XfbVarying> xfbVaryings;
Vector<Uint32> xfbStrides;
Vector<Uint32> gsStripTriangles;
Bool gsStripCaptureFixup = false;
GLenum gsInputPrimitive = GL_NONE;
GLenum xfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Int xfbVaryingNameMaxLength = 0;
Bool xfbNeedsScatteredCapture = false;
Uint32 xfbPackedStride = 0;
};
// Blocks until a pending link (P1 stage 4 onwards) has published its artifacts.
// Public because a few call sites have to join without reading anything - see the
// explicit-join list in the P1 design. Today there is never a pending link, so this
// is a no-op; it is wired up when glLinkProgram starts enqueueing.
void JoinLink() const { EnsureLinkJoined(); }
void SetTransformFeedbackVaryings(Vector<String>&& names, GLenum bufferMode) {
m_requestedXfbVaryings = Move(names);
m_requestedXfbBufferMode = bufferMode;
}
GLenum GetTransformFeedbackBufferMode() const { return Artifacts().xfbBufferMode; }
SizeT GetTransformFeedbackVaryingCount() const { return Artifacts().xfbVaryings.size(); }
const XfbVarying* GetTransformFeedbackVarying(SizeT index) const {
return index < Artifacts().xfbVaryings.size() ? &Artifacts().xfbVaryings[index] : nullptr;
}
const Vector<XfbVarying>& GetTransformFeedbackVaryings() const { return Artifacts().xfbVaryings; }
// Stride of one captured vertex in the given capture buffer slot.
Uint32 GetTransformFeedbackStride(Uint32 bufferIndex) const {
return bufferIndex < Artifacts().xfbStrides.size() ? Artifacts().xfbStrides[bufferIndex] : 0;
}
SizeT GetTransformFeedbackBufferCount() const { return Artifacts().xfbStrides.size(); }
Int GetTransformFeedbackVaryingMaxLength() const { return Artifacts().xfbVaryingNameMaxLength; }
// True when the capture layout uses gl_SkipComponents / gl_NextBuffer
// (ARB_transform_feedback3), which no ES driver can express: it can only pack every
// captured varying into one record with no gaps. A backend that captures through
// such a driver has to capture into scratch storage and scatter the records into the
// application's buffers itself, using packedOffsetBytes as the source offset and
// (bufferIndex, offsetBytes, stride) as the destination.
Bool NeedsScatteredTransformFeedbackCapture() const { return Artifacts().xfbNeedsScatteredCapture; }
// Bytes one gap-free captured record occupies.
Uint32 GetTransformFeedbackPackedStride() const { return Artifacts().xfbPackedStride; }
// True when the capture stage is a triangle-strip geometry shader with a
// statically-known emit sequence: the Vulkan capture order then needs the GL
// odd-triangle vertex swap after EndTransformFeedback.
Bool HasGsTriangleStripCaptureFixup() const { return Artifacts().gsStripCaptureFixup; }
// Triangles per strip, in emission order, for ONE geometry invocation.
const Vector<Uint32>& GetGsStripTriangles() const { return Artifacts().gsStripTriangles; }
// GL_GEOMETRY_INPUT_TYPE of the linked geometry stage (GL_POINTS, GL_LINES,
// GL_LINES_ADJACENCY, GL_TRIANGLES or GL_TRIANGLES_ADJACENCY), or GL_NONE when the
// program has no geometry stage. Draws must present a compatible primitive type.
GLenum GetGeometryInputType() const { return Artifacts().gsInputPrimitive; }
Uint GetExternalIndex() const { return m_externalIndex; }
// Globally-unique, never-reused id for this program object's lifetime. Unlike the GL
// name (external index), which is freed to a LIFO list and immediately handed back by
// the next glCreateProgram, this distinguishes a deleted-and-recreated program from the
// original, so an identity cache can't false-hit on name recycling.
Uint64 GetLifetimeId() const { return m_lifetimeId; }
private:
// ---- The one and only join gate for link output (P1 invariant I5) ----
// Blocks until a pending link has finished and its LinkArtifacts have been
// published into m_artifacts. Today no link is ever pending - glLinkProgram still
// runs the whole body inline - so this is an unconditional no-op, and the whole
// Artifacts() indirection compiles away. It exists NOW so that the ~120 readers of
// link output are already routed through it when stage 4 makes it block: the edit
// that turns links asynchronous then touches this function and nothing else.
//
// Defined inline (not in ProgramObject.cpp) on purpose: this is called from every
// Artifacts() read - ~1200 call sites project-wide - and the project never builds
// with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line empty body would leave a
// real cross-TU call at every one of them instead of folding away. Stage 4's
// version, which actually blocks, moves the wait itself out-of-line behind a
// `m_pendingLink` check that stays inline here.
void EnsureLinkJoined() const {}
LinkArtifacts& Artifacts() {
EnsureLinkJoined();
return m_artifacts;
}
const LinkArtifacts& Artifacts() const {
EnsureLinkJoined();
return m_artifacts;
}
void ResetLinkArtifacts();
// GL-thread-only companion to ResetLinkArtifacts (see its definition).
void BumpLinkObservableVersions();
// Builds the GL-facing reflection surface from the linked TProgram. Returns false
// (with the artifacts' infoLog set and link artifacts reset) when reflection itself fails or an
// explicit-uniform-location conflict makes the link invalid.
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
// Resolves the requested transform feedback varyings against the linked
// vertex stage; fails the link (GL semantics) on unknown or duplicate
// names or exceeded capture limits.
Bool ResolveTransformFeedbackVaryings();
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
// The former GenerateBinary, split around DoReflection's data dependencies:
// SPIR-V must be generated BEFORE buildReflection touches the linked TProgram (its
// live-variable analysis mutates the intermediates enough to change
// GlslangToSpv output), while the glUniform*-to-global-UBO routing tables are
// sized and keyed by reflection results (maxUniformLocation, uniformLocations)
// and so must run AFTER it.
void GenerateSpirv();
void BuildGlobalUboRouting();
void AddDefaultFragmentShaderIfMissing();
Bool ValidateFragmentOutputLocations();
static Uint64 AllocateLifetimeId();
// ---- GL-thread-owned state: never joins ----
// Most of this is never produced by a link at all. The three version counters
// (m_backendStateVersion / m_uboContentVersion / m_linkVersion) ARE
// link-observable, but they are bumped exclusively on the GL thread
// (BumpLinkObservableVersions in Link()'s prologue and glProgramBinary's
// failure path) - the link BODY, which stage 4 moves to a worker, never
// writes them.
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = 0;
// The attach lists are mutated only in Link()'s GL-thread prologue, which is why
// glGetAttachedShaders / GL_ATTACHED_SHADERS / the orphan-shader sweep need no join.
Vector<SharedPtr<ShaderObject>> m_shaders;
Vector<SharedPtr<ShaderObject>> m_detachedShaders; // Store detached shaders and remove on next link
// Link INPUTS (all "take effect at the next link" per GL): glBindAttribLocation,
// glBindFragDataLocation(Indexed), glTransformFeedbackVaryings, and the draw-buffer
// count stamped in by the entry point. A pending link snapshots these at enqueue.
UnorderedMap<String, Uint> m_explicitAttribLocations;
UnorderedMap<String, Uint> m_explicitFragDataLocation;
// Dual-source blend color index per output name (glBindFragDataLocationIndexed); snapshotted
// into the linked map at link time, like the location maps above.
UnorderedMap<String, Uint> m_explicitFragDataIndex;
Int m_maxFragmentOutputColorNumber = 8;
Vector<String> m_requestedXfbVaryings;
GLenum m_requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS;
String m_infoLog;
Bool m_deleteStatus = false; Bool m_deleteStatus = false;
Bool m_linkStatus = false;
Bool m_binaryRetrievableHint = false; Bool m_binaryRetrievableHint = false;
Bool m_separable = false; Bool m_separable = false;
Bool m_validateStatus = true; Bool m_validateStatus = true;
@@ -638,17 +781,9 @@ namespace MobileGL::MG_State::GLState {
Uint32 m_uboContentVersion = 0; Uint32 m_uboContentVersion = 0;
Uint32 m_linkVersion = 0; Uint32 m_linkVersion = 0;
// Transform feedback: request (applies at next link) and linked snapshot. // ---- Link OUTPUT ----
Vector<String> m_requestedXfbVaryings; // Written by the link and by the post-link setters GL allows (glUniform1i's sampler
GLenum m_requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS; // unit, glUniformBlockBinding). Reachable only through Artifacts(); see LinkArtifacts.
Vector<XfbVarying> m_xfbVaryings; LinkArtifacts m_artifacts;
Vector<Uint32> m_xfbStrides;
Vector<Uint32> m_gsStripTriangles;
Bool m_gsStripCaptureFixup = false;
GLenum m_gsInputPrimitive = GL_NONE;
GLenum m_xfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Int m_xfbVaryingNameMaxLength = 0;
Bool m_xfbNeedsScatteredCapture = false;
Uint32 m_xfbPackedStride = 0;
}; };
} // namespace MobileGL::MG_State::GLState } // namespace MobileGL::MG_State::GLState
@@ -11,7 +11,7 @@
namespace MobileGL::MG_State::GLState { namespace MobileGL::MG_State::GLState {
Uint ProgramState::CreateProgram() { Uint ProgramState::CreateProgram() {
Uint programId = 0; Uint programId = 0;
m_programIndexGenerator.Generate(1, &programId); m_programShaderNameGenerator.Generate(1, &programId);
EnsureIndexAvail(programId, m_programObjects); EnsureIndexAvail(programId, m_programObjects);
auto programObject = MakeShared<ProgramObject>(programId); auto programObject = MakeShared<ProgramObject>(programId);
if (programObject == nullptr) return 0; if (programObject == nullptr) return 0;
@@ -43,7 +43,7 @@ namespace MobileGL::MG_State::GLState {
// that were flagged with glDeleteShader while still attached. // that were flagged with glDeleteShader while still attached.
const Vector<SharedPtr<ShaderObject>> attachedShaders = programObject->GetAttachedShaders(); const Vector<SharedPtr<ShaderObject>> attachedShaders = programObject->GetAttachedShaders();
programObject.reset(); programObject.reset();
m_programIndexGenerator.Delete(program); m_programShaderNameGenerator.Delete(program);
for (const auto& shader : attachedShaders) { for (const auto& shader : attachedShaders) {
const Uint shaderName = shader->GetExternalIndex(); const Uint shaderName = shader->GetExternalIndex();
if (CheckIndexAvail(shaderName, m_shaderObjects) && m_shaderObjects[shaderName] == shader) { if (CheckIndexAvail(shaderName, m_shaderObjects) && m_shaderObjects[shaderName] == shader) {
@@ -77,9 +77,9 @@ namespace MobileGL::MG_State::GLState {
Uint ProgramState::CreateShader(ShaderStage stage) { Uint ProgramState::CreateShader(ShaderStage stage) {
Uint shaderId = 0; Uint shaderId = 0;
m_shaderIndexGenerator.Generate(1, &shaderId); m_programShaderNameGenerator.Generate(1, &shaderId);
EnsureIndexAvail(shaderId, m_shaderObjects); EnsureIndexAvail(shaderId, m_shaderObjects);
auto shaderObject = MakeShared<ShaderObject>(stage, shaderId); auto shaderObject = MakeShared<ShaderObject>(stage, shaderId, m_shaderPreprocessCache);
if (shaderObject == nullptr) return 0; if (shaderObject == nullptr) return 0;
m_shaderObjects[shaderId] = shaderObject; m_shaderObjects[shaderId] = shaderObject;
return shaderId; return shaderId;
@@ -121,7 +121,7 @@ namespace MobileGL::MG_State::GLState {
if (shaderObject == nullptr || !shaderObject->GetDeleteStatus()) return; if (shaderObject == nullptr || !shaderObject->GetDeleteStatus()) return;
if (ShaderHasGLVisibleAttachment(shaderObject)) return; if (ShaderHasGLVisibleAttachment(shaderObject)) return;
shaderObject.reset(); shaderObject.reset();
m_shaderIndexGenerator.Delete(shader); m_programShaderNameGenerator.Delete(shader);
} }
Bool ProgramState::ValidateShaderObject(Uint shader) const { Bool ProgramState::ValidateShaderObject(Uint shader) const {
@@ -10,6 +10,7 @@
#include <Includes.h> #include <Includes.h>
#include <MG_Util/Miscellany/IndexGenerator.h> #include <MG_Util/Miscellany/IndexGenerator.h>
#include "ProgramObject.h" #include "ProgramObject.h"
#include "ShaderPreprocessCache.h"
namespace MobileGL::MG_State::GLState { namespace MobileGL::MG_State::GLState {
class ProgramState { class ProgramState {
@@ -33,6 +34,11 @@ namespace MobileGL::MG_State::GLState {
const SharedPtr<ProgramObject>& GetCurrentProgram() const { return m_currentProgram; } const SharedPtr<ProgramObject>& GetCurrentProgram() const { return m_currentProgram; }
// P0b layer 2. Exposed for tests and diagnostics; the GL frontend never touches it
// directly - shader objects reach it through the pointer they are handed at
// CreateShader().
ShaderPreprocessCache& GetShaderPreprocessCache() { return *m_shaderPreprocessCache; }
private: private:
Bool ShaderHasGLVisibleAttachment(const SharedPtr<ShaderObject>& shaderObject) const; Bool ShaderHasGLVisibleAttachment(const SharedPtr<ShaderObject>& shaderObject) const;
// Frees the name slot and releases orphaned attached shaders; the immediate half // Frees the name slot and releases orphaned attached shaders; the immediate half
@@ -52,10 +58,19 @@ namespace MobileGL::MG_State::GLState {
vec.resize(idx + 1); vec.resize(idx + 1);
} }
IndexGenerator<Uint> m_programIndexGenerator; // Programs and shaders share one GL name space (GL 3.3 core 2.11: a shader
Vector<SharedPtr<ProgramObject>> m_programObjects; // name passed where a program is expected must be recognized as a shader and
// rejected with INVALID_OPERATION, and vice versa). One generator for both
// object kinds keeps the names disjoint; the object tables stay separate.
IndexGenerator<Uint> m_programShaderNameGenerator;
IndexGenerator<Uint> m_shaderIndexGenerator; // P0b layer 2: every shader object created here is handed shared ownership of this
// cache, so its lifetime no longer depends on member destruction order (P1: an
// in-flight compile job may outlive the context). The FIRST-member declaration is
// kept anyway - it costs nothing and documents the intent.
SharedPtr<ShaderPreprocessCache> m_shaderPreprocessCache = MakeShared<ShaderPreprocessCache>();
Vector<SharedPtr<ProgramObject>> m_programObjects;
Vector<SharedPtr<ShaderObject>> m_shaderObjects; Vector<SharedPtr<ShaderObject>> m_shaderObjects;
SharedPtr<ProgramObject> m_currentProgram; SharedPtr<ProgramObject> m_currentProgram;
@@ -7,12 +7,15 @@
// End of Source File Header // End of Source File Header
#include "ShaderObject.h" #include "ShaderObject.h"
#include "ShaderPreprocessCache.h"
#include <MG_Util/ShaderTranspiler/Types.h> #include <MG_Util/ShaderTranspiler/Types.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h> #include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h> #include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h> #include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h> #include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h>
#include <MG_Backend/BackendObjects.h>
#include <charconv>
namespace { namespace {
struct ComputeLocalSize { struct ComputeLocalSize {
@@ -71,16 +74,31 @@ namespace {
return result; return result;
} }
// Hoisted out of ParseComputeLocalSize: constructing a std::regex costs far more than
// running it over a small source, and it was being rebuilt on every compute compile. A
// const regex carries no mutable state, so sharing one instance is safe.
static const std::regex kComputeLocalSizePattern(R"(local_size_([xyz])\s*=\s*([0-9]+))");
static ComputeLocalSize ParseComputeLocalSize(const MobileGL::String& source) { static ComputeLocalSize ParseComputeLocalSize(const MobileGL::String& source) {
ComputeLocalSize localSize; ComputeLocalSize localSize;
const MobileGL::String uncommentedSource = StripGlslComments(source); const MobileGL::String uncommentedSource = StripGlslComments(source);
const std::regex localSizePattern(R"(local_size_([xyz])\s*=\s*([0-9]+))");
for (std::sregex_iterator it(uncommentedSource.begin(), uncommentedSource.end(), localSizePattern), end; for (std::sregex_iterator it(uncommentedSource.begin(), uncommentedSource.end(), kComputeLocalSizePattern),
end;
it != end; ++it) { it != end; ++it) {
const char axis = (*it)[1].str()[0]; const char axis = (*it)[1].str()[0];
const auto value = static_cast<unsigned long long>(std::stoull((*it)[2].str())); // The [0-9]+ capture is unbounded, so `local_size_x = 99999999999999999999999`
const MobileGL::Uint clampedValue = value > UINT_MAX ? UINT_MAX : static_cast<MobileGL::Uint>(value); // is a legal match. std::stoull would throw std::out_of_range on it and let the
// exception escape glCompileShader; std::from_chars reports the overflow instead.
// An overflowing literal saturates to UINT_MAX, which the device-limit check
// below rejects anyway - the same verdict a non-overflowing huge value gets.
const MobileGL::String digits = (*it)[2].str();
unsigned long long value = 0;
const std::from_chars_result parsed =
std::from_chars(digits.data(), digits.data() + digits.size(), value);
const MobileGL::Uint clampedValue = (parsed.ec != std::errc() || value > UINT_MAX)
? UINT_MAX
: static_cast<MobileGL::Uint>(value);
// TODO: Replace this literal layout scanner with parser/AST-backed validation so expressions and // TODO: Replace this literal layout scanner with parser/AST-backed validation so expressions and
// specialization-id layouts are handled consistently with glslang. // specialization-id layouts are handled consistently with glslang.
@@ -97,107 +115,231 @@ namespace {
return localSize; return localSize;
} }
static MobileGL::Uint GetComputeWorkGroupSizeLimit(MobileGL::Uint index) { // The device limits come from the CompileEnv snapshot, never from a live driver query.
constexpr MobileGL::Uint kFrontendMinComputeWorkGroupSizes[] = {1024, 1024, 64}; // GL_MAX_COMPUTE_WORK_GROUP_SIZE is a real GLES call on the DirectGLES backend: issued
MobileGL::Int backendValue = 0; // off the context thread it would silently no-op and turn a legal local_size_z into
if (MobileGL::MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v) { // COMPILE_STATUS=FALSE. CaptureCompileEnv() issues it once, on the GL thread.
MobileGL::MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, index, static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(
&backendValue); const MobileGL::String& source, const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
}
// TODO: Share these exposed compute limit helpers with GL_Getter.cpp instead of duplicating the frontend minima.
return std::max(static_cast<MobileGL::Uint>(std::max(backendValue, 0)),
kFrontendMinComputeWorkGroupSizes[index]);
}
static unsigned long long GetComputeWorkGroupInvocationLimit() {
constexpr unsigned long long kFrontendMaxComputeWorkGroupInvocations = 1024;
if (!MobileGL::MG_Backend::pActiveBackendObject) return kFrontendMaxComputeWorkGroupInvocations;
return std::max(static_cast<unsigned long long>(std::max(
MobileGL::MG_Backend::pActiveBackendObject->GetDynamicParameters()
.MaxComputeWorkGroupInvocations,
0)),
kFrontendMaxComputeWorkGroupInvocations);
}
static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(const MobileGL::String& source) {
const ComputeLocalSize localSize = ParseComputeLocalSize(source); const ComputeLocalSize localSize = ParseComputeLocalSize(source);
if (!localSize.declared) return std::nullopt; if (!localSize.declared) return std::nullopt;
if (localSize.x > GetComputeWorkGroupSizeLimit(0) || localSize.y > GetComputeWorkGroupSizeLimit(1) || if (localSize.x > env.maxComputeWorkGroupSize[0] || localSize.y > env.maxComputeWorkGroupSize[1] ||
localSize.z > GetComputeWorkGroupSizeLimit(2)) { localSize.z > env.maxComputeWorkGroupSize[2]) {
return "Compute shader local_size exceeds GL_MAX_COMPUTE_WORK_GROUP_SIZE."; return "Compute shader local_size exceeds GL_MAX_COMPUTE_WORK_GROUP_SIZE.";
} }
const unsigned long long invocations = static_cast<unsigned long long>(localSize.x) * localSize.y * localSize.z; const unsigned long long invocations = static_cast<unsigned long long>(localSize.x) * localSize.y * localSize.z;
if (invocations > GetComputeWorkGroupInvocationLimit()) { if (invocations > env.maxComputeWorkGroupInvocations) {
return "Compute shader local_size product exceeds GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS."; return "Compute shader local_size product exceeds GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS.";
} }
return std::nullopt; return std::nullopt;
} }
// The half of ShaderObject::Compile() that depends on nothing but the source text and
// the stage: preprocessing, the two lexical rejections, and the two lexical
// side-channel extractions. Split out so P0b layer 2 can memoize exactly this and
// nothing else - the glslang parse stays per-object because its TShader is
// consume-once. Deliberately free of any per-object state so the memo is sound.
//
// The former caveat is gone: the compute local-size verdict reads `env` rather than the
// live backend, and env.fingerprint is part of the P0b cache key, so a memo can never be
// returned against limits other than the ones it was computed against.
static MobileGL::MG_State::GLState::ShaderPreprocessResult RunSourceOnlyPipeline(
const MobileGL::ShaderStage stage, const MobileGL::String& source,
const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
using namespace MobileGL;
using namespace MobileGL::MG_Util::ShaderTranspiler;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
MobileGL::MG_State::GLState::ShaderPreprocessResult result;
result.preprocessedSource = source;
PreprocessShaderSource(stage, result.preprocessedSource, env);
if (stage == ShaderStage::Compute) {
if (const std::optional<String> localSizeError =
ValidateComputeLocalSizeLimits(result.preprocessedSource, env)) {
result.outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
result.infoLog = *localSizeError;
return result;
}
}
if (const std::optional<String> reservedError = FindReservedIdentifierViolation(result.preprocessedSource)) {
result.outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
result.infoLog = *reservedError;
return result;
}
// The parse this feeds runs in the link-compatible configuration (Vulkan-client
// env with relaxed rules): the TShader it produces is what glLinkProgram links and
// what the backends' SPIR-V is generated from - there is no second, GL-client
// parse. The GL frontend semantics the relaxed parse cannot provide are restored
// on top: explicit default-block uniform locations through the lexical
// side-channels below, dead-uniform/global-UBO filtering in
// ProgramObject::DoReflection.
result.explicitUniformLocations = ExtractExplicitUniformLocations(result.preprocessedSource);
result.explicitOpaqueBindings = ExtractExplicitOpaqueBindings(result.preprocessedSource);
result.outcome = ShaderPreprocessOutcome::Preprocessed;
return result;
}
} }
namespace MobileGL::MG_State::GLState { namespace MobileGL::MG_State::GLState {
void ShaderObject::SetShaderSource(const String& source) { void ShaderObject::SetShaderSource(const String& source) {
// P0b layer 1. glShaderSource always REPLACES the source, but replacing it with a
// byte-identical one cannot change what a compile would produce: the whole
// pipeline below (preprocess -> lexical checks -> glslang parse) is a pure
// function of (stage, source) plus context-lifetime backend limits. So keeping the
// compiled state is not an optimization that changes observable behaviour - the
// COMPILE_STATUS, the info log and the reflection a caller can query are exactly
// what a real recompile would have rebuilt, byte for byte.
if (SourceMatchesCompiledState(source)) return;
m_source = source; m_source = source;
m_shader.reset(); InvalidateCompiledState();
m_compileStatus = false;
m_infoLog.clear();
} }
void ShaderObject::SetShaderSource(String&& source) { void ShaderObject::SetShaderSource(String&& source) {
if (SourceMatchesCompiledState(source)) return;
m_source = Move(source); m_source = Move(source);
m_shader.reset(); InvalidateCompiledState();
m_compileStatus = false; }
m_infoLog.clear();
Bool ShaderObject::SourceMatchesCompiledState(const String& candidate) const {
if (!m_hasCompiledState) return false;
if (candidate.length() != m_compiledSourceLength) return false;
if (ShaderPreprocessCache::HashSource(candidate) != m_compiledSourceHash) return false;
// The hash is a fast reject only; confirm against the actual stored text. While
// m_hasCompiledState holds, m_source IS the source that produced the state.
return candidate == m_source;
}
void ShaderObject::RememberCompiledSource(const Uint64 sourceHash) {
m_hasCompiledState = true;
m_compiledSourceHash = sourceHash;
m_compiledSourceLength = m_source.length();
}
// EnsureCompileJoined() is defined inline in ShaderObject.h (see the comment there for
// why: no LTO, and it is called from every Compiled() read).
void ShaderObject::InvalidateCompiledState() {
// The compile artifacts are exactly what one Compile() writes, so discarding them
// wholesale IS the invalidation. (Kept as an explicit reset rather than a
// default-construct so the intent survives a future field addition.)
Compiled() = CompileArtifacts{};
m_hasCompiledState = false;
m_compiledSourceHash = 0;
m_compiledSourceLength = 0;
} }
void ShaderObject::Compile() { void ShaderObject::Compile() {
using namespace MG_Util::ShaderTranspiler; using namespace MG_Util::ShaderTranspiler;
String compileSource = m_source;
MG_Util::ShaderTranspiler::PreprocessShaderSource(m_stage, compileSource);
if (m_stage == ShaderStage::Compute) { // P0b layer 1: the state this object holds was produced by a previous Compile() of
const std::optional<String> localSizeError = ValidateComputeLocalSizeLimits(compileSource); // the exact source it still holds, so a recompile is a no-op. This covers the
if (localSizeError) { // failure case too - the info log stays queryable because nothing is cleared.
m_compileStatus = false; //
m_shader.reset(); // shaderConsumedByLink interaction: if the stored TShader already fed a link,
m_infoLog = *localSizeError; // the no-op leaves preprocessedSource and both side-channel maps intact, which
return; // is precisely what TakeShaderForLink's on-demand re-parse needs. A real recompile
} // would have handed the next link a fresh parse; the no-op hands it a fresh
} // re-parse of the identical source instead. Same result, one parse either way.
if (m_hasCompiledState) return;
const std::optional<String> reservedError = InvalidateCompiledState();
MG_Util::ShaderTranspiler::FindReservedIdentifierViolation(compileSource);
if (reservedError) { const Uint64 sourceHash = ShaderPreprocessCache::HashSource(m_source);
m_compileStatus = false;
m_shader.reset(); // The compile-environment snapshot, taken here on the GL thread. Everything below
m_infoLog = *reservedError; // reads the device through it and never through pActiveBackendObject, which is what
// makes the whole body movable onto a worker in stage 3.
CompileArtifacts& compiled = Compiled();
compiled.env = MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
const MG_Util::ShaderTranspiler::CompileEnv& env = *compiled.env;
// P0b layer 2: another shader object in this context may already have run the
// source-only half over byte-identical text under the same environment.
ShaderPreprocessResultPtr cached =
m_preprocessCache ? m_preprocessCache->Find(m_stage, sourceHash, m_source, env.fingerprint) : nullptr;
SharedPtr<ShaderPreprocessResult> fresh;
if (!cached) fresh = MakeShared<ShaderPreprocessResult>(RunSourceOnlyPipeline(m_stage, m_source, env));
const ShaderPreprocessResult& shared = cached ? *cached : *fresh;
const Bool shouldPopulateCache = !cached && m_preprocessCache != nullptr;
if (!shared.Preprocessed()) {
// Rejected lexically, or a glslang failure this context has already seen for
// this exact source (ParseFailed) - either way the parse can be skipped.
compiled.infoLog = shared.infoLog;
if (shouldPopulateCache) {
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
RememberCompiledSource(sourceHash);
return; return;
} }
// Compile for OpenGL here, so that we can do validation and link
// like a real OpenGL driver at linking stage
// Will compile for other backends later.
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage), ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = compileSource, .sourceStr = shared.preprocessedSource,
.flags = ShaderCompileBits::CompileForOpenGL}; .flags = 0,
.env = &env};
auto result = ShaderCompiler::CompileShader(attrib); auto result = ShaderCompiler::CompileShader(attrib);
if (result) { if (result) {
m_compileStatus = true; compiled.compileStatus = true;
m_shader = result.value(); compiled.shader = result.value();
m_infoLog.clear(); // Copy, not move: `shared` may alias a cache entry that has to outlive us, and
} else { // `fresh` is about to be handed to the cache.
m_compileStatus = false; compiled.preprocessedSource = shared.preprocessedSource;
m_shader.reset(); compiled.explicitUniformLocations = shared.explicitUniformLocations;
m_infoLog = result.error().log; compiled.explicitOpaqueBindings = shared.explicitOpaqueBindings;
MGLOG_D("ShaderObject::Compile: Shader %d compilation failed.\nSource:\n%s\nInfoLog:\n%s\nSetting " compiled.infoLog.clear();
"m_compileStatus = false as a result.", if (shouldPopulateCache) {
m_externalIndex, compileSource.c_str(), m_infoLog.c_str()); m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
} }
} else {
compiled.infoLog = result.error().log;
MGLOG_D("ShaderObject::Compile: Shader %d compilation failed.\nSource:\n%s\nInfoLog:\n%s\nSetting "
"compileStatus = false as a result.",
m_externalIndex, shared.preprocessedSource.c_str(), compiled.infoLog.c_str());
if (shouldPopulateCache) {
fresh->outcome = ShaderPreprocessOutcome::ParseFailed;
fresh->infoLog = compiled.infoLog;
fresh->explicitUniformLocations.clear();
fresh->explicitOpaqueBindings.clear();
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
}
RememberCompiledSource(sourceHash);
}
SharedPtr<glslang::TShader> ShaderObject::TakeShaderForLink(String& outReparseLog) {
CompileArtifacts& compiled = Compiled();
if (compiled.shader && !compiled.shaderConsumedByLink) {
compiled.shaderConsumedByLink = true;
return compiled.shader;
}
// The stored parse already fed a link, whose mapIO mutated its intermediate.
// Re-parse the preprocessed source through the identical configuration; this
// costs one glslang parse, which is exactly what GenerateBinary used to spend
// here on EVERY link rather than only on reuse.
using namespace MG_Util::ShaderTranspiler;
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = compiled.preprocessedSource,
.flags = 0,
// Re-parse against the SAME environment the original parse used,
// not against whatever the backend reports now.
.env = compiled.env.get()};
auto result = ShaderCompiler::CompileShader(attrib);
if (!result) {
// Should be unreachable: the same source parsed successfully at Compile().
outReparseLog = result.error().log;
MGLOG_E("ShaderObject::TakeShaderForLink: re-parse of shader %d failed:\n%s", m_externalIndex,
outReparseLog.c_str());
return nullptr;
}
return result.value();
} }
void ShaderObject::MarkAsDeleted() { void ShaderObject::MarkAsDeleted() {
@@ -8,6 +8,7 @@
#pragma once #pragma once
#include <Includes.h> #include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL { namespace MobileGL {
enum class ShaderStage { enum class ShaderStage {
@@ -22,34 +23,146 @@ namespace MobileGL {
}; };
namespace MG_State::GLState { namespace MG_State::GLState {
// P0b layer 2. Declared, not included: the cache keys on ShaderStage, so including
// its header here would be circular.
class ShaderPreprocessCache;
class ShaderObject { class ShaderObject {
public: public:
ShaderObject(const ShaderStage stage, Uint externalIndex) // `preprocessCache` is the owning context's cross-object memo (P0b layer 2);
: m_stage(stage), m_externalIndex(externalIndex) {} // null is fully supported and simply means "no sharing" - that is what the
// context-less internal shader objects (the default FS, the blit pipeline) use.
// Shared ownership rather than a raw pointer: once compiles run on a worker the
// job outlives neither the object nor the context deterministically, and the
// cache has to stay alive for whoever is still reading it.
ShaderObject(const ShaderStage stage, Uint externalIndex,
SharedPtr<ShaderPreprocessCache> preprocessCache = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(Move(preprocessCache)) {}
void SetShaderSource(const String& source); void SetShaderSource(const String& source);
void SetShaderSource(String&& source); void SetShaderSource(String&& source);
void Compile(); void Compile();
void MarkAsDeleted(); void MarkAsDeleted();
// Hands out a link-consumable TShader. glslang's mapIO mutates the TShader's
// aliased intermediate, so the parse stored by Compile() may feed exactly one
// link; every later link (relink, or the same shader attached to a second
// program) gets a fresh parse of the stored preprocessed source through the
// byte-identical CompileShader path (including the legacy-460 retry). Only
// callable while GetCompileStatus() is true. Returns null only if that
// re-parse fails - outReparseLog then carries its diagnostics.
SharedPtr<glslang::TShader> TakeShaderForLink(String& outReparseLog);
Uint GetExternalIndex() const { return m_externalIndex; } Uint GetExternalIndex() const { return m_externalIndex; }
ShaderStage GetShaderStage() const { return m_stage; } ShaderStage GetShaderStage() const { return m_stage; }
const String& GetShaderSource() const { return m_source; } const String& GetShaderSource() const { return m_source; }
const SharedPtr<glslang::TShader>& GetCompiledShader() const { return m_shader; } const SharedPtr<glslang::TShader>& GetCompiledShader() const { return Compiled().shader; }
const String& GetInfoLog() const { return m_infoLog; } const String& GetInfoLog() const { return Compiled().infoLog; }
const UnorderedMap<String, Uint>& GetUniformLocations() const { return m_uniforms; } const UnorderedMap<String, Uint>& GetUniformLocations() const { return Compiled().uniforms; }
Bool GetCompileStatus() const { return m_compileStatus; } // Explicit layout(location = N) qualifiers on this shader's default-block
// uniforms, captured lexically at Compile() because the relaxed parse drops
// them from reflection (see ExtractExplicitUniformLocations).
const UnorderedMap<String, Int>& GetExplicitUniformLocations() const {
return Compiled().explicitUniformLocations;
}
// Explicit layout(binding = N) on sampler/image uniforms - their initial
// texture/image units - captured lexically for the same reason (see
// ExtractExplicitOpaqueBindings).
const UnorderedMap<String, Uint>& GetExplicitOpaqueBindings() const {
return Compiled().explicitOpaqueBindings;
}
Bool GetCompileStatus() const { return Compiled().compileStatus; }
Bool GetDeleteStatus() const { return m_deleteStatus; } Bool GetDeleteStatus() const { return m_deleteStatus; }
// Blocks until a pending compile (P1 stage 3 onwards) has published its
// artifacts. Public for the few sites that must join without reading anything.
// A no-op today - nothing is ever pending.
void JoinCompile() const { EnsureCompileJoined(); }
// True while this object holds the outcome (success OR failure) of a previous
// Compile() of exactly the source it currently holds - i.e. while the P0b
// layer-1 memo is armed and a glCompileShader would be a no-op. Diagnostics
// and tests only; nothing in the GL frontend branches on it.
//
// Deliberately does NOT join: the memo bookkeeping below is GL-thread-owned and
// says nothing about whether a worker has finished, which is exactly the
// property GL_COMPLETION_STATUS_KHR needs when stage 3 lands.
Bool HasMemoizedCompile() const { return m_hasCompiledState; }
private: private:
// ---- P1: everything a compile PRODUCES, in one block ----
//
// Same rule as ProgramObject::LinkArtifacts: this is exactly what
// InvalidateCompiledState() clears, i.e. exactly what one run of Compile()
// writes. Stage 3 lifts this struct wholesale into ShaderCompileTask, where a
// worker fills it in and the GL thread reads it through the same gate.
struct CompileArtifacts {
// The CompileEnv snapshot this compile ran against. Held so the
// consume-once re-parse in TakeShaderForLink() reproduces the original
// parse exactly, instead of re-reading whatever the backend says now.
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
SharedPtr<glslang::TShader> shader;
// The source Compile() actually parsed (after PreprocessShaderSource), kept
// for TakeShaderForLink's re-parse so a later link never depends on the
// preprocessor being deterministic across backend-state changes.
String preprocessedSource;
UnorderedMap<String, Uint> uniforms;
UnorderedMap<String, Int> explicitUniformLocations;
UnorderedMap<String, Uint> explicitOpaqueBindings;
Bool shaderConsumedByLink = false;
String infoLog;
Bool compileStatus = false;
};
// ---- The one and only join gate for compile output (P1 invariant I5) ----
// Blocks until a pending compile has published into m_compiled. Today nothing
// is ever pending - glCompileShader still runs the whole body inline - so this
// is an unconditional no-op. It exists NOW so that every reader of compile
// output is already routed through it when stage 3 makes it block.
//
// Defined inline (not in ShaderObject.cpp): called from every Compiled() read,
// and the project never builds with LTO, so an out-of-line empty body would be
// a real cross-TU call at each of those call sites instead of folding away.
void EnsureCompileJoined() const {}
CompileArtifacts& Compiled() {
EnsureCompileJoined();
return m_compiled;
}
const CompileArtifacts& Compiled() const {
EnsureCompileJoined();
return m_compiled;
}
void InvalidateCompiledState();
// ---- P0b layer 1: per-object no-op recompile ----
// True iff `candidate` is byte-identical to the source that produced the
// compiled state this object is currently holding. The stored hash and length
// are only a fast reject; the answer is always confirmed against the full
// stored text, so no behaviour rides on a 64-bit hash.
Bool SourceMatchesCompiledState(const String& candidate) const;
// Arms the layer-1 memo for the source that Compile() just processed.
void RememberCompiledSource(Uint64 sourceHash);
// ---- GL-thread-owned state: never produced by a compile, so it never joins ----
const Uint m_externalIndex = 0; const Uint m_externalIndex = 0;
const ShaderStage m_stage; const ShaderStage m_stage;
// glShaderSource text. A worker only ever reads the snapshot handed to it, so
// GL_SHADER_SOURCE_LENGTH and glGetShaderSource never join.
String m_source; String m_source;
SharedPtr<glslang::TShader> m_shader;
UnorderedMap<String, Uint> m_uniforms;
String m_infoLog; // P0b layer 2: the owning context's cross-object memo, or null.
const SharedPtr<ShaderPreprocessCache> m_preprocessCache;
// P0b layer 1. m_hasCompiledState is the invariant "m_source is byte-identical
// to the source that produced the compile artifacts"; it is armed at the end of
// every Compile() and disarmed by InvalidateCompiledState(). Stage 3 replaces
// all three with a pointer compare against the in-flight job's source snapshot.
Bool m_hasCompiledState = false;
Uint64 m_compiledSourceHash = 0;
SizeT m_compiledSourceLength = 0;
Bool m_deleteStatus = false; Bool m_deleteStatus = false;
Bool m_compileStatus = false;
// ---- Compile OUTPUT ---- reachable only through Compiled().
CompileArtifacts m_compiled;
}; };
} // namespace MG_State::GLState } // namespace MG_State::GLState
} // namespace MobileGL } // namespace MobileGL
@@ -0,0 +1,86 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ShaderPreprocessCache.h"
namespace MobileGL::MG_State::GLState {
ShaderPreprocessResultPtr ShaderPreprocessCache::Find(const ShaderStage stage, const Uint64 sourceHash,
const String& source, const Uint64 envFingerprint) const {
const Key key{.stage = stage,
.sourceHash = sourceHash,
.sourceLength = source.length(),
.envFingerprint = envFingerprint};
const std::lock_guard<std::mutex> lock(m_mutex);
const auto it = m_index.find(key);
if (it == m_index.end()) return nullptr;
// Never let correctness ride on a 64-bit hash: confirm the hit byte for byte.
// Lengths already matched (they are part of the key), so this is a plain memcmp.
const Entry& entry = *it->second;
if (entry.originalSource != source) return nullptr;
// A copy of the SharedPtr, taken under the lock: the payload now outlives any
// eviction the caller races with.
return entry.result;
}
void ShaderPreprocessCache::Insert(const ShaderStage stage, const Uint64 sourceHash, const String& source,
const Uint64 envFingerprint, ShaderPreprocessResultPtr result) {
if (!result) return;
const SizeT entryBytes = EntryBytes(source, *result);
// A single source bigger than the whole budget would evict every other entry and
// then itself; refuse it instead of thrashing the cache empty.
if (entryBytes > kMaxStoredSourceBytes) return;
const Key key{.stage = stage,
.sourceHash = sourceHash,
.sourceLength = source.length(),
.envFingerprint = envFingerprint};
const std::lock_guard<std::mutex> lock(m_mutex);
if (const auto existing = m_index.find(key); existing != m_index.end()) {
// Either a re-insert of the same source (harmless) or a genuine hash collision
// with a different source. Both are resolved by letting the newcomer win: one
// entry per key keeps the index a plain map, and a collision is astronomically
// rare enough that the loser simply misses.
EraseEntryLocked(existing->second);
}
m_entries.push_back(Entry{.key = key, .originalSource = source, .result = Move(result)});
m_index[key] = std::prev(m_entries.end());
m_storedSourceBytes += entryBytes;
EvictUntilWithinBudgetLocked();
}
void ShaderPreprocessCache::Clear() {
const std::lock_guard<std::mutex> lock(m_mutex);
m_entries.clear();
m_index.clear();
m_storedSourceBytes = 0;
}
void ShaderPreprocessCache::EraseEntryLocked(const EntryList::iterator it) {
const SizeT bytes = EntryBytes(it->originalSource, *it->result);
m_storedSourceBytes = bytes > m_storedSourceBytes ? 0 : m_storedSourceBytes - bytes;
m_index.erase(it->key);
m_entries.erase(it);
}
void ShaderPreprocessCache::EvictUntilWithinBudgetLocked() {
// FIFO: the oldest insertion goes first. Insert() already refuses entries larger
// than the byte budget, so this loop always terminates with at least the entry
// that was just added still resident.
while (!m_entries.empty() &&
(m_entries.size() > kMaxEntries || m_storedSourceBytes > kMaxStoredSourceBytes)) {
EraseEntryLocked(m_entries.begin());
}
}
} // namespace MobileGL::MG_State::GLState
@@ -0,0 +1,165 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <list>
#include <mutex>
#include <MG_State/GLState/ProgramState/ShaderObject.h>
namespace MobileGL::MG_State::GLState {
// Where the shared, source-only half of ShaderObject::Compile() stopped. The two
// rejection verdicts are kept apart (rather than collapsed into "failed") so a hit
// reproduces the original diagnosis, not just the original info log.
enum class ShaderPreprocessOutcome : Uint8 {
// The source-only half ran clean; preprocessedSource and both maps are valid.
Preprocessed,
// ValidateComputeLocalSizeLimits rejected it (compute only).
ComputeLocalSizeRejected,
// FindReservedIdentifierViolation rejected it.
ReservedIdentifierRejected,
// The source-only half was clean but glslang rejected the preprocessed source.
// Memoizing this saves the parse itself on every later object with that source.
ParseFailed,
};
// Everything ShaderObject::Compile() derives from the source text alone, i.e.
// everything that is identical for two shader objects holding byte-identical source.
struct ShaderPreprocessResult {
ShaderPreprocessOutcome outcome = ShaderPreprocessOutcome::Preprocessed;
// Valid unless the preprocessor itself never ran; kept even for the rejection
// outcomes because that is the text the diagnostics refer to.
String preprocessedSource;
UnorderedMap<String, Int> explicitUniformLocations;
UnorderedMap<String, Uint> explicitOpaqueBindings;
// The compile info log to publish; empty when outcome == Preprocessed.
String infoLog;
Bool Preprocessed() const { return outcome == ShaderPreprocessOutcome::Preprocessed; }
};
// Cache hits hand out shared ownership, not a raw pointer into the entry list. That is
// what makes the cache safe once compiles run concurrently: a reader keeps its payload
// alive across any eviction, and a 107 KB preprocessedSource is never copied on a hit.
using ShaderPreprocessResultPtr = SharedPtr<const ShaderPreprocessResult>;
// P0b layer 2: a per-context, bounded memo of the source-only half of shader
// compilation, keyed by (stage, xxhash64(source), source length).
//
// Motivation: in the Iris shader-pack corpus ~21% of every glCompileShader in a trace
// is a *different* shader object holding byte-identical source (packs glue the same
// common/composite GLSL into many program stages), so the preprocess + reserved-
// identifier scan + explicit-location/binding extraction runs over the same megabytes
// again and again. Layer 1 (in ShaderObject) covers the same object recompiled with
// unchanged source; this covers the cross-object case.
//
// What is NOT cached: the glslang parse. glslang's TShader is consume-once (mapIO
// mutates the aliased intermediate at link), so every shader object still needs its
// own parse; only the text-processing half is shared.
//
// Correctness: the 64-bit hash is a lookup accelerator only. Every hit re-compares the
// full stored original source with memcmp before it is honored, so a hash collision
// degrades to a miss, never to a wrong answer. That is why the full original text is
// stored rather than a prefix/suffix digest - the cache is bounded, so the cost is.
//
// Eviction: FIFO (insertion order), bounded by BOTH an entry count and a stored-source
// byte budget, whichever binds first. FIFO rather than LRU because shader-pack loading
// is a burst of mostly-distinct sources whose reuse clusters around insertion time;
// LRU's extra list splice on every hit buys nothing measurable here, and FIFO keeps
// Find() a genuinely const, read-only operation.
class ShaderPreprocessCache {
public:
static constexpr SizeT kMaxEntries = 128;
static constexpr SizeT kMaxStoredSourceBytes = 8u * 1024u * 1024u;
// Returns the memoized result for this exact source under this exact compile
// environment, or null on a miss. The returned SharedPtr owns its payload, so it
// stays valid for as long as the caller holds it - across Insert(), Clear(), and
// across the destruction of the cache itself.
//
// envFingerprint joins the key because the source-only pipeline's compute
// local-size verdict is computed against CompileEnv's device limits: a memo must
// never outlive the environment it was computed against (memo-hazard rule).
ShaderPreprocessResultPtr Find(ShaderStage stage, Uint64 sourceHash, const String& source,
Uint64 envFingerprint) const;
// Memoizes `result` for this source. A source whose own storage cost already
// exceeds the byte budget is simply not cached (caching it would evict everything
// else and then itself).
void Insert(ShaderStage stage, Uint64 sourceHash, const String& source, Uint64 envFingerprint,
ShaderPreprocessResultPtr result);
void Clear();
static Uint64 HashSource(const String& source) {
return static_cast<Uint64>(XXH64(source.data(), source.length(), 0));
}
SizeT GetEntryCount() const {
const std::lock_guard<std::mutex> lock(m_mutex);
return m_entries.size();
}
SizeT GetStoredSourceBytes() const {
const std::lock_guard<std::mutex> lock(m_mutex);
return m_storedSourceBytes;
}
private:
struct Key {
ShaderStage stage = ShaderStage::Unknown;
Uint64 sourceHash = 0;
SizeT sourceLength = 0;
Uint64 envFingerprint = 0;
Bool operator==(const Key& other) const {
return stage == other.stage && sourceHash == other.sourceHash &&
sourceLength == other.sourceLength && envFingerprint == other.envFingerprint;
}
};
struct KeyHasher {
SizeT operator()(const Key& key) const {
// The source hash already spreads well; fold the two discriminators in so
// that same-hash-different-stage/length keys land in different buckets.
Uint64 mixed = key.sourceHash;
mixed ^= static_cast<Uint64>(key.sourceLength) + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
mixed ^= static_cast<Uint64>(static_cast<Int>(key.stage)) * 0xff51afd7ed558ccdull;
mixed ^= key.envFingerprint + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
return static_cast<SizeT>(mixed);
}
};
struct Entry {
Key key;
// The full original (pre-preprocess) source, kept so a hit can be confirmed by
// comparison instead of trusting the hash.
String originalSource;
ShaderPreprocessResultPtr result;
};
using EntryList = std::list<Entry>;
static SizeT EntryBytes(const String& source, const ShaderPreprocessResult& result) {
return source.length() + result.preprocessedSource.length();
}
void EvictUntilWithinBudgetLocked();
void EraseEntryLocked(EntryList::iterator it);
// P1: every public entry point takes this. The lock alone would NOT have been
// enough - the old Find() handed back a raw pointer into an entry that a
// concurrent Insert()'s FIFO eviction could erase while the caller was still
// reading it. Shared ownership of the payload is what closes that hole; the mutex
// only protects the containers below.
mutable std::mutex m_mutex;
EntryList m_entries; // front = oldest (FIFO victim)
UnorderedMap<Key, EntryList::iterator, KeyHasher> m_index;
SizeT m_storedSourceBytes = 0;
};
} // namespace MobileGL::MG_State::GLState
+1
View File
@@ -74,6 +74,7 @@ add_subdirectory(Program)
add_subdirectory(Query) add_subdirectory(Query)
add_subdirectory(Pipeline) add_subdirectory(Pipeline)
add_subdirectory(ShaderTranspiler) add_subdirectory(ShaderTranspiler)
add_subdirectory(Util)
if (ENABLE_INTEGRATION_TESTS) if (ENABLE_INTEGRATION_TESTS)
add_subdirectory(Backend/DirectVulkan) add_subdirectory(Backend/DirectVulkan)
endif() endif()
+1
View File
@@ -9,6 +9,7 @@ add_executable(
${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp ${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp
${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/SpvcSession.cpp ${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/SpvcSession.cpp
${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/glslang/UniformTraverser.cpp ${MGL_ROOT}/MobileGL/MG_Util/ShaderTranspiler/glslang/UniformTraverser.cpp
${MGL_ROOT}/MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
) )
target_include_directories(ProgramUtilTest PRIVATE target_include_directories(ProgramUtilTest PRIVATE
+714
View File
@@ -19,6 +19,7 @@
#include "MG_Impl/GLImpl/Getter/GL_Getter.h" #include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h" #include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h" #include "MG_State/GLState/Core.h"
#include "MG_State/GLState/ProgramState/ShaderPreprocessCache.h"
#include "MG_Util/ShaderTranspiler/ShaderCompiler.h" #include "MG_Util/ShaderTranspiler/ShaderCompiler.h"
using namespace MobileGL; using namespace MobileGL;
@@ -346,6 +347,29 @@ void main() {
EXPECT_EQ(GetError(), GL_NO_ERROR); EXPECT_EQ(GetError(), GL_NO_ERROR);
} }
TEST_F(ProgramTest, OutOfRangeComputeLocalSizeLiteralFailsCompileInsteadOfThrowing) {
// The layout scanner's digit capture is unbounded, so a literal wider than 64 bits is a
// legal match. It must saturate and be rejected through COMPILE_STATUS; if the integer
// conversion throws instead, the exception escapes glCompileShader entirely.
char infoLog[1024] = "";
const char* csSrc = R"(#version 460 core
layout(local_size_x = 99999999999999999999999) in;
void main() {
}
)";
GLuint cs = CreateShader(GL_COMPUTE_SHADER);
ShaderSource(cs, 1, &csSrc, nullptr);
CompileShader(cs);
GLint csStatus = GL_TRUE;
GetShaderiv(cs, GL_COMPILE_STATUS, &csStatus);
EXPECT_EQ(csStatus, GL_FALSE);
GetShaderInfoLog(cs, sizeof(infoLog), nullptr, infoLog);
EXPECT_NE(String(infoLog).find("GL_MAX_COMPUTE_WORK_GROUP_SIZE"), String::npos) << infoLog;
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
TEST_F(ProgramTest, DirectVulkanStorageBlockUsesShaderLayoutBinding) { TEST_F(ProgramTest, DirectVulkanStorageBlockUsesShaderLayoutBinding) {
char infoLog[1024] = ""; char infoLog[1024] = "";
const char* csSrc = R"(#version 460 core const char* csSrc = R"(#version 460 core
@@ -2365,3 +2389,693 @@ void main() { o_color = vec4(1.0); }
EXPECT_EQ(IsShader(fs), GL_FALSE); EXPECT_EQ(IsShader(fs), GL_FALSE);
EXPECT_EQ(GetError(), GL_NO_ERROR); EXPECT_EQ(GetError(), GL_NO_ERROR);
} }
// ---- P0a single-parse regression tests ----
// glCompileShader now performs the one link-compatible (relaxed Vulkan-rules) parse;
// these pin the GL frontend semantics that parse cannot provide by itself.
namespace {
GLuint CompileShaderChecked(GLenum type, const char* source) {
char infoLog[1024] = "";
GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
CompileShader(shader);
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
GetShaderInfoLog(shader, sizeof(infoLog), nullptr, infoLog);
EXPECT_EQ(status, GL_TRUE) << infoLog;
return shader;
}
GLuint LinkVsFs(GLuint vs, GLuint fs, GLint expectedLinkStatus) {
char infoLog[2048] = "";
GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
GLint linkStatus = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &linkStatus);
GetProgramInfoLog(program, sizeof(infoLog), nullptr, infoLog);
EXPECT_EQ(linkStatus, expectedLinkStatus) << infoLog;
return program;
}
} // namespace
// The relaxed parse sweeps every DECLARED default-block uniform into MGL_GLOBAL_UBO,
// including ones no stage reads. GL requires those to be inactive: absent from the
// glGetActiveUniform enumeration and -1 from glGetUniformLocation. The synthesized
// MGL_GLOBAL_UBO itself must not surface as a GL uniform block either.
TEST_F(ProgramTest, DeclaredButUnreadUniformIsInactiveAndGlobalUboStaysHidden) {
const char* vsSource = R"(#version 330 core
uniform mat4 uUsedMat;
uniform vec4 uDeadVec;
void main() { gl_Position = uUsedMat * vec4(1.0); }
)";
const char* fsSource = R"(#version 330 core
uniform vec4 uUsedColor;
uniform float uDeadFloat;
out vec4 fragColor;
void main() { fragColor = uUsedColor; }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
GLint activeUniforms = 0;
GetProgramiv(program, GL_ACTIVE_UNIFORMS, &activeUniforms);
EXPECT_EQ(activeUniforms, 2);
EXPECT_NE(GetUniformLocation(program, "uUsedMat"), -1);
EXPECT_NE(GetUniformLocation(program, "uUsedColor"), -1);
EXPECT_EQ(GetUniformLocation(program, "uDeadVec"), -1);
EXPECT_EQ(GetUniformLocation(program, "uDeadFloat"), -1);
EXPECT_EQ(UniformIndexByName(program, "uDeadVec"), GL_INVALID_INDEX);
char nameBuf[64] = "";
for (GLint i = 0; i < activeUniforms; ++i) {
GLsizei nameLen = 0;
GLint size = 0;
GLenum type = 0;
GetActiveUniform(program, static_cast<GLuint>(i), sizeof(nameBuf), &nameLen, &size, &type, nameBuf);
EXPECT_TRUE(std::strcmp(nameBuf, "uDeadVec") != 0 && std::strcmp(nameBuf, "uDeadFloat") != 0)
<< nameBuf;
}
// No named blocks are declared, so GL must see zero uniform blocks - the global
// UBO the transpiler materializes is an implementation artifact.
GLint activeBlocks = 0;
GetProgramiv(program, GL_ACTIVE_UNIFORM_BLOCKS, &activeBlocks);
EXPECT_EQ(activeBlocks, 0);
EXPECT_EQ(GetUniformBlockIndex(program, "MGL_GLOBAL_UBO"), GL_INVALID_INDEX);
// Default-block uniforms report block index -1 and offset -1 even though the
// relaxed parse physically placed them in the global UBO.
const GLuint usedMat = UniformIndexByName(program, "uUsedMat");
ASSERT_NE(usedMat, GL_INVALID_INDEX);
EXPECT_EQ(QueryUniformiv(program, usedMat, GL_UNIFORM_BLOCK_INDEX), -1);
EXPECT_EQ(QueryUniformiv(program, usedMat, GL_UNIFORM_OFFSET), -1);
EXPECT_EQ(QueryUniformiv(program, usedMat, GL_UNIFORM_ARRAY_STRIDE), -1);
EXPECT_EQ(QueryUniformiv(program, usedMat, GL_UNIFORM_MATRIX_STRIDE), -1);
EXPECT_EQ(QueryUniformiv(program, usedMat, GL_UNIFORM_IS_ROW_MAJOR), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Distinct uniforms whose explicit locations overlap across stages must fail the
// link (ARB_explicit_uniform_location). The GL-client parse used to reject this at
// glslang mapIO; the relaxed parse drops the qualifiers, so the location assigner
// enforces it - this is the experiment's synthetic divergence case.
TEST_F(ProgramTest, ExplicitUniformLocationOverlapAcrossStagesFailsLink) {
const char* vsSource = R"(#version 460 core
layout(location = 3) uniform vec4 uVec[4];
void main() { gl_Position = uVec[0] + uVec[3]; }
)";
const char* fsSource = R"(#version 460 core
layout(location = 5) uniform float uF;
out vec4 fragColor;
void main() { fragColor = vec4(uF); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_FALSE);
char infoLog[1024] = "";
GLsizei logLength = 0;
GetProgramInfoLog(program, sizeof(infoLog), &logLength, infoLog);
EXPECT_GT(logLength, 0);
}
// The same uniform declared with different explicit locations in two stages is a
// link error as well.
TEST_F(ProgramTest, ConflictingExplicitUniformLocationsOnSameUniformFailLink) {
const char* vsSource = R"(#version 460 core
layout(location = 2) uniform vec4 uShared;
void main() { gl_Position = uShared; }
)";
const char* fsSource = R"(#version 460 core
layout(location = 4) uniform vec4 uShared;
out vec4 fragColor;
void main() { fragColor = uShared; }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
(void)LinkVsFs(vs, fs, GL_FALSE);
}
// Same-location explicit declarations of the SAME uniform in both stages stay
// linkable, and both explicit locations (opaque and non-opaque) are honored.
TEST_F(ProgramTest, ExplicitUniformLocationsHonoredForPlainAndOpaqueUniforms) {
const char* vsSource = R"(#version 460 core
layout(location = 11) uniform mat4 uMvp;
void main() { gl_Position = uMvp * vec4(1.0); }
)";
const char* fsSource = R"(#version 460 core
layout(location = 7) uniform sampler2D uTex;
layout(location = 11) uniform mat4 uMvp;
out vec4 fragColor;
void main() { fragColor = texture(uTex, uMvp[0].xy); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_EQ(GetUniformLocation(program, "uMvp"), 11);
EXPECT_EQ(GetUniformLocation(program, "uTex"), 7);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// A glslang-auto-assigned opaque location may collide with a source-explicit plain
// uniform location under the relaxed parse (glslang no longer sees the plain
// uniform's qualifier). The assigner must relocate the auto one, not fail the link.
TEST_F(ProgramTest, AutoOpaqueLocationCollidingWithExplicitPlainLocationRelocates) {
const char* vsSource = R"(#version 460 core
layout(location = 0) uniform mat4 uM;
void main() { gl_Position = uM * vec4(1.0); }
)";
const char* fsSource = R"(#version 460 core
uniform sampler2D uTex;
out vec4 fragColor;
void main() { fragColor = texture(uTex, vec2(0.5)); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
const GLint mLoc = GetUniformLocation(program, "uM");
const GLint texLoc = GetUniformLocation(program, "uTex");
EXPECT_EQ(mLoc, 0);
ASSERT_NE(texLoc, -1);
EXPECT_NE(texLoc, mLoc);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Relinking a program and linking the same compiled shaders into a second program
// both re-consume the stored single parse (glslang mapIO mutates a linked TShader,
// so reuse goes through the consume-once re-parse path). Reflection must be intact
// every time, without any glCompileShader in between.
TEST_F(ProgramTest, RelinkAndSecondProgramReuseCompiledShaders) {
const char* vsSource = R"(#version 330 core
uniform mat4 uMvp;
in vec3 aPos;
void main() { gl_Position = uMvp * vec4(aPos, 1.0); }
)";
const char* fsSource = R"(#version 330 core
uniform sampler2D uTex;
uniform vec4 uTint;
out vec4 fragColor;
void main() { fragColor = texture(uTex, vec2(0.5)) * uTint; }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program1 = LinkVsFs(vs, fs, GL_TRUE);
GLint activeUniforms1 = 0;
GetProgramiv(program1, GL_ACTIVE_UNIFORMS, &activeUniforms1);
EXPECT_EQ(activeUniforms1, 3);
EXPECT_NE(GetUniformLocation(program1, "uMvp"), -1);
// Relink: consumes the re-parse path.
LinkProgram(program1);
GLint relinkStatus = GL_FALSE;
char infoLog[1024] = "";
GetProgramiv(program1, GL_LINK_STATUS, &relinkStatus);
GetProgramInfoLog(program1, sizeof(infoLog), nullptr, infoLog);
ASSERT_EQ(relinkStatus, GL_TRUE) << infoLog;
GLint activeUniformsRelink = 0;
GetProgramiv(program1, GL_ACTIVE_UNIFORMS, &activeUniformsRelink);
EXPECT_EQ(activeUniformsRelink, 3);
EXPECT_NE(GetUniformLocation(program1, "uTint"), -1);
// Same shaders into a fresh program.
GLuint program2 = LinkVsFs(vs, fs, GL_TRUE);
GLint activeUniforms2 = 0;
GetProgramiv(program2, GL_ACTIVE_UNIFORMS, &activeUniforms2);
EXPECT_EQ(activeUniforms2, 3);
EXPECT_NE(GetUniformLocation(program2, "uTex"), -1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Programs and shaders share one GL name space (GL 3.3 core 2.11). A name must
// never be handed out as both, and a shader name passed where a program is
// expected is INVALID_OPERATION (KHR-GL30.get_uniform_tests.get_uniform relies
// on this; a name-collided linked program used to swallow the error).
TEST_F(ProgramTest, ProgramAndShaderNamesShareOneNameSpace) {
GLuint program = CreateProgram();
GLuint vs = CreateShader(GL_VERTEX_SHADER);
GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
EXPECT_NE(program, vs);
EXPECT_NE(program, fs);
EXPECT_NE(vs, fs);
EXPECT_EQ(IsProgram(vs), GL_FALSE);
EXPECT_EQ(IsShader(program), GL_FALSE);
GLfloat floatValue = 0.0f;
GetUniformfv(vs, 0, &floatValue);
EXPECT_EQ(GetError(), static_cast<GLenum>(GL_INVALID_OPERATION));
GLint intValue = 0;
GetUniformiv(fs, 0, &intValue);
EXPECT_EQ(GetError(), static_cast<GLenum>(GL_INVALID_OPERATION));
// A never-allocated name is INVALID_VALUE, distinguishing the two cases.
GetUniformfv(program + vs + fs + 100, 0, &floatValue);
EXPECT_EQ(GetError(), static_cast<GLenum>(GL_INVALID_VALUE));
DeleteShader(vs);
DeleteShader(fs);
DeleteProgram(program);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---- builtin-shadowing OpName pass (P0c) ----
// Desktop GLSL lets a pack redefine builtins; ESSL 3.x forbids it, so the rename
// now happens as a SPIR-V OpName pass in SanitizeAndOptimizeBinary instead of the
// old whole-source string scan. These pin the pass end-to-end: real sources through
// glCompileShader/glLinkProgram, generated SPIR-V transpiled to the ESSL the Espryt
// driver would see.
namespace {
Vector<MobileGL::String> TranspileProgramSpirvToEssl(GLuint program) {
Vector<MobileGL::String> esslModules;
auto programObj = MG_State::pGLContext->GetProgramObject(program);
for (auto& spirvCode : programObj->GetGeneratedSpirv()) {
MG_Util::ShaderTranspiler::SpvcSession spvcSession(
spirvCode, MG_Util::ShaderTranspiler::SessionUsageBit::Transpile);
spvc_compiler_options options;
spvcSession.CreateOptions(&options);
spvc_compiler_options_set_uint(options, SPVC_COMPILER_OPTION_GLSL_VERSION, 320);
spvc_compiler_options_set_bool(options, SPVC_COMPILER_OPTION_GLSL_ES, SPVC_TRUE);
spvc_compiler_options_set_bool(options, SPVC_COMPILER_OPTION_GLSL_VULKAN_SEMANTICS, SPVC_FALSE);
spvcSession.SetOptions(options);
const char* result = nullptr;
spvcSession.Compile(&result);
EXPECT_NE(result, nullptr) << spvcSession.GetLastErrorString();
esslModules.push_back(result ? result : "");
}
return esslModules;
}
} // namespace
// The two blind spots of the old string scan, eliminated by construction: a
// MULTILINE definition (bliss-shaped "float fma\n(...)"), and names outside the
// old 5-entry list: sinh, as a NEW overload no builtin signature matches, so it
// parses fine and the SPIR-V OpName backstop does the rename. (An EXACT-signature
// sinh redefinition is parse-rejected by glslang - on HEAD too - and is therefore
// deliberately NOT lexically rescued; see kLexicalPreemptRenameNames.)
// min3/max3 keep their historical coverage.
TEST_F(ProgramTest, BuiltinShadowingFunctionsRenamedInEsslOutput) {
const char* vsSource = R"(#version 330 core
void main() { gl_Position = vec4(0.0, 0.0, 0.0, 1.0); }
)";
const char* fsSource = R"(#version 330 core
out vec4 fragColor;
float fma
(float a, float b, float c) { return a * b + c; }
float sinh(float x, float y) { return x * y; }
float round(float x) { return floor(x + 0.5); }
float min3(float a, float b, float c) { return min(min(a, b), c); }
void main() {
fragColor = vec4(fma(0.1, 0.2, 0.3), sinh(0.4, 2.0), round(1.25), min3(0.1, 0.2, 0.3));
}
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
for (const auto& essl : TranspileProgramSpirvToEssl(program)) {
if (essl.find("fragColor") == String::npos) continue; // fragment module only
EXPECT_NE(essl.find("mg_fma("), String::npos) << essl;
EXPECT_NE(essl.find("mg_sinh("), String::npos) << essl;
EXPECT_NE(essl.find("mg_round("), String::npos) << essl;
EXPECT_NE(essl.find("mg_min3("), String::npos) << essl;
EXPECT_EQ(essl.find("float fma("), String::npos) << essl;
EXPECT_EQ(essl.find("float sinh("), String::npos) << essl;
EXPECT_EQ(essl.find("float round("), String::npos) << essl;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Pure builtin USAGE (plus a commented-out definition) must stay untouched: builtin
// calls never resolve to a user function id in SPIR-V, so no mg_ name may appear.
TEST_F(ProgramTest, BuiltinUsageWithoutShadowingDefinitionKeepsBuiltinCalls) {
const char* vsSource = R"(#version 330 core
void main() { gl_Position = vec4(0.0, 0.0, 0.0, 1.0); }
)";
// 400, not 330: the builtin fma() really is called here, and it is only core from GLSL 4.00
// (at 330 it needs GL_ARB_gpu_shader5). The shadowing case above can stay at 330 precisely
// because the rename means no call to the builtin survives.
const char* fsSource = R"(#version 400 core
// float round(float x) { return floor(x + 0.5); }
out vec4 fragColor;
void main() {
fragColor = vec4(round(1.25), fma(0.1, 0.2, 0.3), tanh(0.5), 1.0);
}
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
for (const auto& essl : TranspileProgramSpirvToEssl(program)) {
EXPECT_EQ(essl.find("mg_"), String::npos) << essl;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---- the three shapes the lexical pre-empt pass must NOT touch (P0c) ----
// The source-level rename runs only for the handful of names glslang's relaxed
// parse rejects outright; everything else waits for the OpName pass, which cannot
// over-fire. These pin the three ways a lexical scan gets it wrong. All of them
// would fail as "no matching overloaded function found" - an over-detection is
// unrecoverable because the source never reaches SPIR-V.
namespace {
// "pow(" as a real builtin call, i.e. not the tail of "mg_pow(".
bool ContainsUnprefixedCall(const MobileGL::String& essl, const MobileGL::String& name) {
const MobileGL::String needle = name + "(";
for (SizeT pos = essl.find(needle); pos != String::npos; pos = essl.find(needle, pos + 1)) {
const char before = pos == 0 ? ' ' : essl[pos - 1];
const bool isIdentifierChar =
std::isalnum(static_cast<unsigned char>(before)) != 0 || before == '_';
if (!isIdentifierChar) return true;
}
return false;
}
} // namespace
// B1: preprocessor-asymmetric braces desync a raw brace-depth counter (each arm of
// the #ifdef closes the function), and "return" is lexically an identifier - so
// "return clamp(...)" reads as a top-level definition "<type> <builtin> (". A
// shader that shadows nothing must survive intact.
TEST_F(ProgramTest, StatementKeywordCallInPreprocessorAsymmetricBracesIsNotAShadowingDefinition) {
const char* vsSource = R"(#version 330 core
void main() { gl_Position = vec4(0.0, 0.0, 0.0, 1.0); }
)";
const char* fsSource = R"(#version 330 core
uniform vec3 uP;
out vec4 fragColor;
float getShadow(vec3 v) {
#ifdef SHADOW_OFF
return 1.0;
}
#else
return round(dot(v, v));
}
#endif
void main() { fragColor = vec4(getShadow(uP) * clamp(uP.x, 0.0, 1.0)); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
for (const auto& essl : TranspileProgramSpirvToEssl(program)) {
if (essl.find("fragColor") == String::npos) continue; // fragment module only
EXPECT_EQ(essl.find("mg_"), String::npos) << essl;
// SPIRV-Cross lowers GLSL.std.450 FClamp to its NaN-correct min/max/isnan form, so the
// surviving evidence of the builtin call is that pair, not the spelling "clamp(". The
// stronger guard is above it: a renamed mg_clamp would not have compiled at all.
EXPECT_TRUE(ContainsUnprefixedCall(essl, "min")) << essl;
EXPECT_TRUE(ContainsUnprefixedCall(essl, "max")) << essl;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// B2: the scan is preprocessor-blind, so a definition in a DEAD #if branch would
// poison every live call to the real builtin. #version 120 normalizes to 330, so
// __VERSION__ is 330 and the compat shim is dropped by glslang - the definition
// never exists, and nothing may be renamed.
TEST_F(ProgramTest, ShadowingDefinitionInDeadPreprocessorBranchLeavesLiveBuiltinCalls) {
const char* vsSource = R"(#version 120
#if __VERSION__ < 140
mat4 inverse(mat4 m) { return m; }
#endif
uniform mat4 uM;
uniform vec4 uV;
void main() { gl_Position = inverse(uM) * uV; }
)";
const char* fsSource = R"(#version 330 core
out vec4 fragColor;
void main() { fragColor = vec4(1.0); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
for (const auto& essl : TranspileProgramSpirvToEssl(program)) {
if (essl.find("gl_Position") == String::npos) continue; // vertex module only
EXPECT_EQ(essl.find("mg_"), String::npos) << essl;
EXPECT_TRUE(ContainsUnprefixedCall(essl, "inverse")) << essl;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// B3: the idiomatic reason to shadow a builtin is to ADD an overload and delegate
// to the real one. A blanket call-site rewrite would turn the body's builtin call
// into mg_pow(vec3, vec3), which has no overload. The OpName backstop renames the
// user function id only, so the delegation still resolves to GLSL.std.450 Pow.
TEST_F(ProgramTest, OverloadDelegatingToShadowedBuiltinKeepsItsBuiltinCall) {
const char* vsSource = R"(#version 330 core
void main() { gl_Position = vec4(0.0, 0.0, 0.0, 1.0); }
)";
const char* fsSource = R"(#version 330 core
uniform vec3 uBase;
out vec4 fragColor;
vec3 pow(vec3 v, float e) { return pow(v, vec3(e)); }
void main() { fragColor = vec4(pow(uBase, 2.2), 1.0); }
)";
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, vsSource);
GLuint fs = CompileShaderChecked(GL_FRAGMENT_SHADER, fsSource);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
for (const auto& essl : TranspileProgramSpirvToEssl(program)) {
if (essl.find("fragColor") == String::npos) continue; // fragment module only
EXPECT_NE(essl.find("mg_pow("), String::npos) << essl;
EXPECT_TRUE(ContainsUnprefixedCall(essl, "pow")) << essl;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------
// P0b: source-hash dedupe for shader recompiles.
// Layer 1 - the same shader object re-sourced with byte-identical text keeps its
// compiled state, and glCompileShader on it is a no-op.
// Layer 2 - two DIFFERENT shader objects holding byte-identical text share the
// source-only half of the pipeline (preprocess + lexical checks +
// side-channel extraction) through the context's ShaderPreprocessCache,
// while each still gets its own glslang parse.
// ---------------------------------------------------------------------------
namespace {
const char* kP0bVs = R"(#version 330 core
uniform mat4 uModel;
uniform vec4 uTint;
void main() { gl_Position = uModel * uTint; }
)";
const char* kP0bFs = R"(#version 330 core
uniform vec4 uColor;
out vec4 fragColor;
void main() { fragColor = uColor; }
)";
// Same stage, different declared uniform: makes "did it actually recompile?"
// observable through reflection rather than through internal state.
const char* kP0bAltFs = R"(#version 330 core
uniform vec4 uOtherColor;
out vec4 fragColor;
void main() { fragColor = uOtherColor; }
)";
const char* kP0bBrokenFs = R"(#version 330 core
out vec4 fragColor;
void main() { fragColor = notADeclaredThing; }
)";
GLuint MakeShaderWithSource(GLenum type, const char* source) {
GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
return shader;
}
GLint QueryCompileStatus(GLuint shader) {
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
String QueryShaderInfoLog(GLuint shader) {
GLint length = 0;
GetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return String();
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetShaderInfoLog(shader, length, &written, buffer.data());
return String(buffer.data(), static_cast<size_t>(written));
}
Bool ShaderHasMemoizedCompile(GLuint shader) {
const auto& shaderObject = MG_State::pGLContext->GetShaderObject(shader);
EXPECT_NE(shaderObject, nullptr);
return shaderObject != nullptr && shaderObject->HasMemoizedCompile();
}
} // namespace
// Layer 1, success path: re-sourcing with identical text and recompiling must leave
// COMPILE_STATUS, the info log and every downstream consumer exactly as they were -
// including a program that links the shader AFTER the redundant recompile.
TEST_F(ProgramTest, RecompileWithIdenticalSourceKeepsCompiledStateAndStillLinks) {
GLuint vs = MakeShaderWithSource(GL_VERTEX_SHADER, kP0bVs);
GLuint fs = MakeShaderWithSource(GL_FRAGMENT_SHADER, kP0bFs);
CompileShader(vs);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(vs), GL_TRUE) << QueryShaderInfoLog(vs);
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
const String vsLogBefore = QueryShaderInfoLog(vs);
EXPECT_TRUE(ShaderHasMemoizedCompile(vs));
// A first link consumes the stored TShader; the redundant recompile below must not
// disturb the preprocessed source that TakeShaderForLink re-parses from.
GLuint firstProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(firstProgram, "uColor"), 0);
// glShaderSource with byte-identical text, then glCompileShader: both no-ops.
ShaderSource(vs, 1, &kP0bVs, nullptr);
EXPECT_TRUE(ShaderHasMemoizedCompile(vs)) << "identical re-source must not invalidate the compiled state";
CompileShader(vs);
ShaderSource(fs, 1, &kP0bFs, nullptr);
CompileShader(fs);
EXPECT_EQ(QueryCompileStatus(vs), GL_TRUE);
EXPECT_EQ(QueryCompileStatus(fs), GL_TRUE);
EXPECT_EQ(QueryShaderInfoLog(vs), vsLogBefore);
// The original source text is still what glGetShaderSource reports.
GLint sourceLength = 0;
GetShaderiv(vs, GL_SHADER_SOURCE_LENGTH, &sourceLength);
ASSERT_GT(sourceLength, 1);
std::vector<GLchar> sourceBuffer(static_cast<size_t>(sourceLength));
GLsizei written = 0;
GetShaderSource(vs, sourceLength, &written, sourceBuffer.data());
EXPECT_EQ(String(sourceBuffer.data(), static_cast<size_t>(written)), String(kP0bVs));
// A second program built from the same, redundantly recompiled shaders links and
// reflects - i.e. TakeShaderForLink's re-parse path survived the no-op.
GLuint secondProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(secondProgram, "uColor"), 0);
EXPECT_GE(GetUniformLocation(secondProgram, "uModel"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Layer 1 must not swallow a REAL source change: different text invalidates, and the
// change is visible in what the next link reflects.
TEST_F(ProgramTest, DifferentSourceAfterCompileInvalidatesCompiledState) {
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint fs = MakeShaderWithSource(GL_FRAGMENT_SHADER, kP0bFs);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
GLuint firstProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(firstProgram, "uColor"), 0);
EXPECT_EQ(GetUniformLocation(firstProgram, "uOtherColor"), -1);
// New text -> compiled state gone, and glCompileShader is mandatory again.
ShaderSource(fs, 1, &kP0bAltFs, nullptr);
EXPECT_FALSE(ShaderHasMemoizedCompile(fs));
EXPECT_EQ(QueryCompileStatus(fs), GL_FALSE);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
GLuint secondProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(secondProgram, "uOtherColor"), 0);
EXPECT_EQ(GetUniformLocation(secondProgram, "uColor"), -1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Layer 2: byte-identical source in two distinct shader objects. Both must compile,
// and each must own an independent TShader - if the parse were shared, the second
// link would be handed an intermediate that the first link's mapIO already mutated.
TEST_F(ProgramTest, TwoShaderObjectsWithIdenticalSourceLinkIndependently) {
GLuint vsA = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint fsA = CompileShaderChecked(GL_FRAGMENT_SHADER, kP0bFs);
GLuint vsB = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint fsB = CompileShaderChecked(GL_FRAGMENT_SHADER, kP0bFs);
ASSERT_NE(vsA, vsB);
ASSERT_NE(fsA, fsB);
const auto& objectA = MG_State::pGLContext->GetShaderObject(vsA);
const auto& objectB = MG_State::pGLContext->GetShaderObject(vsB);
ASSERT_NE(objectA, nullptr);
ASSERT_NE(objectB, nullptr);
EXPECT_EQ(objectA->GetShaderSource(), objectB->GetShaderSource());
// Independent parses despite the shared preprocess.
EXPECT_NE(objectA->GetCompiledShader(), objectB->GetCompiledShader());
EXPECT_NE(objectA->GetCompiledShader(), nullptr);
EXPECT_NE(objectB->GetCompiledShader(), nullptr);
GLuint programA = LinkVsFs(vsA, fsA, GL_TRUE);
GLuint programB = LinkVsFs(vsB, fsB, GL_TRUE);
for (GLuint program : {programA, programB}) {
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
EXPECT_GE(GetUniformLocation(program, "uModel"), 0);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Failure memoization: a compile that failed stays failed, with the SAME log, when
// recompiled against the same source; a real fix to the source still takes effect.
// The second object pins the cached-ParseFailed path (layer 2), which skips the parse
// entirely and must reproduce the identical verdict.
TEST_F(ProgramTest, FailedCompileIsMemoizedAndStillRecoversOnGoodSource) {
GLuint fs = MakeShaderWithSource(GL_FRAGMENT_SHADER, kP0bBrokenFs);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_FALSE);
const String failureLog = QueryShaderInfoLog(fs);
EXPECT_FALSE(failureLog.empty());
// Layer 1: identical re-source + recompile keeps the failure AND the log queryable.
ShaderSource(fs, 1, &kP0bBrokenFs, nullptr);
CompileShader(fs);
EXPECT_EQ(QueryCompileStatus(fs), GL_FALSE);
EXPECT_EQ(QueryShaderInfoLog(fs), failureLog);
// Layer 2: a second object with the same broken source reports the same failure.
GLuint otherFs = MakeShaderWithSource(GL_FRAGMENT_SHADER, kP0bBrokenFs);
CompileShader(otherFs);
EXPECT_EQ(QueryCompileStatus(otherFs), GL_FALSE);
EXPECT_EQ(QueryShaderInfoLog(otherFs), failureLog);
// A genuine fix still compiles and links.
ShaderSource(fs, 1, &kP0bFs, nullptr);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
EXPECT_TRUE(QueryShaderInfoLog(fs).empty());
GLuint vs = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint program = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
}
// Layer 2 under eviction: push more distinct sources through the context than the
// cache can hold, then confirm nothing broke and a fresh duplicate pair still works.
TEST_F(ProgramTest, PreprocessCacheOverflowKeepsCompilingCorrectly) {
const SizeT overflow = MG_State::GLState::ShaderPreprocessCache::kMaxEntries + 8;
for (SizeT i = 0; i < overflow; ++i) {
const String source = "#version 330 core\nuniform vec4 uColor" + ToString(i) +
";\nout vec4 fragColor;\nvoid main() { fragColor = uColor" + ToString(i) + "; }\n";
const char* sourcePtr = source.c_str();
GLuint shader = MakeShaderWithSource(GL_FRAGMENT_SHADER, sourcePtr);
CompileShader(shader);
ASSERT_EQ(QueryCompileStatus(shader), GL_TRUE) << QueryShaderInfoLog(shader) << "\n" << source;
DeleteShader(shader);
}
// Everything inserted above has long since been evicted; a brand-new duplicate
// pair must still take the layer-2 path and produce two working programs.
GLuint vsA = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint fsA = CompileShaderChecked(GL_FRAGMENT_SHADER, kP0bFs);
GLuint vsB = CompileShaderChecked(GL_VERTEX_SHADER, kP0bVs);
GLuint fsB = CompileShaderChecked(GL_FRAGMENT_SHADER, kP0bFs);
GLuint programA = LinkVsFs(vsA, fsA, GL_TRUE);
GLuint programB = LinkVsFs(vsB, fsB, GL_TRUE);
EXPECT_GE(GetUniformLocation(programA, "uColor"), 0);
EXPECT_GE(GetUniformLocation(programB, "uColor"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
+450 -49
View File
@@ -20,6 +20,8 @@
#include <MG_Util/ShaderTranspiler/SpirvPasses/RenameSamplerFunctionParameterPass.h> #include <MG_Util/ShaderTranspiler/SpirvPasses/RenameSamplerFunctionParameterPass.h>
#include <MG_Util/ShaderTranspiler/Types.h> #include <MG_Util/ShaderTranspiler/Types.h>
#include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h> #include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h>
#include <MG_State/GLState/ProgramState/ShaderPreprocessCache.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <spirv-tools/libspirv.hpp> #include <spirv-tools/libspirv.hpp>
#include <spirv-tools/optimizer.hpp> #include <spirv-tools/optimizer.hpp>
@@ -615,25 +617,6 @@ void main() {
} }
} }
// The builtin-shadowing rename only fires when the shader really defines its own round/tanh/etc.
// Deciding that from a commented-out definition renames every genuine call to the builtin to a
// mg_ name that nothing defines, which fails to link.
TEST_F(ProgramUtilTest, PreprocessIgnoresCommentedOutBuiltinShadowingDefinition) {
using namespace MG_Util::ShaderTranspiler;
String source = R"(#version 330 core
// float round(float x) { return floor(x + 0.5); }
out vec4 fragColor;
void main() {
fragColor = vec4(round(1.25));
}
)";
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_NE(source.find("round(1.25)"), String::npos) << "call was renamed from a comment:\n" << source;
EXPECT_EQ(source.find("mg_round"), String::npos);
}
// A block-commented extension directive must not be treated as a real one - the int64 filter turns // A block-commented extension directive must not be treated as a real one - the int64 filter turns
// unsupported directives into #error, so reading one out of a comment manufactures a compile // unsupported directives into #error, so reading one out of a comment manufactures a compile
// failure for a shader that never asked for the extension. // failure for a shader that never asked for the extension.
@@ -844,36 +827,6 @@ void main() {
} }
} }
TEST_F(ProgramUtilTest, PreprocessFragmentShaderRenamesMin3Max3Helpers) {
using namespace MG_Util::ShaderTranspiler;
String source = R"(#version 460 core
out vec4 fragColor;
float min3(float a, float b, float c) { return min(min(a, b), c); }
float max3(float a, float b, float c) { return max(max(a, b), c); }
void main() {
float dark = min3(0.1, 0.2, 0.3);
float bright = max3(max3(0.1, 0.2, 0.3), 0.4, 0.5);
fragColor = vec4(dark, bright, 0.0, 1.0);
})";
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_NE(source.find("float mg_min3("), String::npos);
EXPECT_NE(source.find("float mg_max3("), String::npos);
EXPECT_NE(source.find("mg_min3(0.1, 0.2, 0.3)"), String::npos);
EXPECT_NE(source.find("mg_max3(mg_max3(0.1, 0.2, 0.3), 0.4, 0.5)"), String::npos);
EXPECT_EQ(source.find("float min3("), String::npos);
EXPECT_EQ(source.find("float max3("), String::npos);
ShaderAttrib attrib{.shaderType = GL_FRAGMENT_SHADER, .sourceStr = source};
auto res = ShaderCompiler::CompileShader(attrib);
if (!res) {
FAIL() << "errc: " << res.error().errc << "\nlog: " << res.error().log << "\nsource:\n" << source;
}
}
const char* vs = R"(#version 150 const char* vs = R"(#version 150
@@ -2248,3 +2201,451 @@ TEST_F(ProgramUtilTest, RewriteLinearSubgroupPrefixScanRejectsPartialOrUnsafeTem
"float other = shuffleNV(1.0f, 0u, 32u);\n "); "float other = shuffleNV(1.0f, 0u, 32u);\n ");
expectUnchanged(std::move(nvShuffleCall)); expectUnchanged(std::move(nvShuffleCall));
} }
// The LEXICAL half must fire at the source level (before the parse) for the
// preempt-list names - the end-to-end ESSL tests cannot tell which half did the
// rename, and for these names the parse would fail without the source rewrite.
TEST_F(ProgramUtilTest, PreprocessRenamesLexicalPreemptShadowingInSource) {
using namespace MG_Util::ShaderTranspiler;
String source = R"(#version 460 core
out vec4 fragColor;
float min3(float a, float b, float c) { return min(min(a, b), c); }
void main() {
fragColor = vec4(min3(0.1, 0.2, 0.3));
}
)";
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_NE(source.find("float mg_min3("), String::npos) << source;
EXPECT_NE(source.find("mg_min3(0.1, 0.2, 0.3)"), String::npos) << source;
EXPECT_EQ(source.find("float min3("), String::npos) << source;
}
// GLSL has no multi-line string or character literal, so a lone apostrophe never opens one - it is
// an English contraction, in a comment or in a diagnostic directive. MaskCommentsAndQuotedText used
// to disagree: it entered its quoted-text region on the apostrophe and, having no end-of-line rule,
// stayed there to the end of the file, blanking everything after it for every consumer of the mask
// (the tokenizer, the #version inspection, the explicit-location and opaque-binding extractors).
//
// Apostrophes inside comments were never affected - the comment region claims them first - but that
// is exactly the property the fix must not break, so pin it.
TEST_F(ProgramUtilTest, PreprocessKeepsApostrophesInsideCommentsHarmless) {
using namespace MG_Util::ShaderTranspiler;
String source = R"(#version 460 core
// don't do this: the sampler isn't bound before the first frame
/* and here's a block comment whose apostrophes shouldn't matter either */
uniform sampler2D tex;
in vec2 uv;
out vec4 fragColor;
void main() {
fragColor = texture(tex, uv);
}
)";
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_NE(source.find("void main()"), String::npos) << "shader body was blanked:\n" << source;
EXPECT_NE(source.find("fragColor = texture(tex, uv);"), String::npos) << source;
ShaderAttrib attrib{.shaderType = GL_FRAGMENT_SHADER, .sourceStr = source};
auto res = ShaderCompiler::CompileShader(attrib);
if (!res) {
FAIL() << "errc: " << res.error().errc << "\nlog: " << res.error().log << "\nsource:\n" << source;
}
}
// The case the old masker actually broke: an apostrophe in real (non-comment) text. Everything after
// it looked like string interior, so ExtractExplicitUniformLocations tokenized a blank source and
// handed the GL location assigner an empty map - the uniform silently lost its explicit location.
TEST_F(ProgramUtilTest, PreprocessApostropheInDirectiveKeepsLaterCodeVisibleToExtractors) {
using namespace MG_Util::ShaderTranspiler;
String source = R"(#version 460 core
#pragma MG_NOTE(this pack can't run without explicit locations)
layout(location = 7) uniform vec4 tint;
in vec2 uv;
out vec4 fragColor;
void main() {
fragColor = tint * uv.x;
}
)";
PreprocessShaderSource(ShaderStage::Fragment, source);
const UnorderedMap<String, Int> locations = ExtractExplicitUniformLocations(source);
ASSERT_EQ(locations.count("tint"), 1u) << "extractor went blind past the apostrophe:\n" << source;
EXPECT_EQ(locations.at("tint"), 7);
ShaderAttrib attrib{.shaderType = GL_FRAGMENT_SHADER, .sourceStr = source};
auto res = ShaderCompiler::CompileShader(attrib);
if (!res) {
FAIL() << "errc: " << res.error().errc << "\nlog: " << res.error().log << "\nsource:\n" << source;
}
}
// PreprocessShaderSource used to rediscover "where does the #version directive end?" once per
// injection - up to five whole-source masks and line scans per compile for one offset. It now takes
// the anchor once, from the pass that creates it, and tracks it.
//
// These three sources drive every consumer of that anchor: NormalizeLineDirectives (both the
// keep branch and the drop-ahead-of-#version branch), ModernizeLegacyGLSL's gl_FragColor
// injection, and InjectDepthRangeBuiltinShim's. The expected texts are the byte-exact output of
// the pre-memo implementation, captured from it - the change is pure memoization and is allowed to
// move no byte at all.
//
// Case B and case C are the two ways the anchor moves out from under the memo, and are why it is
// tracked rather than simply cached: B deletes a #line that precedes the version directive, and C
// has ModernizeLegacyGLSL's raw ReplaceIdentifier rewrite "varying"/"texture2D" inside a comment
// banner ahead of it, pulling the anchor six bytes left. An offset cached blindly would put the
// injected declaration six bytes inside the version line.
TEST_F(ProgramUtilTest, PreprocessLegacyFragmentShaderOutputIsByteStableAcrossTheVersionAnchor) {
using namespace MG_Util::ShaderTranspiler;
const char* kLegacyBody = R"(#line 30
varying vec2 uv;
uniform sampler2D tex;
void main() {
float d = gl_DepthRange.diff;
gl_FragColor = texture2D(tex, uv) * d;
}
)";
const char* kExpectedBody =
"#version 330 core /*mobilegl-normalized-legacy*/\n"
"struct mg_DepthRangeParameters { float near; float far; float diff; };\n"
"const mg_DepthRangeParameters mg_DepthRange = mg_DepthRangeParameters(0.0, 1.0, 1.0);\n"
"#define gl_DepthRange mg_DepthRange\n"
"out vec4 mg_FragColor;\n"
"#line 30\n"
"in vec2 uv;\n"
"uniform sampler2D tex;\n"
"\n"
"void main() {\n"
" float d = gl_DepthRange.diff;\n"
" mg_FragColor = texture(tex, uv) * d;\n"
"}\n";
{
SCOPED_TRACE("A: version directive at offset 0");
String source = String("#version 120\n") + kLegacyBody;
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_EQ(source, String(kExpectedBody));
ShaderAttrib attrib{.shaderType = GL_FRAGMENT_SHADER, .sourceStr = source};
auto res = ShaderCompiler::CompileShader(attrib);
if (!res) {
FAIL() << "errc: " << res.error().errc << "\nlog: " << res.error().log << "\nsource:\n" << source;
}
}
{
SCOPED_TRACE("B: a #line ahead of the version directive is dropped, shortening the prefix");
String source = String("// pack preamble\n#line 1 \"world.fsh\"\n#version 120\n") + kLegacyBody;
PreprocessShaderSource(ShaderStage::Fragment, source);
// The dropped directive leaves its newline behind, so line numbering is untouched.
EXPECT_EQ(source, String("// pack preamble\n\n") + kExpectedBody);
}
{
SCOPED_TRACE("C: a comment banner ahead of the version directive is itself rewritten");
String source = R"(/* legacy varying / texture2D helpers */
#version 120
varying vec2 uv;
uniform sampler2D tex;
void main() {
gl_FragColor = texture2D(tex, uv);
}
)";
PreprocessShaderSource(ShaderStage::Fragment, source);
EXPECT_EQ(source, String("/* legacy in / texture helpers */\n"
"#version 330 core /*mobilegl-normalized-legacy*/\n"
"out vec4 mg_FragColor;\n"
"in vec2 uv;\n"
"uniform sampler2D tex;\n"
"void main() {\n"
" mg_FragColor = texture(tex, uv);\n"
"}\n"));
ShaderAttrib attrib{.shaderType = GL_FRAGMENT_SHADER, .sourceStr = source};
auto res = ShaderCompiler::CompileShader(attrib);
if (!res) {
FAIL() << "errc: " << res.error().errc << "\nlog: " << res.error().log << "\nsource:\n" << source;
}
}
}
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// P1: CompileEnv - the compile pipeline's snapshot of everything outside
// (stage, source). These pin the two properties the rest of P1 rides on: the
// compute limits really are carried in the snapshot (an off-thread
// GL_MAX_COMPUTE_WORK_GROUP_SIZE query would silently return 0 and reject a
// legal local_size), and the fingerprint really does move when they do.
// ---------------------------------------------------------------------------
TEST_F(ProgramUtilTest, CompileEnvCarriesComputeLimitsAndFrontendMinima) {
using MobileGL::MG_Util::ShaderTranspiler::CaptureCompileEnv;
const auto env = CaptureCompileEnv();
ASSERT_NE(env, nullptr);
// With no backend the snapshot is the frontend minimum, never zero - the value an
// off-thread GetIntegeri_v would have left behind.
EXPECT_GE(env->maxComputeWorkGroupSize[0], 1024u);
EXPECT_GE(env->maxComputeWorkGroupSize[1], 1024u);
EXPECT_GE(env->maxComputeWorkGroupSize[2], 64u);
EXPECT_GE(env->maxComputeWorkGroupInvocations, 1024u);
EXPECT_NE(env->fingerprint, 0u);
}
TEST_F(ProgramUtilTest, CompileEnvFingerprintTracksEveryInput) {
using MobileGL::MG_Util::ShaderTranspiler::CompileEnv;
using MobileGL::MG_Util::ShaderTranspiler::ComputeCompileEnvFingerprint;
CompileEnv base;
const Uint64 baseline = ComputeCompileEnvFingerprint(base);
EXPECT_EQ(ComputeCompileEnvFingerprint(base), baseline) << "fingerprint must be deterministic";
// A device that allows a bigger workgroup than the frontend minimum is a DIFFERENT
// compile environment: a memo taken under the smaller limit must not be reusable.
CompileEnv biggerZ = base;
biggerZ.maxComputeWorkGroupSize[2] = 256;
EXPECT_NE(ComputeCompileEnvFingerprint(biggerZ), baseline);
CompileEnv moreInvocations = base;
moreInvocations.maxComputeWorkGroupInvocations = 2048;
EXPECT_NE(ComputeCompileEnvFingerprint(moreInvocations), baseline);
CompileEnv otherBackend = base;
otherBackend.backend = MobileGL::BackendType::DirectVulkan;
EXPECT_NE(ComputeCompileEnvFingerprint(otherBackend), baseline);
CompileEnv otherLimits = base;
otherLimits.params.MaxVertexAttribs = 31;
EXPECT_NE(ComputeCompileEnvFingerprint(otherLimits), baseline);
CompileEnv otherExtensions = base;
otherExtensions.advertisedExtensions.push_back(MobileGL::E_GL_ARB_gpu_shader_int64);
EXPECT_NE(ComputeCompileEnvFingerprint(otherExtensions), baseline);
CompileEnv otherQuirk = base;
otherQuirk.subgroupPrefixScanQuirk = MobileGL::MG_Config::QuirkOverride::ForceOn;
EXPECT_NE(ComputeCompileEnvFingerprint(otherQuirk), baseline);
}
// The no-backend fallback must stay exactly what the pipeline used to do inline:
// everything counts as advertised, because there is nothing to gate against.
TEST_F(ProgramUtilTest, CompileEnvWithoutBackendAdvertisesEverything) {
using MobileGL::MG_Util::ShaderTranspiler::CompileEnv;
CompileEnv env;
EXPECT_FALSE(env.HasBackend());
EXPECT_TRUE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
env.backend = MobileGL::BackendType::DirectGLES;
EXPECT_TRUE(env.HasBackend());
EXPECT_FALSE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
env.advertisedExtensions.push_back(MobileGL::E_GL_ARB_gpu_shader_int64);
EXPECT_TRUE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
}
// P0b layer 2: ShaderPreprocessCache, tested directly. The GL-level behaviour it
// enables is covered end to end in ProgramTest; these pin the container itself,
// where the interesting cases (hash collisions, both eviction budgets) are hard
// to provoke through glCompileShader.
// ---------------------------------------------------------------------------
namespace {
using MobileGL::MG_State::GLState::ShaderPreprocessCache;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
using MobileGL::MG_State::GLState::ShaderPreprocessResult;
using MobileGL::MG_State::GLState::ShaderPreprocessResultPtr;
// The env fingerprint every test below keys against, unless it is specifically
// exercising the fingerprint itself.
constexpr MobileGL::Uint64 kEnvA = 0x1111'2222'3333'4444ull;
constexpr MobileGL::Uint64 kEnvB = 0x5555'6666'7777'8888ull;
ShaderPreprocessResultPtr MakeResult(const String& preprocessed) {
auto result = MakeShared<ShaderPreprocessResult>();
result->outcome = ShaderPreprocessOutcome::Preprocessed;
result->preprocessedSource = preprocessed;
result->explicitUniformLocations["uMarker"] = 7;
result->explicitOpaqueBindings["sMarker"] = 3;
return result;
}
} // namespace
TEST_F(ProgramUtilTest, ShaderPreprocessCacheRoundTripsAndSeparatesStages) {
ShaderPreprocessCache cache;
const String source = "// a shader\nvoid main() {}\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA), nullptr);
cache.Insert(ShaderStage::Vertex, hash, source, kEnvA, MakeResult("vertex-preprocessed"));
const ShaderPreprocessResultPtr hit = cache.Find(ShaderStage::Vertex, hash, source, kEnvA);
ASSERT_NE(hit, nullptr);
EXPECT_TRUE(hit->Preprocessed());
EXPECT_EQ(hit->preprocessedSource, "vertex-preprocessed");
const auto uniformIt = hit->explicitUniformLocations.find("uMarker");
ASSERT_NE(uniformIt, hit->explicitUniformLocations.end());
EXPECT_EQ(uniformIt->second, 7);
const auto bindingIt = hit->explicitOpaqueBindings.find("sMarker");
ASSERT_NE(bindingIt, hit->explicitOpaqueBindings.end());
EXPECT_EQ(bindingIt->second, 3u);
// Byte-identical source, different stage: a different key, so still a miss. Two
// stages sharing one entry would hand a fragment shader a vertex preprocess.
EXPECT_EQ(cache.Find(ShaderStage::Fragment, hash, source, kEnvA), nullptr);
cache.Insert(ShaderStage::Fragment, hash, source, kEnvA, MakeResult("fragment-preprocessed"));
const ShaderPreprocessResultPtr fragmentHit = cache.Find(ShaderStage::Fragment, hash, source, kEnvA);
ASSERT_NE(fragmentHit, nullptr);
EXPECT_EQ(fragmentHit->preprocessedSource, "fragment-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA)->preprocessedSource, "vertex-preprocessed");
EXPECT_EQ(cache.GetEntryCount(), 2u);
}
TEST_F(ProgramUtilTest, ShaderPreprocessCacheMemoizesRejectionVerdictsDistinctly) {
ShaderPreprocessCache cache;
const String reservedSource = "int packed;\n";
const String localSizeSource = "layout(local_size_x = 99999) in;\n";
auto reserved = MakeShared<ShaderPreprocessResult>();
reserved->outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
reserved->infoLog = "reserved identifier";
auto localSize = MakeShared<ShaderPreprocessResult>();
localSize->outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
localSize->infoLog = "local_size too big";
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource, kEnvA,
Move(reserved));
cache.Insert(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource, kEnvA,
Move(localSize));
const ShaderPreprocessResultPtr reservedHit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource, kEnvA);
ASSERT_NE(reservedHit, nullptr);
EXPECT_FALSE(reservedHit->Preprocessed());
EXPECT_EQ(reservedHit->outcome, ShaderPreprocessOutcome::ReservedIdentifierRejected);
EXPECT_EQ(reservedHit->infoLog, "reserved identifier");
const ShaderPreprocessResultPtr localSizeHit =
cache.Find(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource, kEnvA);
ASSERT_NE(localSizeHit, nullptr);
EXPECT_EQ(localSizeHit->outcome, ShaderPreprocessOutcome::ComputeLocalSizeRejected);
EXPECT_EQ(localSizeHit->infoLog, "local_size too big");
}
// Correctness must not ride on a 64-bit hash. Feed two different sources of the same
// length under a forged, identical hash: the entry stores the full original text, so
// the impostor lookup must miss instead of returning the wrong preprocess.
TEST_F(ProgramUtilTest, ShaderPreprocessCacheRejectsForgedHashCollision) {
ShaderPreprocessCache cache;
const String real = "void main() { int a = 1; }\n";
const String impostor = "void main() { int a = 2; }\n";
ASSERT_EQ(real.length(), impostor.length());
ASSERT_NE(real, impostor);
const Uint64 forgedHash = 0xdeadbeefcafef00dull;
cache.Insert(ShaderStage::Vertex, forgedHash, real, kEnvA, MakeResult("real-preprocessed"));
ASSERT_NE(cache.Find(ShaderStage::Vertex, forgedHash, real, kEnvA), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, impostor, kEnvA), nullptr);
// The colliding newcomer wins the slot rather than being silently dropped, so it
// is the previous occupant that degrades to a miss - never a wrong hit.
cache.Insert(ShaderStage::Vertex, forgedHash, impostor, kEnvA, MakeResult("impostor-preprocessed"));
const ShaderPreprocessResultPtr impostorHit = cache.Find(ShaderStage::Vertex, forgedHash, impostor, kEnvA);
ASSERT_NE(impostorHit, nullptr);
EXPECT_EQ(impostorHit->preprocessedSource, "impostor-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, real, kEnvA), nullptr);
EXPECT_EQ(cache.GetEntryCount(), 1u);
}
// P1: the compile environment joins the key. A memo computed against one backend's
// GL_MAX_COMPUTE_WORK_GROUP_* limits must never be handed back after the environment
// changed (backend swap), which is exactly what CompileEnv::fingerprint keys on.
TEST_F(ProgramUtilTest, ShaderPreprocessCacheMissesOnChangedEnvFingerprint) {
ShaderPreprocessCache cache;
const String source = "layout(local_size_x = 512) in;\nvoid main() {}\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
cache.Insert(ShaderStage::Compute, hash, source, kEnvA, MakeResult("env-a-preprocessed"));
ASSERT_NE(cache.Find(ShaderStage::Compute, hash, source, kEnvA), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvB), nullptr);
// Both environments can coexist; neither can see the other's verdict.
cache.Insert(ShaderStage::Compute, hash, source, kEnvB, MakeResult("env-b-preprocessed"));
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvA)->preprocessedSource, "env-a-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvB)->preprocessedSource, "env-b-preprocessed");
EXPECT_EQ(cache.GetEntryCount(), 2u);
}
// A hit hands out shared ownership, so the payload survives the eviction of its entry.
// Under the old raw-pointer API this read was a use-after-free the moment two compiles
// ran concurrently.
TEST_F(ProgramUtilTest, ShaderPreprocessCacheHitOutlivesEviction) {
ShaderPreprocessCache cache;
const String source = "void main() { int keep = 1; }\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
cache.Insert(ShaderStage::Vertex, hash, source, kEnvA, MakeResult("survivor"));
const ShaderPreprocessResultPtr held = cache.Find(ShaderStage::Vertex, hash, source, kEnvA);
ASSERT_NE(held, nullptr);
cache.Clear();
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA), nullptr);
EXPECT_EQ(held->preprocessedSource, "survivor");
}
TEST_F(ProgramUtilTest, ShaderPreprocessCacheEvictsFifoOnEntryCap) {
ShaderPreprocessCache cache;
Vector<String> sources;
const SizeT overflow = ShaderPreprocessCache::kMaxEntries + 8;
for (SizeT i = 0; i < overflow; ++i) {
sources.push_back("void main() { int a = " + ToString(i) + "; }\n");
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources.back()), sources.back(), kEnvA,
MakeResult("pp" + ToString(i)));
EXPECT_LE(cache.GetEntryCount(), ShaderPreprocessCache::kMaxEntries);
}
EXPECT_EQ(cache.GetEntryCount(), ShaderPreprocessCache::kMaxEntries);
// FIFO: the first `overflow - kMaxEntries` insertions are gone, the rest resident.
for (SizeT i = 0; i < overflow; ++i) {
const ShaderPreprocessResultPtr hit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources[i]), sources[i], kEnvA);
if (i < overflow - ShaderPreprocessCache::kMaxEntries) {
EXPECT_EQ(hit, nullptr) << "entry " << i << " should have been evicted";
} else {
ASSERT_NE(hit, nullptr) << "entry " << i << " should still be resident";
EXPECT_EQ(hit->preprocessedSource, "pp" + ToString(i));
}
}
cache.Clear();
EXPECT_EQ(cache.GetEntryCount(), 0u);
EXPECT_EQ(cache.GetStoredSourceBytes(), 0u);
}
TEST_F(ProgramUtilTest, ShaderPreprocessCacheHonorsByteBudget) {
ShaderPreprocessCache cache;
// Well under the entry cap, well over the byte budget: the byte budget must be the
// one that binds, and the accounting must come back down as entries are evicted.
const SizeT chunk = ShaderPreprocessCache::kMaxStoredSourceBytes / 8;
for (SizeT i = 0; i < 24; ++i) {
String source(chunk, static_cast<char>('a' + (i % 26)));
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(source), source, kEnvA, MakeResult(""));
EXPECT_LE(cache.GetStoredSourceBytes(), ShaderPreprocessCache::kMaxStoredSourceBytes);
EXPECT_LT(cache.GetEntryCount(), ShaderPreprocessCache::kMaxEntries);
}
// A single source larger than the whole budget is refused outright: caching it
// would evict every other entry and then immediately itself.
const SizeT before = cache.GetEntryCount();
const String oversized(ShaderPreprocessCache::kMaxStoredSourceBytes + 1, 'z');
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized, kEnvA, MakeResult(""));
EXPECT_EQ(cache.GetEntryCount(), before);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized, kEnvA), nullptr);
}
+20
View File
@@ -0,0 +1,20 @@
cmake_minimum_required(VERSION 3.14)
add_executable(
JobNodeTest
JobNodeTest.cpp
)
target_include_directories(JobNodeTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
JobNodeTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
include(GoogleTest)
gtest_discover_tests(JobNodeTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
+522
View File
@@ -0,0 +1,522 @@
// MobileGL - MobileGL/MG_Test/Util/JobNodeTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include <gtest/gtest.h>
#include <chrono>
#include <stdexcept>
#include "Includes.h"
#include <Config.h>
#include <MG_Util/Async/JobNode.h>
#include <MG_Util/Async/ShaderCompilePool.h>
using namespace MobileGL;
using namespace MobileGL::MG_Util::Async;
namespace {
// Every test drives its own pool instance rather than ShaderCompilePool::Get(): the
// process-wide pool is stopped permanently by StopAndDrain (that is the teardown
// contract), so a test that drained the singleton would poison every test after it.
constexpr Uint kTestThreads = 4;
// A job whose body does exactly what the test tells it to. `ran` counts executions so
// "enqueued once, ran once" is checkable, and the optional gate lets a test hold a job
// inside its body while it inspects the node from the outside.
class TestJob final : public JobNode {
public:
explicit TestJob(std::function<void(TestJob&)> body = {}) : m_body(Move(body)) {}
std::atomic<Uint> ran{0};
std::atomic<Bool> observedCancelledInBody{false};
std::atomic<Bool> observedCancelledStateInBody{false};
protected:
void RunBody() override {
ran.fetch_add(1, std::memory_order_acq_rel);
if (m_body) m_body(*this);
observedCancelledInBody.store(IsCancellationRequested(), std::memory_order_release);
// A running body sees the request, not the outcome: the node is still Running
// until it returns, which is exactly the cooperative contract.
observedCancelledStateInBody.store(IsCancelled(), std::memory_order_release);
}
private:
std::function<void(TestJob&)> m_body;
};
// A manual gate, so a test can pin a job in Running and observe the node meanwhile.
class Gate {
public:
void Open() {
{
const std::lock_guard<std::mutex> lock(m_mutex);
m_open = true;
}
m_cv.notify_all();
}
void Wait() {
std::unique_lock<std::mutex> lock(m_mutex);
m_cv.wait(lock, [this] { return m_open; });
}
private:
std::mutex m_mutex;
std::condition_variable m_cv;
Bool m_open = false;
};
Bool WaitUntil(const std::function<Bool()>& predicate,
const std::chrono::milliseconds timeout = std::chrono::seconds(10)) {
const auto deadline = std::chrono::steady_clock::now() + timeout;
while (std::chrono::steady_clock::now() < deadline) {
if (predicate()) return true;
std::this_thread::sleep_for(std::chrono::milliseconds(1));
}
return predicate();
}
} // namespace
// ---------------------------------------------------------------------------------------
// Pool lifecycle
// ---------------------------------------------------------------------------------------
TEST(ShaderCompilePoolLifecycle, ConstructingAPoolStartsNoThreadUntilSomethingIsPosted) {
ShaderCompilePool pool(kTestThreads);
EXPECT_EQ(pool.GetThreadCount(), kTestThreads);
EXPECT_EQ(pool.GetMaxConcurrency(), kTestThreads);
// Nothing observable to assert about thread creation from here; what this pins is that
// construction is side-effect free and the pool destructs cleanly without ever running.
}
TEST(ShaderCompilePoolLifecycle, StopAndDrainIsIdempotentAndSafeOnAnUnusedPool) {
ShaderCompilePool pool(kTestThreads);
pool.StopAndDrain();
pool.StopAndDrain();
SUCCEED();
}
TEST(ShaderCompilePoolLifecycle, AStoppedPoolRunsPostedJobsInlineOnTheCallingThread) {
ShaderCompilePool pool(kTestThreads);
pool.StopAndDrain();
const auto callingThread = std::this_thread::get_id();
std::thread::id bodyThread{};
auto job = MakeShared<TestJob>([&](TestJob&) { bodyThread = std::this_thread::get_id(); });
pool.Post(job);
// Terminal by the time Post returned - the whole point of the stopped-is-synchronous
// rule: a late entry point after teardown still gets a correct result, it just gets it
// without resurrecting a worker thread.
EXPECT_TRUE(job->IsComplete());
EXPECT_EQ(job->ran.load(), 1u);
EXPECT_EQ(bodyThread, callingThread);
}
TEST(ShaderCompilePoolLifecycle, SetMaxConcurrencyIsClampedToTheThreadCount) {
ShaderCompilePool pool(kTestThreads);
pool.SetMaxConcurrency(0);
EXPECT_EQ(pool.GetMaxConcurrency(), 1u);
pool.SetMaxConcurrency(1000);
EXPECT_EQ(pool.GetMaxConcurrency(), kTestThreads);
pool.SetMaxConcurrency(2);
EXPECT_EQ(pool.GetMaxConcurrency(), 2u);
}
TEST(ShaderCompilePoolLifecycle, DetectedThreadCountIsPositive) {
EXPECT_GE(DetectShaderCompileThreadCount(), 1u);
}
TEST(ShaderCompilePoolLifecycle, AsyncIsOffByDefaultInThisStage) {
// Stage 1 ships the machinery wired to nothing. If this ever fails without the default
// constant having been deliberately flipped, something enabled async by accident.
EXPECT_EQ(MG_Config::Features.AsyncShaderCompile, MG_Config::QuirkOverride::Auto);
EXPECT_FALSE(kAsyncShaderCompileDefault);
EXPECT_FALSE(AsyncShaderCompileEnabled());
}
// ---------------------------------------------------------------------------------------
// Submit and join
// ---------------------------------------------------------------------------------------
TEST(JobNodeSubmit, PostedJobRunsOnAPoolThreadAndWaitJoinsIt) {
ShaderCompilePool pool(kTestThreads);
std::atomic<Bool> sawPoolThread{false};
auto job = MakeShared<TestJob>(
[&](TestJob&) { sawPoolThread.store(ShaderCompilePool::IsPoolThread(), std::memory_order_release); });
pool.Post(job);
job->Wait();
EXPECT_TRUE(job->IsTerminal());
EXPECT_TRUE(job->IsComplete());
EXPECT_FALSE(job->IsCancelled());
EXPECT_EQ(job->ran.load(), 1u);
EXPECT_TRUE(sawPoolThread.load());
// The joining thread is not a pool thread - the assert inside Wait() depends on it.
EXPECT_FALSE(ShaderCompilePool::IsPoolThread());
}
TEST(JobNodeSubmit, WaitOnAnAlreadyTerminalJobReturnsImmediately) {
ShaderCompilePool pool(kTestThreads);
auto job = MakeShared<TestJob>();
pool.Post(job);
job->Wait();
job->Wait();
EXPECT_EQ(job->ran.load(), 1u);
}
TEST(JobNodeSubmit, RunInlineExecutesOnTheCallingThreadWithoutAPool) {
auto job = MakeShared<TestJob>();
job->RunInline();
EXPECT_TRUE(job->IsComplete());
EXPECT_EQ(job->ran.load(), 1u);
}
TEST(JobNodeSubmit, ManyJobsAllComplete) {
constexpr Uint kJobs = 256;
ShaderCompilePool pool(kTestThreads);
Vector<SharedPtr<TestJob>> jobs;
jobs.reserve(kJobs);
std::atomic<Uint> completed{0};
for (Uint i = 0; i < kJobs; ++i) {
jobs.push_back(MakeShared<TestJob>([&](TestJob&) { completed.fetch_add(1, std::memory_order_acq_rel); }));
pool.Post(jobs.back());
}
for (const auto& job : jobs) job->Wait();
EXPECT_EQ(completed.load(), kJobs);
for (const auto& job : jobs) {
EXPECT_TRUE(job->IsComplete());
EXPECT_EQ(job->ran.load(), 1u);
}
}
TEST(JobNodeSubmit, ConcurrencyBudgetIsNeverExceeded) {
constexpr Uint kBudget = 2;
constexpr Uint kJobs = 64;
ShaderCompilePool pool(kTestThreads);
pool.SetMaxConcurrency(kBudget);
std::atomic<Uint> inFlight{0};
std::atomic<Uint> peak{0};
Vector<SharedPtr<TestJob>> jobs;
jobs.reserve(kJobs);
for (Uint i = 0; i < kJobs; ++i) {
jobs.push_back(MakeShared<TestJob>([&](TestJob&) {
const Uint current = inFlight.fetch_add(1, std::memory_order_acq_rel) + 1;
Uint observed = peak.load(std::memory_order_acquire);
while (current > observed && !peak.compare_exchange_weak(observed, current)) {
}
std::this_thread::sleep_for(std::chrono::milliseconds(1));
inFlight.fetch_sub(1, std::memory_order_acq_rel);
}));
pool.Post(jobs.back());
}
for (const auto& job : jobs) job->Wait();
// This is also the memory bound: it is what stops a 300-program pack load from putting
// 300 glslang arenas in flight at once.
EXPECT_LE(peak.load(), kBudget);
EXPECT_GE(peak.load(), 1u);
}
// ---------------------------------------------------------------------------------------
// OnTerminal and dependency ordering
// ---------------------------------------------------------------------------------------
TEST(JobNodeContinuation, OnTerminalOnAnAlreadyTerminalNodeRunsInlineBeforeItReturns) {
auto job = MakeShared<TestJob>();
job->RunInline();
ASSERT_TRUE(job->IsTerminal());
Bool ranInline = false;
const auto callingThread = std::this_thread::get_id();
std::thread::id continuationThread{};
job->OnTerminal([&] {
ranInline = true;
continuationThread = std::this_thread::get_id();
});
EXPECT_TRUE(ranInline);
EXPECT_EQ(continuationThread, callingThread);
}
TEST(JobNodeContinuation, EveryContinuationFiresExactlyOnce) {
constexpr Uint kContinuations = 8;
ShaderCompilePool pool(kTestThreads);
Gate gate;
auto job = MakeShared<TestJob>([&](TestJob&) { gate.Wait(); });
pool.Post(job);
std::atomic<Uint> fired{0};
for (Uint i = 0; i < kContinuations; ++i) {
job->OnTerminal([&] { fired.fetch_add(1, std::memory_order_acq_rel); });
}
gate.Open();
job->Wait();
// Registered while the job was pending or running, so all of them are handed to the
// finishing thread; a late one would have run inline instead. Either way: once each.
EXPECT_TRUE(WaitUntil([&] { return fired.load() == kContinuations; }));
EXPECT_EQ(fired.load(), kContinuations);
// A continuation registered after the fact still fires, exactly once, inline.
job->OnTerminal([&] { fired.fetch_add(1, std::memory_order_acq_rel); });
EXPECT_EQ(fired.load(), kContinuations + 1);
}
TEST(JobNodeContinuation, DependencyCounterReachesZeroExactlyOnceAndOnlyAfterEveryDependency) {
// The shape ProgramLinkTask::SubmitAfter uses: the dependent is posted by whichever
// thread drives the counter to zero, so it is enqueued only once every dependency is
// terminal - which is why no job body ever has to wait on another job.
constexpr Uint kDeps = 16;
ShaderCompilePool pool(kTestThreads);
Vector<SharedPtr<TestJob>> deps;
deps.reserve(kDeps);
for (Uint i = 0; i < kDeps; ++i) deps.push_back(MakeShared<TestJob>());
std::atomic<Int> remaining{static_cast<Int>(kDeps) + 1}; // +1 guard: nothing fires mid-registration
std::atomic<Uint> released{0};
std::atomic<Bool> allDepsTerminalAtRelease{false};
const auto settle = [&] {
if (remaining.fetch_sub(1, std::memory_order_acq_rel) == 1) {
Bool allTerminal = true;
for (const auto& dep : deps) allTerminal = allTerminal && dep->IsTerminal();
allDepsTerminalAtRelease.store(allTerminal, std::memory_order_release);
released.fetch_add(1, std::memory_order_acq_rel);
}
};
for (const auto& dep : deps) {
pool.Post(dep);
dep->OnTerminal(settle);
}
settle(); // release the guard
EXPECT_TRUE(WaitUntil([&] { return released.load() == 1u; }));
EXPECT_EQ(released.load(), 1u);
EXPECT_TRUE(allDepsTerminalAtRelease.load());
for (const auto& dep : deps) EXPECT_TRUE(dep->IsComplete());
}
TEST(JobNodeContinuation, AlreadyTerminalDependenciesStillSettleTheCounterExactlyOnce) {
// Same counter, but every dependency is terminal before registration, so every
// continuation runs inline on the registering thread.
constexpr Uint kDeps = 4;
Vector<SharedPtr<TestJob>> deps;
for (Uint i = 0; i < kDeps; ++i) {
deps.push_back(MakeShared<TestJob>());
deps.back()->RunInline();
}
std::atomic<Int> remaining{static_cast<Int>(kDeps) + 1};
Uint released = 0;
const auto settle = [&] {
if (remaining.fetch_sub(1, std::memory_order_acq_rel) == 1) ++released;
};
for (const auto& dep : deps) dep->OnTerminal(settle);
settle();
EXPECT_EQ(released, 1u);
}
// ---------------------------------------------------------------------------------------
// Cancellation
// ---------------------------------------------------------------------------------------
TEST(JobNodeCancel, CancelBeforeAnyDispatchSettlesTheNodeAndSkipsTheBody) {
ShaderCompilePool pool(kTestThreads);
auto job = MakeShared<TestJob>();
job->Cancel();
EXPECT_TRUE(job->IsCancelled());
EXPECT_TRUE(job->IsTerminal());
EXPECT_FALSE(job->IsComplete());
// Posting an already-cancelled node is a no-op, not a second run.
pool.Post(job);
job->Wait();
EXPECT_EQ(job->ran.load(), 0u);
EXPECT_TRUE(job->IsCancelled());
}
TEST(JobNodeCancel, CancelWhileTheBodyIsRunningLetsItFinishAndReportsCancelled) {
ShaderCompilePool pool(kTestThreads);
Gate gate;
std::atomic<Bool> entered{false};
auto job = MakeShared<TestJob>([&](TestJob&) {
entered.store(true, std::memory_order_release);
gate.Wait();
});
pool.Post(job);
ASSERT_TRUE(WaitUntil([&] { return entered.load(); }));
job->Cancel();
// A running body is not interrupted - cancellation is cooperative - so the node is
// still Running until the body returns.
EXPECT_FALSE(job->IsTerminal());
gate.Open();
job->Wait();
EXPECT_EQ(job->ran.load(), 1u);
EXPECT_TRUE(job->IsCancelled());
EXPECT_FALSE(job->IsComplete());
EXPECT_TRUE(job->observedCancelledInBody.load());
EXPECT_FALSE(job->observedCancelledStateInBody.load());
}
TEST(JobNodeCancel, CancelAfterCompletionDoesNotUndoTheResult) {
ShaderCompilePool pool(kTestThreads);
auto job = MakeShared<TestJob>();
pool.Post(job);
job->Wait();
ASSERT_TRUE(job->IsComplete());
job->Cancel();
// The request is recorded, but a settled result is never retroactively undone.
EXPECT_TRUE(job->IsCancellationRequested());
EXPECT_TRUE(job->IsComplete());
EXPECT_FALSE(job->IsCancelled());
EXPECT_EQ(job->ran.load(), 1u);
}
TEST(JobNodeCancel, CancelReleasesContinuationsSoDependentsAreNotStranded) {
auto job = MakeShared<TestJob>();
std::atomic<Uint> fired{0};
job->OnTerminal([&] { fired.fetch_add(1, std::memory_order_acq_rel); });
job->Cancel();
EXPECT_EQ(fired.load(), 1u);
job->Wait(); // must not hang: a cancelled pending node is terminal
EXPECT_TRUE(job->IsCancelled());
}
// ---------------------------------------------------------------------------------------
// Exceptions
// ---------------------------------------------------------------------------------------
TEST(JobNodeException, AnExceptionEscapingABodyCancelsTheJobInsteadOfTerminating) {
// Asio propagates an exception out of thread_pool::run(), which is std::terminate for
// the process. Containing it at the job boundary is what makes that impossible.
ShaderCompilePool pool(kTestThreads);
auto job = MakeShared<TestJob>([](TestJob&) { throw std::runtime_error("boom"); });
pool.Post(job);
job->Wait();
EXPECT_TRUE(job->IsTerminal());
EXPECT_TRUE(job->IsCancelled());
EXPECT_FALSE(job->IsComplete());
ASSERT_EQ(job->diagnostics.logLines.size(), 1u);
EXPECT_NE(job->diagnostics.logLines[0].find("boom"), String::npos);
}
TEST(JobNodeException, ANonStandardExceptionIsContainedToo) {
ShaderCompilePool pool(kTestThreads);
auto job = MakeShared<TestJob>([](TestJob&) { throw 42; });
pool.Post(job);
job->Wait();
EXPECT_TRUE(job->IsCancelled());
ASSERT_EQ(job->diagnostics.logLines.size(), 1u);
}
TEST(JobNodeException, AThrowingJobDoesNotPoisonTheWorkerForLaterJobs) {
ShaderCompilePool pool(kTestThreads);
auto thrower = MakeShared<TestJob>([](TestJob&) { throw std::runtime_error("boom"); });
pool.Post(thrower);
thrower->Wait();
auto healthy = MakeShared<TestJob>();
pool.Post(healthy);
healthy->Wait();
EXPECT_TRUE(healthy->IsComplete());
}
// ---------------------------------------------------------------------------------------
// Drain
// ---------------------------------------------------------------------------------------
TEST(ShaderCompilePoolDrain, StopAndDrainWithAThousandQueuedJobsLeavesNoneRunningOrPending) {
constexpr Uint kJobs = 1000;
ShaderCompilePool pool(kTestThreads);
pool.SetMaxConcurrency(1); // keep the vast majority queued behind the budget
Vector<SharedPtr<TestJob>> jobs;
jobs.reserve(kJobs);
for (Uint i = 0; i < kJobs; ++i) {
jobs.push_back(MakeShared<TestJob>());
pool.Post(jobs.back());
}
pool.StopAndDrain();
// Every node is terminal, so nothing can be waiting on a worker that will never come.
Uint complete = 0;
Uint cancelled = 0;
for (const auto& job : jobs) {
ASSERT_TRUE(job->IsTerminal());
if (job->IsComplete()) ++complete;
if (job->IsCancelled()) ++cancelled;
EXPECT_LE(job->ran.load(), 1u);
}
EXPECT_EQ(complete + cancelled, kJobs);
EXPECT_GT(cancelled, 0u); // the drain really did abandon queued work rather than run it
}
TEST(ShaderCompilePoolDrain, StopAndDrainWaitsForARunningBodyToReturn) {
ShaderCompilePool pool(kTestThreads);
Gate gate;
std::atomic<Bool> entered{false};
std::atomic<Bool> left{false};
auto job = MakeShared<TestJob>([&](TestJob&) {
entered.store(true, std::memory_order_release);
gate.Wait();
left.store(true, std::memory_order_release);
});
pool.Post(job);
ASSERT_TRUE(WaitUntil([&] { return entered.load(); }));
std::thread opener([&] {
std::this_thread::sleep_for(std::chrono::milliseconds(20));
gate.Open();
});
pool.StopAndDrain();
opener.join();
// This is the guarantee library teardown relies on: once StopAndDrain returns, no worker
// is still inside a body that could touch glslang's process globals.
EXPECT_TRUE(left.load());
EXPECT_TRUE(job->IsTerminal());
}
TEST(ShaderCompilePoolDrain, JobsPostedAfterADrainStillRun) {
ShaderCompilePool pool(kTestThreads);
pool.StopAndDrain();
auto job = MakeShared<TestJob>();
pool.Post(job);
EXPECT_TRUE(job->IsComplete());
EXPECT_EQ(job->ran.load(), 1u);
}
+125
View File
@@ -0,0 +1,125 @@
// MobileGL - MobileGL/MG_Util/Async/JobNode.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "JobNode.h"
#include "ShaderCompilePool.h"
namespace MobileGL::MG_Util::Async {
namespace {
Bool IsTerminalState(const JobState state) {
return state == JobState::Complete || state == JobState::Cancelled;
}
} // namespace
Bool JobNode::IsTerminal() const { return IsTerminalState(m_state.load(std::memory_order_acquire)); }
Bool JobNode::IsComplete() const { return m_state.load(std::memory_order_acquire) == JobState::Complete; }
Bool JobNode::IsCancelled() const { return m_state.load(std::memory_order_acquire) == JobState::Cancelled; }
Bool JobNode::IsCancellationRequested() const { return m_cancelled.load(std::memory_order_acquire); }
JobState JobNode::State() const { return m_state.load(std::memory_order_acquire); }
// The single place a node changes state. Keeping every transition here is what makes the
// continuation list exactly-once: the same critical section that publishes the terminal
// state also takes ownership of the callbacks, so a concurrent OnTerminal either lands in
// the list before the swap or sees the terminal state and runs inline - never neither and
// never both.
Bool JobNode::TryTransition(const JobState from, const JobState to) {
const Bool terminal = IsTerminalState(to);
Vector<std::function<void()>> continuations;
{
const std::lock_guard<std::mutex> lock(m_mutex);
if (m_state.load(std::memory_order_relaxed) != from) return false;
m_state.store(to, std::memory_order_release);
if (terminal) continuations.swap(m_continuations);
}
if (!terminal) return true;
m_cv.notify_all();
// Run continuations OUTSIDE the lock: a continuation is free to call back into this
// node (IsComplete, State) and, in the link-dependency case, to post the dependent
// job to the pool from whichever thread drove this node terminal.
for (auto& continuation : continuations) {
if (continuation) continuation();
}
return true;
}
void JobNode::Run() {
if (m_cancelled.load(std::memory_order_acquire)) {
TryTransition(JobState::Pending, JobState::Cancelled);
return;
}
// Loses to a concurrent Cancel() that already took the node terminal, and to a second
// dispatch of the same node. Either way there is nothing left to do.
if (!TryTransition(JobState::Pending, JobState::Running)) return;
try {
RunBody();
} catch (const std::exception& e) {
// Asio propagates an exception escaping a handler out of thread_pool::run(),
// which means std::terminate for the whole process. Every job boundary contains
// it and reports the job as Cancelled; the joining GL thread then sees a node
// that produced no result, which is the same shape as an abandoned node.
diagnostics.logLines.push_back(std::format("Job body threw: {}", e.what()));
TryTransition(JobState::Running, JobState::Cancelled);
return;
} catch (...) {
diagnostics.logLines.emplace_back("Job body threw a non-std exception");
TryTransition(JobState::Running, JobState::Cancelled);
return;
}
// Debug-only tripwire for the design's section 6 invariant: a compile or link body
// must not need to raise a GL error. Anything that does belongs in the GL-thread
// prologue of CompileShader_State / LinkProgram_State, next to the active-XFB relink
// rejection that already works that way.
MOBILEGL_ASSERT(diagnostics.errors.empty(),
"JobNode: a job body recorded %zu deferred GL error(s); compile and link bodies must not "
"raise GL errors (see the P1 design, section 6)",
diagnostics.errors.size());
TryTransition(JobState::Running,
m_cancelled.load(std::memory_order_acquire) ? JobState::Cancelled : JobState::Complete);
}
void JobNode::RunInline() { Run(); }
void JobNode::Wait() {
// Invariant I4, mechanically enforced: no job body ever blocks on another job, so the
// pool can never deadlock with all its workers waiting on each other.
MOBILEGL_ASSERT(!ShaderCompilePool::IsPoolThread(),
"JobNode::Wait() called from a pool thread; job dependencies must be resolved by posting "
"late (SubmitAfter), never by waiting from inside a body");
std::unique_lock<std::mutex> lock(m_mutex);
m_cv.wait(lock, [this] { return IsTerminalState(m_state.load(std::memory_order_relaxed)); });
}
void JobNode::Cancel() {
m_cancelled.store(true, std::memory_order_release);
// A node that never reached a worker settles right here. Doing this rather than
// waiting for a dispatch that may never come is what lets every cancel site
// (glShaderSource over a pending compile, glDeleteProgram, teardown) drop the node
// without a wait and without stranding a dependent link job behind it.
TryTransition(JobState::Pending, JobState::Cancelled);
}
void JobNode::OnTerminal(std::function<void()> fn) {
if (!fn) return;
{
const std::lock_guard<std::mutex> lock(m_mutex);
if (!IsTerminalState(m_state.load(std::memory_order_relaxed))) {
m_continuations.push_back(Move(fn));
return;
}
}
fn();
}
} // namespace MobileGL::MG_Util::Async
+116
View File
@@ -0,0 +1,116 @@
// MobileGL - MobileGL/MG_Util/Async/JobNode.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_Util/Types.h>
#include <MG_State/GLState/ErrorState/ErrorCode.h>
#include <MG_State/GLState/ErrorState/ErrorInfo.h>
#include <condition_variable>
namespace MobileGL::MG_Util::Async {
enum class JobState : Uint8 {
Pending, // constructed, not started; may still be sitting in a queue
Running, // a worker is inside RunBody()
Complete, // RunBody() returned normally and the node's outputs are readable
Cancelled, // abandoned before it started, cancelled mid-run, or threw
};
// A GL error a job body wants to raise. Nothing in the compile/link pipeline produces
// one today (see the design's section 6: GL defines compile/link *failure* as
// COMPILE_STATUS/LINK_STATUS plus an info log, not as a GL error, which is exactly why
// asynchronous compilation is legal at all), and JobNode::Finish asserts the vector is
// still empty in debug builds. The mechanism exists so that the day a body genuinely
// needs to raise one, the fix is to append here and let the join replay it on the GL
// thread - not to reach for pGLContext->RecordError() from a worker.
struct DeferredError {
Uint64 sequence = 0; // job-global monotonic counter, assigned at record time
ErrorCode code = ErrorCode::NoError;
UniquePtr<ErrorInfo> info;
};
struct JobDiagnostics {
Vector<DeferredError> errors; // replayed, in ascending `sequence`, by the join
Vector<String> logLines; // worker-side MGLOG text, flushed in order by the join
};
// The scheduling primitive every asynchronous compile and link is built on. A node owns
// its inputs and its outputs; a worker reads only the former and writes only the latter,
// which is what makes the "no worker touches GL-thread state" invariant structural
// rather than review-enforced.
//
// State machine, and the only legal transitions:
// Pending -> Running (a worker picked the node up)
// Pending -> Cancelled (cancelled before any worker started it)
// Running -> Complete (RunBody() returned normally)
// Running -> Cancelled (cancelled mid-run, or RunBody() threw)
// Complete and Cancelled are terminal and the node is immutable afterwards, so every
// reader that observed IsTerminal() may read the outputs without further synchronization.
class JobNode {
public:
JobNode() = default;
virtual ~JobNode() = default;
JobNode(const JobNode&) = delete;
JobNode& operator=(const JobNode&) = delete;
// Lock-free and non-blocking - safe from any thread, including a pool thread.
Bool IsTerminal() const;
Bool IsComplete() const; // Complete only; this is what backs GL_COMPLETION_STATUS_KHR
Bool IsCancelled() const; // settled AS cancelled - the outcome, not the request
JobState State() const;
// The cancellation *request*, which is what a body polls to bail out early: a
// running job stays Running until its body returns, so IsCancelled() is still false
// at that point. Kept separate from IsCancelled() precisely so the two questions
// ("should I stop?" and "did it end up cancelled?") cannot be confused.
Bool IsCancellationRequested() const;
// Blocks until the node is terminal. GL thread only: a job body that waited on
// another job could deadlock the whole pool, so this asserts it is not called from a
// pool thread. Dependencies are resolved by posting late (see ProgramLinkTask::
// SubmitAfter), never by waiting from inside a body.
void Wait();
// Cooperative and non-blocking. A node that has not started yet goes terminal
// immediately, so anything waiting on it or chained behind it is released rather
// than stranded; a running node is flagged and settles as Cancelled when its body
// returns. Because every node owns its inputs and writes only into itself, an
// abandoned node is always safe to simply drop - the caller never waits.
void Cancel();
// Runs `fn` once, when this node goes terminal. If the node is ALREADY terminal,
// `fn` runs on the calling thread before OnTerminal returns. Exactly-once in both
// directions: the callback is either handed to the finishing thread or run inline,
// never both.
void OnTerminal(std::function<void()> fn);
// Runs the body on the calling thread. The synchronous path (async disabled,
// context-less internal shaders, a pool that has been stopped) goes through here, so
// that "inline" and "on a worker" differ only in which thread executes RunBody().
void RunInline();
JobDiagnostics diagnostics;
protected:
virtual void RunBody() = 0;
private:
friend class ShaderCompilePool;
// Pool entry point: cancel check -> RunBody() (exceptions contained) -> Finish().
void Run();
Bool TryTransition(JobState from, JobState to);
mutable std::mutex m_mutex;
std::condition_variable m_cv;
std::atomic<JobState> m_state{JobState::Pending};
std::atomic<Bool> m_cancelled{false};
Vector<std::function<void()>> m_continuations;
};
} // namespace MobileGL::MG_Util::Async
@@ -0,0 +1,227 @@
// MobileGL - MobileGL/MG_Util/Async/ShaderCompilePool.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ShaderCompilePool.h"
#include <Config.h>
#include <asio/post.hpp>
#include <asio/thread_pool.hpp>
#include <cstdio>
#include <deque>
namespace MobileGL::MG_Util::Async {
namespace {
// The memory ceiling, not a throughput guess: peak RSS during a pack load scales as
// (workers x largest glslang arena), and a shaderpack stage arena is large enough
// that four concurrent ones is already as much as a phone should be asked for.
constexpr Uint kMaxAutoShaderCompileThreads = 4;
// A core counts as "big" if its cpufreq ceiling is within 15% of the fastest core's.
// On a symmetric desktop that is every core; on a big.LITTLE phone it selects the
// cluster the GL thread itself runs on.
constexpr Uint64 kBigCoreFrequencyPercent = 85;
thread_local Bool tl_isPoolThread = false;
// Mirrors DirectGLES's InProcessTeardown()/EnsureProcessTeardownSentinel(): once the
// process has entered exit(), starting a worker thread is unsafe (cross-translation
// -unit static destruction order is unspecified, and glslang's process globals may
// already be gone). The flag is latched by an atexit handler registered lazily on
// first pool use, so it is guaranteed to run before any static destructor.
Bool g_processTeardown = false;
std::once_flag g_teardownSentinelOnce;
Bool InProcessTeardown() { return g_processTeardown; }
void EnsureProcessTeardownSentinel() {
std::call_once(g_teardownSentinelOnce, [] { std::atexit(+[] { g_processTeardown = true; }); });
}
Uint64 ReadCpuMaxFrequencyKHz(const Uint cpu) {
const String path =
std::format("/sys/devices/system/cpu/cpu{}/cpufreq/cpuinfo_max_freq", cpu);
std::FILE* file = std::fopen(path.c_str(), "r");
if (file == nullptr) return 0;
unsigned long long value = 0;
const int scanned = std::fscanf(file, "%llu", &value);
std::fclose(file);
return scanned == 1 ? static_cast<Uint64>(value) : 0;
}
Uint DetectBigCoreCount() {
const Uint cpuCount = std::max(1u, std::thread::hardware_concurrency());
Vector<Uint64> frequencies;
frequencies.reserve(cpuCount);
for (Uint cpu = 0; cpu < cpuCount; ++cpu) {
const Uint64 frequency = ReadCpuMaxFrequencyKHz(cpu);
if (frequency == 0) break;
frequencies.push_back(frequency);
}
// Windows, macOS, and containers that hide the cpufreq tree land here, as does a
// partially readable tree: with no asymmetry information the honest answer is
// "every core is a big core", and the [1, 4] clamp bounds it anyway.
if (frequencies.size() != cpuCount) return cpuCount;
const Uint64 peak = *std::max_element(frequencies.begin(), frequencies.end());
const Uint64 threshold = peak * kBigCoreFrequencyPercent / 100;
Uint bigCores = 0;
for (const Uint64 frequency : frequencies) {
if (frequency >= threshold) ++bigCores;
}
return bigCores > 0 ? bigCores : cpuCount;
}
} // namespace
Bool AsyncShaderCompileEnabled() {
switch (MG_Config::Features.AsyncShaderCompile) {
case MG_Config::QuirkOverride::ForceOn: return true;
case MG_Config::QuirkOverride::ForceOff: return false;
case MG_Config::QuirkOverride::Auto: break;
}
return kAsyncShaderCompileDefault;
}
Uint DetectShaderCompileThreadCount() {
if (const Uint32 configured = MG_Config::Features.AsyncShaderCompileThreads; configured > 0) {
// An explicit request is honoured as given - it is the escape hatch for measuring
// scaling and for working around a device - so it is not squeezed into [1, 4].
return configured;
}
return std::clamp(DetectBigCoreCount(), 1u, kMaxAutoShaderCompileThreads);
}
struct ShaderCompilePool::Impl {
explicit Impl(const Uint threads) : threadCount(std::max(1u, threads)), maxConcurrency(threadCount) {}
const Uint threadCount;
std::mutex mutex;
// Created on the first dispatched Post, never in the constructor: asio::thread_pool
// spawns its threads eagerly, and a build with async off must not pay for threads it
// will never use.
UniquePtr<asio::thread_pool> pool;
std::deque<SharedPtr<JobNode>> queue;
Uint inFlight = 0;
Uint maxConcurrency;
std::atomic<Bool> stopped{false};
// Callers hold `mutex`. Hands as many queued nodes to Asio as the concurrency budget
// allows. Posting under the lock is safe and is what keeps `pool` from being moved
// out by a concurrent StopAndDrain between the decision and the dispatch: asio::post
// only enqueues, it never runs the handler on the calling thread, so it cannot
// re-enter this mutex.
void DispatchLocked() {
while (!queue.empty() && inFlight < maxConcurrency && !stopped.load(std::memory_order_acquire)) {
SharedPtr<JobNode> node = Move(queue.front());
queue.pop_front();
++inFlight;
asio::post(*pool, [this, node = Move(node)]() mutable { RunOnWorker(Move(node)); });
}
}
void RunOnWorker(SharedPtr<JobNode> node) {
tl_isPoolThread = true;
// A node that was already handed to Asio when StopAndDrain ran still arrives
// here; cancelling it first turns the dispatch into a state transition instead of
// a full compile, so the drain's join() returns promptly.
if (stopped.load(std::memory_order_acquire)) node->Cancel();
node->Run();
node.reset();
const std::lock_guard<std::mutex> lock(mutex);
--inFlight;
DispatchLocked();
}
};
ShaderCompilePool::ShaderCompilePool(const Uint threadCount) : m_impl(MakeUnique<Impl>(threadCount)) {}
ShaderCompilePool::~ShaderCompilePool() { StopAndDrain(); }
ShaderCompilePool& ShaderCompilePool::Get() {
// Leak-at-exit, like the other MobileGL singletons: a process that exits without
// eglTerminate hands the threads to the OS rather than joining them from a static
// destructor, where the rest of the library may already be gone.
static ShaderCompilePool* pool = new ShaderCompilePool(DetectShaderCompileThreadCount());
return *pool;
}
Bool ShaderCompilePool::IsPoolThread() { return tl_isPoolThread; }
Uint ShaderCompilePool::GetThreadCount() const { return m_impl->threadCount; }
Uint ShaderCompilePool::GetMaxConcurrency() const {
const std::lock_guard<std::mutex> lock(m_impl->mutex);
return m_impl->maxConcurrency;
}
void ShaderCompilePool::SetMaxConcurrency(const Uint n) {
const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
// Raising the budget releases whatever the old one was holding back.
if (m_impl->pool) m_impl->DispatchLocked();
}
void ShaderCompilePool::Post(SharedPtr<JobNode> node) {
if (!node) return;
EnsureProcessTeardownSentinel();
{
const std::lock_guard<std::mutex> lock(m_impl->mutex);
if (!m_impl->stopped.load(std::memory_order_acquire) && !InProcessTeardown()) {
if (!m_impl->pool) m_impl->pool = MakeUnique<asio::thread_pool>(m_impl->threadCount);
m_impl->queue.push_back(Move(node));
m_impl->DispatchLocked();
return;
}
}
// A stopped pool is a synchronous pool, not a black hole: the node still runs, just
// on the caller's thread. Everything downstream already handles "terminal by the time
// Post returns", because that is exactly what the inline path looks like. Run it
// outside the lock - a body, or a continuation it releases, is free to Post again.
node->RunInline();
}
void ShaderCompilePool::StopAndDrain() {
// asio::thread_pool::join() from a pool thread would deadlock on itself, and the
// whole point of this call is that the GL thread waits for the workers.
MOBILEGL_ASSERT(!IsPoolThread(), "ShaderCompilePool::StopAndDrain() called from a pool thread");
std::deque<SharedPtr<JobNode>> abandoned;
UniquePtr<asio::thread_pool> pool;
{
const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->stopped.store(true, std::memory_order_release);
abandoned.swap(m_impl->queue);
pool = Move(m_impl->pool);
}
// Queued but never dispatched: settle them so anything chained behind them is
// released rather than waiting for a worker that will never pick them up.
for (const auto& node : abandoned) {
if (node) node->Cancel();
}
if (pool) {
pool->join(); // returns once every handler already handed to Asio has finished
pool.reset();
}
const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->inFlight = 0;
// The pool stays stopped, so ShaderCompilePool::Get() keeps returning a stopped,
// synchronous pool for the rest of the process. That is deliberate for the teardown
// path this exists to serve; if a future stage wants eglTerminate followed by a fresh
// eglInitialize to get its worker threads back, the re-arm belongs in
// MobileGL::Initialize(), next to glslang::InitializeProcess().
}
} // namespace MobileGL::MG_Util::Async
@@ -0,0 +1,82 @@
// MobileGL - MobileGL/MG_Util/Async/ShaderCompilePool.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_Util/Types.h>
#include <MG_Util/Async/JobNode.h>
// This header deliberately includes NO Asio header: asio::thread_pool lives behind the pimpl
// in ShaderCompilePool.cpp. Asio stays a private implementation detail of one translation
// unit, so no consumer target (MG_Test, MG_IntegrationTest, MG_Benchmark - each with its own
// target_include_directories) needs the Asio include path, and no consumer pays its compile
// time. Do not add one here.
namespace MobileGL::MG_Util::Async {
// Stage 1 ships the whole machinery switched off: the pool is constructible and tested,
// but nothing in the GL pipeline posts to it. The flip to true happens only after the
// real-client soak in the final stage, because the riskiest part of asynchronous
// compilation is not the joins - it is that Iris and Sodium change their submission
// schedule the moment GL_KHR_parallel_shader_compile is advertised, and a recorded trace
// can never cover that path.
inline constexpr Bool kAsyncShaderCompileDefault = false;
// MOBILEGL_ASYNC_SHADER_COMPILE forces the answer either way; unset keeps the built-in
// default above. Falsy is a complete kill switch: it reverts the threading *and* (from
// the extension stage on) withdraws GL_KHR_parallel_shader_compile, so the application
// behaviour change goes with it.
Bool AsyncShaderCompileEnabled();
// min(4, big cores), where a big core is one whose cpufreq ceiling is within 15% of the
// machine maximum; the whole CPU count where that sysfs tree is absent. Clamped to [1, 4]
// because peak RSS scales as workers x largest glslang arena, and four
// Complementary-sized arenas is already the memory ceiling worth accepting on a phone.
// MOBILEGL_ASYNC_SHADER_COMPILE_THREADS overrides it outright.
Uint DetectShaderCompileThreadCount();
class ShaderCompilePool {
public:
explicit ShaderCompilePool(Uint threadCount);
~ShaderCompilePool();
ShaderCompilePool(const ShaderCompilePool&) = delete;
ShaderCompilePool& operator=(const ShaderCompilePool&) = delete;
// Process-wide pool, leak-at-exit like pGLContext. Sized by
// DetectShaderCompileThreadCount() on first use; no thread is created until the first
// Post, so a build that never enables async never starts one.
static ShaderCompilePool& Get();
// True only on a thread owned by some ShaderCompilePool. Backs the two asserts that
// hold the design's invariants up: no GL/EGL reach-back from a worker, and no job
// body waiting on another job.
static Bool IsPoolThread();
// Dispatches the node, or queues it behind the concurrency budget. A stopped pool -
// and one whose process is exiting - runs the node inline on the calling thread
// instead, so a late entry point can never resurrect worker threads.
void Post(SharedPtr<JobNode> node);
// Cancels everything still queued and joins everything already running. This is the
// one cancellation path that waits, and it must run before glslang::FinalizeProcess()
// and before pGLContext is destroyed: in-flight jobs hold their own inputs safely,
// but they share glslang's process globals, which teardown is about to free.
void StopAndDrain();
Uint GetThreadCount() const;
Uint GetMaxConcurrency() const;
// Bounded concurrency doubles as the memory bound, and is how
// glMaxShaderCompilerThreadsKHR(n) is honoured: a 300-program pack load cannot put
// 300 glslang arenas in flight at once. Clamped to [1, thread count].
void SetMaxConcurrency(Uint n);
private:
struct Impl;
UniquePtr<Impl> m_impl;
};
} // namespace MobileGL::MG_Util::Async
@@ -0,0 +1,98 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "CompileEnv.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_State/GLState/Core.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
namespace {
void HashBytes(Uint64& state, const void* data, const SizeT length) {
state = static_cast<Uint64>(XXH64(data, length, state));
}
template <typename T>
void HashValue(Uint64& state, const T& value) {
static_assert(std::is_trivially_copyable_v<T>);
HashBytes(state, &value, sizeof(T));
}
} // namespace
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env) {
Uint64 state = 0x9e3779b97f4a7c15ull;
HashValue(state, env.maxComputeWorkGroupSize[0]);
HashValue(state, env.maxComputeWorkGroupSize[1]);
HashValue(state, env.maxComputeWorkGroupSize[2]);
HashValue(state, env.maxComputeWorkGroupInvocations);
HashValue(state, env.backend);
// DynamicBackendParameters is a plain aggregate of scalars; hashing its object
// representation is deliberate - it means a new limit cannot be added without also
// changing the fingerprint, which is exactly the memo-hazard property wanted here.
HashBytes(state, &env.params, sizeof(env.params));
if (!env.advertisedExtensions.empty()) {
HashBytes(state, env.advertisedExtensions.data(),
env.advertisedExtensions.size() * sizeof(GLExtension));
}
HashValue(state, env.subgroupPrefixScanQuirk);
return state;
}
SharedPtr<const CompileEnv> CaptureCompileEnv() {
auto env = MakeShared<CompileEnv>();
const auto& activeBackend = MG_Backend::pActiveBackendObject;
if (activeBackend) {
env->backend = activeBackend->GetBackendType();
env->params = activeBackend->GetDynamicParameters();
env->advertisedExtensions = activeBackend->GetRendererInfo().RendererGLInfo.Extensions;
}
// GL_MAX_COMPUTE_WORK_GROUP_SIZE. This is a REAL driver call on DirectGLES; it must
// happen here, on the context thread, and exactly once per context. The frontend
// minimum is the floor, matching what GL_Getter reports.
// TODO: Share these exposed compute limit helpers with GL_Getter.cpp instead of duplicating the frontend minima.
constexpr Uint kFrontendMinComputeWorkGroupSizes[3] = {1024, 1024, 64};
for (Uint index = 0; index < 3; ++index) {
Int backendValue = 0;
if (MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v) {
MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, index,
&backendValue);
}
env->maxComputeWorkGroupSize[index] =
std::max(static_cast<Uint>(std::max(backendValue, 0)), kFrontendMinComputeWorkGroupSizes[index]);
}
constexpr Uint64 kFrontendMaxComputeWorkGroupInvocations = 1024;
env->maxComputeWorkGroupInvocations =
activeBackend ? std::max(static_cast<Uint64>(std::max(env->params.MaxComputeWorkGroupInvocations, 0)),
kFrontendMaxComputeWorkGroupInvocations)
: kFrontendMaxComputeWorkGroupInvocations;
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return env;
}
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv() {
// Function-local static, not a namespace-scope one: the fingerprint has to be
// computed, and this must not run before MG_Config is loaded.
static const SharedPtr<const CompileEnv> kDefault = [] {
auto env = MakeShared<CompileEnv>();
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return SharedPtr<const CompileEnv>(Move(env));
}();
return kDefault;
}
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv() {
if (MG_State::pGLContext) return MG_State::pGLContext->GetCompileEnv();
return GetDefaultCompileEnv();
}
} // namespace MobileGL::MG_Util::ShaderTranspiler
@@ -0,0 +1,81 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <Config.h>
#include <MG_Backend/BackendObject.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
// Everything the shader compile/link pipeline reads from OUTSIDE its own (stage, source)
// inputs: backend identity, backend limits, the advertised extension list, and the one
// config quirk the source rewriter branches on.
//
// Why it exists (P1): every one of those reads is a reach-back into
// MG_Backend::pActiveBackendObject / gBackendFunctionsTable, and one of them
// (GL_MAX_COMPUTE_WORK_GROUP_SIZE) is a *real driver call* that on the DirectGLES
// backend silently no-ops off the context thread - which would turn a perfectly legal
// `local_size_z` into COMPILE_STATUS=FALSE the moment compilation moved to a worker.
// Snapshotting the whole set once per context, on the GL thread, removes every
// reach-back at once and makes the pipeline a pure function of (stage, source, env).
//
// Lifetime: captured lazily on first use by GLState::GLContext::GetCompileEnv(), and
// RE-captured if the active backend object changes. Immutable once published; held by
// value/`SharedPtr<const CompileEnv>` so a worker can never observe a torn update.
//
// Memo-hazard rule: `fingerprint` hashes every member above it and is part of the P0b
// ShaderPreprocessCache key, so a memo computed against one env can never be returned
// against another. ADDING A FIELD HERE MEANS ADDING IT TO ComputeFingerprint().
struct CompileEnv {
// --- compute limits: the ONLY former real-driver read in the pipeline ---
// GL_MAX_COMPUTE_WORK_GROUP_SIZE, already max()'d with the frontend minimum.
Uint maxComputeWorkGroupSize[3] = {1024, 1024, 64};
// GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, likewise.
Uint64 maxComputeWorkGroupInvocations = 1024;
// --- backend identity + limits ---
// Unknown means "no backend was active at capture time". Every consumer keeps the
// exact no-backend fallback it had before: extensions read as advertised, limits
// read as the frontend defaults.
BackendType backend = BackendType::Unknown;
MG_Backend::DynamicBackendParameters params{}; // by value, never by reference
Vector<GLExtension> advertisedExtensions;
// --- config the source rewriter branches on ---
MG_Config::QuirkOverride subgroupPrefixScanQuirk = MG_Config::QuirkOverride::Auto;
Uint64 fingerprint = 0; // set by CaptureCompileEnv()
Bool HasBackend() const { return backend != BackendType::Unknown; }
// Matches the historical rule exactly: with no active backend every extension counts
// as advertised, because the frontend then has nothing to gate against.
Bool IsExtensionAdvertised(GLExtension extension) const {
if (!HasBackend()) return true;
return std::find(advertisedExtensions.begin(), advertisedExtensions.end(), extension) !=
advertisedExtensions.end();
}
};
// Hashes every semantically relevant member. Public so a test can assert that two
// different envs really do produce different P0b cache keys.
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env);
// GL thread only: this is where the GL_MAX_COMPUTE_WORK_GROUP_SIZE queries live now.
SharedPtr<const CompileEnv> CaptureCompileEnv();
// The env a context-less caller gets: exactly what CaptureCompileEnv() would produce
// with no active backend. Used by the unit tests that drive the transpiler directly and
// by the internal shader objects that compile before any context exists.
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv();
// The env of the current GL context, or GetDefaultCompileEnv() when there is none.
// GL thread only (it may trigger a capture). This is the compatibility shim for the
// handful of entry points that still resolve their env implicitly; the pipeline itself
// always takes an explicit `const CompileEnv&`.
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv();
} // namespace MobileGL::MG_Util::ShaderTranspiler
@@ -0,0 +1,123 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/EsslBuiltinFunctionNames.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <algorithm>
#include <string_view>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Builtin-shadowing rename: TWO tables, split by FAILURE LAYER.
//
// A desktop pack may redefine a builtin; ESSL 3.x forbids the redefinition, so every
// such helper is renamed to mg_<name>. That rename happens in two places, and which
// names belong in which place is decided by *where the failure would occur*, not by
// how thorough the table looks:
//
// - kEsslBuiltinFunctionNames (below, the full ~146-name ESSL 3.20 set plus the
// GL_AMD/EXT trinary min3/mid3/max3) drives the SPIR-V OpName backstop pass. That
// pass is safe BY CONSTRUCTION for any name: it renames function ids, and builtin
// calls are GLSL.std.450 instructions that can never resolve to a user OpFunction.
// Overloads are distinct ids (so an overload delegating to the real builtin keeps
// working), dead preprocessor branches never reach SPIR-V, and there is no lexical
// guessing to over-fire. Everything that CAN wait for the IR belongs here only.
//
// - kLexicalPreemptRenameNames (a strict handful-of-names subset) drives the source-level
// scan in ShaderSourceProcessor::RenameBuiltinShadowingFunctions. That scan exists
// for exactly one reason: glslang's relaxed parse rejects some shadowing overload
// shapes at PARSE time ("overloaded functions must have the same parameter
// precision qualifiers"), and a shadowed builtin can itself need an extension the
// declared #version does not enable (fma() at #version 330 wants
// GL_ARB_gpu_shader5) - such a shader never produces SPIR-V, so the backstop never
// sees it. Only names empirically observed to hit that parse-level rejection go
// here. A lexical scan is preprocessor-blind and cannot see overload sets, so it
// can over-fire (rename a live call whose definition sits in a dead #if branch, or
// rewrite an overload's delegating call to the real builtin) - and over-detection
// is UNRECOVERABLE, because the source never reaches the backstop. Keeping this
// table minimal keeps that exposure at its historical scope.
//
// "main" is deliberately absent from both.
inline constexpr std::string_view kEsslBuiltinFunctionNames[] = {
"EmitVertex", "EndPrimitive",
"abs", "acos", "acosh", "all", "any", "asin", "asinh", "atan", "atanh",
"atomicAdd", "atomicAnd", "atomicCompSwap", "atomicCounter",
"atomicCounterDecrement", "atomicCounterIncrement", "atomicExchange",
"atomicMax", "atomicMin", "atomicOr", "atomicXor",
"barrier", "bitCount", "bitfieldExtract", "bitfieldInsert", "bitfieldReverse",
"ceil", "clamp", "cos", "cosh", "cross",
"dFdx", "dFdy", "degrees", "determinant", "distance", "dot",
"equal", "exp", "exp2",
"faceforward", "findLSB", "findMSB", "floatBitsToInt", "floatBitsToUint",
"floor", "fma", "fract", "frexp", "fwidth",
"greaterThan", "greaterThanEqual", "groupMemoryBarrier",
"imageAtomicAdd", "imageAtomicAnd", "imageAtomicCompSwap",
"imageAtomicExchange", "imageAtomicMax", "imageAtomicMin", "imageAtomicOr",
"imageAtomicXor", "imageLoad", "imageSize", "imageStore", "imulExtended",
"intBitsToFloat", "interpolateAtCentroid", "interpolateAtOffset",
"interpolateAtSample", "inverse", "inversesqrt", "isinf", "isnan",
"ldexp", "length", "lessThan", "lessThanEqual", "log", "log2",
"matrixCompMult", "max", "max3", "memoryBarrier",
"memoryBarrierAtomicCounter", "memoryBarrierBuffer", "memoryBarrierImage",
"memoryBarrierShared", "mid3", "min", "min3", "mix", "mod", "modf",
"normalize", "not", "notEqual",
"outerProduct",
"packHalf2x16", "packSnorm2x16", "packSnorm4x8", "packUnorm2x16",
"packUnorm4x8", "pow",
"radians", "reflect", "refract", "round", "roundEven",
"sign", "sin", "sinh", "smoothstep", "sqrt", "step",
"tan", "tanh", "texelFetch", "texelFetchOffset", "texture",
"textureGather", "textureGatherOffset", "textureGatherOffsets",
"textureGrad", "textureGradOffset", "textureLod", "textureLodOffset",
"textureOffset", "textureProj", "textureProjGrad", "textureProjGradOffset",
"textureProjLod", "textureProjLodOffset", "textureProjOffset", "textureSize",
"transpose", "trunc",
"uaddCarry", "uintBitsToFloat", "umulExtended", "unpackHalf2x16",
"unpackSnorm2x16", "unpackSnorm4x8", "unpackUnorm2x16", "unpackUnorm4x8",
"usubBorrow",
};
inline bool IsEsslBuiltinFunctionName(std::string_view name) {
return std::binary_search(std::begin(kEsslBuiltinFunctionNames),
std::end(kEsslBuiltinFunctionNames), name);
}
// The parse-level subset, sorted for std::binary_search.
//
// What decides membership, measured against this glslang: a redefinition whose
// signature EXACTLY matches a builtin overload is rejected at parse time
// ("overloaded functions must have the same parameter precision qualifiers", because
// the builtin declaration carries precision qualifiers and the user's does not), so
// it never produces SPIR-V and the OpName backstop never gets a turn. A definition
// that merely ADDS an overload (a signature the builtin set does not have, e.g.
// vec3 pow(vec3, float)) parses fine and is the backstop's job. Probed across the
// full table with a float(float) redefinition, 38 names are rejected that way - so
// membership here is not "everything that could ever be rejected", it is the set
// actually seen in shipped content plus whatever the test suite pins:
// fma, tanh - the bliss shaderpack's from-scratch helpers
// round, min3, max3 - the historical string-scan list this pass replaced
// An EXACT-signature redefinition of any of the other 33 probed-rejected names
// (sinh, floor, sqrt, ...) never compiled on MobileGL HEAD either - the old
// 5-name string scan did not rescue them - so leaving them out preserves the
// status quo for that (never-working) shape while keeping the dead-#if /
// overload-delegation exposure at exactly its historical scope.
// Adding a name is not free: it buys a parse-time rescue at the cost of lexical
// over-detection risk on every shader that merely *calls* that builtin (a definition
// in a dead #if branch, or an overload delegating to the real builtin). Add one only
// with evidence that real content redefines it with a builtin-identical signature.
inline constexpr std::string_view kLexicalPreemptRenameNames[] = {
"fma", "max3", "min3", "round", "tanh",
};
inline bool IsLexicalPreemptRenameName(std::string_view name) {
return std::binary_search(std::begin(kLexicalPreemptRenameNames),
std::end(kLexicalPreemptRenameNames), name);
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -15,6 +15,7 @@
#include "SpirvPasses/EliminateFloatEqualsZeroPass.h" #include "SpirvPasses/EliminateFloatEqualsZeroPass.h"
#include "SpirvPasses/FlattenInterfaceStructPass.h" #include "SpirvPasses/FlattenInterfaceStructPass.h"
#include "SpirvPasses/RenameSamplerFunctionParameterPass.h" #include "SpirvPasses/RenameSamplerFunctionParameterPass.h"
#include "SpirvPasses/RenameBuiltinShadowingFunctionsPass.h"
#include "SpirvPasses/DecomposeWorkgroupVec3Pass.h" #include "SpirvPasses/DecomposeWorkgroupVec3Pass.h"
#include "SpirvPasses/DecoratePositionInvariantPass.h" #include "SpirvPasses/DecoratePositionInvariantPass.h"
#include "SpirvPasses/LowerDrawParametersPass.h" #include "SpirvPasses/LowerDrawParametersPass.h"
@@ -36,7 +37,10 @@
namespace MobileGL { namespace MobileGL {
namespace MG_Util { namespace MG_Util {
namespace ShaderTranspiler { namespace ShaderTranspiler {
TBuiltInResource BuildTBuiltInResource() { // `env` is the compile-time backend snapshot; null means "resolve from the live
// backend", which is what the standalone/test entry points do. The pipeline always
// passes one, so a worker never reaches pActiveBackendObject through here.
TBuiltInResource BuildTBuiltInResource(const CompileEnv* env) {
TBuiltInResource Resources{}; TBuiltInResource Resources{};
Resources.maxLights = 32; Resources.maxLights = 32;
Resources.maxClipPlanes = 6; Resources.maxClipPlanes = 6;
@@ -138,7 +142,8 @@ namespace MobileGL {
const MG_Backend::DynamicBackendParameters fallbackParameters{}; const MG_Backend::DynamicBackendParameters fallbackParameters{};
const auto& activeBackend = MG_Backend::pActiveBackendObject; const auto& activeBackend = MG_Backend::pActiveBackendObject;
const auto& dynamicParameters = const auto& dynamicParameters =
activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters; env ? env->params
: (activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters);
Resources.maxImageUnits = dynamicParameters.MaxImageUnits; Resources.maxImageUnits = dynamicParameters.MaxImageUnits;
Resources.maxCombinedImageUnitsAndFragmentOutputs = Resources.maxCombinedImageUnitsAndFragmentOutputs =
dynamicParameters.MaxImageUnits + dynamicParameters.MaxDrawBuffers; dynamicParameters.MaxImageUnits + dynamicParameters.MaxDrawBuffers;
@@ -166,7 +171,8 @@ namespace MobileGL {
// copies that could drift apart. // copies that could drift apart.
static Result<SharedPtr<glslang::TShader>> ParseShaderSource(EShLanguage lang, GLenum shaderType, static Result<SharedPtr<glslang::TShader>> ParseShaderSource(EShLanguage lang, GLenum shaderType,
const String& source, const String& source,
Flags<ShaderCompileBits> flags) { Flags<ShaderCompileBits> flags,
const CompileEnv* env) {
SharedPtr<glslang::TShader> res; SharedPtr<glslang::TShader> res;
auto& tshader = res; auto& tshader = res;
tshader = MakeShared<glslang::TShader>(lang); tshader = MakeShared<glslang::TShader>(lang);
@@ -193,7 +199,7 @@ namespace MobileGL {
tshader->setAutoMapLocations(true); tshader->setAutoMapLocations(true);
tshader->setAutoMapBindings(true); tshader->setAutoMapBindings(true);
tshader->setGlobalUniformBlockName(GLOBAL_UBO_NAME); tshader->setGlobalUniformBlockName(GLOBAL_UBO_NAME);
auto resources = BuildTBuiltInResource(); auto resources = BuildTBuiltInResource(env);
if (!tshader->parse(&resources, 460, ECoreProfile, if (!tshader->parse(&resources, 460, ECoreProfile,
/*forceDefaultVersionAndProfile: */ false, /*forceDefaultVersionAndProfile: */ false,
/*forwardCompatible: */ true, EShMsgDefault)) { /*forwardCompatible: */ true, EShMsgDefault)) {
@@ -219,7 +225,7 @@ namespace MobileGL {
} }
const String source(attrib.sourceStr); const String source(attrib.sourceStr);
auto result = ParseShaderSource(lang, shaderType, source, attrib.flags); auto result = ParseShaderSource(lang, shaderType, source, attrib.flags, attrib.env);
if (result) return result; if (result) return result;
// Legacy desktop sources are normalized to "#version 330 core" (with a marker on the // Legacy desktop sources are normalized to "#version 330 core" (with a marker on the
@@ -235,7 +241,7 @@ namespace MobileGL {
return result; return result;
} }
auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags); auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags, attrib.env);
if (!retryResult) return result; if (!retryResult) return result;
MGLOG_D("CompileShader: %s only parsed after retargeting its legacy #version to 460", MGLOG_D("CompileShader: %s only parsed after retargeting its legacy #version to 460",
@@ -310,6 +316,8 @@ namespace MobileGL {
optimizer.RegisterPass(CreateRemoveUnusedInterfaceVariablesPass()); optimizer.RegisterPass(CreateRemoveUnusedInterfaceVariablesPass());
optimizer.RegisterPass(FlattenInterfaceStructPass::CreateFlattenInterfaceStructPass()); optimizer.RegisterPass(FlattenInterfaceStructPass::CreateFlattenInterfaceStructPass());
optimizer.RegisterPass(RenameSamplerFunctionParameterPass::CreateRenameSamplerFunctionParameterPass()); optimizer.RegisterPass(RenameSamplerFunctionParameterPass::CreateRenameSamplerFunctionParameterPass());
optimizer.RegisterPass(
RenameBuiltinShadowingFunctionsPass::CreateRenameBuiltinShadowingFunctionsPass());
optimizer.RegisterPass(EliminateFloatEqualsZeroPass::CreateEliminateFloatEqualsZeroPass()); optimizer.RegisterPass(EliminateFloatEqualsZeroPass::CreateEliminateFloatEqualsZeroPass());
optimizer.RegisterPass(DecomposeWorkgroupVec3Pass::CreateDecomposeWorkgroupVec3Pass()); optimizer.RegisterPass(DecomposeWorkgroupVec3Pass::CreateDecomposeWorkgroupVec3Pass());
@@ -10,10 +10,15 @@
#include <algorithm> #include <algorithm>
#include <cctype> #include <cctype>
#include <climits>
#include <cstdlib>
#include <initializer_list> #include <initializer_list>
#include <utility> #include <utility>
#include <Config.h> #include <Config.h>
#include <MG_Backend/BackendObjects.h> #include <MG_Backend/BackendObjects.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include "EsslBuiltinFunctionNames.h"
namespace { namespace {
using MobileGL::SizeT; using MobileGL::SizeT;
@@ -28,6 +33,20 @@ namespace {
return (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || ch == '_'; return (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || ch == '_';
} }
// Return a copy of `source` with every comment and string-literal interior blanked to spaces.
//
// The passes that follow answer lexical questions ("is this identifier real code?", "where does
// the #version line end?"), so comment and literal text has to stop being visible to them - but
// it must not be *deleted*: replacing the bytes with spaces keeps every offset 1:1 with the
// original, so an edit collected against the mask applies verbatim to the source, and keeping
// newlines means glslang's diagnostics still point at the line the application wrote.
//
// It also has to be lexically stateful. A banner line such as
//
// //*** lighting pass ***
//
// contains "/*" one byte in, and a naive search for that opener treats the rest of the file as
// an unterminated comment.
MobileGL::String MaskCommentsAndQuotedText(const MobileGL::String& source) { MobileGL::String MaskCommentsAndQuotedText(const MobileGL::String& source) {
enum class Region { Code, SingleLineComment, MultiLineComment, QuotedText }; enum class Region { Code, SingleLineComment, MultiLineComment, QuotedText };
@@ -81,9 +100,17 @@ namespace {
continue; continue;
} }
if (ch != '\n' && ch != '\r') { // GLSL has no multi-line string literals, so a quote that reaches end of line was never
masked[pos] = ' '; // a literal to begin with - most likely an apostrophe in a #error or #pragma message.
// Ending the region here keeps one stray apostrophe from swallowing the rest of the file
// for every consumer of this mask: the tokenizer, the #version inspection, and the
// explicit-location / opaque-binding extractors all go blind past that point otherwise.
if (ch == '\n' || ch == '\r') {
region = Region::Code;
continue;
} }
masked[pos] = ' ';
if (escaped) { if (escaped) {
escaped = false; escaped = false;
} else if (ch == '\\') { } else if (ch == '\\') {
@@ -96,77 +123,6 @@ namespace {
return masked; return masked;
} }
// Blank out block comments in place, leaving line comments and every other byte where it is.
//
// The passes that follow scan the source as raw text, so block comments have to stop being
// visible to them - but they must not be *deleted*: replacing the bytes with spaces keeps every
// later offset valid and keeps newlines, so glslang's diagnostics still point at the line the
// application wrote. It also has to be lexically aware. A banner line such as
//
// //*** lighting pass ***
//
// contains "/*" one byte in, and a naive search for that opener treats the rest of the file as
// an unterminated comment.
void BlankBlockComments(MobileGL::String& source) {
enum class Region { Code, SingleLineComment, MultiLineComment, QuotedText };
Region region = Region::Code;
char quote = '\0';
bool escaped = false;
for (SizeT pos = 0; pos < source.size(); pos++) {
const char ch = source[pos];
const char next = pos + 1 < source.size() ? source[pos + 1] : '\0';
if (region == Region::Code) {
if (ch == '/' && next == '/') {
pos++;
region = Region::SingleLineComment;
} else if (ch == '/' && next == '*') {
source[pos] = ' ';
source[pos + 1] = ' ';
pos++;
region = Region::MultiLineComment;
} else if (ch == '"' || ch == '\'') {
quote = ch;
escaped = false;
region = Region::QuotedText;
}
continue;
}
if (region == Region::SingleLineComment) {
if (ch == '\n' || ch == '\r') region = Region::Code;
continue;
}
if (region == Region::MultiLineComment) {
if (ch == '*' && next == '/') {
source[pos] = ' ';
source[pos + 1] = ' ';
pos++;
region = Region::Code;
} else if (ch != '\n' && ch != '\r') {
source[pos] = ' ';
}
continue;
}
// GLSL has no multi-line string literals, so a quote that reaches end of line was never
// a literal to begin with - most likely an apostrophe in a #error or #pragma message.
// Ending the region here keeps one stray apostrophe from swallowing the rest of the file.
if (ch == '\n' || ch == '\r') {
region = Region::Code;
} else if (escaped) {
escaped = false;
} else if (ch == '\\') {
escaped = true;
} else if (ch == quote) {
region = Region::Code;
}
}
}
struct CodeToken { struct CodeToken {
String text; String text;
SizeT begin = 0; SizeT begin = 0;
@@ -725,7 +681,17 @@ namespace {
: "#version 460 core\n"; : "#version 460 core\n";
} }
void NormalizeVersionDirective(MobileGL::String& source, const ShaderLanguageInfo& info) { // Rewrites the #version directive and returns the offset just past it in the rewritten source -
// the anchor every later injection inserts at.
//
// The offset is returned rather than rediscovered because this function is the only place that
// knows it for free; recovering it costs a whole-source mask plus a line scan
// (FindAfterVersionDirective -> InspectShaderLanguage). Each branch below leaves the bytes
// ahead of the directive untouched apart from the BOM erase, and each replacement text is
// exactly one newline-terminated line, so the arithmetic is exact in all three cases.
SizeT NormalizeVersionDirective(MobileGL::String& source, const ShaderLanguageInfo& info) {
const SizeT bomBytes = info.hasUtf8Bom ? 3 : 0;
// A malformed #version (329, 331, bad profile, float/trailing tokens) is left exactly as the // A malformed #version (329, 331, bad profile, float/trailing tokens) is left exactly as the
// application wrote it so glslang rejects it - rewriting it to "#version 330 core" would // application wrote it so glslang rejects it - rewriting it to "#version 330 core" would
// silently legalize the CTS directive.version_* rejection cases. Still drop a leading BOM so // silently legalize the CTS directive.version_* rejection cases. Still drop a leading BOM so
@@ -734,7 +700,8 @@ namespace {
if (info.hasUtf8Bom) { if (info.hasUtf8Bom) {
source.erase(0, 3); source.erase(0, 3);
} }
return; // The directive keeps its text and only slides left by the erased BOM.
return info.versionDirectiveEnd - bomBytes;
} }
const MobileGL::String replacement = GetNormalizedVersionDirective(info); const MobileGL::String replacement = GetNormalizedVersionDirective(info);
@@ -744,16 +711,35 @@ namespace {
if (info.hasUtf8Bom) { if (info.hasUtf8Bom) {
source.erase(0, 3); source.erase(0, 3);
} }
return; // Only whitespace can precede the directive on its own line, so the replacement occupies
// the whole rest of that line and ends it.
return info.versionDirectiveStart - bomBytes + replacement.size();
} }
if (info.hasUtf8Bom) { if (info.hasUtf8Bom) {
source.erase(0, 3); source.erase(0, 3);
} }
source.insert(0, replacement); source.insert(0, replacement);
return replacement.size();
} }
bool HasSingleLineFunctionDefinition(const MobileGL::String& source, const MobileGL::String& functionName) { // Start of the physical line containing `offset`, never scanning before `lowerBound`.
SizeT FindPhysicalLineStart(const MobileGL::String& source, SizeT offset, SizeT lowerBound) {
if (offset == 0) {
return lowerBound;
}
const SizeT newline = source.rfind('\n', offset - 1);
if (newline == MobileGL::String::npos || newline + 1 < lowerBound) {
return lowerBound;
}
return newline + 1;
}
// Half-open [begin, end) byte ranges of the preprocessor directive lines, in source order.
// A directive is one logical line: a trailing backslash splices the next physical line into it.
Vector<std::pair<SizeT, SizeT>> FindDirectiveLineRanges(const MobileGL::String& source) {
Vector<std::pair<SizeT, SizeT>> ranges;
SizeT lineStart = 0; SizeT lineStart = 0;
while (lineStart < source.size()) { while (lineStart < source.size()) {
SizeT lineEnd = source.find('\n', lineStart); SizeT lineEnd = source.find('\n', lineStart);
@@ -761,71 +747,177 @@ namespace {
lineEnd = source.size(); lineEnd = source.size();
} }
SizeT functionPos = source.find(functionName, lineStart); SizeT probe = lineStart;
while (functionPos != MobileGL::String::npos && functionPos < lineEnd) {
const bool hasLeftBoundary = functionPos == 0 || !IsIdentifierChar(source[functionPos - 1]);
const SizeT functionEnd = functionPos + functionName.size();
const bool hasRightBoundary = functionEnd >= source.size() || !IsIdentifierChar(source[functionEnd]);
if (hasLeftBoundary && hasRightBoundary) {
SizeT probe = functionEnd;
while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) { while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++; probe++;
} }
if (probe < lineEnd && source[probe] == '(') { if (probe >= lineEnd || source[probe] != '#') {
const SizeT closingParen = source.find(')', probe);
if (closingParen != MobileGL::String::npos && closingParen < lineEnd) {
probe = closingParen + 1;
while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++;
}
if (probe < lineEnd && source[probe] == '{') {
return true;
}
}
}
}
functionPos = source.find(functionName, functionPos + functionName.size());
}
lineStart = lineEnd + 1; lineStart = lineEnd + 1;
}
return false;
}
void RenameFunctionInvocations(MobileGL::String& source, const MobileGL::String& from, const MobileGL::String& to) {
SizeT pos = 0;
while ((pos = source.find(from, pos)) != MobileGL::String::npos) {
const bool hasLeftBoundary = pos == 0 || !IsIdentifierChar(source[pos - 1]);
const SizeT end = pos + from.size();
const bool hasRightBoundary = end >= source.size() || !IsIdentifierChar(source[end]);
SizeT probe = end;
while (probe < source.size() && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++;
}
if (hasLeftBoundary && hasRightBoundary && probe < source.size() && source[probe] == '(') {
source.replace(pos, from.size(), to);
pos += to.size();
continue; continue;
} }
pos = end; SizeT directiveEnd = lineEnd;
while (directiveEnd < source.size()) {
// directiveEnd sits on a '\n'; a backslash immediately before it (modulo the \r of
// a CRLF file and trailing blanks) splices the following physical line in.
// The scan must not leave the physical line that directiveEnd terminates: a
// whitespace-only spliced line would otherwise let the back-scan reach the
// backslash of the PREVIOUS line and swallow one extra real line of code.
const SizeT physicalLineStart = FindPhysicalLineStart(source, directiveEnd, lineStart);
SizeT back = directiveEnd;
while (back > physicalLineStart && std::isspace(static_cast<unsigned char>(source[back - 1]))) {
back--;
}
if (back == physicalLineStart || source[back - 1] != '\\') {
break;
}
SizeT splicedEnd = source.find('\n', directiveEnd + 1);
if (splicedEnd == MobileGL::String::npos) {
splicedEnd = source.size();
}
directiveEnd = splicedEnd;
}
ranges.push_back({lineStart, directiveEnd});
lineStart = directiveEnd + 1;
}
return ranges;
}
bool IsInDirectiveLine(const Vector<std::pair<SizeT, SizeT>>& ranges, SizeT offset) {
// Ranges are disjoint and sorted, so the only candidate is the last one starting at or
// before the offset.
const auto next = std::upper_bound(ranges.begin(), ranges.end(), offset,
[](SizeT value, const std::pair<SizeT, SizeT>& range) {
return value < range.first;
});
return next != ranges.begin() && offset < std::prev(next)->second;
}
// No GLSL type name is a statement keyword, so "<keyword> <builtin> (" is never a definition -
// it is `return clamp(...)`, `else round(...)`, `do fma(...)`, a `case` label expression. The
// if/for/while/switch entries cannot precede a call in valid GLSL either (a '(' always follows
// them directly), and are listed defensively. Sorted for std::binary_search.
constexpr std::string_view kStatementKeywordsBeforeCall[] = {
"case", "do", "else", "for", "if", "return", "switch", "while",
};
bool IsStatementKeywordToken(const CodeToken& token) {
return std::binary_search(std::begin(kStatementKeywordsBeforeCall),
std::end(kStatementKeywordsBeforeCall), std::string_view(token.text));
}
// A brace counter over raw tokens is preprocessor-blind: it counts the braces of BOTH arms of
// an #ifdef, so the classic "early return inside one arm, closing brace in each arm" idiom
// desyncs it. A desynced depth turns statements into apparent top-level definitions, and an
// over-detection is unrecoverable (the source never reaches the SPIR-V backstop). A file whose
// braces do not net to zero, or whose running depth ever dips below zero, is therefore not
// trustworthy for depth-based detection at all.
bool HasBalancedBraces(const Vector<CodeToken>& tokens) {
SizeT depth = 0;
for (const CodeToken& token : tokens) {
if (token.text.size() != 1) continue;
if (token.text[0] == '{') {
depth++;
} else if (token.text[0] == '}') {
if (depth == 0) return false;
depth--;
}
}
return depth == 0;
}
// Some shader packs define their own helpers under builtin GLSL names - round(), fma(),
// min3(), tanh(). Desktop GLSL allows that shadowing; ESSL 3.x forbids the redefinition, so
// every such helper is renamed to mg_<name> together with all of its call sites.
//
// Scope is deliberately NARROW: only kLexicalPreemptRenameNames, the handful of names whose
// shadowing definitions glslang's relaxed parse rejects outright ("overloaded functions must
// have the same parameter precision qualifiers"), or which need an extension the declared
// #version does not enable (fma() at #version 330 wants GL_ARB_gpu_shader5). Those shaders
// never produce SPIR-V, so only a source-level rename can save them. Everything else is left
// to the SPIR-V OpName pass in SanitizeAndOptimizeBinary, which is safe by construction -
// see EsslBuiltinFunctionNames.h for the full failure-layer split. A lexical scan is
// preprocessor-blind and overload-blind, so widening this table trades a rescue nobody needs
// for an unrecoverable over-detection risk on every shader that merely calls the builtin.
//
// Cost: ONE tokenize for the whole job, and nothing further at all in the overwhelmingly
// common no-shadowing case. The path this replaces probed the entire source once per
// candidate name, which measured ~68% of a Complementary-scale pack's compile time.
void RenameBuiltinShadowingFunctions(MobileGL::String& source) {
const Vector<CodeToken> tokens = TokenizeCode(source);
if (tokens.size() < 3) {
return;
}
// Desynced depth -> skip the lexical half entirely and let the backstop handle whatever
// this file shadows. Missing a definition is recoverable; inventing one is not.
if (!HasBalancedBraces(tokens)) {
return;
}
const Vector<std::pair<SizeT, SizeT>> directiveRanges = FindDirectiveLineRanges(source);
// Pass A - collect the shadowed names. A definition or prototype at brace depth 0 reads
// as "<type-identifier> <builtin-name> (", which is what separates it from a call in a
// global initializer ("const float PI = radians(180.0);", where the previous token is '=').
// Token positions ignore layout, so a definition split across lines is found the same way.
Vector<MobileGL::String> shadowedNames;
SizeT braceDepth = 0;
for (SizeT i = 0; i + 1 < tokens.size(); i++) {
const CodeToken& token = tokens[i];
if (token.text.size() == 1) {
if (token.text[0] == '{') {
braceDepth++;
continue;
}
if (token.text[0] == '}') {
if (braceDepth > 0) braceDepth--;
continue;
}
}
if (braceDepth != 0 || i == 0 || tokens[i + 1].text != "(" || !IsIdentifierToken(tokens[i - 1])) {
continue;
}
// IsIdentifierToken is purely lexical, so "return"/"else"/"do"/"case" pass it. None of
// them is a return type, so "return round(x)" is a CALL, not a definition.
// A directive tail ('#endif' tokenizes to '#' + 'endif') is not a return type;
// without this, a balanced-but-desynced file could see it as one.
if (IsInDirectiveLine(directiveRanges, tokens[i - 1].begin)) {
continue;
}
if (IsStatementKeywordToken(tokens[i - 1])) {
continue;
}
// "#define FOO fma(x, y, z)" defines FOO, not fma.
if (!MobileGL::MG_Util::ShaderTranspiler::IsLexicalPreemptRenameName(token.text) ||
IsInDirectiveLine(directiveRanges, token.begin)) {
continue;
}
if (std::find(shadowedNames.begin(), shadowedNames.end(), token.text) == shadowedNames.end()) {
shadowedNames.push_back(token.text);
} }
} }
void RenameBuiltinShadowingFunction(MobileGL::String& source, const char* from, const char* to) { if (shadowedNames.empty()) {
const MobileGL::String fromName = from;
// Decide from a comment-free view. A commented-out definition is not a definition, and
// acting on one renames every genuine call to the builtin to a name nothing defines - which
// then fails to resolve. Line comments survive BlankBlockComments, so this matters.
if (!HasSingleLineFunctionDefinition(MaskCommentsAndQuotedText(source), fromName)) {
return; return;
} }
RenameFunctionInvocations(source, fromName, to); // Pass B - rename the definition, its prototypes and every call. Only a name followed by
// '(' is the function; the same spelling as a variable must keep its own identity.
// Directive lines DO participate: a macro body calling the renamed helper has to follow it.
Vector<SizeT> insertOffsets;
for (SizeT i = 0; i + 1 < tokens.size(); i++) {
if (tokens[i + 1].text != "(") {
continue;
}
if (std::find(shadowedNames.begin(), shadowedNames.end(), tokens[i].text) != shadowedNames.end()) {
insertOffsets.push_back(tokens[i].begin);
}
}
// Back to front, so each recorded offset is still valid when it is used.
for (auto offset = insertOffsets.rbegin(); offset != insertOffsets.rend(); ++offset) {
source.insert(*offset, "mg_");
}
} }
void ReplaceIdentifier(MobileGL::String& source, const MobileGL::String& from, const MobileGL::String& to) { void ReplaceIdentifier(MobileGL::String& source, const MobileGL::String& from, const MobileGL::String& to) {
@@ -843,65 +935,64 @@ namespace {
} }
} }
void RemoveDefineForIdentifier(MobileGL::String& source, const MobileGL::String& identifier) {
SizeT lineStart = 0;
while (lineStart < source.size()) {
SizeT lineEnd = source.find('\n', lineStart);
const bool hasLineBreak = lineEnd != MobileGL::String::npos;
if (!hasLineBreak) {
lineEnd = source.size();
}
SizeT probe = lineStart;
while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++;
}
if (probe < lineEnd && source[probe] == '#') {
probe++;
while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++;
}
constexpr const char* defineToken = "define";
constexpr SizeT defineLen = 6;
const bool hasDefine = probe + defineLen <= lineEnd &&
source.compare(probe, defineLen, defineToken) == 0 &&
(probe + defineLen == lineEnd ||
!IsIdentifierChar(source[probe + defineLen]));
if (hasDefine) {
probe += defineLen;
while (probe < lineEnd && std::isspace(static_cast<unsigned char>(source[probe]))) {
probe++;
}
const bool hasIdentifier = probe + identifier.size() <= lineEnd &&
source.compare(probe, identifier.size(), identifier) == 0 &&
(probe + identifier.size() == lineEnd ||
!IsIdentifierChar(source[probe + identifier.size()]));
if (hasIdentifier) {
source.erase(lineStart, lineEnd - lineStart + (hasLineBreak ? 1 : 0));
continue;
}
}
}
lineStart = lineEnd + (hasLineBreak ? 1 : 0);
}
}
SizeT FindAfterVersionDirective(const MobileGL::String& source) { SizeT FindAfterVersionDirective(const MobileGL::String& source) {
const ShaderLanguageInfo info = InspectShaderLanguage(source); const ShaderLanguageInfo info = InspectShaderLanguage(source);
return info.HasVersionDirective() ? info.versionDirectiveEnd : 0; return info.HasVersionDirective() ? info.versionDirectiveEnd : 0;
} }
// Holds the offset just past the #version directive - the anchor every injected declaration is
// inserted at - across the passes of one PreprocessShaderSource call.
//
// Four consumers want that one number, and each used to buy it with its own
// FindAfterVersionDirective, i.e. its own whole-source mask plus line scan. Taking it once and
// handing it down turns up to five InspectShaderLanguage sweeps per compile into one.
//
// It stays EXACT rather than merely cached. The memo is handed out only while the bytes ahead
// of the anchor are byte-for-byte what they were when it was taken, and that is precisely the
// condition under which a fresh FindAfterVersionDirective returns the same answer: the whole
// version line, and every line the scan looks at before reaching it, lies inside that prefix,
// so an unchanged prefix means the same directive is still found ending at the same offset.
// The guard is load-bearing, not decoration - passes really do rewrite ahead of the anchor.
// NormalizeLineDirectives deletes #line directives that precede the version line, and
// ModernizeLegacyGLSL's ReplaceIdentifier is raw text and so rewrites inside a leading comment
// banner. When the guard trips the offset is simply recomputed, which is the pre-memo behavior.
//
// The one-argument constructor is that pre-memo behavior in full, for any caller that has a
// source but no anchor to hand.
class AfterVersionAnchor {
public:
explicit AfterVersionAnchor(const MobileGL::String& source) { Recompute(source); }
AfterVersionAnchor(const MobileGL::String& source, SizeT offset) { Adopt(source, offset); }
SizeT Get(const MobileGL::String& source) {
if (source.size() < m_offset || source.compare(0, m_offset, m_prefix) != 0) {
Recompute(source);
}
return m_offset;
}
private:
void Recompute(const MobileGL::String& source) { Adopt(source, FindAfterVersionDirective(source)); }
void Adopt(const MobileGL::String& source, SizeT offset) {
m_offset = offset;
m_prefix.assign(source, 0, offset);
}
SizeT m_offset = 0;
MobileGL::String m_prefix;
};
// GLSL's #line takes integer expressions only, but plenty of shader-pack preprocessors emit the // GLSL's #line takes integer expressions only, but plenty of shader-pack preprocessors emit the
// C form with a quoted filename. Deleting every #line outright made those harmless - at the cost // C form with a quoted filename. Deleting every #line outright made those harmless - at the cost
// of __LINE__ reporting the position in MobileGL's rewritten text rather than the one the pack // of __LINE__ reporting the position in MobileGL's rewritten text rather than the one the pack
// author wrote, and of every later diagnostic pointing at the wrong line. Dropping just the // author wrote, and of every later diagnostic pointing at the wrong line. Dropping just the
// quoted operand keeps the directive doing its job and still hands glslang something it accepts. // quoted operand keeps the directive doing its job and still hands glslang something it accepts.
void NormalizeLineDirectives(MobileGL::String& source) { //
// `versionEnd` is the after-version anchor for the current `source` (AfterVersionAnchor::Get);
// this pass only reads the source ahead of its own rewrites, so the plain offset is enough.
void NormalizeLineDirectives(MobileGL::String& source, SizeT versionEnd) {
const MobileGL::String masked = MaskCommentsAndQuotedText(source); const MobileGL::String masked = MaskCommentsAndQuotedText(source);
const SizeT versionEnd = FindAfterVersionDirective(source);
MobileGL::String result; MobileGL::String result;
result.reserve(source.size()); result.reserve(source.size());
@@ -946,15 +1037,6 @@ namespace {
source = std::move(result); source = std::move(result);
} }
bool IsExtensionAdvertised(MobileGL::GLExtension extension) {
const auto& activeBackendObject = MobileGL::MG_Backend::pActiveBackendObject;
if (!activeBackendObject) {
return true;
}
const auto& extensions = activeBackendObject->GetRendererInfo().RendererGLInfo.Extensions;
return std::find(extensions.begin(), extensions.end(), extension) != extensions.end();
}
MobileGL::String TrimDirectiveToken(const MobileGL::String& token) { MobileGL::String TrimDirectiveToken(const MobileGL::String& token) {
SizeT start = 0; SizeT start = 0;
@@ -969,8 +1051,9 @@ namespace {
return token.substr(start, end - start); return token.substr(start, end - start);
} }
void FilterUnsupportedGpuShaderInt64(MobileGL::String& source) { void FilterUnsupportedGpuShaderInt64(const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env,
if (IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) { MobileGL::String& source) {
if (env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) {
return; return;
} }
@@ -1118,7 +1201,12 @@ namespace {
} }
} }
void ModernizeLegacyGLSL(MobileGL::ShaderStage stage, MobileGL::String& source) { // `afterVersion` tracks the anchor the two injections below insert at. It is passed as the
// tracker rather than a bare offset because this pass rewrites identifiers first, and those
// rewrites are raw text: a leading comment banner mentioning `varying` or `texture2D` moves the
// anchor, and the tracker notices.
void ModernizeLegacyGLSL(MobileGL::ShaderStage stage, MobileGL::String& source,
AfterVersionAnchor& afterVersion) {
// Precision qualifiers (highp/mediump/lowp and default-precision statements) are legal and // Precision qualifiers (highp/mediump/lowp and default-precision statements) are legal and
// ignored in the normalized desktop core profiles, so glslang handles them natively. // ignored in the normalized desktop core profiles, so glslang handles them natively.
@@ -1139,16 +1227,17 @@ namespace {
const bool usesFragData = source.find("gl_FragData") != MobileGL::String::npos; const bool usesFragData = source.find("gl_FragData") != MobileGL::String::npos;
if (usesFragColor) { if (usesFragColor) {
ReplaceIdentifier(source, "gl_FragColor", "mg_FragColor"); ReplaceIdentifier(source, "gl_FragColor", "mg_FragColor");
source.insert(FindAfterVersionDirective(source), "out vec4 mg_FragColor;\n"); source.insert(afterVersion.Get(source), "out vec4 mg_FragColor;\n");
} }
if (usesFragData) { if (usesFragData) {
ReplaceIdentifier(source, "gl_FragData", "mg_FragData"); ReplaceIdentifier(source, "gl_FragData", "mg_FragData");
source.insert(FindAfterVersionDirective(source), "layout(location = 0) out vec4 mg_FragData[8];\n"); source.insert(afterVersion.Get(source), "layout(location = 0) out vec4 mg_FragData[8];\n");
} }
} }
} }
void InjectDepthRangeBuiltinShim(MobileGL::ShaderStage stage, MobileGL::String& source) { void InjectDepthRangeBuiltinShim(MobileGL::ShaderStage stage, MobileGL::String& source,
AfterVersionAnchor& afterVersion) {
if (stage != MobileGL::ShaderStage::Fragment) return; if (stage != MobileGL::ShaderStage::Fragment) return;
if (source.find("gl_DepthRange") == MobileGL::String::npos) return; if (source.find("gl_DepthRange") == MobileGL::String::npos) return;
if (source.find("mg_DepthRangeParameters") != MobileGL::String::npos) return; if (source.find("mg_DepthRangeParameters") != MobileGL::String::npos) return;
@@ -1157,7 +1246,7 @@ namespace {
"struct mg_DepthRangeParameters { float near; float far; float diff; };\n" "struct mg_DepthRangeParameters { float near; float far; float diff; };\n"
"const mg_DepthRangeParameters mg_DepthRange = mg_DepthRangeParameters(0.0, 1.0, 1.0);\n" "const mg_DepthRangeParameters mg_DepthRange = mg_DepthRangeParameters(0.0, 1.0, 1.0);\n"
"#define gl_DepthRange mg_DepthRange\n"; "#define gl_DepthRange mg_DepthRange\n";
source.insert(FindAfterVersionDirective(source), shim); source.insert(afterVersion.Get(source), shim);
} }
} // namespace } // namespace
@@ -1218,7 +1307,9 @@ namespace MobileGL {
// instead of open-coding them in PreprocessShaderSource. // instead of open-coding them in PreprocessShaderSource.
struct ShaderSourceQuirk { struct ShaderSourceQuirk {
const char* name; const char* name;
MG_Config::QuirkOverride (*GetOverride)(); // Reads the override out of the captured env, never out of the live
// MG_Config table: a worker must see the same config the GL thread saw.
MG_Config::QuirkOverride (*GetOverride)(const CompileEnv&);
Bool (*DeviceApplies)(const ShaderSourceQuirkContext&); Bool (*DeviceApplies)(const ShaderSourceQuirkContext&);
Bool (*Apply)(const ShaderSourceQuirkContext&, String&); Bool (*Apply)(const ShaderSourceQuirkContext&, String&);
}; };
@@ -1227,7 +1318,7 @@ namespace MobileGL {
{ {
// MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN // MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN
"subgroup-prefix-scan-rewrite", "subgroup-prefix-scan-rewrite",
[] { return MG_Config::Features.SubgroupPrefixScanQuirk; }, [](const CompileEnv& env) { return env.subgroupPrefixScanQuirk; },
[](const ShaderSourceQuirkContext& ctx) { [](const ShaderSourceQuirkContext& ctx) {
// Qualcomm's Vulkan driver miscompiles the recognized float // Qualcomm's Vulkan driver miscompiles the recognized float
// InclusiveScan pattern for native subgroups wider than the // InclusiveScan pattern for native subgroups wider than the
@@ -1243,20 +1334,21 @@ namespace MobileGL {
}, },
}; };
void ApplyShaderSourceQuirks(ShaderStage stage, String& source) { void ApplyShaderSourceQuirks(const CompileEnv& env, ShaderStage stage, String& source) {
const auto& activeBackend = MG_Backend::pActiveBackendObject; // No backend at capture time means no device to match a quirk against,
if (!activeBackend) { // and (as before) no quirk can fire - not even a forced one, because
// every Apply reads device parameters that do not exist yet.
if (!env.HasBackend()) {
return; return;
} }
const auto& dynamicParameters = activeBackend->GetDynamicParameters();
const ShaderSourceQuirkContext quirkContext{ const ShaderSourceQuirkContext quirkContext{
stage, stage,
activeBackend->GetBackendType(), env.backend,
dynamicParameters.GpuVendor, env.params.GpuVendor,
dynamicParameters.SubgroupSize, env.params.SubgroupSize,
}; };
for (const ShaderSourceQuirk& quirk : kShaderSourceQuirks) { for (const ShaderSourceQuirk& quirk : kShaderSourceQuirks) {
const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride(); const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride(env);
if (quirkOverride == MG_Config::QuirkOverride::ForceOff) { if (quirkOverride == MG_Config::QuirkOverride::ForceOff) {
continue; continue;
} }
@@ -1273,10 +1365,18 @@ namespace MobileGL {
} // namespace } // namespace
void PreprocessShaderSource(ShaderStage stage, String& source) { void PreprocessShaderSource(ShaderStage stage, String& source) {
// Normalize while the inspector's source span still refers to the untouched input. Later passes PreprocessShaderSource(stage, source, *GetCurrentCompileEnv());
// remove comments and directives, so any subsequent insertion re-inspects the current source. }
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env) {
// Normalize while the inspector's source span still refers to the untouched input.
const ShaderLanguageInfo originalLanguage = InspectShaderLanguage(source); const ShaderLanguageInfo originalLanguage = InspectShaderLanguage(source);
NormalizeVersionDirective(source, originalLanguage);
// Four passes below inject just past the #version directive, and each of them used
// to locate that anchor for itself - a whole-source mask plus line scan apiece, up
// to five per compile for one offset. NormalizeVersionDirective hands back the
// anchor it just created and the tracker keeps it honest from there.
AfterVersionAnchor afterVersion(source, NormalizeVersionDirective(source, originalLanguage));
// Comments are left intact for glslang's own preprocessor: a block comment is a single // Comments are left intact for glslang's own preprocessor: a block comment is a single
// preprocessing token that collapses to one space even across newlines and inside a // preprocessing token that collapses to one space even across newlines and inside a
@@ -1285,7 +1385,7 @@ namespace MobileGL {
// preprocessor multiline_comment_define / redefine_object / function_redefinition). // preprocessor multiline_comment_define / redefine_object / function_redefinition).
// Every MobileGL pass that must ignore comment/string text already masks them locally // Every MobileGL pass that must ignore comment/string text already masks them locally
// via MaskCommentsAndQuotedText/TokenizeCode, so the source we hand glslang keeps them. // via MaskCommentsAndQuotedText/TokenizeCode, so the source we hand glslang keeps them.
NormalizeLineDirectives(source); NormalizeLineDirectives(source, afterVersion.Get(source));
// noperspective is intentionally NOT touched here. It is core in desktop GLSL (1.30+) // noperspective is intentionally NOT touched here. It is core in desktop GLSL (1.30+)
// and maps to the core SPIR-V NoPerspective decoration, which DirectVulkan renders // and maps to the core SPIR-V NoPerspective decoration, which DirectVulkan renders
@@ -1295,20 +1395,15 @@ namespace MobileGL {
// identifier that merely contained the word. The GLES fallback for devices without // identifier that merely contained the word. The GLES fallback for devices without
// the extension lives in the backend, where device capabilities are known. // the extension lives in the backend, where device capabilities are known.
FilterUnsupportedGpuShaderInt64(source); FilterUnsupportedGpuShaderInt64(env, source);
CoerceUniformBlockPackingToStd140(source); CoerceUniformBlockPackingToStd140(source);
// Some shader packs define helpers with built-in GLSL names such as round(), tanh(), or fma(). RenameBuiltinShadowingFunctions(source);
// These may pass OpenGL-style validation but fail when recompiled for Vulkan/SPIR-V generation.
RenameBuiltinShadowingFunction(source, "round", "mg_round");
RenameBuiltinShadowingFunction(source, "tanh", "mg_tanh");
RenameBuiltinShadowingFunction(source, "fma", "mg_fma");
RenameBuiltinShadowingFunction(source, "min3", "mg_min3");
RenameBuiltinShadowingFunction(source, "max3", "mg_max3");
ModernizeLegacyGLSL(stage, source);
InjectDepthRangeBuiltinShim(stage, source);
ApplyShaderSourceQuirks(stage, source); ModernizeLegacyGLSL(stage, source, afterVersion);
InjectDepthRangeBuiltinShim(stage, source, afterVersion);
ApplyShaderSourceQuirks(env, stage, source);
} }
Bool RetargetLegacyVersionDirectiveTo460(String& source) { Bool RetargetLegacyVersionDirectiveTo460(String& source) {
@@ -1441,6 +1536,283 @@ namespace MobileGL {
return std::nullopt; return std::nullopt;
} }
namespace {
bool IsNonLayoutQualifierKeyword(const String& text) {
static const char* kQualifiers[] = {
"highp", "mediump", "lowp", "precise", "const", "flat",
"noperspective", "smooth", "centroid", "sample", "patch", "invariant",
"coherent", "volatile", "restrict", "readonly", "writeonly", "subroutine",
};
for (const char* qualifier : kQualifiers) {
if (text == qualifier) return true;
}
return false;
}
bool IsDecimalIntegerToken(const String& text) {
if (text.empty()) return false;
return std::all_of(text.begin(), text.end(),
[](char ch) { return ch >= '0' && ch <= '9'; });
}
// Parses one brace-free depth-0 statement [begin, end) and records its
// declarators when it is a uniform declaration carrying an integral
// layout(location = N). Multi-declarator statements assign consecutive
// locations, each declarator advancing by its array element count
// (ARB_explicit_uniform_location rules). Anything the narrow grammar does
// not recognize is skipped, never guessed at.
void RecordUniformDeclarationLocations(const Vector<CodeToken>& tokens, SizeT begin, SizeT end,
MobileGL::UnorderedMap<String, MobileGL::Int>& locations) {
using MobileGL::Int;
long long location = -1;
bool sawUniform = false;
SizeT declaratorBegin = end;
for (SizeT k = begin; k < end;) {
const String& text = tokens[k].text;
if (text == "layout" && k + 1 < end && tokens[k + 1].text == "(") {
SizeT j = k + 2;
Int parenDepth = 1;
while (j < end && parenDepth > 0) {
const String& layoutToken = tokens[j].text;
if (layoutToken == "(") {
++parenDepth;
} else if (layoutToken == ")") {
--parenDepth;
} else if (parenDepth == 1 && layoutToken == "location" && j + 2 < end &&
tokens[j + 1].text == "=" && IsDecimalIntegerToken(tokens[j + 2].text)) {
location = std::min(std::strtoll(tokens[j + 2].text.c_str(), nullptr, 10),
static_cast<long long>(INT_MAX / 2));
j += 2;
}
++j;
}
k = j;
continue;
}
if (text == "uniform") {
sawUniform = true;
++k;
continue;
}
if (sawUniform && location >= 0 && IsIdentifierToken(tokens[k]) &&
!IsNonLayoutQualifierKeyword(text)) {
declaratorBegin = k + 1; // 'text' is the type; declarators follow
break;
}
++k;
}
if (!sawUniform || location < 0 || declaratorBegin >= end) return;
long long nextLocation = location;
for (SizeT k = declaratorBegin; k < end;) {
if (!IsIdentifierToken(tokens[k])) return; // malformed; record nothing further
const String& name = tokens[k].text;
++k;
long long span = 1;
while (k < end && tokens[k].text == "[") {
++k;
long long dimension = 1;
if (k < end && IsDecimalIntegerToken(tokens[k].text)) {
dimension = std::strtoll(tokens[k].text.c_str(), nullptr, 10);
++k;
}
if (k >= end || tokens[k].text != "]") return; // sized by expression; bail out
++k;
span *= std::max(1ll, std::min(dimension, static_cast<long long>(INT_MAX / 2)));
}
// Keep the first sighting: a duplicate can only come from alternative
// preprocessor branches declaring the same name.
locations.emplace(name, static_cast<Int>(std::min(
nextLocation, static_cast<long long>(INT_MAX / 2))));
nextLocation += span;
if (k >= end) break;
if (tokens[k].text == "=") { // skip an initializer up to the declarator comma
Int nestingDepth = 0;
++k;
while (k < end) {
const String& initializerToken = tokens[k].text;
if (initializerToken == "(" || initializerToken == "[") {
++nestingDepth;
} else if (initializerToken == ")" || initializerToken == "]") {
--nestingDepth;
} else if (initializerToken == "," && nestingDepth == 0) {
break;
}
++k;
}
}
if (k >= end) break;
if (tokens[k].text != ",") return;
++k;
}
}
// Parses one brace-free depth-0 statement [begin, end) and records its
// declarators when it is a sampler/image uniform declaration carrying an
// integral layout(binding = N). Such a binding is a GL texture/image unit,
// which the Vulkan-client relaxed parse strips before mapIO can observe it
// (it is not a valid descriptor binding there), so it is extracted lexically
// and restored as the uniform's initial unit. Every declarator in the
// statement shares the qualifier's binding, matching what the GL-client
// mapIO used to capture from the shared type qualifier. Anything the narrow
// grammar does not recognize is skipped, never guessed at.
void RecordOpaqueDeclarationBindings(const Vector<CodeToken>& tokens, SizeT begin, SizeT end,
MobileGL::UnorderedMap<String, MobileGL::Uint>& bindings) {
using MobileGL::Int;
long long binding = -1;
bool sawUniform = false;
SizeT declaratorBegin = end;
for (SizeT k = begin; k < end;) {
const String& text = tokens[k].text;
if (text == "layout" && k + 1 < end && tokens[k + 1].text == "(") {
SizeT j = k + 2;
Int parenDepth = 1;
while (j < end && parenDepth > 0) {
const String& layoutToken = tokens[j].text;
if (layoutToken == "(") {
++parenDepth;
} else if (layoutToken == ")") {
--parenDepth;
} else if (parenDepth == 1 && layoutToken == "binding" && j + 2 < end &&
tokens[j + 1].text == "=" && IsDecimalIntegerToken(tokens[j + 2].text)) {
binding = std::min(std::strtoll(tokens[j + 2].text.c_str(), nullptr, 10),
static_cast<long long>(INT_MAX / 2));
j += 2;
}
++j;
}
k = j;
continue;
}
if (text == "uniform") {
sawUniform = true;
++k;
continue;
}
if (sawUniform && binding >= 0 && IsIdentifierToken(tokens[k]) &&
!IsNonLayoutQualifierKeyword(text)) {
// 'text' is the type. Only sampler/image opaques carry unit
// bindings; on anything else (e.g. atomic_uint, whose binding
// is a counter-buffer index) record nothing.
if (text.find("sampler") == String::npos && text.find("image") == String::npos) return;
declaratorBegin = k + 1;
break;
}
++k;
}
if (!sawUniform || binding < 0 || declaratorBegin >= end) return;
for (SizeT k = declaratorBegin; k < end;) {
if (!IsIdentifierToken(tokens[k])) return; // malformed; record nothing further
const String& name = tokens[k].text;
++k;
while (k < end && tokens[k].text == "[") {
++k;
if (k < end && IsDecimalIntegerToken(tokens[k].text)) ++k;
if (k >= end || tokens[k].text != "]") return; // sized by expression; bail out
++k;
}
bindings[name] = static_cast<MobileGL::Uint>(binding);
if (k >= end) break;
if (tokens[k].text != ",") return; // opaque declarators cannot take initializers
++k;
}
}
} // namespace
UnorderedMap<String, Uint> ExtractExplicitOpaqueBindings(const String& source) {
UnorderedMap<String, Uint> bindings;
// Fast path: without the qualifier keyword there is nothing to extract.
if (source.find("binding") == String::npos) return bindings;
const Vector<CodeToken> tokens = TokenizeCode(source);
const SizeT count = tokens.size();
Int braceDepth = 0;
SizeT pos = 0;
while (pos < count) {
const String& text = tokens[pos].text;
if (text == "{") {
++braceDepth;
++pos;
continue;
}
if (text == "}") {
if (braceDepth > 0) --braceDepth;
++pos;
continue;
}
if (braceDepth != 0 || text == ";") {
++pos;
continue;
}
// A depth-0 statement runs to its ';'. One that opens a brace instead is
// a function definition or an interface/uniform block: a block's binding
// is a buffer binding point, not a texture unit, so skip both alike.
SizeT statementEnd = pos;
while (statementEnd < count && tokens[statementEnd].text != ";" &&
tokens[statementEnd].text != "{") {
++statementEnd;
}
if (statementEnd >= count || tokens[statementEnd].text == "{") {
pos = statementEnd;
continue;
}
RecordOpaqueDeclarationBindings(tokens, pos, statementEnd, bindings);
pos = statementEnd + 1;
}
return bindings;
}
UnorderedMap<String, Int> ExtractExplicitUniformLocations(const String& source) {
UnorderedMap<String, Int> locations;
// Fast path: without the qualifier keyword there is nothing to extract.
if (source.find("location") == String::npos) return locations;
const Vector<CodeToken> tokens = TokenizeCode(source);
const SizeT count = tokens.size();
Int braceDepth = 0;
SizeT pos = 0;
while (pos < count) {
const String& text = tokens[pos].text;
if (text == "{") {
++braceDepth;
++pos;
continue;
}
if (text == "}") {
if (braceDepth > 0) --braceDepth;
++pos;
continue;
}
if (braceDepth != 0 || text == ";") {
++pos;
continue;
}
// A depth-0 statement runs to its ';'. One that opens a brace instead is a
// function definition or an interface/uniform block: neither can declare a
// default-block uniform location, so hand the '{' back to the depth tracker.
SizeT statementEnd = pos;
while (statementEnd < count && tokens[statementEnd].text != ";" &&
tokens[statementEnd].text != "{") {
++statementEnd;
}
if (statementEnd >= count || tokens[statementEnd].text == "{") {
pos = statementEnd;
continue;
}
RecordUniformDeclarationLocations(tokens, pos, statementEnd, locations);
pos = statementEnd + 1;
}
return locations;
}
} // namespace ShaderTranspiler } // namespace ShaderTranspiler
} // namespace MG_Util } // namespace MG_Util
} // namespace MobileGL } // namespace MobileGL
@@ -9,6 +9,7 @@
#pragma once #pragma once
#include <Includes.h> #include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderObject.h> #include <MG_State/GLState/ProgramState/ShaderObject.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL { namespace MobileGL {
enum class ShaderProfile { enum class ShaderProfile {
@@ -19,6 +20,14 @@ namespace MobileGL {
namespace MG_Util { namespace MG_Util {
namespace ShaderTranspiler { namespace ShaderTranspiler {
// The whole source-rewriting pipeline. `env` is the compile-time snapshot of
// everything outside (stage, source) this reads - advertised extensions and the
// device-quirk inputs - so the transformation is a pure function of its three
// arguments and can run on a worker thread.
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env);
// Convenience overload that resolves the current context's env itself. GL thread
// only, and deliberately not used by the compile pipeline: it exists for the unit
// tests and diagnostics that drive the preprocessor standalone.
void PreprocessShaderSource(ShaderStage stage, String& source); void PreprocessShaderSource(ShaderStage stage, String& source);
// Some desktop-captured compute shaders build a workgroup-wide linear prefix scan // Some desktop-captured compute shaders build a workgroup-wide linear prefix scan
@@ -45,6 +54,28 @@ namespace MobileGL {
// "row_major" outside a layout(...) list, the image*Shadow family). Returns the // "row_major" outside a layout(...) list, the image*Shadow family). Returns the
// compile-error text for the first violation, or nullopt for a clean source. // compile-error text for the first violation, or nullopt for a clean source.
std::optional<String> FindReservedIdentifierViolation(const String& source); std::optional<String> FindReservedIdentifierViolation(const String& source);
// Explicit layout(location = N) qualifiers on default-block uniform declarations,
// keyed by declared name (no "[0]" suffix). Multi-declarator statements assign
// consecutive locations, advancing by the array element count.
//
// Exists because the single link-compatible parse runs under relaxed Vulkan rules,
// where glslang's vkRelaxedRemapUniformVariable moves plain uniforms into
// MGL_GLOBAL_UBO and DISCARDS their location qualifiers ("ignoring layout qualifier
// for uniform location"); opaque uniforms keep theirs. This lexical side-channel
// restores the discarded locations to the GL location assigner
// (ProgramObject::DoReflection). It scans preprocessor-visible text, so a
// declaration inside an inactive #if branch is still recorded - harmless unless a
// pack declares the same uniform with different explicit locations in alternative
// branches (none observed; explicit uniform locations have zero incidence in the
// shader-pack corpus, this is an ARB_explicit_uniform_location conformance surface).
UnorderedMap<String, Int> ExtractExplicitUniformLocations(const String& source);
// Explicit layout(binding = N) on sampler/image uniforms, i.e. their initial
// texture/image units. The Vulkan-client relaxed parse strips these before
// mapIO can capture them, so they are recovered lexically (same narrow
// grammar discipline as ExtractExplicitUniformLocations).
UnorderedMap<String, Uint> ExtractExplicitOpaqueBindings(const String& source);
} // namespace ShaderTranspiler } // namespace ShaderTranspiler
} // namespace MG_Util } // namespace MG_Util
} // namespace MobileGL } // namespace MobileGL
@@ -0,0 +1,62 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RenameBuiltinShadowingFunctionsPass.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "RenameBuiltinShadowingFunctionsPass.h"
#include <string>
#include <string_view>
#include "../EsslBuiltinFunctionNames.h"
#include "spirv.hpp"
#include "source/opt/def_use_manager.h"
#include "source/opt/instruction.h"
#include "source/opt/ir_context.h"
#include "source/opt/module.h"
#include "source/util/make_unique.h"
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
spvtools::opt::Pass::Status RenameBuiltinShadowingFunctionsPass::Process() {
Bool modified = false;
auto* irContext = context();
auto* defUseMgr = irContext->get_def_use_mgr();
for (auto& debugInst : irContext->debugs2()) {
if (debugInst.opcode() != spv::Op::OpName || debugInst.NumInOperands() < 2) {
continue;
}
const auto* target = defUseMgr->GetDef(debugInst.GetSingleWordInOperand(0));
if (target == nullptr || target->opcode() != spv::Op::OpFunction) {
continue;
}
// glslang mangles function OpNames as "name(<paramcodes>"; the base name is
// everything before the '(' (entry points like "main" carry no mangling).
const std::string mangled = debugInst.GetInOperand(1).AsString();
const std::string_view baseName =
std::string_view(mangled).substr(0, mangled.find('('));
if (!IsEsslBuiltinFunctionName(baseName)) {
continue;
}
debugInst.SetInOperand(1, spvtools::utils::MakeVector<spvtools::opt::Operand::OperandData>(
"mg_" + mangled));
modified = true;
}
return modified ? Status::SuccessWithChange : Status::SuccessWithoutChange;
}
spvtools::Optimizer::PassToken
RenameBuiltinShadowingFunctionsPass::CreateRenameBuiltinShadowingFunctionsPass() {
return spvtools::Optimizer::PassToken(MakeUnique<RenameBuiltinShadowingFunctionsPass>());
}
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -0,0 +1,46 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/SpirvPasses/RenameBuiltinShadowingFunctionsPass.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include "source/opt/pass.h"
#include "spirv-tools/optimizer.hpp"
#include <Includes.h>
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// Desktop GLSL lets a shader redefine a builtin function (round, fma, ...) and
// shadow it; ESSL 3.x forbids the redefinition, so when SPIRV-Cross re-emits the
// function under its original OpName a strict ES driver rejects the shader with a
// redefinition error. Prefix the OpName of every user-defined function whose base
// name collides with an ESSL builtin (plus the min3/max3 trinary extension names)
// with "mg_". Renaming a user function is always semantics-preserving: its
// definition and every call site go through the same result id, while calls to
// the real builtin never resolve to a user function id in SPIR-V.
//
// This is the BACKSTOP half of the rename. The primary half is the lexical
// RenameBuiltinShadowingFunctions in ShaderSourceProcessor, which has to run
// before the parse - glslang's relaxed parse rejects some shadowing overload
// shapes outright, and a shadowed builtin may itself need an extension the
// declared #version does not enable. This pass catches what a lexical scan
// cannot see (macro-expanded definitions) and is idempotent: an already
// renamed mg_* name is not in the builtin table.
//
// Both halves share MG_Util/ShaderTranspiler/EsslBuiltinFunctionNames.h, so the
// covered name set cannot drift between them.
class RenameBuiltinShadowingFunctionsPass : public spvtools::opt::Pass {
public:
const char* name() const override { return "rename-builtin-shadowing-functions"; }
Status Process() override;
static spvtools::Optimizer::PassToken CreateRenameBuiltinShadowingFunctionsPass();
};
} // namespace ShaderTranspiler
} // namespace MG_Util
} // namespace MobileGL
@@ -138,6 +138,9 @@ namespace MobileGL {
std::swap(this->resources, that.resources); std::swap(this->resources, that.resources);
std::swap(this->reflectModule, that.reflectModule); std::swap(this->reflectModule, that.reflectModule);
std::swap(this->reflectModuleValid, that.reflectModuleValid); std::swap(this->reflectModuleValid, that.reflectModuleValid);
// ParseMetaData() fills `metadata`, and GetMetadata() is read through the
// moved-to session: leaving it behind silently returns an empty reflection.
std::swap(this->metadata, that.metadata);
} }
SpvcSession& SpvcSession::operator=(SpvcSession&& that) { SpvcSession& SpvcSession::operator=(SpvcSession&& that) {
@@ -149,6 +152,7 @@ namespace MobileGL {
std::swap(this->resources, that.resources); std::swap(this->resources, that.resources);
std::swap(this->reflectModule, that.reflectModule); std::swap(this->reflectModule, that.reflectModule);
std::swap(this->reflectModuleValid, that.reflectModuleValid); std::swap(this->reflectModuleValid, that.reflectModuleValid);
std::swap(this->metadata, that.metadata);
return *this; return *this;
} }
@@ -8,6 +8,7 @@
#pragma once #pragma once
#include <Includes.h> #include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL { namespace MobileGL {
namespace MG_Util { namespace MG_Util {
@@ -25,6 +26,11 @@ namespace MobileGL {
GLenum shaderType; GLenum shaderType;
StringView sourceStr; StringView sourceStr;
Flags<ShaderCompileBits> flags; Flags<ShaderCompileBits> flags;
// The compile-time backend snapshot the glslang resource limits come from.
// Null means "read them off the live backend object" - only legal on the GL
// thread, and only used by the standalone/test entry points. Non-owning: the
// env outlives the attrib (it is a per-context SharedPtr).
const CompileEnv* env = nullptr;
}; };
struct ProgramAttrib { struct ProgramAttrib {