swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-09-08 04:08:32 +09:00
Code Issues Packages Projects Releases Wiki Activity

[Refactor] (MG_State, MG_Util): join-by-construction link/compile artifacts (P1 stage 2)

Browse Source 
Still fully synchronous - EnsureLinkJoined()/EnsureCompileJoined() are empty
inline no-ops (verified to fold away at every one of the ~1200 call sites;
this project builds without LTO) - but every read of link- or compile-produced
state now goes through a private accessor the compiler enforces, so when
stage 4 moves the bodies onto pool workers, 'which reads must join' is a
type-system fact instead of a 400-line audit.

- ProgramObject: the 31 fields ResetLinkArtifacts clears plus the 5 link
  outputs it forgot (infoLog, linkedFragData{Location,Index}, the geometry
  strip-capture pair) move into a nested LinkArtifacts behind Artifacts().
  ResetLinkArtifacts is now a worker-safe pure clear; the link-observable
  version bumps (backendState/link/uboContent) move to a GL-thread-only
  BumpLinkObservableVersions() called once from Link()'s prologue and from
  glProgramBinary's mandated failure - the link body never writes them, so
  a stage-4 worker cannot lose an invalidation against the draw path.
- ShaderObject: compile artifacts (TShader, preprocessed source, side-channel
  maps, status/log, consume-once flag) behind Compiled(); the P0b layer-1
  memo trio deliberately stays outside as the future non-joining
  COMPLETION_STATUS_KHR fast path.
- CompileEnv (new): a GL-thread snapshot of everything the compile pipeline
  used to read live from the backend mid-parse - compute limits (the
  GetIntegeri_v reach-back is gone from the worker path), advertised
  extensions, device quirks, TBuiltInResource inputs. Captured lazily per
  backend activation; the consume-once re-parse now runs against the same
  env as the original parse.
- The GL-thread prologue / worker-body boundary is marked in Link() where
  the stage sort ends; everything below is a pure function of the snapshot.

Public getter signatures unchanged - MG_Impl and both backends compile
untouched. Unit 476/476, Program suites 117/117, DirectGLES retrace 38/39 on
llvmpipe (the one failure is the known pre-existing non-CI iterationrp case;
the NVIDIA userspace driver was updated out from under the running kernel
module mid-session, so GLX there is down until a reboot).
This commit is contained in:
BZLZHH
2026-08-08 07:12:37 -04:00
parent 8191075133
commit c93e5fa409
18 changed files with 1186 additions and 654 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CMakeLists.txt
+1
View File
@@ -190,6 +190,7 @@ set(SOURCE_FILES
MobileGL/MG_Util/Classifiers/TextureEnumClassifier.cpp
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp
MobileGL/MG_Util/ShaderTranspiler/SpvcSession.cpp
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp
MobileGL/MG_State/GLState/Core.cpp
+14
View File
@@ -9,6 +9,8 @@
#include "Core.h"
#include "MG_State/GLState/RenderbufferState/RenderbufferObject.h"
#include "MG_State/EGLState/Core.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <Config.h>
namespace MobileGL::MG_State {
@@ -24,6 +26,18 @@ namespace MobileGL::MG_State {
}
namespace GLState {
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv>& GLContext::GetCompileEnv() {
const void* backend = static_cast<const void*>(MG_Backend::pActiveBackendObject.get());
if (!m_compileEnv || m_compileEnvBackend != backend) {
// First use, or the backend was swapped underneath us. Re-capturing rolls the
// fingerprint, so every P0b preprocess memo computed against the old backend's
// limits becomes structurally unreachable instead of silently reusable.
m_compileEnv = MG_Util::ShaderTranspiler::CaptureCompileEnv();
m_compileEnvBackend = backend;
}
return m_compileEnv;
}
// Error
void GLContext::RecordError(ErrorCode code, UniquePtr<ErrorInfo> info) {
m_errorState.RecordError(code, Move(info));
MobileGL/MG_State/GLState/Core.h
+18
View File
@@ -21,6 +21,10 @@
#include "VertexArrayState/VertexArrayState.h"
#include "RenderbufferState/RenderbufferState.h"
namespace MobileGL::MG_Util::ShaderTranspiler {
struct CompileEnv;
}
namespace MobileGL {
namespace MG_State {
void Init();
@@ -380,6 +384,15 @@ namespace MobileGL {
Bool ValidateRenderbufferName(Uint index) const;
Bool ValidateRenderbufferObject(Uint index) const;
// P1: the shader compile/link pipeline's snapshot of everything it reads from
// outside its own (stage, source) inputs. Captured lazily here because it
// cannot be captured in MG_State::Init() - that runs BEFORE MG_Backend::Init(),
// so there is no backend to query yet. Re-captured whenever the active backend
// object changes, which also rolls the fingerprint and therefore invalidates
// every P0b preprocess memo keyed against the old one.
// GL thread only.
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv>& GetCompileEnv();
private:
// State Components
ErrorState m_errorState;
@@ -437,6 +450,11 @@ namespace MobileGL {
FramebufferState m_framebufferState;
SamplerState m_samplerState;
RenderbufferState m_renderbufferState;
mutable SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> m_compileEnv;
// Identity of the backend object m_compileEnv was captured against; a plain
// pointer compare, never dereferenced.
const void* m_compileEnvBackend = nullptr;
};
} // namespace GLState
MobileGL/MG_State/GLState/ProgramState/ProgramObject.cpp
+287 -263
View File
File diff suppressed because it is too large Load Diff
MobileGL/MG_State/GLState/ProgramState/ProgramObject.h
+258 -189
View File
@@ -44,28 +44,28 @@ namespace MobileGL::MG_State::GLState {
Vector<SharedPtr<ShaderObject>>& GetAttachedShaders();
const Vector<SharedPtr<ShaderObject>>& GetAttachedShaders() const;
const String& GetInfoLog() const { return m_infoLog; }
const String& GetInfoLog() const { return Artifacts().infoLog; }
// glCreateShaderProgramv folds the shader's compile log into the program's log, which
// is the only place a caller can read it from once the shader name is gone.
void AppendInfoLog(const String& text) {
if (text.empty()) return;
if (!m_infoLog.empty() && m_infoLog.back() != '\n') m_infoLog += '\n';
m_infoLog += text;
if (!Artifacts().infoLog.empty() && Artifacts().infoLog.back() != '\n') Artifacts().infoLog += '\n';
Artifacts().infoLog += text;
}
Int GetUniformMaxLength() const { return m_uniformNameMaxLength; }
Uint GetUniformCount() const { return m_activeUniformCount; }
Uint GetMaxUniformLocation() const { return m_maxUniformLocation; }
Int GetUniformMaxLength() const { return Artifacts().uniformNameMaxLength; }
Uint GetUniformCount() const { return Artifacts().activeUniformCount; }
Uint GetMaxUniformLocation() const { return Artifacts().maxUniformLocation; }
Int GetUniformLocation(const String& name) const {
const auto it = m_uniformLocations.find(name);
if (it != m_uniformLocations.end()) return (Int)it->second;
const auto it = Artifacts().uniformLocations.find(name);
if (it != Artifacts().uniformLocations.end()) return (Int)it->second;
// Reflection stores GL-style names: an array uniform is keyed "arr[0]" (its base
// location). A bare "arr" query resolves to that entry; an "arr[k]" query resolves
// to base + k because DoReflection reserves one location per array element.
if (name.empty()) return -1;
if (name.back() != ']') {
const auto suffixedIt = m_uniformLocations.find(name + "[0]");
if (suffixedIt != m_uniformLocations.end()) return (Int)suffixedIt->second;
const auto suffixedIt = Artifacts().uniformLocations.find(name + "[0]");
if (suffixedIt != Artifacts().uniformLocations.end()) return (Int)suffixedIt->second;
return -1;
}
if (name.length() < 4) return -1;
@@ -78,19 +78,19 @@ namespace MobileGL::MG_State::GLState {
element = element * 10 + static_cast<Uint>(name[i] - '0');
if (element > 0x0FFFFFFFu) return -1;
}
auto baseIt = m_uniformLocations.find(name.substr(0, bracket) + "[0]");
if (baseIt == m_uniformLocations.end()) {
auto baseIt = Artifacts().uniformLocations.find(name.substr(0, bracket) + "[0]");
if (baseIt == Artifacts().uniformLocations.end()) {
// Legacy key without the "[0]" suffix (defensive; reflection normally
// stores the suffixed form for arrays).
baseIt = m_uniformLocations.find(name.substr(0, bracket));
if (baseIt == m_uniformLocations.end()) return -1;
baseIt = Artifacts().uniformLocations.find(name.substr(0, bracket));
if (baseIt == Artifacts().uniformLocations.end()) return -1;
}
const Int base = (Int)baseIt->second;
if (!IsValidUniformLocation(base)) return -1;
const Int index = m_uniformIndexInTProgram[base];
const Int index = Artifacts().uniformIndexInTProgram[base];
// "[k]" only addresses arrays ("scalar[0]" is not a uniform name), and only
// in-range elements.
const glslang::TType* type = m_program->getUniform(index).getType();
const glslang::TType* type = Artifacts().program->getUniform(index).getType();
if (type == nullptr || !type->isArray()) return -1;
if (static_cast<GLint>(element) >= GetUniformArraySizeByTIndex(index)) return -1;
const Int location = base + (Int)element;
@@ -101,7 +101,7 @@ namespace MobileGL::MG_State::GLState {
// True when both locations are element slots of the same uniform variable.
Bool UniformLocationsAliasSameUniform(Int a, Int b) const {
if (!IsValidUniformLocation(a) || !IsValidUniformLocation(b)) return false;
return m_uniformIndexInTProgram[a] == m_uniformIndexInTProgram[b];
return Artifacts().uniformIndexInTProgram[a] == Artifacts().uniformIndexInTProgram[b];
}
// ---- GL index <-> glslang TProgram index translation ----
@@ -111,22 +111,22 @@ namespace MobileGL::MG_State::GLState {
// index spaces; every public "index"-taking getter translates through them, so
// GL and backend consumers keep seeing exactly the pre-P0a surface.
Int TProgramUniformIndex(Uint glIndex) const {
return m_glUniformIndexToTProgram[glIndex];
return Artifacts().glUniformIndexToTProgram[glIndex];
}
Int GlUniformIndexFromTProgram(Int tIndex) const {
if (tIndex < 0 || tIndex >= static_cast<Int>(m_tProgramUniformIndexToGl.size())) return -1;
return m_tProgramUniformIndexToGl[tIndex];
if (tIndex < 0 || tIndex >= static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size())) return -1;
return Artifacts().tProgramUniformIndexToGl[tIndex];
}
Int GlBlockIndexFromTProgram(Int tBlockIndex) const {
if (tBlockIndex < 0 || tBlockIndex >= static_cast<Int>(m_tProgramBlockIndexToGl.size())) return -1;
return m_tProgramBlockIndexToGl[tBlockIndex];
if (tBlockIndex < 0 || tBlockIndex >= static_cast<Int>(Artifacts().tProgramBlockIndexToGl.size())) return -1;
return Artifacts().tProgramBlockIndexToGl[tBlockIndex];
}
Int GetActiveUniformIndex(const String& name) const {
const Int tProgramCount = static_cast<Int>(m_tProgramUniformIndexToGl.size());
const Int uniformIndex = m_program->getUniformIndex(name.c_str());
const Int tProgramCount = static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
const Int uniformIndex = Artifacts().program->getUniformIndex(name.c_str());
if (uniformIndex >= 0 && uniformIndex < tProgramCount &&
m_program->getUniform(uniformIndex).name == name) {
Artifacts().program->getUniform(uniformIndex).name == name) {
return GlUniformIndexFromTProgram(uniformIndex);
}
@@ -135,9 +135,9 @@ namespace MobileGL::MG_State::GLState {
// robustness against non-suffixed reflection entries.
if (!name.empty() && name.back() != ']') {
const String suffixedName = name + "[0]";
const Int suffixedIndex = m_program->getUniformIndex(suffixedName.c_str());
const Int suffixedIndex = Artifacts().program->getUniformIndex(suffixedName.c_str());
if (suffixedIndex >= 0 && suffixedIndex < tProgramCount &&
m_program->getUniform(suffixedIndex).name == suffixedName) {
Artifacts().program->getUniform(suffixedIndex).name == suffixedName) {
return GlUniformIndexFromTProgram(suffixedIndex);
}
return -1;
@@ -145,28 +145,28 @@ namespace MobileGL::MG_State::GLState {
if (name.length() <= 3 || name.compare(name.length() - 3, 3, "[0]") != 0) return -1;
const String baseName = name.substr(0, name.length() - 3);
const Int baseIndex = m_program->getUniformIndex(baseName.c_str());
const Int baseIndex = Artifacts().program->getUniformIndex(baseName.c_str());
if (baseIndex < 0 || baseIndex >= tProgramCount) return -1;
return m_program->getUniform(baseIndex).name == baseName ? GlUniformIndexFromTProgram(baseIndex)
return Artifacts().program->getUniform(baseIndex).name == baseName ? GlUniformIndexFromTProgram(baseIndex)
: -1;
}
Bool IsValidUniformLocation(Int location) const {
if (location < 0 || location > static_cast<Int>(m_maxUniformLocation)) return false;
if (static_cast<SizeT>(location) >= m_uniformIndexInTProgram.size()) return false;
const Int uniformIndexInProgram = m_uniformIndexInTProgram[location];
if (location < 0 || location > static_cast<Int>(Artifacts().maxUniformLocation)) return false;
if (static_cast<SizeT>(location) >= Artifacts().uniformIndexInTProgram.size()) return false;
const Int uniformIndexInProgram = Artifacts().uniformIndexInTProgram[location];
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
uniformIndexInProgram >= 0 &&
uniformIndexInProgram < static_cast<Int>(m_tProgramUniformIndexToGl.size());
uniformIndexInProgram < static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
}
GLenum GetUniformType(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]);
auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.glDefineType;
}
GLenum GetActiveUniformType(Uint index) const {
auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
return uniform.glDefineType;
}
@@ -174,9 +174,9 @@ namespace MobileGL::MG_State::GLState {
// glslang's TObjectReflection.size only carries the element count for a NON-block array; for
// a block array member it reports 1, so take the count from the TType, which is authoritative
// for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
// m_uniformIndexInTProgram stores).
// the artifacts' uniformIndexInTProgram stores).
GLint GetUniformArraySizeByTIndex(Int tIndex) const {
const auto& uniform = m_program->getUniform(tIndex);
const auto& uniform = Artifacts().program->getUniform(tIndex);
const glslang::TType* type = uniform.getType();
if (type != nullptr && type->isSizedArray()) {
return type->getOuterArraySize();
@@ -189,7 +189,7 @@ namespace MobileGL::MG_State::GLState {
}
Int GetActiveUniformBlockIndex(Uint index) const {
auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
// Members of the synthesized global UBO are default-block uniforms to GL: -1.
return GlBlockIndexFromTProgram(uniform.index);
}
@@ -198,7 +198,7 @@ namespace MobileGL::MG_State::GLState {
// uniform. The relaxed parse gives global-UBO members real byte offsets, but GL must keep
// seeing them as default-block uniforms, so gate on the GL-visible block index.
GLint GetActiveUniformOffset(Uint index) const {
const auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
return uniform.offset;
}
@@ -212,7 +212,7 @@ namespace MobileGL::MG_State::GLState {
// generated SPIR-V lay the array out with std140 16-byte-rounded strides. MobileGL's UBO
// layout is always std140, where every array element stride rounds up to a vec4.
GLint GetActiveUniformArrayStride(Uint index) const {
const auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isArray()) return 0;
@@ -232,13 +232,13 @@ namespace MobileGL::MG_State::GLState {
// check suffices; the getUniformBlock() fallback is defensive for a config that instead leaves
// an inheriting member's layoutMatrix == ElmNone.
GLint GetActiveUniformIsRowMajor(Uint index) const {
const auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (GlBlockIndexFromTProgram(uniform.index) < 0) return 0;
const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isMatrix()) return 0;
glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix;
if (layoutMatrix == glslang::ElmNone) {
layoutMatrix = m_program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
layoutMatrix = Artifacts().program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
}
return (layoutMatrix == glslang::ElmRowMajor) ? 1 : 0;
}
@@ -250,13 +250,13 @@ namespace MobileGL::MG_State::GLState {
// out as std140 (packed/shared are coerced), so this matches the offsets glslang reports. For
// every GL 3.3 float matrix this evaluates to 16, independent of majorness.
GLint GetActiveUniformMatrixStride(Uint index) const {
const auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
const auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
if (GlBlockIndexFromTProgram(uniform.index) < 0) return -1;
const glslang::TType* type = uniform.getType();
if (type == nullptr || !type->isMatrix()) return 0;
glslang::TLayoutMatrix layoutMatrix = type->getQualifier().layoutMatrix;
if (layoutMatrix == glslang::ElmNone) {
layoutMatrix = m_program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
layoutMatrix = Artifacts().program->getUniformBlock(uniform.index).getType()->getQualifier().layoutMatrix;
}
const bool rowMajor = (layoutMatrix == glslang::ElmRowMajor);
const int strideVectorComponents = rowMajor ? type->getMatrixCols() : type->getMatrixRows();
@@ -268,50 +268,50 @@ namespace MobileGL::MG_State::GLState {
}
const glslang::TType* GetUniformTType(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]);
auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.getType();
}
Bool IsUniformOpaqueAtLocation(Uint location) const { return GetUniformTType(location)->isOpaque(); }
const String& GetUniformName(Uint location) const {
auto& uniform = m_program->getUniform(m_uniformIndexInTProgram[location]);
auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
return uniform.name;
}
const String& GetActiveUniformName(Uint index) const {
auto& uniform = m_program->getUniform(TProgramUniformIndex(index));
auto& uniform = Artifacts().program->getUniform(TProgramUniformIndex(index));
return uniform.name;
}
// Sentinel for a uniform location without global-UBO backing storage (should not
// survive linking: GenerateBinary falls back to tail-allocated scratch storage).
static constexpr Uint kInvalidUniformOffset = ~0u;
Uint GetUniformOffset(Uint location) const { return m_uniformOffsets[location]; }
Uint GetUniformOffset(Uint location) const { return Artifacts().uniformOffsets[location]; }
Uint GetUniformSizesInBytes(Uint location) const { return MG_Util::GetGLTypeSize(GetUniformType(location)); }
Int GetAttributeLocation(const String& name) {
const auto it = std::find(m_attribs.begin(), m_attribs.end(), name);
return (it == m_attribs.end()) ? -1 : (Int)std::distance(m_attribs.begin(), it);
const auto it = std::find(Artifacts().attribs.begin(), Artifacts().attribs.end(), name);
return (it == Artifacts().attribs.end()) ? -1 : (Int)std::distance(Artifacts().attribs.begin(), it);
}
Uint32 GetActiveAttributeLocationMask() const {
Uint32 mask = 0;
const SizeT count = std::min<SizeT>(m_attribs.size(), 32);
const SizeT count = std::min<SizeT>(Artifacts().attribs.size(), 32);
for (SizeT index = 0; index < count; ++index) {
if (!m_attribs[index].empty()) {
if (!Artifacts().attribs[index].empty()) {
mask |= (1u << index);
}
}
return mask;
}
Uint32 GetActiveFragmentOutputLocationMask() const {
if (!m_program) {
if (!Artifacts().program) {
return 0;
}
Uint32 mask = 0;
const Int outputCount = m_program->getNumPipeOutputs();
const Int outputCount = Artifacts().program->getNumPipeOutputs();
for (Int index = 0; index < outputCount; ++index) {
const Int location = static_cast<Int>(m_program->getPipeOutput(index).layoutLocation());
const Int location = static_cast<Int>(Artifacts().program->getPipeOutput(index).layoutLocation());
if (location >= 0 && location < 32) {
mask |= (1u << location);
}
@@ -319,38 +319,38 @@ namespace MobileGL::MG_State::GLState {
return mask;
}
Int GetActiveFragmentOutputCount() const {
return m_program ? m_program->getNumPipeOutputs() : 0;
return Artifacts().program ? Artifacts().program->getNumPipeOutputs() : 0;
}
const String& GetActiveFragmentOutputName(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetActiveFragmentOutputName: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()),
MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetActiveFragmentOutputName: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetActiveFragmentOutputName: index=%u out of range", index);
return m_program->getPipeOutput(static_cast<Int>(index)).name;
return Artifacts().program->getPipeOutput(static_cast<Int>(index)).name;
}
Int GetFragmentOutputLocation(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetFragmentOutputLocation: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()),
MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetFragmentOutputLocation: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetFragmentOutputLocation: index=%u out of range",
index);
return static_cast<Int>(m_program->getPipeOutput(static_cast<Int>(index)).layoutLocation());
return static_cast<Int>(Artifacts().program->getPipeOutput(static_cast<Int>(index)).layoutLocation());
}
GLint GetActiveFragmentOutputArraySize(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetActiveFragmentOutputArraySize: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()),
MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetActiveFragmentOutputArraySize: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetActiveFragmentOutputArraySize: index=%u out of range", index);
return m_program->getPipeOutput(static_cast<Int>(index)).size;
return Artifacts().program->getPipeOutput(static_cast<Int>(index)).size;
}
GLenum GetFragmentOutputType(Uint index) const {
MOBILEGL_ASSERT(m_program != nullptr, "ProgramObject::GetFragmentOutputType: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(m_program->getNumPipeOutputs()),
MOBILEGL_ASSERT(Artifacts().program != nullptr, "ProgramObject::GetFragmentOutputType: program is null");
MOBILEGL_ASSERT(index < static_cast<Uint>(Artifacts().program->getNumPipeOutputs()),
"ProgramObject::GetFragmentOutputType: index=%u out of range",
index);
return m_program->getPipeOutput(static_cast<Int>(index)).glDefineType;
return Artifacts().program->getPipeOutput(static_cast<Int>(index)).glDefineType;
}
GLenum GetAttribType(Uint index) const { return m_attribTypes[index]; }
const String& GetAttribName(Uint index) const { return m_attribs[index]; }
GLenum GetActiveAttribType(Uint index) const { return m_program->getPipeInput(static_cast<Int>(index)).glDefineType; }
GLint GetActiveAttribArraySize(Uint index) const { return m_program->getPipeInput(static_cast<Int>(index)).size; }
GLenum GetAttribType(Uint index) const { return Artifacts().attribTypes[index]; }
const String& GetAttribName(Uint index) const { return Artifacts().attribs[index]; }
GLenum GetActiveAttribType(Uint index) const { return Artifacts().program->getPipeInput(static_cast<Int>(index)).glDefineType; }
GLint GetActiveAttribArraySize(Uint index) const { return Artifacts().program->getPipeInput(static_cast<Int>(index)).size; }
// The Vulkan-semantics parse reflects the vertex builtins under their SPIR-V names;
// GL must keep reporting the GL spellings (glGetActiveAttrib and the program-input
// resource queries enumerate builtins).
@@ -362,11 +362,11 @@ namespace MobileGL::MG_State::GLState {
return name;
}
const String& GetActiveAttribName(Uint index) const {
return NormalizeBuiltinPipeInputName(m_program->getPipeInput(static_cast<Int>(index)).name);
return NormalizeBuiltinPipeInputName(Artifacts().program->getPipeInput(static_cast<Int>(index)).name);
}
void* MapUBO() { return m_globalUboScratch.data(); }
const void* GetUBOData() const { return m_globalUboScratch.data(); }
Uint GetUBOSize() const { return static_cast<Uint>(m_globalUboScratch.size()); }
void* MapUBO() { return Artifacts().globalUboScratch.data(); }
const void* GetUBOData() const { return Artifacts().globalUboScratch.data(); }
Uint GetUBOSize() const { return static_cast<Uint>(Artifacts().globalUboScratch.size()); }
// Content version of the CPU-side global-UBO shadow: writers bump it so backends
// can skip re-uploading an unchanged UBO on every draw. ~0u is reserved as the
// backends' "never uploaded" sentinel, so skip over it on wrap.
@@ -412,20 +412,20 @@ namespace MobileGL::MG_State::GLState {
}
void SetUniformSamplerOrImageUnitIndex(Uint location, Int unit) {
if (location >= m_uniformSamplerOrImageUnitIndex.size() ||
m_uniformSamplerOrImageUnitIndex[location] == unit) {
if (location >= Artifacts().uniformSamplerOrImageUnitIndex.size() ||
Artifacts().uniformSamplerOrImageUnitIndex[location] == unit) {
return;
}
m_uniformSamplerOrImageUnitIndex[location] = unit;
Artifacts().uniformSamplerOrImageUnitIndex[location] = unit;
++m_backendStateVersion;
}
Int GetUniformSamplerOrImageUnitIndex(Uint location) const {
return m_uniformSamplerOrImageUnitIndex[location];
return Artifacts().uniformSamplerOrImageUnitIndex[location];
}
Bool GetDeleteStatus() const { return m_deleteStatus; }
Bool GetLinkStatus() const { return m_linkStatus; }
Bool GetLinkStatus() const { return Artifacts().linkStatus; }
// GL_PROGRAM_BINARY_RETRIEVABLE_HINT. MobileGL exposes no program binary format
// (GL_NUM_PROGRAM_BINARY_FORMATS is 0), so the hint is pure state - which is all
// ARB_get_program_binary requires of it.
@@ -439,26 +439,27 @@ namespace MobileGL::MG_State::GLState {
// glProgramBinary always fails here (there is no format it could accept) and the
// spec then requires the program's LINK_STATUS to read FALSE.
void MarkLinkFailedByProgramBinary() {
BumpLinkObservableVersions();
ResetLinkArtifacts();
m_infoLog = "No program binary format is supported.";
Artifacts().infoLog = "No program binary format is supported.";
}
Bool GetValidateStatus() const { return m_validateStatus; }
Int GetActiveAtomicCounterCount() const { return m_program->getNumAtomicCounters(); }
Int GetActiveAttributesCount() const { return m_program->getNumPipeInputs(); }
Int GetActiveAtomicCounterCount() const { return Artifacts().program->getNumAtomicCounters(); }
Int GetActiveAttributesCount() const { return Artifacts().program->getNumPipeInputs(); }
// GL-visible uniform blocks only: the synthesized MGL_GLOBAL_UBO the relaxed parse
// materializes for default-block uniforms is filtered out by DoReflection.
Int GetActiveUniformBlocksCount() const { return static_cast<Int>(m_glBlockIndexToTProgram.size()); }
GLuint GetComputeLocalSize(Uint dim) const { return m_program->getLocalSize(static_cast<Int>(dim)); }
Int GetActiveAttributesMaxLength() const { return m_attribInNameMaxLength; }
Int GetActiveUniformBlocksMaxNameLength() const { return m_uniformBlockNameMaxLength; }
Int GetActiveUniformBlocksCount() const { return static_cast<Int>(Artifacts().glBlockIndexToTProgram.size()); }
GLuint GetComputeLocalSize(Uint dim) const { return Artifacts().program->getLocalSize(static_cast<Int>(dim)); }
Int GetActiveAttributesMaxLength() const { return Artifacts().attribInNameMaxLength; }
Int GetActiveUniformBlocksMaxNameLength() const { return Artifacts().uniformBlockNameMaxLength; }
Uint GetUniformBlockIndex(const char* name) const {
auto it = m_uniformBlockIndexByName.find(name);
if (it != m_uniformBlockIndexByName.end()) return it->second;
auto it = Artifacts().uniformBlockIndexByName.find(name);
if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
// Instances of an arrayed block are reflected as "Block[0]".."Block[N-1]";
// a bare "Block" query resolves to the first instance per GL semantics.
const String suffixedName = String(name) + "[0]";
it = m_uniformBlockIndexByName.find(suffixedName);
if (it != m_uniformBlockIndexByName.end()) return it->second;
it = Artifacts().uniformBlockIndexByName.find(suffixedName);
if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
return 0xFFFFFFFFu; // GL_INVALID_INDEX
}
Bool IsActiveUniformBlock(Uint index) const {
@@ -471,11 +472,11 @@ namespace MobileGL::MG_State::GLState {
// (like a std140 struct) occupies a vec4-rounded size, and that is what the
// backend compiles: ES drivers reject draws whose bound UBO range is smaller
// than the block (a block ending in ivec3 reported 12 while the driver needs 16).
return (m_program->getUniformBlock(m_glBlockIndexToTProgram[index]).size + 15u) & ~15u;
return (Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]).size + 15u) & ~15u;
}
const String& GetUniformBlockName(Uint index) const {
auto& ubo = m_program->getUniformBlock(m_glBlockIndexToTProgram[index]);
auto& ubo = Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]);
return ubo.name;
}
@@ -487,8 +488,8 @@ namespace MobileGL::MG_State::GLState {
if (name.empty() || name.back() != ']') return index;
const SizeT bracket = name.rfind('[');
if (bracket == String::npos) return index;
const auto it = m_uniformBlockIndexByName.find(name.substr(0, bracket) + "[0]");
if (it != m_uniformBlockIndexByName.end()) return it->second;
const auto it = Artifacts().uniformBlockIndexByName.find(name.substr(0, bracket) + "[0]");
if (it != Artifacts().uniformBlockIndexByName.end()) return it->second;
return index;
}
@@ -499,31 +500,31 @@ namespace MobileGL::MG_State::GLState {
Int GetUniformBlockActiveUniformCount(Uint index) const {
const Int ownerIndex = static_cast<Int>(GetUniformBlockMemberOwnerIndex(index));
Int count = 0;
for (Uint uniformIndex = 0; uniformIndex < m_activeUniformCount; ++uniformIndex) {
for (Uint uniformIndex = 0; uniformIndex < Artifacts().activeUniformCount; ++uniformIndex) {
if (GetActiveUniformBlockIndex(uniformIndex) == ownerIndex) ++count;
}
return count;
}
Bool IsUniformBlockReferencedByStage(Uint index, EShLanguage stage) const {
const auto& ubo = m_program->getUniformBlock(m_glBlockIndexToTProgram[index]);
const auto& ubo = Artifacts().program->getUniformBlock(Artifacts().glBlockIndexToTProgram[index]);
const auto stageMask = static_cast<EShLanguageMask>(1 << stage);
return (ubo.stages & stageMask) != 0;
}
// Set by glUniformBlockBinding
void SetUniformBlockBinding(Uint index, Uint binding) {
if (index >= m_uniformBlockBinding.size() || m_uniformBlockBinding[index] == static_cast<Int>(binding)) {
if (index >= Artifacts().uniformBlockBinding.size() || Artifacts().uniformBlockBinding[index] == static_cast<Int>(binding)) {
return;
}
m_uniformBlockBinding[index] = static_cast<Int>(binding);
Artifacts().uniformBlockBinding[index] = static_cast<Int>(binding);
++m_backendStateVersion;
}
Uint GetUniformBlockBinding(Uint index) const { return m_uniformBlockBinding[index]; }
Uint GetUniformBlockBinding(Uint index) const { return Artifacts().uniformBlockBinding[index]; }
Vector<Vector<unsigned>>& GetGeneratedSpirv() { return m_generatedSpirv; }
const Vector<Vector<unsigned>>& GetGeneratedSpirv() const { return m_generatedSpirv; }
Vector<Vector<unsigned>>& GetGeneratedSpirv() { return Artifacts().generatedSpirv; }
const Vector<Vector<unsigned>>& GetGeneratedSpirv() const { return Artifacts().generatedSpirv; }
Int GetShaderIndexByStage(ShaderStage stage) const {
auto it = std::find_if(m_shaders.begin(), m_shaders.end(), [stage](const SharedPtr<ShaderObject>& shader) {
@@ -545,41 +546,134 @@ namespace MobileGL::MG_State::GLState {
// layout captures into; see NeedsScatteredTransformFeedbackCapture.
Uint32 packedOffsetBytes = 0;
};
// ---- P1: everything a link PRODUCES, in one movable block ----
//
// The membership rule is mechanical, not editorial: this is exactly the field list
// ResetLinkArtifacts() clears (plus the four it forgot to - infoLog,
// linkedFragDataLocation/Index and the geometry strip-capture pair - which are just
// as much link output). Nothing else belongs here.
//
// Why a struct: once glLinkProgram runs on a worker (P1 stage 4) the worker writes
// its OWN LinkArtifacts and the GL thread publishes it with a single move, instead
// of thirty cross-thread field assignments. Until then this is a pure refactor.
//
// Access rule (invariant I5): the member below is private and reachable ONLY
// through ProgramObject::Artifacts(), which calls EnsureLinkJoined() first. That is
// what makes "every read of link output joins the pending link" a property the
// compiler checks rather than a review item - a new reader cannot spell the field
// without going through the gate.
struct LinkArtifacts {
SharedPtr<glslang::TProgram> program;
Vector<Vector<unsigned>> generatedSpirv;
// Attributes (Vertex in)
Vector<String> attribs;
Vector<GLenum> attribTypes;
// FragData (Frag out): the per-link snapshot of the explicit request maps.
UnorderedMap<String, Uint> linkedFragDataLocation;
UnorderedMap<String, Uint> linkedFragDataIndex;
// GL-facing index spaces (see the translation helpers above): GL active-uniform
// index <-> glslang TProgram uniform index, GL uniform-block index <-> TProgram
// block index. -1 marks a TProgram entry GL does not expose (dead default-block
// uniforms swept into MGL_GLOBAL_UBO by the relaxed parse, and that block itself).
Vector<Int> glUniformIndexToTProgram;
Vector<Int> tProgramUniformIndexToGl;
Vector<Int> glBlockIndexToTProgram;
Vector<Int> tProgramBlockIndexToGl;
// Per-link merged snapshot of the attached shaders' lexically extracted
// layout(location = N) default-block uniform qualifiers (the relaxed parse drops
// them from reflection; the DoReflection assigner restores them from here).
UnorderedMap<String, Int> linkedExplicitUniformLocations;
UnorderedMap<String, Uint> uniformLocations;
// Ordered by location,
// aka. uniformIndexInTProgram[loc] == "uniform index of TProgram at location `loc`"
Vector<Int> uniformIndexInTProgram;
// ditto. Will be set at glUniform1i
Vector<Int> uniformSamplerOrImageUnitIndex;
UnorderedMap<String, Uint> explicitOpaqueUniformBindings;
// Ordered by uniform block index
// index is DIFFERENT from binding!!!
//
// Let's define UniformBlockIndex == the order at glslang getUniformBlock()
// aka `i = glGetUniformBlockIndex(prog, "BlockName")` implies:
// `prog->getUniformBlock(i) == "BlockName"`
// These stuff are present for GL semantics, not for backend inspection
// These may change after-link (because GL spec decided to have `glUniformBlockBinding`)
UnorderedMap<String, Uint> uniformBlockIndexByName;
Vector<Int> uniformBlockBinding;
// Need to be reflected after linking of SPIR-V binary
Vector<Uint> uniformOffsets;
Vector<Uint> uniformSizesInBytes;
Vector<Uint8> globalUboScratch;
Uint activeUniformCount = 0;
Uint maxUniformLocation = 0;
Int uniformNameMaxLength = 0;
Int attribInNameMaxLength = 0;
Int uniformBlockNameMaxLength = 0;
String infoLog;
Bool linkStatus = false;
// Transform feedback: the linked snapshot (the request lives outside, on the
// GL-thread-owned side).
Vector<XfbVarying> xfbVaryings;
Vector<Uint32> xfbStrides;
Vector<Uint32> gsStripTriangles;
Bool gsStripCaptureFixup = false;
GLenum gsInputPrimitive = GL_NONE;
GLenum xfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Int xfbVaryingNameMaxLength = 0;
Bool xfbNeedsScatteredCapture = false;
Uint32 xfbPackedStride = 0;
};
// Blocks until a pending link (P1 stage 4 onwards) has published its artifacts.
// Public because a few call sites have to join without reading anything - see the
// explicit-join list in the P1 design. Today there is never a pending link, so this
// is a no-op; it is wired up when glLinkProgram starts enqueueing.
void JoinLink() const { EnsureLinkJoined(); }
void SetTransformFeedbackVaryings(Vector<String>&& names, GLenum bufferMode) {
m_requestedXfbVaryings = Move(names);
m_requestedXfbBufferMode = bufferMode;
}
GLenum GetTransformFeedbackBufferMode() const { return m_xfbBufferMode; }
SizeT GetTransformFeedbackVaryingCount() const { return m_xfbVaryings.size(); }
GLenum GetTransformFeedbackBufferMode() const { return Artifacts().xfbBufferMode; }
SizeT GetTransformFeedbackVaryingCount() const { return Artifacts().xfbVaryings.size(); }
const XfbVarying* GetTransformFeedbackVarying(SizeT index) const {
return index < m_xfbVaryings.size() ? &m_xfbVaryings[index] : nullptr;
return index < Artifacts().xfbVaryings.size() ? &Artifacts().xfbVaryings[index] : nullptr;
}
const Vector<XfbVarying>& GetTransformFeedbackVaryings() const { return m_xfbVaryings; }
const Vector<XfbVarying>& GetTransformFeedbackVaryings() const { return Artifacts().xfbVaryings; }
// Stride of one captured vertex in the given capture buffer slot.
Uint32 GetTransformFeedbackStride(Uint32 bufferIndex) const {
return bufferIndex < m_xfbStrides.size() ? m_xfbStrides[bufferIndex] : 0;
return bufferIndex < Artifacts().xfbStrides.size() ? Artifacts().xfbStrides[bufferIndex] : 0;
}
SizeT GetTransformFeedbackBufferCount() const { return m_xfbStrides.size(); }
Int GetTransformFeedbackVaryingMaxLength() const { return m_xfbVaryingNameMaxLength; }
SizeT GetTransformFeedbackBufferCount() const { return Artifacts().xfbStrides.size(); }
Int GetTransformFeedbackVaryingMaxLength() const { return Artifacts().xfbVaryingNameMaxLength; }
// True when the capture layout uses gl_SkipComponents / gl_NextBuffer
// (ARB_transform_feedback3), which no ES driver can express: it can only pack every
// captured varying into one record with no gaps. A backend that captures through
// such a driver has to capture into scratch storage and scatter the records into the
// application's buffers itself, using packedOffsetBytes as the source offset and
// (bufferIndex, offsetBytes, stride) as the destination.
Bool NeedsScatteredTransformFeedbackCapture() const { return m_xfbNeedsScatteredCapture; }
Bool NeedsScatteredTransformFeedbackCapture() const { return Artifacts().xfbNeedsScatteredCapture; }
// Bytes one gap-free captured record occupies.
Uint32 GetTransformFeedbackPackedStride() const { return m_xfbPackedStride; }
Uint32 GetTransformFeedbackPackedStride() const { return Artifacts().xfbPackedStride; }
// True when the capture stage is a triangle-strip geometry shader with a
// statically-known emit sequence: the Vulkan capture order then needs the GL
// odd-triangle vertex swap after EndTransformFeedback.
Bool HasGsTriangleStripCaptureFixup() const { return m_gsStripCaptureFixup; }
Bool HasGsTriangleStripCaptureFixup() const { return Artifacts().gsStripCaptureFixup; }
// Triangles per strip, in emission order, for ONE geometry invocation.
const Vector<Uint32>& GetGsStripTriangles() const { return m_gsStripTriangles; }
const Vector<Uint32>& GetGsStripTriangles() const { return Artifacts().gsStripTriangles; }
// GL_GEOMETRY_INPUT_TYPE of the linked geometry stage (GL_POINTS, GL_LINES,
// GL_LINES_ADJACENCY, GL_TRIANGLES or GL_TRIANGLES_ADJACENCY), or GL_NONE when the
// program has no geometry stage. Draws must present a compatible primitive type.
GLenum GetGeometryInputType() const { return m_gsInputPrimitive; }
GLenum GetGeometryInputType() const { return Artifacts().gsInputPrimitive; }
Uint GetExternalIndex() const { return m_externalIndex; }
// Globally-unique, never-reused id for this program object's lifetime. Unlike the GL
@@ -589,99 +683,82 @@ namespace MobileGL::MG_State::GLState {
Uint64 GetLifetimeId() const { return m_lifetimeId; }
private:
// ---- The one and only join gate for link output (P1 invariant I5) ----
// Blocks until a pending link has finished and its LinkArtifacts have been
// published into m_artifacts. Today no link is ever pending - glLinkProgram still
// runs the whole body inline - so this is an unconditional no-op, and the whole
// Artifacts() indirection compiles away. It exists NOW so that the ~120 readers of
// link output are already routed through it when stage 4 makes it block: the edit
// that turns links asynchronous then touches this function and nothing else.
//
// Defined inline (not in ProgramObject.cpp) on purpose: this is called from every
// Artifacts() read - ~1200 call sites project-wide - and the project never builds
// with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line empty body would leave a
// real cross-TU call at every one of them instead of folding away. Stage 4's
// version, which actually blocks, moves the wait itself out-of-line behind a
// `m_pendingLink` check that stays inline here.
void EnsureLinkJoined() const {}
LinkArtifacts& Artifacts() {
EnsureLinkJoined();
return m_artifacts;
}
const LinkArtifacts& Artifacts() const {
EnsureLinkJoined();
return m_artifacts;
}
void ResetLinkArtifacts();
// GL-thread-only companion to ResetLinkArtifacts (see its definition).
void BumpLinkObservableVersions();
// Builds the GL-facing reflection surface from the linked TProgram. Returns false
// (with m_infoLog set and link artifacts reset) when reflection itself fails or an
// (with the artifacts' infoLog set and link artifacts reset) when reflection itself fails or an
// explicit-uniform-location conflict makes the link invalid.
Bool DoReflection();
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
// Resolves the requested transform feedback varyings against the linked
// vertex stage; fails the link (GL semantics) on unknown or duplicate
// names or exceeded capture limits.
Bool ResolveTransformFeedbackVaryings();
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
// The former GenerateBinary, split around DoReflection's data dependencies:
// SPIR-V must be generated BEFORE buildReflection touches m_program (its
// SPIR-V must be generated BEFORE buildReflection touches the linked TProgram (its
// live-variable analysis mutates the intermediates enough to change
// GlslangToSpv output), while the glUniform*-to-global-UBO routing tables are
// sized and keyed by reflection results (m_maxUniformLocation,
// m_uniformLocations) and so must run AFTER it.
// sized and keyed by reflection results (maxUniformLocation, uniformLocations)
// and so must run AFTER it.
void GenerateSpirv();
void BuildGlobalUboRouting();
void WaitUntilGenerationCompleted() const;
void AddDefaultFragmentShaderIfMissing();
Bool ValidateFragmentOutputLocations();
static Uint64 AllocateLifetimeId();
// ---- GL-thread-owned state: never joins ----
// Most of this is never produced by a link at all. The three version counters
// (m_backendStateVersion / m_uboContentVersion / m_linkVersion) ARE
// link-observable, but they are bumped exclusively on the GL thread
// (BumpLinkObservableVersions in Link()'s prologue and glProgramBinary's
// failure path) - the link BODY, which stage 4 moves to a worker, never
// writes them.
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = 0;
// The attach lists are mutated only in Link()'s GL-thread prologue, which is why
// glGetAttachedShaders / GL_ATTACHED_SHADERS / the orphan-shader sweep need no join.
Vector<SharedPtr<ShaderObject>> m_shaders;
Vector<SharedPtr<ShaderObject>> m_detachedShaders; // Store detached shaders and remove on next link
SharedPtr<glslang::TProgram> m_program;
Vector<Vector<unsigned>> m_generatedSpirv;
// Attributes (Vertex in)
// Link INPUTS (all "take effect at the next link" per GL): glBindAttribLocation,
// glBindFragDataLocation(Indexed), glTransformFeedbackVaryings, and the draw-buffer
// count stamped in by the entry point. A pending link snapshots these at enqueue.
UnorderedMap<String, Uint> m_explicitAttribLocations;
Vector<String> m_attribs;
Vector<GLenum> m_attribTypes;
// FragData (Frag out)
UnorderedMap<String, Uint> m_explicitFragDataLocation;
UnorderedMap<String, Uint> m_linkedFragDataLocation;
// Dual-source blend color index per output name (glBindFragDataLocationIndexed); snapshotted
// into the linked map at link time, like the location maps above.
UnorderedMap<String, Uint> m_explicitFragDataIndex;
UnorderedMap<String, Uint> m_linkedFragDataIndex;
Int m_maxFragmentOutputColorNumber = 8;
Vector<String> m_requestedXfbVaryings;
GLenum m_requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS;
// Uniforms
// GL-facing index spaces (see the translation helpers above): GL active-uniform
// index <-> glslang TProgram uniform index, GL uniform-block index <-> TProgram
// block index. -1 marks a TProgram entry GL does not expose (dead default-block
// uniforms swept into MGL_GLOBAL_UBO by the relaxed parse, and that block itself).
Vector<Int> m_glUniformIndexToTProgram;
Vector<Int> m_tProgramUniformIndexToGl;
Vector<Int> m_glBlockIndexToTProgram;
Vector<Int> m_tProgramBlockIndexToGl;
// Per-link merged snapshot of the attached shaders' lexically extracted
// layout(location = N) default-block uniform qualifiers (the relaxed parse drops
// them from reflection; the DoReflection assigner restores them from here).
UnorderedMap<String, Int> m_linkedExplicitUniformLocations;
UnorderedMap<String, Uint> m_uniformLocations;
// Ordered by location,
// aka. m_uniformIndexInTProgram[loc] == "uniform index of TProgram at location `loc`"
Vector<Int> m_uniformIndexInTProgram;
// ditto. Will be set at glUniform1i
Vector<Int> m_uniformSamplerOrImageUnitIndex;
UnorderedMap<String, Uint> m_explicitOpaqueUniformBindings;
// Ordered by uniform block index
// index is DIFFERENT from binding!!!
//
// Let's define UniformBlockIndex == the order at glslang getUniformBlock()
// aka `i = glGetUniformBlockIndex(prog, "BlockName")` implies:
// `prog->getUniformBlock(i) == "BlockName"`
// These stuff are present for GL semantics, not for backend inspection
// These may change after-link (because GL spec decided to have `glUniformBlockBinding`)
UnorderedMap<String, Uint> m_uniformBlockIndexByName;
Vector<Int> m_uniformBlockBinding;
// Need to be reflected after linking of SPIR-V binary
Vector<Uint> m_uniformOffsets;
Vector<Uint> m_uniformSizesInBytes;
Vector<Uint8> m_globalUboScratch;
Uint m_activeUniformCount = 0;
Uint m_maxUniformLocation = 0;
Int m_uniformNameMaxLength = 0;
Int m_attribInNameMaxLength = 0;
Int m_uniformBlockNameMaxLength = 0;
String m_infoLog;
Bool m_deleteStatus = false;
Bool m_linkStatus = false;
Bool m_binaryRetrievableHint = false;
Bool m_separable = false;
Bool m_validateStatus = true;
@@ -704,17 +781,9 @@ namespace MobileGL::MG_State::GLState {
Uint32 m_uboContentVersion = 0;
Uint32 m_linkVersion = 0;
// Transform feedback: request (applies at next link) and linked snapshot.
Vector<String> m_requestedXfbVaryings;
GLenum m_requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Vector<XfbVarying> m_xfbVaryings;
Vector<Uint32> m_xfbStrides;
Vector<Uint32> m_gsStripTriangles;
Bool m_gsStripCaptureFixup = false;
GLenum m_gsInputPrimitive = GL_NONE;
GLenum m_xfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Int m_xfbVaryingNameMaxLength = 0;
Bool m_xfbNeedsScatteredCapture = false;
Uint32 m_xfbPackedStride = 0;
// ---- Link OUTPUT ----
// Written by the link and by the post-link setters GL allows (glUniform1i's sampler
// unit, glUniformBlockBinding). Reachable only through Artifacts(); see LinkArtifacts.
LinkArtifacts m_artifacts;
};
} // namespace MobileGL::MG_State::GLState
MobileGL/MG_State/GLState/ProgramState/ProgramState.cpp
+1 -1
View File
@@ -79,7 +79,7 @@ namespace MobileGL::MG_State::GLState {
Uint shaderId = 0;
m_programShaderNameGenerator.Generate(1, &shaderId);
EnsureIndexAvail(shaderId, m_shaderObjects);
auto shaderObject = MakeShared<ShaderObject>(stage, shaderId, &m_shaderPreprocessCache);
auto shaderObject = MakeShared<ShaderObject>(stage, shaderId, m_shaderPreprocessCache);
if (shaderObject == nullptr) return 0;
m_shaderObjects[shaderId] = shaderObject;
return shaderId;
MobileGL/MG_State/GLState/ProgramState/ProgramState.h
+6 -5
View File
@@ -37,7 +37,7 @@ namespace MobileGL::MG_State::GLState {
// P0b layer 2. Exposed for tests and diagnostics; the GL frontend never touches it
// directly - shader objects reach it through the pointer they are handed at
// CreateShader().
ShaderPreprocessCache& GetShaderPreprocessCache() { return m_shaderPreprocessCache; }
ShaderPreprocessCache& GetShaderPreprocessCache() { return *m_shaderPreprocessCache; }
private:
Bool ShaderHasGLVisibleAttachment(const SharedPtr<ShaderObject>& shaderObject) const;
@@ -64,10 +64,11 @@ namespace MobileGL::MG_State::GLState {
// object kinds keeps the names disjoint; the object tables stay separate.
IndexGenerator<Uint> m_programShaderNameGenerator;
// P0b layer 2: every shader object created here is handed a pointer to this cache.
// Declared FIRST on purpose - members are destroyed in reverse declaration order,
// so the cache outlives every shader object holding a pointer to it.
ShaderPreprocessCache m_shaderPreprocessCache;
// P0b layer 2: every shader object created here is handed shared ownership of this
// cache, so its lifetime no longer depends on member destruction order (P1: an
// in-flight compile job may outlive the context). The FIRST-member declaration is
// kept anyway - it costs nothing and documents the intent.
SharedPtr<ShaderPreprocessCache> m_shaderPreprocessCache = MakeShared<ShaderPreprocessCache>();
Vector<SharedPtr<ProgramObject>> m_programObjects;
Vector<SharedPtr<ShaderObject>> m_shaderObjects;
MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp
+74 -77
View File
@@ -12,8 +12,8 @@
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h>
#include <MG_Backend/BackendObjects.h>
#include <charconv>
@@ -115,41 +115,22 @@ namespace {
return localSize;
}
static MobileGL::Uint GetComputeWorkGroupSizeLimit(MobileGL::Uint index) {
constexpr MobileGL::Uint kFrontendMinComputeWorkGroupSizes[] = {1024, 1024, 64};
MobileGL::Int backendValue = 0;
if (MobileGL::MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v) {
MobileGL::MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, index,
&backendValue);
}
// TODO: Share these exposed compute limit helpers with GL_Getter.cpp instead of duplicating the frontend minima.
return std::max(static_cast<MobileGL::Uint>(std::max(backendValue, 0)),
kFrontendMinComputeWorkGroupSizes[index]);
}
static unsigned long long GetComputeWorkGroupInvocationLimit() {
constexpr unsigned long long kFrontendMaxComputeWorkGroupInvocations = 1024;
if (!MobileGL::MG_Backend::pActiveBackendObject) return kFrontendMaxComputeWorkGroupInvocations;
return std::max(static_cast<unsigned long long>(std::max(
MobileGL::MG_Backend::pActiveBackendObject->GetDynamicParameters()
.MaxComputeWorkGroupInvocations,
0)),
kFrontendMaxComputeWorkGroupInvocations);
}
static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(const MobileGL::String& source) {
// The device limits come from the CompileEnv snapshot, never from a live driver query.
// GL_MAX_COMPUTE_WORK_GROUP_SIZE is a real GLES call on the DirectGLES backend: issued
// off the context thread it would silently no-op and turn a legal local_size_z into
// COMPILE_STATUS=FALSE. CaptureCompileEnv() issues it once, on the GL thread.
static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(
const MobileGL::String& source, const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
const ComputeLocalSize localSize = ParseComputeLocalSize(source);
if (!localSize.declared) return std::nullopt;
if (localSize.x > GetComputeWorkGroupSizeLimit(0) || localSize.y > GetComputeWorkGroupSizeLimit(1) ||
localSize.z > GetComputeWorkGroupSizeLimit(2)) {
if (localSize.x > env.maxComputeWorkGroupSize[0] || localSize.y > env.maxComputeWorkGroupSize[1] ||
localSize.z > env.maxComputeWorkGroupSize[2]) {
return "Compute shader local_size exceeds GL_MAX_COMPUTE_WORK_GROUP_SIZE.";
}
const unsigned long long invocations = static_cast<unsigned long long>(localSize.x) * localSize.y * localSize.z;
if (invocations > GetComputeWorkGroupInvocationLimit()) {
if (invocations > env.maxComputeWorkGroupInvocations) {
return "Compute shader local_size product exceeds GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS.";
}
@@ -162,23 +143,23 @@ namespace {
// nothing else - the glslang parse stays per-object because its TShader is
// consume-once. Deliberately free of any per-object state so the memo is sound.
//
// Caveat, documented rather than defended against: the compute local-size verdict also
// reads the active backend's GL_MAX_COMPUTE_WORK_GROUP_* limits. Those are fixed for
// the lifetime of a context, and the cache is per-context, so the memo cannot outlive
// the limits it was computed against.
// The former caveat is gone: the compute local-size verdict reads `env` rather than the
// live backend, and env.fingerprint is part of the P0b cache key, so a memo can never be
// returned against limits other than the ones it was computed against.
static MobileGL::MG_State::GLState::ShaderPreprocessResult RunSourceOnlyPipeline(
const MobileGL::ShaderStage stage, const MobileGL::String& source) {
const MobileGL::ShaderStage stage, const MobileGL::String& source,
const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
using namespace MobileGL;
using namespace MobileGL::MG_Util::ShaderTranspiler;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
MobileGL::MG_State::GLState::ShaderPreprocessResult result;
result.preprocessedSource = source;
PreprocessShaderSource(stage, result.preprocessedSource);
PreprocessShaderSource(stage, result.preprocessedSource, env);
if (stage == ShaderStage::Compute) {
if (const std::optional<String> localSizeError =
ValidateComputeLocalSizeLimits(result.preprocessedSource)) {
ValidateComputeLocalSizeLimits(result.preprocessedSource, env)) {
result.outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
result.infoLog = *localSizeError;
return result;
@@ -240,14 +221,14 @@ namespace MobileGL::MG_State::GLState {
m_compiledSourceLength = m_source.length();
}
// EnsureCompileJoined() is defined inline in ShaderObject.h (see the comment there for
// why: no LTO, and it is called from every Compiled() read).
void ShaderObject::InvalidateCompiledState() {
m_shader.reset();
m_preprocessedSource.clear();
m_explicitUniformLocations.clear();
m_explicitOpaqueBindings.clear();
m_shaderConsumedByLink = false;
m_compileStatus = false;
m_infoLog.clear();
// The compile artifacts are exactly what one Compile() writes, so discarding them
// wholesale IS the invalidation. (Kept as an explicit reset rather than a
// default-construct so the intent survives a future field addition.)
Compiled() = CompileArtifacts{};
m_hasCompiledState = false;
m_compiledSourceHash = 0;
m_compiledSourceLength = 0;
@@ -260,8 +241,8 @@ namespace MobileGL::MG_State::GLState {
// the exact source it still holds, so a recompile is a no-op. This covers the
// failure case too - the info log stays queryable because nothing is cleared.
//
// m_shaderConsumedByLink interaction: if the stored TShader already fed a link,
// the no-op leaves m_preprocessedSource and both side-channel maps intact, which
// shaderConsumedByLink interaction: if the stored TShader already fed a link,
// the no-op leaves preprocessedSource and both side-channel maps intact, which
// is precisely what TakeShaderForLink's on-demand re-parse needs. A real recompile
// would have handed the next link a fresh parse; the no-op hands it a fresh
// re-parse of the identical source instead. Same result, one parse either way.
@@ -271,59 +252,72 @@ namespace MobileGL::MG_State::GLState {
const Uint64 sourceHash = ShaderPreprocessCache::HashSource(m_source);
// The compile-environment snapshot, taken here on the GL thread. Everything below
// reads the device through it and never through pActiveBackendObject, which is what
// makes the whole body movable onto a worker in stage 3.
CompileArtifacts& compiled = Compiled();
compiled.env = MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
const MG_Util::ShaderTranspiler::CompileEnv& env = *compiled.env;
// P0b layer 2: another shader object in this context may already have run the
// source-only half over byte-identical text.
const ShaderPreprocessResult* cached =
m_preprocessCache != nullptr ? m_preprocessCache->Find(m_stage, sourceHash, m_source) : nullptr;
ShaderPreprocessResult fresh;
if (cached == nullptr) fresh = RunSourceOnlyPipeline(m_stage, m_source);
const ShaderPreprocessResult& shared = cached != nullptr ? *cached : fresh;
const Bool shouldPopulateCache = cached == nullptr && m_preprocessCache != nullptr;
// source-only half over byte-identical text under the same environment.
ShaderPreprocessResultPtr cached =
m_preprocessCache ? m_preprocessCache->Find(m_stage, sourceHash, m_source, env.fingerprint) : nullptr;
SharedPtr<ShaderPreprocessResult> fresh;
if (!cached) fresh = MakeShared<ShaderPreprocessResult>(RunSourceOnlyPipeline(m_stage, m_source, env));
const ShaderPreprocessResult& shared = cached ? *cached : *fresh;
const Bool shouldPopulateCache = !cached && m_preprocessCache != nullptr;
if (!shared.Preprocessed()) {
// Rejected lexically, or a glslang failure this context has already seen for
// this exact source (ParseFailed) - either way the parse can be skipped.
m_infoLog = shared.infoLog;
if (shouldPopulateCache) m_preprocessCache->Insert(m_stage, sourceHash, m_source, Move(fresh));
compiled.infoLog = shared.infoLog;
if (shouldPopulateCache) {
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
RememberCompiledSource(sourceHash);
return;
}
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = shared.preprocessedSource,
.flags = 0};
.flags = 0,
.env = &env};
auto result = ShaderCompiler::CompileShader(attrib);
if (result) {
m_compileStatus = true;
m_shader = result.value();
compiled.compileStatus = true;
compiled.shader = result.value();
// Copy, not move: `shared` may alias a cache entry that has to outlive us, and
// `fresh` is about to be handed to the cache.
m_preprocessedSource = shared.preprocessedSource;
m_explicitUniformLocations = shared.explicitUniformLocations;
m_explicitOpaqueBindings = shared.explicitOpaqueBindings;
m_infoLog.clear();
if (shouldPopulateCache) m_preprocessCache->Insert(m_stage, sourceHash, m_source, Move(fresh));
} else {
m_infoLog = result.error().log;
MGLOG_D("ShaderObject::Compile: Shader %d compilation failed.\nSource:\n%s\nInfoLog:\n%s\nSetting "
"m_compileStatus = false as a result.",
m_externalIndex, shared.preprocessedSource.c_str(), m_infoLog.c_str());
compiled.preprocessedSource = shared.preprocessedSource;
compiled.explicitUniformLocations = shared.explicitUniformLocations;
compiled.explicitOpaqueBindings = shared.explicitOpaqueBindings;
compiled.infoLog.clear();
if (shouldPopulateCache) {
fresh.outcome = ShaderPreprocessOutcome::ParseFailed;
fresh.infoLog = m_infoLog;
fresh.explicitUniformLocations.clear();
fresh.explicitOpaqueBindings.clear();
m_preprocessCache->Insert(m_stage, sourceHash, m_source, Move(fresh));
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
} else {
compiled.infoLog = result.error().log;
MGLOG_D("ShaderObject::Compile: Shader %d compilation failed.\nSource:\n%s\nInfoLog:\n%s\nSetting "
"compileStatus = false as a result.",
m_externalIndex, shared.preprocessedSource.c_str(), compiled.infoLog.c_str());
if (shouldPopulateCache) {
fresh->outcome = ShaderPreprocessOutcome::ParseFailed;
fresh->infoLog = compiled.infoLog;
fresh->explicitUniformLocations.clear();
fresh->explicitOpaqueBindings.clear();
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
}
RememberCompiledSource(sourceHash);
}
SharedPtr<glslang::TShader> ShaderObject::TakeShaderForLink(String& outReparseLog) {
if (m_shader && !m_shaderConsumedByLink) {
m_shaderConsumedByLink = true;
return m_shader;
CompileArtifacts& compiled = Compiled();
if (compiled.shader && !compiled.shaderConsumedByLink) {
compiled.shaderConsumedByLink = true;
return compiled.shader;
}
// The stored parse already fed a link, whose mapIO mutated its intermediate.
@@ -332,8 +326,11 @@ namespace MobileGL::MG_State::GLState {
// here on EVERY link rather than only on reuse.
using namespace MG_Util::ShaderTranspiler;
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = m_preprocessedSource,
.flags = 0};
.sourceStr = compiled.preprocessedSource,
.flags = 0,
// Re-parse against the SAME environment the original parse used,
// not against whatever the backend reports now.
.env = compiled.env.get()};
auto result = ShaderCompiler::CompileShader(attrib);
if (!result) {
// Should be unreachable: the same source parsed successfully at Compile().
MobileGL/MG_State/GLState/ProgramState/ShaderObject.h
+77 -23
View File
@@ -8,6 +8,7 @@
#pragma once
#include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL {
enum class ShaderStage {
@@ -31,9 +32,12 @@ namespace MobileGL {
// `preprocessCache` is the owning context's cross-object memo (P0b layer 2);
// null is fully supported and simply means "no sharing" - that is what the
// context-less internal shader objects (the default FS, the blit pipeline) use.
// Shared ownership rather than a raw pointer: once compiles run on a worker the
// job outlives neither the object nor the context deterministically, and the
// cache has to stay alive for whoever is still reading it.
ShaderObject(const ShaderStage stage, Uint externalIndex,
ShaderPreprocessCache* preprocessCache = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(preprocessCache) {}
SharedPtr<ShaderPreprocessCache> preprocessCache = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(Move(preprocessCache)) {}
void SetShaderSource(const String& source);
void SetShaderSource(String&& source);
void Compile();
@@ -51,29 +55,83 @@ namespace MobileGL {
Uint GetExternalIndex() const { return m_externalIndex; }
ShaderStage GetShaderStage() const { return m_stage; }
const String& GetShaderSource() const { return m_source; }
const SharedPtr<glslang::TShader>& GetCompiledShader() const { return m_shader; }
const String& GetInfoLog() const { return m_infoLog; }
const UnorderedMap<String, Uint>& GetUniformLocations() const { return m_uniforms; }
const SharedPtr<glslang::TShader>& GetCompiledShader() const { return Compiled().shader; }
const String& GetInfoLog() const { return Compiled().infoLog; }
const UnorderedMap<String, Uint>& GetUniformLocations() const { return Compiled().uniforms; }
// Explicit layout(location = N) qualifiers on this shader's default-block
// uniforms, captured lexically at Compile() because the relaxed parse drops
// them from reflection (see ExtractExplicitUniformLocations).
const UnorderedMap<String, Int>& GetExplicitUniformLocations() const {
return m_explicitUniformLocations;
return Compiled().explicitUniformLocations;
}
// Explicit layout(binding = N) on sampler/image uniforms - their initial
// texture/image units - captured lexically for the same reason (see
// ExtractExplicitOpaqueBindings).
const UnorderedMap<String, Uint>& GetExplicitOpaqueBindings() const { return m_explicitOpaqueBindings; }
Bool GetCompileStatus() const { return m_compileStatus; }
const UnorderedMap<String, Uint>& GetExplicitOpaqueBindings() const {
return Compiled().explicitOpaqueBindings;
}
Bool GetCompileStatus() const { return Compiled().compileStatus; }
Bool GetDeleteStatus() const { return m_deleteStatus; }
// Blocks until a pending compile (P1 stage 3 onwards) has published its
// artifacts. Public for the few sites that must join without reading anything.
// A no-op today - nothing is ever pending.
void JoinCompile() const { EnsureCompileJoined(); }
// True while this object holds the outcome (success OR failure) of a previous
// Compile() of exactly the source it currently holds - i.e. while the P0b
// layer-1 memo is armed and a glCompileShader would be a no-op. Diagnostics
// and tests only; nothing in the GL frontend branches on it.
//
// Deliberately does NOT join: the memo bookkeeping below is GL-thread-owned and
// says nothing about whether a worker has finished, which is exactly the
// property GL_COMPLETION_STATUS_KHR needs when stage 3 lands.
Bool HasMemoizedCompile() const { return m_hasCompiledState; }
private:
// ---- P1: everything a compile PRODUCES, in one block ----
//
// Same rule as ProgramObject::LinkArtifacts: this is exactly what
// InvalidateCompiledState() clears, i.e. exactly what one run of Compile()
// writes. Stage 3 lifts this struct wholesale into ShaderCompileTask, where a
// worker fills it in and the GL thread reads it through the same gate.
struct CompileArtifacts {
// The CompileEnv snapshot this compile ran against. Held so the
// consume-once re-parse in TakeShaderForLink() reproduces the original
// parse exactly, instead of re-reading whatever the backend says now.
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
SharedPtr<glslang::TShader> shader;
// The source Compile() actually parsed (after PreprocessShaderSource), kept
// for TakeShaderForLink's re-parse so a later link never depends on the
// preprocessor being deterministic across backend-state changes.
String preprocessedSource;
UnorderedMap<String, Uint> uniforms;
UnorderedMap<String, Int> explicitUniformLocations;
UnorderedMap<String, Uint> explicitOpaqueBindings;
Bool shaderConsumedByLink = false;
String infoLog;
Bool compileStatus = false;
};
// ---- The one and only join gate for compile output (P1 invariant I5) ----
// Blocks until a pending compile has published into m_compiled. Today nothing
// is ever pending - glCompileShader still runs the whole body inline - so this
// is an unconditional no-op. It exists NOW so that every reader of compile
// output is already routed through it when stage 3 makes it block.
//
// Defined inline (not in ShaderObject.cpp): called from every Compiled() read,
// and the project never builds with LTO, so an out-of-line empty body would be
// a real cross-TU call at each of those call sites instead of folding away.
void EnsureCompileJoined() const {}
CompileArtifacts& Compiled() {
EnsureCompileJoined();
return m_compiled;
}
const CompileArtifacts& Compiled() const {
EnsureCompileJoined();
return m_compiled;
}
void InvalidateCompiledState();
// ---- P0b layer 1: per-object no-op recompile ----
// True iff `candidate` is byte-identical to the source that produced the
@@ -84,31 +142,27 @@ namespace MobileGL {
// Arms the layer-1 memo for the source that Compile() just processed.
void RememberCompiledSource(Uint64 sourceHash);
// ---- GL-thread-owned state: never produced by a compile, so it never joins ----
const Uint m_externalIndex = 0;
const ShaderStage m_stage;
// glShaderSource text. A worker only ever reads the snapshot handed to it, so
// GL_SHADER_SOURCE_LENGTH and glGetShaderSource never join.
String m_source;
// The source Compile() actually parsed (after PreprocessShaderSource), kept
// for TakeShaderForLink's re-parse so a later link never depends on the
// preprocessor being deterministic across backend-state changes.
String m_preprocessedSource;
SharedPtr<glslang::TShader> m_shader;
UnorderedMap<String, Uint> m_uniforms;
UnorderedMap<String, Int> m_explicitUniformLocations;
UnorderedMap<String, Uint> m_explicitOpaqueBindings;
Bool m_shaderConsumedByLink = false;
// P0b layer 2: the owning context's cross-object memo, or null. Not owned.
ShaderPreprocessCache* const m_preprocessCache = nullptr;
// P0b layer 2: the owning context's cross-object memo, or null.
const SharedPtr<ShaderPreprocessCache> m_preprocessCache;
// P0b layer 1. m_hasCompiledState is the invariant "m_source is byte-identical
// to the source that produced m_compileStatus/m_infoLog/m_shader"; it is armed
// at the end of every Compile() and disarmed by InvalidateCompiledState().
// to the source that produced the compile artifacts"; it is armed at the end of
// every Compile() and disarmed by InvalidateCompiledState(). Stage 3 replaces
// all three with a pointer compare against the in-flight job's source snapshot.
Bool m_hasCompiledState = false;
Uint64 m_compiledSourceHash = 0;
SizeT m_compiledSourceLength = 0;
String m_infoLog;
Bool m_deleteStatus = false;
Bool m_compileStatus = false;
// ---- Compile OUTPUT ---- reachable only through Compiled().
CompileArtifacts m_compiled;
};
} // namespace MG_State::GLState
} // namespace MobileGL
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
+28 -13
View File
@@ -9,9 +9,14 @@
#include "ShaderPreprocessCache.h"
namespace MobileGL::MG_State::GLState {
const ShaderPreprocessResult* ShaderPreprocessCache::Find(const ShaderStage stage, const Uint64 sourceHash,
const String& source) const {
const Key key{.stage = stage, .sourceHash = sourceHash, .sourceLength = source.length()};
ShaderPreprocessResultPtr ShaderPreprocessCache::Find(const ShaderStage stage, const Uint64 sourceHash,
const String& source, const Uint64 envFingerprint) const {
const Key key{.stage = stage,
.sourceHash = sourceHash,
.sourceLength = source.length(),
.envFingerprint = envFingerprint};
const std::lock_guard<std::mutex> lock(m_mutex);
const auto it = m_index.find(key);
if (it == m_index.end()) return nullptr;
@@ -20,52 +25,62 @@ namespace MobileGL::MG_State::GLState {
const Entry& entry = *it->second;
if (entry.originalSource != source) return nullptr;
return &entry.result;
// A copy of the SharedPtr, taken under the lock: the payload now outlives any
// eviction the caller races with.
return entry.result;
}
void ShaderPreprocessCache::Insert(const ShaderStage stage, const Uint64 sourceHash, const String& source,
ShaderPreprocessResult result) {
const SizeT entryBytes = EntryBytes(source, result);
const Uint64 envFingerprint, ShaderPreprocessResultPtr result) {
if (!result) return;
const SizeT entryBytes = EntryBytes(source, *result);
// A single source bigger than the whole budget would evict every other entry and
// then itself; refuse it instead of thrashing the cache empty.
if (entryBytes > kMaxStoredSourceBytes) return;
const Key key{.stage = stage, .sourceHash = sourceHash, .sourceLength = source.length()};
const Key key{.stage = stage,
.sourceHash = sourceHash,
.sourceLength = source.length(),
.envFingerprint = envFingerprint};
const std::lock_guard<std::mutex> lock(m_mutex);
if (const auto existing = m_index.find(key); existing != m_index.end()) {
// Either a re-insert of the same source (harmless) or a genuine hash collision
// with a different source. Both are resolved by letting the newcomer win: one
// entry per key keeps the index a plain map, and a collision is astronomically
// rare enough that the loser simply misses.
EraseEntry(existing->second);
EraseEntryLocked(existing->second);
}
m_entries.push_back(Entry{.key = key, .originalSource = source, .result = Move(result)});
m_index[key] = std::prev(m_entries.end());
m_storedSourceBytes += entryBytes;
EvictUntilWithinBudget();
EvictUntilWithinBudgetLocked();
}
void ShaderPreprocessCache::Clear() {
const std::lock_guard<std::mutex> lock(m_mutex);
m_entries.clear();
m_index.clear();
m_storedSourceBytes = 0;
}
void ShaderPreprocessCache::EraseEntry(const EntryList::iterator it) {
const SizeT bytes = EntryBytes(it->originalSource, it->result);
void ShaderPreprocessCache::EraseEntryLocked(const EntryList::iterator it) {
const SizeT bytes = EntryBytes(it->originalSource, *it->result);
m_storedSourceBytes = bytes > m_storedSourceBytes ? 0 : m_storedSourceBytes - bytes;
m_index.erase(it->key);
m_entries.erase(it);
}
void ShaderPreprocessCache::EvictUntilWithinBudget() {
void ShaderPreprocessCache::EvictUntilWithinBudgetLocked() {
// FIFO: the oldest insertion goes first. Insert() already refuses entries larger
// than the byte budget, so this loop always terminates with at least the entry
// that was just added still resident.
while (!m_entries.empty() &&
(m_entries.size() > kMaxEntries || m_storedSourceBytes > kMaxStoredSourceBytes)) {
EraseEntry(m_entries.begin());
EraseEntryLocked(m_entries.begin());
}
}
} // namespace MobileGL::MG_State::GLState
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.h
+40 -15
View File
@@ -9,6 +9,7 @@
#pragma once
#include <Includes.h>
#include <list>
#include <mutex>
#include <MG_State/GLState/ProgramState/ShaderObject.h>
namespace MobileGL::MG_State::GLState {
@@ -42,6 +43,11 @@ namespace MobileGL::MG_State::GLState {
Bool Preprocessed() const { return outcome == ShaderPreprocessOutcome::Preprocessed; }
};
// Cache hits hand out shared ownership, not a raw pointer into the entry list. That is
// what makes the cache safe once compiles run concurrently: a reader keeps its payload
// alive across any eviction, and a 107 KB preprocessedSource is never copied on a hit.
using ShaderPreprocessResultPtr = SharedPtr<const ShaderPreprocessResult>;
// P0b layer 2: a per-context, bounded memo of the source-only half of shader
// compilation, keyed by (stage, xxhash64(source), source length).
//
@@ -71,14 +77,22 @@ namespace MobileGL::MG_State::GLState {
static constexpr SizeT kMaxEntries = 128;
static constexpr SizeT kMaxStoredSourceBytes = 8u * 1024u * 1024u;
// Returns the memoized result for this exact source, or null on a miss. The
// returned pointer stays valid until the next Insert()/Clear() on this cache.
const ShaderPreprocessResult* Find(ShaderStage stage, Uint64 sourceHash, const String& source) const;
// Returns the memoized result for this exact source under this exact compile
// environment, or null on a miss. The returned SharedPtr owns its payload, so it
// stays valid for as long as the caller holds it - across Insert(), Clear(), and
// across the destruction of the cache itself.
//
// envFingerprint joins the key because the source-only pipeline's compute
// local-size verdict is computed against CompileEnv's device limits: a memo must
// never outlive the environment it was computed against (memo-hazard rule).
ShaderPreprocessResultPtr Find(ShaderStage stage, Uint64 sourceHash, const String& source,
Uint64 envFingerprint) const;
// Memoizes `result` for this source. A source whose own storage cost already
// exceeds the byte budget is simply not cached (caching it would evict everything
// else and then itself).
void Insert(ShaderStage stage, Uint64 sourceHash, const String& source, ShaderPreprocessResult result);
void Insert(ShaderStage stage, Uint64 sourceHash, const String& source, Uint64 envFingerprint,
ShaderPreprocessResultPtr result);
void Clear();
@@ -86,17 +100,25 @@ namespace MobileGL::MG_State::GLState {
return static_cast<Uint64>(XXH64(source.data(), source.length(), 0));
}
SizeT GetEntryCount() const { return m_entries.size(); }
SizeT GetStoredSourceBytes() const { return m_storedSourceBytes; }
SizeT GetEntryCount() const {
const std::lock_guard<std::mutex> lock(m_mutex);
return m_entries.size();
}
SizeT GetStoredSourceBytes() const {
const std::lock_guard<std::mutex> lock(m_mutex);
return m_storedSourceBytes;
}
private:
struct Key {
ShaderStage stage = ShaderStage::Unknown;
Uint64 sourceHash = 0;
SizeT sourceLength = 0;
Uint64 envFingerprint = 0;
Bool operator==(const Key& other) const {
return stage == other.stage && sourceHash == other.sourceHash && sourceLength == other.sourceLength;
return stage == other.stage && sourceHash == other.sourceHash &&
sourceLength == other.sourceLength && envFingerprint == other.envFingerprint;
}
};
@@ -107,6 +129,7 @@ namespace MobileGL::MG_State::GLState {
Uint64 mixed = key.sourceHash;
mixed ^= static_cast<Uint64>(key.sourceLength) + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
mixed ^= static_cast<Uint64>(static_cast<Int>(key.stage)) * 0xff51afd7ed558ccdull;
mixed ^= key.envFingerprint + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
return static_cast<SizeT>(mixed);
}
};
@@ -116,7 +139,7 @@ namespace MobileGL::MG_State::GLState {
// The full original (pre-preprocess) source, kept so a hit can be confirmed by
// comparison instead of trusting the hash.
String originalSource;
ShaderPreprocessResult result;
ShaderPreprocessResultPtr result;
};
using EntryList = std::list<Entry>;
@@ -125,14 +148,16 @@ namespace MobileGL::MG_State::GLState {
return source.length() + result.preprocessedSource.length();
}
void EraseEntry(EntryList::iterator it);
void EvictUntilWithinBudget();
void EvictUntilWithinBudgetLocked();
// P1: needs a mutex when compiles go async. Everything here is reached from
// glCompileShader on the single GL thread that owns the context, so today the
// cache is deliberately lock-free; the moment shader compilation moves onto a
// worker pool, Find/Insert/Clear all become critical sections (and Find's returned
// pointer stops being safe to hold across an Insert).
void EraseEntryLocked(EntryList::iterator it);
// P1: every public entry point takes this. The lock alone would NOT have been
// enough - the old Find() handed back a raw pointer into an entry that a
// concurrent Insert()'s FIFO eviction could erase while the caller was still
// reading it. Shared ownership of the payload is what closes that hole; the mutex
// only protects the containers below.
mutable std::mutex m_mutex;
EntryList m_entries; // front = oldest (FIFO victim)
UnorderedMap<Key, EntryList::iterator, KeyHasher> m_index;
SizeT m_storedSourceBytes = 0;
MobileGL/MG_Test/Program/ProgramUtilTest.cpp
+154 -39
View File
@@ -21,6 +21,7 @@
#include <MG_Util/ShaderTranspiler/Types.h>
#include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h>
#include <MG_State/GLState/ProgramState/ShaderPreprocessCache.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <spirv-tools/libspirv.hpp>
#include <spirv-tools/optimizer.hpp>
@@ -2378,6 +2379,78 @@ void main() {
}
// ---------------------------------------------------------------------------
// ---------------------------------------------------------------------------
// P1: CompileEnv - the compile pipeline's snapshot of everything outside
// (stage, source). These pin the two properties the rest of P1 rides on: the
// compute limits really are carried in the snapshot (an off-thread
// GL_MAX_COMPUTE_WORK_GROUP_SIZE query would silently return 0 and reject a
// legal local_size), and the fingerprint really does move when they do.
// ---------------------------------------------------------------------------
TEST_F(ProgramUtilTest, CompileEnvCarriesComputeLimitsAndFrontendMinima) {
using MobileGL::MG_Util::ShaderTranspiler::CaptureCompileEnv;
const auto env = CaptureCompileEnv();
ASSERT_NE(env, nullptr);
// With no backend the snapshot is the frontend minimum, never zero - the value an
// off-thread GetIntegeri_v would have left behind.
EXPECT_GE(env->maxComputeWorkGroupSize[0], 1024u);
EXPECT_GE(env->maxComputeWorkGroupSize[1], 1024u);
EXPECT_GE(env->maxComputeWorkGroupSize[2], 64u);
EXPECT_GE(env->maxComputeWorkGroupInvocations, 1024u);
EXPECT_NE(env->fingerprint, 0u);
}
TEST_F(ProgramUtilTest, CompileEnvFingerprintTracksEveryInput) {
using MobileGL::MG_Util::ShaderTranspiler::CompileEnv;
using MobileGL::MG_Util::ShaderTranspiler::ComputeCompileEnvFingerprint;
CompileEnv base;
const Uint64 baseline = ComputeCompileEnvFingerprint(base);
EXPECT_EQ(ComputeCompileEnvFingerprint(base), baseline) << "fingerprint must be deterministic";
// A device that allows a bigger workgroup than the frontend minimum is a DIFFERENT
// compile environment: a memo taken under the smaller limit must not be reusable.
CompileEnv biggerZ = base;
biggerZ.maxComputeWorkGroupSize[2] = 256;
EXPECT_NE(ComputeCompileEnvFingerprint(biggerZ), baseline);
CompileEnv moreInvocations = base;
moreInvocations.maxComputeWorkGroupInvocations = 2048;
EXPECT_NE(ComputeCompileEnvFingerprint(moreInvocations), baseline);
CompileEnv otherBackend = base;
otherBackend.backend = MobileGL::BackendType::DirectVulkan;
EXPECT_NE(ComputeCompileEnvFingerprint(otherBackend), baseline);
CompileEnv otherLimits = base;
otherLimits.params.MaxVertexAttribs = 31;
EXPECT_NE(ComputeCompileEnvFingerprint(otherLimits), baseline);
CompileEnv otherExtensions = base;
otherExtensions.advertisedExtensions.push_back(MobileGL::E_GL_ARB_gpu_shader_int64);
EXPECT_NE(ComputeCompileEnvFingerprint(otherExtensions), baseline);
CompileEnv otherQuirk = base;
otherQuirk.subgroupPrefixScanQuirk = MobileGL::MG_Config::QuirkOverride::ForceOn;
EXPECT_NE(ComputeCompileEnvFingerprint(otherQuirk), baseline);
}
// The no-backend fallback must stay exactly what the pipeline used to do inline:
// everything counts as advertised, because there is nothing to gate against.
TEST_F(ProgramUtilTest, CompileEnvWithoutBackendAdvertisesEverything) {
using MobileGL::MG_Util::ShaderTranspiler::CompileEnv;
CompileEnv env;
EXPECT_FALSE(env.HasBackend());
EXPECT_TRUE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
env.backend = MobileGL::BackendType::DirectGLES;
EXPECT_TRUE(env.HasBackend());
EXPECT_FALSE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
env.advertisedExtensions.push_back(MobileGL::E_GL_ARB_gpu_shader_int64);
EXPECT_TRUE(env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64));
}
// P0b layer 2: ShaderPreprocessCache, tested directly. The GL-level behaviour it
// enables is covered end to end in ProgramTest; these pin the container itself,
// where the interesting cases (hash collisions, both eviction budgets) are hard
@@ -2387,13 +2460,19 @@ namespace {
using MobileGL::MG_State::GLState::ShaderPreprocessCache;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
using MobileGL::MG_State::GLState::ShaderPreprocessResult;
using MobileGL::MG_State::GLState::ShaderPreprocessResultPtr;
ShaderPreprocessResult MakeResult(const String& preprocessed) {
ShaderPreprocessResult result;
result.outcome = ShaderPreprocessOutcome::Preprocessed;
result.preprocessedSource = preprocessed;
result.explicitUniformLocations["uMarker"] = 7;
result.explicitOpaqueBindings["sMarker"] = 3;
// The env fingerprint every test below keys against, unless it is specifically
// exercising the fingerprint itself.
constexpr MobileGL::Uint64 kEnvA = 0x1111'2222'3333'4444ull;
constexpr MobileGL::Uint64 kEnvB = 0x5555'6666'7777'8888ull;
ShaderPreprocessResultPtr MakeResult(const String& preprocessed) {
auto result = MakeShared<ShaderPreprocessResult>();
result->outcome = ShaderPreprocessOutcome::Preprocessed;
result->preprocessedSource = preprocessed;
result->explicitUniformLocations["uMarker"] = 7;
result->explicitOpaqueBindings["sMarker"] = 3;
return result;
}
} // namespace
@@ -2403,10 +2482,10 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheRoundTripsAndSeparatesStages) {
const String source = "// a shader\nvoid main() {}\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA), nullptr);
cache.Insert(ShaderStage::Vertex, hash, source, MakeResult("vertex-preprocessed"));
const ShaderPreprocessResult* hit = cache.Find(ShaderStage::Vertex, hash, source);
cache.Insert(ShaderStage::Vertex, hash, source, kEnvA, MakeResult("vertex-preprocessed"));
const ShaderPreprocessResultPtr hit = cache.Find(ShaderStage::Vertex, hash, source, kEnvA);
ASSERT_NE(hit, nullptr);
EXPECT_TRUE(hit->Preprocessed());
EXPECT_EQ(hit->preprocessedSource, "vertex-preprocessed");
@@ -2419,12 +2498,12 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheRoundTripsAndSeparatesStages) {
// Byte-identical source, different stage: a different key, so still a miss. Two
// stages sharing one entry would hand a fragment shader a vertex preprocess.
EXPECT_EQ(cache.Find(ShaderStage::Fragment, hash, source), nullptr);
cache.Insert(ShaderStage::Fragment, hash, source, MakeResult("fragment-preprocessed"));
const ShaderPreprocessResult* fragmentHit = cache.Find(ShaderStage::Fragment, hash, source);
EXPECT_EQ(cache.Find(ShaderStage::Fragment, hash, source, kEnvA), nullptr);
cache.Insert(ShaderStage::Fragment, hash, source, kEnvA, MakeResult("fragment-preprocessed"));
const ShaderPreprocessResultPtr fragmentHit = cache.Find(ShaderStage::Fragment, hash, source, kEnvA);
ASSERT_NE(fragmentHit, nullptr);
EXPECT_EQ(fragmentHit->preprocessedSource, "fragment-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source)->preprocessedSource, "vertex-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA)->preprocessedSource, "vertex-preprocessed");
EXPECT_EQ(cache.GetEntryCount(), 2u);
}
@@ -2433,27 +2512,27 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheMemoizesRejectionVerdictsDistinctly
const String reservedSource = "int packed;\n";
const String localSizeSource = "layout(local_size_x = 99999) in;\n";
ShaderPreprocessResult reserved;
reserved.outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
reserved.infoLog = "reserved identifier";
ShaderPreprocessResult localSize;
localSize.outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
localSize.infoLog = "local_size too big";
auto reserved = MakeShared<ShaderPreprocessResult>();
reserved->outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
reserved->infoLog = "reserved identifier";
auto localSize = MakeShared<ShaderPreprocessResult>();
localSize->outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
localSize->infoLog = "local_size too big";
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource,
std::move(reserved));
cache.Insert(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource,
std::move(localSize));
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource, kEnvA,
Move(reserved));
cache.Insert(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource, kEnvA,
Move(localSize));
const ShaderPreprocessResult* reservedHit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource);
const ShaderPreprocessResultPtr reservedHit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(reservedSource), reservedSource, kEnvA);
ASSERT_NE(reservedHit, nullptr);
EXPECT_FALSE(reservedHit->Preprocessed());
EXPECT_EQ(reservedHit->outcome, ShaderPreprocessOutcome::ReservedIdentifierRejected);
EXPECT_EQ(reservedHit->infoLog, "reserved identifier");
const ShaderPreprocessResult* localSizeHit =
cache.Find(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource);
const ShaderPreprocessResultPtr localSizeHit =
cache.Find(ShaderStage::Compute, ShaderPreprocessCache::HashSource(localSizeSource), localSizeSource, kEnvA);
ASSERT_NE(localSizeHit, nullptr);
EXPECT_EQ(localSizeHit->outcome, ShaderPreprocessOutcome::ComputeLocalSizeRejected);
EXPECT_EQ(localSizeHit->infoLog, "local_size too big");
@@ -2470,28 +2549,64 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheRejectsForgedHashCollision) {
ASSERT_NE(real, impostor);
const Uint64 forgedHash = 0xdeadbeefcafef00dull;
cache.Insert(ShaderStage::Vertex, forgedHash, real, MakeResult("real-preprocessed"));
cache.Insert(ShaderStage::Vertex, forgedHash, real, kEnvA, MakeResult("real-preprocessed"));
ASSERT_NE(cache.Find(ShaderStage::Vertex, forgedHash, real), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, impostor), nullptr);
ASSERT_NE(cache.Find(ShaderStage::Vertex, forgedHash, real, kEnvA), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, impostor, kEnvA), nullptr);
// The colliding newcomer wins the slot rather than being silently dropped, so it
// is the previous occupant that degrades to a miss - never a wrong hit.
cache.Insert(ShaderStage::Vertex, forgedHash, impostor, MakeResult("impostor-preprocessed"));
const ShaderPreprocessResult* impostorHit = cache.Find(ShaderStage::Vertex, forgedHash, impostor);
cache.Insert(ShaderStage::Vertex, forgedHash, impostor, kEnvA, MakeResult("impostor-preprocessed"));
const ShaderPreprocessResultPtr impostorHit = cache.Find(ShaderStage::Vertex, forgedHash, impostor, kEnvA);
ASSERT_NE(impostorHit, nullptr);
EXPECT_EQ(impostorHit->preprocessedSource, "impostor-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, real), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, forgedHash, real, kEnvA), nullptr);
EXPECT_EQ(cache.GetEntryCount(), 1u);
}
// P1: the compile environment joins the key. A memo computed against one backend's
// GL_MAX_COMPUTE_WORK_GROUP_* limits must never be handed back after the environment
// changed (backend swap), which is exactly what CompileEnv::fingerprint keys on.
TEST_F(ProgramUtilTest, ShaderPreprocessCacheMissesOnChangedEnvFingerprint) {
ShaderPreprocessCache cache;
const String source = "layout(local_size_x = 512) in;\nvoid main() {}\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
cache.Insert(ShaderStage::Compute, hash, source, kEnvA, MakeResult("env-a-preprocessed"));
ASSERT_NE(cache.Find(ShaderStage::Compute, hash, source, kEnvA), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvB), nullptr);
// Both environments can coexist; neither can see the other's verdict.
cache.Insert(ShaderStage::Compute, hash, source, kEnvB, MakeResult("env-b-preprocessed"));
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvA)->preprocessedSource, "env-a-preprocessed");
EXPECT_EQ(cache.Find(ShaderStage::Compute, hash, source, kEnvB)->preprocessedSource, "env-b-preprocessed");
EXPECT_EQ(cache.GetEntryCount(), 2u);
}
// A hit hands out shared ownership, so the payload survives the eviction of its entry.
// Under the old raw-pointer API this read was a use-after-free the moment two compiles
// ran concurrently.
TEST_F(ProgramUtilTest, ShaderPreprocessCacheHitOutlivesEviction) {
ShaderPreprocessCache cache;
const String source = "void main() { int keep = 1; }\n";
const Uint64 hash = ShaderPreprocessCache::HashSource(source);
cache.Insert(ShaderStage::Vertex, hash, source, kEnvA, MakeResult("survivor"));
const ShaderPreprocessResultPtr held = cache.Find(ShaderStage::Vertex, hash, source, kEnvA);
ASSERT_NE(held, nullptr);
cache.Clear();
EXPECT_EQ(cache.Find(ShaderStage::Vertex, hash, source, kEnvA), nullptr);
EXPECT_EQ(held->preprocessedSource, "survivor");
}
TEST_F(ProgramUtilTest, ShaderPreprocessCacheEvictsFifoOnEntryCap) {
ShaderPreprocessCache cache;
Vector<String> sources;
const SizeT overflow = ShaderPreprocessCache::kMaxEntries + 8;
for (SizeT i = 0; i < overflow; ++i) {
sources.push_back("void main() { int a = " + ToString(i) + "; }\n");
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources.back()), sources.back(),
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources.back()), sources.back(), kEnvA,
MakeResult("pp" + ToString(i)));
EXPECT_LE(cache.GetEntryCount(), ShaderPreprocessCache::kMaxEntries);
}
@@ -2499,8 +2614,8 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheEvictsFifoOnEntryCap) {
// FIFO: the first `overflow - kMaxEntries` insertions are gone, the rest resident.
for (SizeT i = 0; i < overflow; ++i) {
const ShaderPreprocessResult* hit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources[i]), sources[i]);
const ShaderPreprocessResultPtr hit =
cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(sources[i]), sources[i], kEnvA);
if (i < overflow - ShaderPreprocessCache::kMaxEntries) {
EXPECT_EQ(hit, nullptr) << "entry " << i << " should have been evicted";
} else {
@@ -2521,7 +2636,7 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheHonorsByteBudget) {
const SizeT chunk = ShaderPreprocessCache::kMaxStoredSourceBytes / 8;
for (SizeT i = 0; i < 24; ++i) {
String source(chunk, static_cast<char>('a' + (i % 26)));
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(source), source, MakeResult(""));
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(source), source, kEnvA, MakeResult(""));
EXPECT_LE(cache.GetStoredSourceBytes(), ShaderPreprocessCache::kMaxStoredSourceBytes);
EXPECT_LT(cache.GetEntryCount(), ShaderPreprocessCache::kMaxEntries);
}
@@ -2530,7 +2645,7 @@ TEST_F(ProgramUtilTest, ShaderPreprocessCacheHonorsByteBudget) {
// would evict every other entry and then immediately itself.
const SizeT before = cache.GetEntryCount();
const String oversized(ShaderPreprocessCache::kMaxStoredSourceBytes + 1, 'z');
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized, MakeResult(""));
cache.Insert(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized, kEnvA, MakeResult(""));
EXPECT_EQ(cache.GetEntryCount(), before);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized), nullptr);
EXPECT_EQ(cache.Find(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(oversized), oversized, kEnvA), nullptr);
}
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
+98
View File
@@ -0,0 +1,98 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "CompileEnv.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_State/GLState/Core.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
namespace {
void HashBytes(Uint64& state, const void* data, const SizeT length) {
state = static_cast<Uint64>(XXH64(data, length, state));
}
template <typename T>
void HashValue(Uint64& state, const T& value) {
static_assert(std::is_trivially_copyable_v<T>);
HashBytes(state, &value, sizeof(T));
}
} // namespace
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env) {
Uint64 state = 0x9e3779b97f4a7c15ull;
HashValue(state, env.maxComputeWorkGroupSize[0]);
HashValue(state, env.maxComputeWorkGroupSize[1]);
HashValue(state, env.maxComputeWorkGroupSize[2]);
HashValue(state, env.maxComputeWorkGroupInvocations);
HashValue(state, env.backend);
// DynamicBackendParameters is a plain aggregate of scalars; hashing its object
// representation is deliberate - it means a new limit cannot be added without also
// changing the fingerprint, which is exactly the memo-hazard property wanted here.
HashBytes(state, &env.params, sizeof(env.params));
if (!env.advertisedExtensions.empty()) {
HashBytes(state, env.advertisedExtensions.data(),
env.advertisedExtensions.size() * sizeof(GLExtension));
}
HashValue(state, env.subgroupPrefixScanQuirk);
return state;
}
SharedPtr<const CompileEnv> CaptureCompileEnv() {
auto env = MakeShared<CompileEnv>();
const auto& activeBackend = MG_Backend::pActiveBackendObject;
if (activeBackend) {
env->backend = activeBackend->GetBackendType();
env->params = activeBackend->GetDynamicParameters();
env->advertisedExtensions = activeBackend->GetRendererInfo().RendererGLInfo.Extensions;
}
// GL_MAX_COMPUTE_WORK_GROUP_SIZE. This is a REAL driver call on DirectGLES; it must
// happen here, on the context thread, and exactly once per context. The frontend
// minimum is the floor, matching what GL_Getter reports.
// TODO: Share these exposed compute limit helpers with GL_Getter.cpp instead of duplicating the frontend minima.
constexpr Uint kFrontendMinComputeWorkGroupSizes[3] = {1024, 1024, 64};
for (Uint index = 0; index < 3; ++index) {
Int backendValue = 0;
if (MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v) {
MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, index,
&backendValue);
}
env->maxComputeWorkGroupSize[index] =
std::max(static_cast<Uint>(std::max(backendValue, 0)), kFrontendMinComputeWorkGroupSizes[index]);
}
constexpr Uint64 kFrontendMaxComputeWorkGroupInvocations = 1024;
env->maxComputeWorkGroupInvocations =
activeBackend ? std::max(static_cast<Uint64>(std::max(env->params.MaxComputeWorkGroupInvocations, 0)),
kFrontendMaxComputeWorkGroupInvocations)
: kFrontendMaxComputeWorkGroupInvocations;
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return env;
}
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv() {
// Function-local static, not a namespace-scope one: the fingerprint has to be
// computed, and this must not run before MG_Config is loaded.
static const SharedPtr<const CompileEnv> kDefault = [] {
auto env = MakeShared<CompileEnv>();
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return SharedPtr<const CompileEnv>(Move(env));
}();
return kDefault;
}
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv() {
if (MG_State::pGLContext) return MG_State::pGLContext->GetCompileEnv();
return GetDefaultCompileEnv();
}
} // namespace MobileGL::MG_Util::ShaderTranspiler
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.h
+81
View File
@@ -0,0 +1,81 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <Config.h>
#include <MG_Backend/BackendObject.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
// Everything the shader compile/link pipeline reads from OUTSIDE its own (stage, source)
// inputs: backend identity, backend limits, the advertised extension list, and the one
// config quirk the source rewriter branches on.
//
// Why it exists (P1): every one of those reads is a reach-back into
// MG_Backend::pActiveBackendObject / gBackendFunctionsTable, and one of them
// (GL_MAX_COMPUTE_WORK_GROUP_SIZE) is a *real driver call* that on the DirectGLES
// backend silently no-ops off the context thread - which would turn a perfectly legal
// `local_size_z` into COMPILE_STATUS=FALSE the moment compilation moved to a worker.
// Snapshotting the whole set once per context, on the GL thread, removes every
// reach-back at once and makes the pipeline a pure function of (stage, source, env).
//
// Lifetime: captured lazily on first use by GLState::GLContext::GetCompileEnv(), and
// RE-captured if the active backend object changes. Immutable once published; held by
// value/`SharedPtr<const CompileEnv>` so a worker can never observe a torn update.
//
// Memo-hazard rule: `fingerprint` hashes every member above it and is part of the P0b
// ShaderPreprocessCache key, so a memo computed against one env can never be returned
// against another. ADDING A FIELD HERE MEANS ADDING IT TO ComputeFingerprint().
struct CompileEnv {
// --- compute limits: the ONLY former real-driver read in the pipeline ---
// GL_MAX_COMPUTE_WORK_GROUP_SIZE, already max()'d with the frontend minimum.
Uint maxComputeWorkGroupSize[3] = {1024, 1024, 64};
// GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, likewise.
Uint64 maxComputeWorkGroupInvocations = 1024;
// --- backend identity + limits ---
// Unknown means "no backend was active at capture time". Every consumer keeps the
// exact no-backend fallback it had before: extensions read as advertised, limits
// read as the frontend defaults.
BackendType backend = BackendType::Unknown;
MG_Backend::DynamicBackendParameters params{}; // by value, never by reference
Vector<GLExtension> advertisedExtensions;
// --- config the source rewriter branches on ---
MG_Config::QuirkOverride subgroupPrefixScanQuirk = MG_Config::QuirkOverride::Auto;
Uint64 fingerprint = 0; // set by CaptureCompileEnv()
Bool HasBackend() const { return backend != BackendType::Unknown; }
// Matches the historical rule exactly: with no active backend every extension counts
// as advertised, because the frontend then has nothing to gate against.
Bool IsExtensionAdvertised(GLExtension extension) const {
if (!HasBackend()) return true;
return std::find(advertisedExtensions.begin(), advertisedExtensions.end(), extension) !=
advertisedExtensions.end();
}
};
// Hashes every semantically relevant member. Public so a test can assert that two
// different envs really do produce different P0b cache keys.
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env);
// GL thread only: this is where the GL_MAX_COMPUTE_WORK_GROUP_SIZE queries live now.
SharedPtr<const CompileEnv> CaptureCompileEnv();
// The env a context-less caller gets: exactly what CaptureCompileEnv() would produce
// with no active backend. Used by the unit tests that drive the transpiler directly and
// by the internal shader objects that compile before any context exists.
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv();
// The env of the current GL context, or GetDefaultCompileEnv() when there is none.
// GL thread only (it may trigger a capture). This is the compatibility shim for the
// handful of entry points that still resolve their env implicitly; the pipeline itself
// always takes an explicit `const CompileEnv&`.
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv();
} // namespace MobileGL::MG_Util::ShaderTranspiler
MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp
+11 -6
View File
@@ -37,7 +37,10 @@
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
TBuiltInResource BuildTBuiltInResource() {
// `env` is the compile-time backend snapshot; null means "resolve from the live
// backend", which is what the standalone/test entry points do. The pipeline always
// passes one, so a worker never reaches pActiveBackendObject through here.
TBuiltInResource BuildTBuiltInResource(const CompileEnv* env) {
TBuiltInResource Resources{};
Resources.maxLights = 32;
Resources.maxClipPlanes = 6;
@@ -139,7 +142,8 @@ namespace MobileGL {
const MG_Backend::DynamicBackendParameters fallbackParameters{};
const auto& activeBackend = MG_Backend::pActiveBackendObject;
const auto& dynamicParameters =
activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters;
env ? env->params
: (activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters);
Resources.maxImageUnits = dynamicParameters.MaxImageUnits;
Resources.maxCombinedImageUnitsAndFragmentOutputs =
dynamicParameters.MaxImageUnits + dynamicParameters.MaxDrawBuffers;
@@ -167,7 +171,8 @@ namespace MobileGL {
// copies that could drift apart.
static Result<SharedPtr<glslang::TShader>> ParseShaderSource(EShLanguage lang, GLenum shaderType,
const String& source,
Flags<ShaderCompileBits> flags) {
Flags<ShaderCompileBits> flags,
const CompileEnv* env) {
SharedPtr<glslang::TShader> res;
auto& tshader = res;
tshader = MakeShared<glslang::TShader>(lang);
@@ -194,7 +199,7 @@ namespace MobileGL {
tshader->setAutoMapLocations(true);
tshader->setAutoMapBindings(true);
tshader->setGlobalUniformBlockName(GLOBAL_UBO_NAME);
auto resources = BuildTBuiltInResource();
auto resources = BuildTBuiltInResource(env);
if (!tshader->parse(&resources, 460, ECoreProfile,
/*forceDefaultVersionAndProfile: */ false,
/*forwardCompatible: */ true, EShMsgDefault)) {
@@ -220,7 +225,7 @@ namespace MobileGL {
}
const String source(attrib.sourceStr);
auto result = ParseShaderSource(lang, shaderType, source, attrib.flags);
auto result = ParseShaderSource(lang, shaderType, source, attrib.flags, attrib.env);
if (result) return result;
// Legacy desktop sources are normalized to "#version 330 core" (with a marker on the
@@ -236,7 +241,7 @@ namespace MobileGL {
return result;
}
auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags);
auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags, attrib.env);
if (!retryResult) return result;
MGLOG_D("CompileShader: %s only parsed after retargeting its legacy #version to 460",
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp
+23 -23
View File
@@ -16,6 +16,7 @@
#include <utility>
#include <Config.h>
#include <MG_Backend/BackendObjects.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include "EsslBuiltinFunctionNames.h"
@@ -1036,15 +1037,6 @@ namespace {
source = std::move(result);
}
bool IsExtensionAdvertised(MobileGL::GLExtension extension) {
const auto& activeBackendObject = MobileGL::MG_Backend::pActiveBackendObject;
if (!activeBackendObject) {
return true;
}
const auto& extensions = activeBackendObject->GetRendererInfo().RendererGLInfo.Extensions;
return std::find(extensions.begin(), extensions.end(), extension) != extensions.end();
}
MobileGL::String TrimDirectiveToken(const MobileGL::String& token) {
SizeT start = 0;
@@ -1059,8 +1051,9 @@ namespace {
return token.substr(start, end - start);
}
void FilterUnsupportedGpuShaderInt64(MobileGL::String& source) {
if (IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) {
void FilterUnsupportedGpuShaderInt64(const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env,
MobileGL::String& source) {
if (env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) {
return;
}
@@ -1314,7 +1307,9 @@ namespace MobileGL {
// instead of open-coding them in PreprocessShaderSource.
struct ShaderSourceQuirk {
const char* name;
MG_Config::QuirkOverride (*GetOverride)();
// Reads the override out of the captured env, never out of the live
// MG_Config table: a worker must see the same config the GL thread saw.
MG_Config::QuirkOverride (*GetOverride)(const CompileEnv&);
Bool (*DeviceApplies)(const ShaderSourceQuirkContext&);
Bool (*Apply)(const ShaderSourceQuirkContext&, String&);
};
@@ -1323,7 +1318,7 @@ namespace MobileGL {
{
// MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN
"subgroup-prefix-scan-rewrite",
[] { return MG_Config::Features.SubgroupPrefixScanQuirk; },
[](const CompileEnv& env) { return env.subgroupPrefixScanQuirk; },
[](const ShaderSourceQuirkContext& ctx) {
// Qualcomm's Vulkan driver miscompiles the recognized float
// InclusiveScan pattern for native subgroups wider than the
@@ -1339,20 +1334,21 @@ namespace MobileGL {
},
};
void ApplyShaderSourceQuirks(ShaderStage stage, String& source) {
const auto& activeBackend = MG_Backend::pActiveBackendObject;
if (!activeBackend) {
void ApplyShaderSourceQuirks(const CompileEnv& env, ShaderStage stage, String& source) {
// No backend at capture time means no device to match a quirk against,
// and (as before) no quirk can fire - not even a forced one, because
// every Apply reads device parameters that do not exist yet.
if (!env.HasBackend()) {
return;
}
const auto& dynamicParameters = activeBackend->GetDynamicParameters();
const ShaderSourceQuirkContext quirkContext{
stage,
activeBackend->GetBackendType(),
dynamicParameters.GpuVendor,
dynamicParameters.SubgroupSize,
env.backend,
env.params.GpuVendor,
env.params.SubgroupSize,
};
for (const ShaderSourceQuirk& quirk : kShaderSourceQuirks) {
const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride();
const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride(env);
if (quirkOverride == MG_Config::QuirkOverride::ForceOff) {
continue;
}
@@ -1369,6 +1365,10 @@ namespace MobileGL {
} // namespace
void PreprocessShaderSource(ShaderStage stage, String& source) {
PreprocessShaderSource(stage, source, *GetCurrentCompileEnv());
}
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env) {
// Normalize while the inspector's source span still refers to the untouched input.
const ShaderLanguageInfo originalLanguage = InspectShaderLanguage(source);
@@ -1395,7 +1395,7 @@ namespace MobileGL {
// identifier that merely contained the word. The GLES fallback for devices without
// the extension lives in the backend, where device capabilities are known.
FilterUnsupportedGpuShaderInt64(source);
FilterUnsupportedGpuShaderInt64(env, source);
CoerceUniformBlockPackingToStd140(source);
RenameBuiltinShadowingFunctions(source);
@@ -1403,7 +1403,7 @@ namespace MobileGL {
ModernizeLegacyGLSL(stage, source, afterVersion);
InjectDepthRangeBuiltinShim(stage, source, afterVersion);
ApplyShaderSourceQuirks(stage, source);
ApplyShaderSourceQuirks(env, stage, source);
}
Bool RetargetLegacyVersionDirectiveTo460(String& source) {
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.h
+9
View File
@@ -9,6 +9,7 @@
#pragma once
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderObject.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL {
enum class ShaderProfile {
@@ -19,6 +20,14 @@ namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// The whole source-rewriting pipeline. `env` is the compile-time snapshot of
// everything outside (stage, source) this reads - advertised extensions and the
// device-quirk inputs - so the transformation is a pure function of its three
// arguments and can run on a worker thread.
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env);
// Convenience overload that resolves the current context's env itself. GL thread
// only, and deliberately not used by the compile pipeline: it exists for the unit
// tests and diagnostics that drive the preprocessor standalone.
void PreprocessShaderSource(ShaderStage stage, String& source);
// Some desktop-captured compute shaders build a workgroup-wide linear prefix scan
MobileGL/MG_Util/ShaderTranspiler/Types.h
+6
View File
@@ -8,6 +8,7 @@
#pragma once
#include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL {
namespace MG_Util {
@@ -25,6 +26,11 @@ namespace MobileGL {
GLenum shaderType;
StringView sourceStr;
Flags<ShaderCompileBits> flags;
// The compile-time backend snapshot the glslang resource limits come from.
// Null means "read them off the live backend object" - only legal on the GL
// thread, and only used by the standalone/test entry points. Non-owning: the
// env outlives the attrib (it is a per-context SharedPtr).
const CompileEnv* env = nullptr;
};
struct ProgramAttrib {