swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-09-11 21:58:31 +09:00
Code Issues Packages Projects Releases Wiki Activity

[Refactor] (MG_State, MG_Util): join-by-construction link/compile artifacts (P1 stage 2)

Browse Source 
Still fully synchronous - EnsureLinkJoined()/EnsureCompileJoined() are empty
inline no-ops (verified to fold away at every one of the ~1200 call sites;
this project builds without LTO) - but every read of link- or compile-produced
state now goes through a private accessor the compiler enforces, so when
stage 4 moves the bodies onto pool workers, 'which reads must join' is a
type-system fact instead of a 400-line audit.

- ProgramObject: the 31 fields ResetLinkArtifacts clears plus the 5 link
  outputs it forgot (infoLog, linkedFragData{Location,Index}, the geometry
  strip-capture pair) move into a nested LinkArtifacts behind Artifacts().
  ResetLinkArtifacts is now a worker-safe pure clear; the link-observable
  version bumps (backendState/link/uboContent) move to a GL-thread-only
  BumpLinkObservableVersions() called once from Link()'s prologue and from
  glProgramBinary's mandated failure - the link body never writes them, so
  a stage-4 worker cannot lose an invalidation against the draw path.
- ShaderObject: compile artifacts (TShader, preprocessed source, side-channel
  maps, status/log, consume-once flag) behind Compiled(); the P0b layer-1
  memo trio deliberately stays outside as the future non-joining
  COMPLETION_STATUS_KHR fast path.
- CompileEnv (new): a GL-thread snapshot of everything the compile pipeline
  used to read live from the backend mid-parse - compute limits (the
  GetIntegeri_v reach-back is gone from the worker path), advertised
  extensions, device quirks, TBuiltInResource inputs. Captured lazily per
  backend activation; the consume-once re-parse now runs against the same
  env as the original parse.
- The GL-thread prologue / worker-body boundary is marked in Link() where
  the stage sort ends; everything below is a pure function of the snapshot.

Public getter signatures unchanged - MG_Impl and both backends compile
untouched. Unit 476/476, Program suites 117/117, DirectGLES retrace 38/39 on
llvmpipe (the one failure is the known pre-existing non-CI iterationrp case;
the NVIDIA userspace driver was updated out from under the running kernel
module mid-session, so GLX there is down until a reboot).
This commit is contained in:
BZLZHH
2026-08-08 07:12:37 -04:00
parent 8191075133
commit c93e5fa409
18 changed files with 1186 additions and 654 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
+98
View File
@@ -0,0 +1,98 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "CompileEnv.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_State/GLState/Core.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
namespace {
void HashBytes(Uint64& state, const void* data, const SizeT length) {
state = static_cast<Uint64>(XXH64(data, length, state));
}
template <typename T>
void HashValue(Uint64& state, const T& value) {
static_assert(std::is_trivially_copyable_v<T>);
HashBytes(state, &value, sizeof(T));
}
} // namespace
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env) {
Uint64 state = 0x9e3779b97f4a7c15ull;
HashValue(state, env.maxComputeWorkGroupSize[0]);
HashValue(state, env.maxComputeWorkGroupSize[1]);
HashValue(state, env.maxComputeWorkGroupSize[2]);
HashValue(state, env.maxComputeWorkGroupInvocations);
HashValue(state, env.backend);
// DynamicBackendParameters is a plain aggregate of scalars; hashing its object
// representation is deliberate - it means a new limit cannot be added without also
// changing the fingerprint, which is exactly the memo-hazard property wanted here.
HashBytes(state, &env.params, sizeof(env.params));
if (!env.advertisedExtensions.empty()) {
HashBytes(state, env.advertisedExtensions.data(),
env.advertisedExtensions.size() * sizeof(GLExtension));
}
HashValue(state, env.subgroupPrefixScanQuirk);
return state;
}
SharedPtr<const CompileEnv> CaptureCompileEnv() {
auto env = MakeShared<CompileEnv>();
const auto& activeBackend = MG_Backend::pActiveBackendObject;
if (activeBackend) {
env->backend = activeBackend->GetBackendType();
env->params = activeBackend->GetDynamicParameters();
env->advertisedExtensions = activeBackend->GetRendererInfo().RendererGLInfo.Extensions;
}
// GL_MAX_COMPUTE_WORK_GROUP_SIZE. This is a REAL driver call on DirectGLES; it must
// happen here, on the context thread, and exactly once per context. The frontend
// minimum is the floor, matching what GL_Getter reports.
// TODO: Share these exposed compute limit helpers with GL_Getter.cpp instead of duplicating the frontend minima.
constexpr Uint kFrontendMinComputeWorkGroupSizes[3] = {1024, 1024, 64};
for (Uint index = 0; index < 3; ++index) {
Int backendValue = 0;
if (MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v) {
MG_Backend::gBackendFunctionsTable.GL.GetIntegeri_v(GL_MAX_COMPUTE_WORK_GROUP_SIZE, index,
&backendValue);
}
env->maxComputeWorkGroupSize[index] =
std::max(static_cast<Uint>(std::max(backendValue, 0)), kFrontendMinComputeWorkGroupSizes[index]);
}
constexpr Uint64 kFrontendMaxComputeWorkGroupInvocations = 1024;
env->maxComputeWorkGroupInvocations =
activeBackend ? std::max(static_cast<Uint64>(std::max(env->params.MaxComputeWorkGroupInvocations, 0)),
kFrontendMaxComputeWorkGroupInvocations)
: kFrontendMaxComputeWorkGroupInvocations;
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return env;
}
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv() {
// Function-local static, not a namespace-scope one: the fingerprint has to be
// computed, and this must not run before MG_Config is loaded.
static const SharedPtr<const CompileEnv> kDefault = [] {
auto env = MakeShared<CompileEnv>();
env->subgroupPrefixScanQuirk = MG_Config::Features.SubgroupPrefixScanQuirk;
env->fingerprint = ComputeCompileEnvFingerprint(*env);
return SharedPtr<const CompileEnv>(Move(env));
}();
return kDefault;
}
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv() {
if (MG_State::pGLContext) return MG_State::pGLContext->GetCompileEnv();
return GetDefaultCompileEnv();
}
} // namespace MobileGL::MG_Util::ShaderTranspiler
MobileGL/MG_Util/ShaderTranspiler/CompileEnv.h
+81
View File
@@ -0,0 +1,81 @@
// MobileGL - MobileGL/MG_Util/ShaderTranspiler/CompileEnv.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <Config.h>
#include <MG_Backend/BackendObject.h>
namespace MobileGL::MG_Util::ShaderTranspiler {
// Everything the shader compile/link pipeline reads from OUTSIDE its own (stage, source)
// inputs: backend identity, backend limits, the advertised extension list, and the one
// config quirk the source rewriter branches on.
//
// Why it exists (P1): every one of those reads is a reach-back into
// MG_Backend::pActiveBackendObject / gBackendFunctionsTable, and one of them
// (GL_MAX_COMPUTE_WORK_GROUP_SIZE) is a *real driver call* that on the DirectGLES
// backend silently no-ops off the context thread - which would turn a perfectly legal
// `local_size_z` into COMPILE_STATUS=FALSE the moment compilation moved to a worker.
// Snapshotting the whole set once per context, on the GL thread, removes every
// reach-back at once and makes the pipeline a pure function of (stage, source, env).
//
// Lifetime: captured lazily on first use by GLState::GLContext::GetCompileEnv(), and
// RE-captured if the active backend object changes. Immutable once published; held by
// value/`SharedPtr<const CompileEnv>` so a worker can never observe a torn update.
//
// Memo-hazard rule: `fingerprint` hashes every member above it and is part of the P0b
// ShaderPreprocessCache key, so a memo computed against one env can never be returned
// against another. ADDING A FIELD HERE MEANS ADDING IT TO ComputeFingerprint().
struct CompileEnv {
// --- compute limits: the ONLY former real-driver read in the pipeline ---
// GL_MAX_COMPUTE_WORK_GROUP_SIZE, already max()'d with the frontend minimum.
Uint maxComputeWorkGroupSize[3] = {1024, 1024, 64};
// GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS, likewise.
Uint64 maxComputeWorkGroupInvocations = 1024;
// --- backend identity + limits ---
// Unknown means "no backend was active at capture time". Every consumer keeps the
// exact no-backend fallback it had before: extensions read as advertised, limits
// read as the frontend defaults.
BackendType backend = BackendType::Unknown;
MG_Backend::DynamicBackendParameters params{}; // by value, never by reference
Vector<GLExtension> advertisedExtensions;
// --- config the source rewriter branches on ---
MG_Config::QuirkOverride subgroupPrefixScanQuirk = MG_Config::QuirkOverride::Auto;
Uint64 fingerprint = 0; // set by CaptureCompileEnv()
Bool HasBackend() const { return backend != BackendType::Unknown; }
// Matches the historical rule exactly: with no active backend every extension counts
// as advertised, because the frontend then has nothing to gate against.
Bool IsExtensionAdvertised(GLExtension extension) const {
if (!HasBackend()) return true;
return std::find(advertisedExtensions.begin(), advertisedExtensions.end(), extension) !=
advertisedExtensions.end();
}
};
// Hashes every semantically relevant member. Public so a test can assert that two
// different envs really do produce different P0b cache keys.
Uint64 ComputeCompileEnvFingerprint(const CompileEnv& env);
// GL thread only: this is where the GL_MAX_COMPUTE_WORK_GROUP_SIZE queries live now.
SharedPtr<const CompileEnv> CaptureCompileEnv();
// The env a context-less caller gets: exactly what CaptureCompileEnv() would produce
// with no active backend. Used by the unit tests that drive the transpiler directly and
// by the internal shader objects that compile before any context exists.
const SharedPtr<const CompileEnv>& GetDefaultCompileEnv();
// The env of the current GL context, or GetDefaultCompileEnv() when there is none.
// GL thread only (it may trigger a capture). This is the compatibility shim for the
// handful of entry points that still resolve their env implicitly; the pipeline itself
// always takes an explicit `const CompileEnv&`.
const SharedPtr<const CompileEnv>& GetCurrentCompileEnv();
} // namespace MobileGL::MG_Util::ShaderTranspiler
MobileGL/MG_Util/ShaderTranspiler/ShaderCompiler.cpp
+11 -6
View File
@@ -37,7 +37,10 @@
namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
TBuiltInResource BuildTBuiltInResource() {
// `env` is the compile-time backend snapshot; null means "resolve from the live
// backend", which is what the standalone/test entry points do. The pipeline always
// passes one, so a worker never reaches pActiveBackendObject through here.
TBuiltInResource BuildTBuiltInResource(const CompileEnv* env) {
TBuiltInResource Resources{};
Resources.maxLights = 32;
Resources.maxClipPlanes = 6;
@@ -139,7 +142,8 @@ namespace MobileGL {
const MG_Backend::DynamicBackendParameters fallbackParameters{};
const auto& activeBackend = MG_Backend::pActiveBackendObject;
const auto& dynamicParameters =
activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters;
env ? env->params
: (activeBackend ? activeBackend->GetDynamicParameters() : fallbackParameters);
Resources.maxImageUnits = dynamicParameters.MaxImageUnits;
Resources.maxCombinedImageUnitsAndFragmentOutputs =
dynamicParameters.MaxImageUnits + dynamicParameters.MaxDrawBuffers;
@@ -167,7 +171,8 @@ namespace MobileGL {
// copies that could drift apart.
static Result<SharedPtr<glslang::TShader>> ParseShaderSource(EShLanguage lang, GLenum shaderType,
const String& source,
Flags<ShaderCompileBits> flags) {
Flags<ShaderCompileBits> flags,
const CompileEnv* env) {
SharedPtr<glslang::TShader> res;
auto& tshader = res;
tshader = MakeShared<glslang::TShader>(lang);
@@ -194,7 +199,7 @@ namespace MobileGL {
tshader->setAutoMapLocations(true);
tshader->setAutoMapBindings(true);
tshader->setGlobalUniformBlockName(GLOBAL_UBO_NAME);
auto resources = BuildTBuiltInResource();
auto resources = BuildTBuiltInResource(env);
if (!tshader->parse(&resources, 460, ECoreProfile,
/*forceDefaultVersionAndProfile: */ false,
/*forwardCompatible: */ true, EShMsgDefault)) {
@@ -220,7 +225,7 @@ namespace MobileGL {
}
const String source(attrib.sourceStr);
auto result = ParseShaderSource(lang, shaderType, source, attrib.flags);
auto result = ParseShaderSource(lang, shaderType, source, attrib.flags, attrib.env);
if (result) return result;
// Legacy desktop sources are normalized to "#version 330 core" (with a marker on the
@@ -236,7 +241,7 @@ namespace MobileGL {
return result;
}
auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags);
auto retryResult = ParseShaderSource(lang, shaderType, retrySource, attrib.flags, attrib.env);
if (!retryResult) return result;
MGLOG_D("CompileShader: %s only parsed after retargeting its legacy #version to 460",
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.cpp
+23 -23
View File
@@ -16,6 +16,7 @@
#include <utility>
#include <Config.h>
#include <MG_Backend/BackendObjects.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include "EsslBuiltinFunctionNames.h"
@@ -1036,15 +1037,6 @@ namespace {
source = std::move(result);
}
bool IsExtensionAdvertised(MobileGL::GLExtension extension) {
const auto& activeBackendObject = MobileGL::MG_Backend::pActiveBackendObject;
if (!activeBackendObject) {
return true;
}
const auto& extensions = activeBackendObject->GetRendererInfo().RendererGLInfo.Extensions;
return std::find(extensions.begin(), extensions.end(), extension) != extensions.end();
}
MobileGL::String TrimDirectiveToken(const MobileGL::String& token) {
SizeT start = 0;
@@ -1059,8 +1051,9 @@ namespace {
return token.substr(start, end - start);
}
void FilterUnsupportedGpuShaderInt64(MobileGL::String& source) {
if (IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) {
void FilterUnsupportedGpuShaderInt64(const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env,
MobileGL::String& source) {
if (env.IsExtensionAdvertised(MobileGL::E_GL_ARB_gpu_shader_int64)) {
return;
}
@@ -1314,7 +1307,9 @@ namespace MobileGL {
// instead of open-coding them in PreprocessShaderSource.
struct ShaderSourceQuirk {
const char* name;
MG_Config::QuirkOverride (*GetOverride)();
// Reads the override out of the captured env, never out of the live
// MG_Config table: a worker must see the same config the GL thread saw.
MG_Config::QuirkOverride (*GetOverride)(const CompileEnv&);
Bool (*DeviceApplies)(const ShaderSourceQuirkContext&);
Bool (*Apply)(const ShaderSourceQuirkContext&, String&);
};
@@ -1323,7 +1318,7 @@ namespace MobileGL {
{
// MOBILEGL_QUIRK_SUBGROUP_PREFIX_SCAN
"subgroup-prefix-scan-rewrite",
[] { return MG_Config::Features.SubgroupPrefixScanQuirk; },
[](const CompileEnv& env) { return env.subgroupPrefixScanQuirk; },
[](const ShaderSourceQuirkContext& ctx) {
// Qualcomm's Vulkan driver miscompiles the recognized float
// InclusiveScan pattern for native subgroups wider than the
@@ -1339,20 +1334,21 @@ namespace MobileGL {
},
};
void ApplyShaderSourceQuirks(ShaderStage stage, String& source) {
const auto& activeBackend = MG_Backend::pActiveBackendObject;
if (!activeBackend) {
void ApplyShaderSourceQuirks(const CompileEnv& env, ShaderStage stage, String& source) {
// No backend at capture time means no device to match a quirk against,
// and (as before) no quirk can fire - not even a forced one, because
// every Apply reads device parameters that do not exist yet.
if (!env.HasBackend()) {
return;
}
const auto& dynamicParameters = activeBackend->GetDynamicParameters();
const ShaderSourceQuirkContext quirkContext{
stage,
activeBackend->GetBackendType(),
dynamicParameters.GpuVendor,
dynamicParameters.SubgroupSize,
env.backend,
env.params.GpuVendor,
env.params.SubgroupSize,
};
for (const ShaderSourceQuirk& quirk : kShaderSourceQuirks) {
const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride();
const MG_Config::QuirkOverride quirkOverride = quirk.GetOverride(env);
if (quirkOverride == MG_Config::QuirkOverride::ForceOff) {
continue;
}
@@ -1369,6 +1365,10 @@ namespace MobileGL {
} // namespace
void PreprocessShaderSource(ShaderStage stage, String& source) {
PreprocessShaderSource(stage, source, *GetCurrentCompileEnv());
}
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env) {
// Normalize while the inspector's source span still refers to the untouched input.
const ShaderLanguageInfo originalLanguage = InspectShaderLanguage(source);
@@ -1395,7 +1395,7 @@ namespace MobileGL {
// identifier that merely contained the word. The GLES fallback for devices without
// the extension lives in the backend, where device capabilities are known.
FilterUnsupportedGpuShaderInt64(source);
FilterUnsupportedGpuShaderInt64(env, source);
CoerceUniformBlockPackingToStd140(source);
RenameBuiltinShadowingFunctions(source);
@@ -1403,7 +1403,7 @@ namespace MobileGL {
ModernizeLegacyGLSL(stage, source, afterVersion);
InjectDepthRangeBuiltinShim(stage, source, afterVersion);
ApplyShaderSourceQuirks(stage, source);
ApplyShaderSourceQuirks(env, stage, source);
}
Bool RetargetLegacyVersionDirectiveTo460(String& source) {
MobileGL/MG_Util/ShaderTranspiler/ShaderSourceProcessor.h
+9
View File
@@ -9,6 +9,7 @@
#pragma once
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderObject.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL {
enum class ShaderProfile {
@@ -19,6 +20,14 @@ namespace MobileGL {
namespace MG_Util {
namespace ShaderTranspiler {
// The whole source-rewriting pipeline. `env` is the compile-time snapshot of
// everything outside (stage, source) this reads - advertised extensions and the
// device-quirk inputs - so the transformation is a pure function of its three
// arguments and can run on a worker thread.
void PreprocessShaderSource(ShaderStage stage, String& source, const CompileEnv& env);
// Convenience overload that resolves the current context's env itself. GL thread
// only, and deliberately not used by the compile pipeline: it exists for the unit
// tests and diagnostics that drive the preprocessor standalone.
void PreprocessShaderSource(ShaderStage stage, String& source);
// Some desktop-captured compute shaders build a workgroup-wide linear prefix scan
MobileGL/MG_Util/ShaderTranspiler/Types.h
+6
View File
@@ -8,6 +8,7 @@
#pragma once
#include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL {
namespace MG_Util {
@@ -25,6 +26,11 @@ namespace MobileGL {
GLenum shaderType;
StringView sourceStr;
Flags<ShaderCompileBits> flags;
// The compile-time backend snapshot the glslang resource limits come from.
// Null means "read them off the live backend object" - only legal on the GL
// thread, and only used by the standalone/test entry points. Non-owning: the
// env outlives the attrib (it is a per-context SharedPtr).
const CompileEnv* env = nullptr;
};
struct ProgramAttrib {