Compare commits

...
5 Commits
Author SHA1 Message Date
BZLZHH d98f72447d [Feat, Test] (MG_Backend/DirectVulkan, MG_Test): Magma advertisement + the parallel-compile test net
Same gated push as the Espryt commit. The tests ride here because they
exercise both backends' advertisement paths and every piece of the
extension: ParallelShaderCompileTest (13 unit cases - the held-job proof
that GL_FALSE is observable and a second poll still shows outstanding work,
program equivalent, untouched objects read TRUE, always-TRUE with async
off, unknown pnames still INVALID_ENUM, zero-count join+inline for compiles
AND links, nonzero restores while Initialize() does not, clamping and
0xFFFFFFFF and KHR/ARB sharing one state, the getter vs the budget, the
string tracking configuration through glGetString AND glGetStringi) and
AsyncCompileScenario (5 real-GPU cases per the design: 64-compile polling,
forced-join correctness, string/thread-count checks against a live driver,
zero-count synchronous settlement, and async-vs-sync frames rendered
byte-identical with quadrant signatures so two identically-wrong images
cannot pass). Verified 5/5 on NVIDIA in the full 2x2 backend x flag matrix
with MOBILEGL_ITEST_REQUIRE_GPU=1.
2026-08-08 13:17:58 -04:00
BZLZHH f15cb8900f [Feat] (MG_Backend/DirectGLES): advertise GL_KHR_parallel_shader_compile when async is enabled
Gated on the configuration flag on purpose: the string is the one change a
retrace can never cover (Iris/Sodium pipeline their submissions differently
once they see it), so the kill switch has to withdraw the app-visible
behaviour along with the threading.
2026-08-08 13:17:58 -04:00
BZLZHH bd0def6133 [Feat] (MG_Impl, MG_State, MG_Util): the GL_KHR_parallel_shader_compile surface (P1 stage 5)
GL_COMPLETION_STATUS_KHR in both object getters, reading the non-joining
node-direct state - the one query that must never block is asserted never
to reach a join gate. glMaxShaderCompilerThreadsKHR/ARB share one
implementation: a zero count suspends async FIRST and then joins every
outstanding compile and link this context owns (suspend-before-join is the
only order whose post-condition is 'nothing in flight'), a nonzero count
restores; the suspension is a process latch the extension controls, kept
distinct from the configuration flag that gates the ADVERTISEMENT - an app
that turned threading off has not made the extension disappear.
GL_MAX_SHADER_COMPILER_THREADS_KHR reports the thread count. DriverPost
gains the MobileGL-side async row (PASS/INFO naming the env knob) and an
informational host-driver row backed by a new GLES capability probe.

The extension string itself lands per backend in the two follow-up
commits, keeping this one green stand-alone.
2026-08-08 13:17:58 -04:00
BZLZHH 6f8b7fbc40 [Feat] (MG_State, MG_Util): async program linking on the job graph (P1 stage 4)
glLinkProgram with the flag on snapshots its inputs in a GL-thread prologue
(stage-sorted shaders with their compile nodes taken without joining, env,
explicit locations/fragdata/xfb, draw-buffer count), then runs the whole
link body - glslang link/mapIO, SPIR-V, reflection, routing tables - as a
ProgramLinkTask that auto-posts when its last compile dependency settles
(+1-guarded countdown; no worker ever waits on another job). The publish is
one move of the LinkArtifacts block at the join, with the second version
bump so nothing memoized during the pending window survives.

The consume-once TShader claim moved onto the shared compile node as a CAS:
two link jobs racing for one shader resolve to winner-takes-the-parse,
loser re-parses the preprocessed source against the node's own env -
identical SPIR-V pinned by test for 2 and for 12 sharing programs.

Two deliberate corrections to the design's cancel matrix, both test-proven:
attach/detach do NOT cancel a pending link (the snapshot isolates it, and
glCreateShaderProgramv's link-then-detach would otherwise discard its own
result before anyone read it); and a compile node a pending link depends on
is pinned against the orphan-name sweep - the ordinary LWJGL teardown
compile/attach/link/detach/delete used to cancel the dependency and turn a
must-pass link into GL_FALSE.

Continuations are now throw-contained per-item (a stage-3 leftover made
load-bearing by the first real continuation), and the review's deadlock
find is fixed: the dispatch loop no longer cancels a node while holding the
pool mutex, since that cancel can run OnDepSettled -> Post -> same mutex.

Explicit joins: the draw path (GetProgramForDraw, both the pipeline stage
loop and the plain-UseProgram half) and the composite-link site; destroy
paths cancel-not-join; COMPLETION_STATUS readers stay non-joining.

Gates: 506/506 unit both flag states; AsyncCompile/AsyncLink/AsyncTeardown
suites x10 repeats clean both states (teardown with 128 jobs in flight,
then re-Initialize); full NVIDIA DirectGLES retrace flag on twice - result
sets identical to flag off, zero new deltas. Compile-phase prefix-diff,
flag on vs off: complementary-reimagined 5.21s -> 2.16s, BSL 1.72s ->
0.90s - past the design's final acceptance targets before the KHR
extension is even advertised. Default remains OFF until stage 5+7.
2026-08-08 11:58:38 -04:00
BZLZHH e5fb57f7eb [Feat] (MG_State, MG_Util): async shader compilation behind the default-off flag (P1 stage 3)
glCompileShader with MOBILEGL_ASYNC_SHADER_COMPILE=1 snapshots its inputs on
the GL thread (source SharedPtr, CompileEnv, cache handle) and runs the whole
pure pipeline - preprocess, validators, extractors, glslang parse - as a
ShaderCompileTask on the worker pool, returning immediately. Every read of
compile-produced state joins through the single Compiled() gate; links stay
synchronous this stage and join their attached shaders at the top of the
body. Flag off, the path is the same code run inline.

Mechanics: the job node owns all its inputs (no back-pointer, no lifetime
tie to the shader object), so re-sourcing or deleting a pending shader is
cancel-and-drop, never a wait; glslang worker hygiene is a TLS-allocator
scope guard plus GL-thread builtin prewarm (gated on the flag, latch reset
on Destroy so re-initialization re-warms); worker-side diagnostics defer
through the job and replay on the GL thread at the join, enforced by
IsPoolThread asserts in RecordError and an empty-deferred-errors tripwire.
A body that throws publishes a COMPLETE failed compile (status false, real
info log) rather than an abandoned node, and never memoizes away the retry;
a failed enqueue (OOM) cancels the node instead of stranding the joiner -
including inside the dispatch loop, where the in-flight slot is repaid.
The pool StopAndDrains from an atexit sentinel too: workers still inside
glslang parse while exit() ran static destructors was a real 2-in-5 SIGSEGV,
reproduced and fixed (15/15 clean after).

Backend-internal shader objects (default FS, DirectVulkan blit/mipmap) are
cache-less and always compile inline - compile-and-read-in-one-breath needs
no round trip.

Gates: unit suite 488/488 with the flag off AND on (x5); AsyncCompileTest
(12 e2e cases: pending re-source/delete/recompile, byte-identical failure
logs across modes, 48-compile cache stress) x10 repeats clean both modes;
full NVIDIA DirectGLES retrace identical result sets flag off/on (zero new
deltas); compile-phase timing flat as designed (links still serial - the
parallel win arrives with stage 4's async link + stage 5's
KHR_parallel_shader_compile).
2026-08-08 10:33:50 -04:00
40 changed files with 5439 additions and 1620 deletions
+2
View File
@@ -294,6 +294,8 @@ set(SOURCE_FILES
MobileGL/MG_State/GLState/TextureState/TextureUnit.cpp
MobileGL/MG_State/GLState/TextureState/TextureState.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramObject.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramLinkTask.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderCompileTask.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramState.cpp
+18
View File
@@ -16,6 +16,7 @@
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <atomic>
#include <mutex>
@@ -61,6 +62,10 @@ namespace MobileGL {
// still reference levels adopted from those tables. Finalizing first left live
// glslang objects pointing at freed memory for the rest of the teardown.
glslang::FinalizeProcess();
// Immediately after, and never apart from it: FinalizeProcess just deleted the
// built-in symbol tables the prewarm latch stands for, so leaving it set would
// make the next Initialize() skip a prewarm it genuinely needs.
MG_Util::ShaderTranspiler::ShaderCompiler::ResetPrewarmLatch();
MG_Backend::gBackendFunctionsTable = {};
g_isInitialized = false;
if (logLifecycle) {
@@ -88,6 +93,19 @@ namespace MobileGL {
MG_Impl::Init();
MGLOG_D("MG_Impl initialized");
glslang::InitializeProcess();
// On the GL thread, before any worker can exist. glslang builds its built-in symbol
// tables lazily under a process-wide lock held for the whole build, so without this
// the first concurrent compiles of a shaderpack all serialize behind the very first
// parse and asynchronous compilation looks like it is doing nothing.
//
// Gated on the flag, because the problem it solves only exists when there are
// workers: with compilation synchronous, nothing ever contends for that lock and the
// three throwaway parses buy nothing - they just add to every eglInitialize. Read the
// flag here rather than inside PrewarmBuiltins so ShaderCompiler keeps no dependency
// on the async subsystem (ProgramUtilTest compiles that file without it).
if (MG_Util::Async::AsyncShaderCompileEnabled()) {
MG_Util::ShaderTranspiler::ShaderCompiler::PrewarmBuiltins();
}
MGLOG_D("glslang initialized");
g_isInitialized = true;
MGLOG_I("MobileGL initialized");
@@ -18,6 +18,7 @@
#include <MG_Util/Converters/MGToGL/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToStr/TextureEnumConverter.h>
#include <MG_Util/Texture/TextureFormatProcessor.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <Config.h>
#include <algorithm>
#include <cmath>
@@ -891,6 +892,22 @@ namespace MobileGL::MG_Backend::DirectGLES {
// extension explicitly permits. It is also the only thing that
// exposes glProgramParameteri before GL 4.1.
E_GL_ARB_get_program_binary};
// GL_KHR_parallel_shader_compile is MobileGL's own capability, not the host ES
// driver's: the compiler threads are MobileGL's, and glCompileShader/glLinkProgram
// are serviced entirely inside the frontend. Whether the device driver advertises
// the string is irrelevant here (the POST reports it separately, for the day the
// driver-side link is what gets parallelised).
//
// Gated on the async flag deliberately, and this is the whole reason the gate
// exists. Advertising the string is the one part of asynchronous compilation that a
// recorded trace can never cover: Iris and Sodium change their SUBMISSION SCHEDULE
// the moment they see it - they enqueue whole pipeline batches and poll
// GL_COMPLETION_STATUS_KHR instead of compiling one program at a time - so
// MOBILEGL_ASYNC_SHADER_COMPILE=0 has to withdraw the application-visible behaviour
// change as well as the threading, or the kill switch would only be half a switch.
if (MG_Util::Async::AsyncShaderCompileEnabled()) {
extensions.push_back(E_GL_KHR_parallel_shader_compile);
}
// Only advertised when the device driver actually has usable timer queries
// (GL_EXT_disjoint_timer_query plus its entry points) and the
// MOBILEGL_DISABLE_TIMERQUERY escape hatch is off.
@@ -16,6 +16,7 @@
#include "MG_Util/Converters/MGToStr/TextureEnumConverter.h"
#include "MG_Util/Converters/MGToVk/TextureEnumConverter.h"
#include "MG_Util/Texture/TextureFormatProcessor.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#include <Config.h>
#include <cmath>
@@ -523,6 +524,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (shaderSubgroupSupported && !MG_Config::Features.DisableSubgroup) {
extensions.push_back(E_GL_KHR_shader_subgroup);
}
// GL_KHR_parallel_shader_compile is MobileGL's own capability, not the Vulkan
// device's: the compiler threads belong to MobileGL's shader pool and
// glCompileShader/glLinkProgram are serviced entirely inside the frontend, so there
// is no device feature to condition this on.
//
// Gated on the async flag deliberately, and this is the whole reason the gate
// exists. Advertising the string is the one part of asynchronous compilation that a
// recorded trace can never cover: Iris and Sodium change their SUBMISSION SCHEDULE
// the moment they see it - they enqueue whole pipeline batches and poll
// GL_COMPLETION_STATUS_KHR instead of compiling one program at a time - so
// MOBILEGL_ASYNC_SHADER_COMPILE=0 has to withdraw the application-visible behaviour
// change as well as the threading, or the kill switch would only be half a switch.
if (MG_Util::Async::AsyncShaderCompileEnabled()) {
extensions.push_back(E_GL_KHR_parallel_shader_compile);
}
// GL_ARB_timer_query gates MC's F3 GPU% (LWJGL checks the extension string);
// only advertised when the device actually supports timestamp queries and the
// MOBILEGL_DISABLE_TIMERQUERY escape hatch is off.
@@ -1273,7 +1273,7 @@ DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord4ivARB, GLenum target, const GL
DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord4sARB, GLenum target, GLshort s, GLshort t, GLshort r, GLshort q) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MultiTexCoord4sARB, target, s, t, r, q)
DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord4svARB, GLenum target, const GLshort* v) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MultiTexCoord4svARB, target, v)
DECLARE_GL_FUNCTION_STUB_HEAD(void, GetQueryObjectivARB, GLuint id, GLenum pname, GLint* params) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, GetQueryObjectivARB, id, pname, params)
DECLARE_GL_FUNCTION_STUB_HEAD(void, MaxShaderCompilerThreadsARB, GLuint count) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MaxShaderCompilerThreadsARB, count)
DECLARE_GL_FUNCTION_HEAD(void, MaxShaderCompilerThreadsARB, GLuint count) DECLARE_GL_FUNCTION_END_NO_RETURN(void, MaxShaderCompilerThreadsARB, count)
DECLARE_GL_FUNCTION_STUB_HEAD(void, PointParameterfARB, GLenum pname, GLfloat param) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, PointParameterfARB, pname, param)
DECLARE_GL_FUNCTION_STUB_HEAD(void, PointParameterfvARB, GLenum pname, const GLfloat* params) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, PointParameterfvARB, pname, params)
DECLARE_GL_FUNCTION_STUB_HEAD(void, GetnTexImageARB, GLenum target, GLint level, GLenum format, GLenum type, GLsizei bufSize, void* img) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, GetnTexImageARB, target, level, format, type, bufSize, img)
@@ -1381,7 +1381,7 @@ DECLARE_GL_FUNCTION_STUB_HEAD(void, WindowPos3ivARB, const GLint* v) DECLARE_GL_
DECLARE_GL_FUNCTION_STUB_HEAD(void, WindowPos3sARB, GLshort x, GLshort y, GLshort z) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, WindowPos3sARB, x, y, z)
DECLARE_GL_FUNCTION_STUB_HEAD(void, WindowPos3svARB, const GLshort* v) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, WindowPos3svARB, v)
DECLARE_GL_FUNCTION_STUB_HEAD(void, BlendBarrierKHR, void) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, BlendBarrierKHR, )
DECLARE_GL_FUNCTION_STUB_HEAD(void, MaxShaderCompilerThreadsKHR, GLuint count) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MaxShaderCompilerThreadsKHR, count)
DECLARE_GL_FUNCTION_HEAD(void, MaxShaderCompilerThreadsKHR, GLuint count) DECLARE_GL_FUNCTION_END_NO_RETURN(void, MaxShaderCompilerThreadsKHR, count)
DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord1bOES, GLenum texture, GLbyte s) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MultiTexCoord1bOES, texture, s)
DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord1bvOES, GLenum texture, const GLbyte* coords) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MultiTexCoord1bvOES, texture, coords)
DECLARE_GL_FUNCTION_STUB_HEAD(void, MultiTexCoord2bOES, GLenum texture, GLbyte s, GLbyte t) DECLARE_GL_FUNCTION_STUB_END_NO_RETURN(void, MultiTexCoord2bOES, texture, s, t)
@@ -23,6 +23,7 @@
#include <MG_Util/Converters/MGToGL/RenderStateEnumConverter.h>
#include <MG_State/GLState/FramebufferState/FramebufferObject.h>
#include <MG_Util/Texture/TextureFormatProcessor.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Backend/BackendObjects.h>
namespace MobileGL::MG_Impl::GLImpl {
@@ -1069,6 +1070,20 @@ namespace MobileGL::MG_Impl::GLImpl {
*params = obj ? static_cast<GLint>(obj->GetExternalIndex()) : 0;
return;
}
case GL_MAX_SHADER_COMPILER_THREADS_KHR:
// GL_KHR_parallel_shader_compile (GL_MAX_SHADER_COMPILER_THREADS_ARB is the same
// 0x91B0). The number of threads MobileGL's compile pool would actually use, so
// an application sizing its own submission batches gets a real answer.
//
// Zero when asynchronous compilation is off, which is the honest reply and the
// one the extension defines for an implementation with no compiler threads: the
// extension string is withdrawn in that configuration too, so a conforming
// application never reaches this query, and one that asks anyway is told there
// are none rather than being handed a thread count nothing will use.
*params = MG_Util::Async::AsyncShaderCompileEnabled()
? static_cast<GLint>(MG_Util::Async::ShaderCompilePool::Get().GetThreadCount())
: 0;
return;
case GL_MAX_DEBUG_GROUP_STACK_DEPTH:
*params = 0; // debug-group entrypoints are stubbed
return;
@@ -16,6 +16,7 @@
#include <MG_Util/Converters/GLToMG/ProgramEnumConverter.h>
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/Converters/SPIRVCrossToGL/SpvcTypeConverter.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Backend/BackendObjects.h>
namespace MobileGL::MG_Impl::GLImpl {
@@ -355,6 +356,54 @@ namespace MobileGL::MG_Impl::GLImpl {
shaderObject->Compile();
}
// glMaxShaderCompilerThreadsKHR / glMaxShaderCompilerThreadsARB - one implementation,
// because GL_KHR_parallel_shader_compile and GL_ARB_parallel_shader_compile define the
// same entry point with the same semantics and GetProcAddress.cpp maps both spellings.
//
// The three cases the extension defines, and what each means here:
//
// count == 0 "no compiler threads": compilation must happen on the
// application's thread. Everything already in flight is joined
// first, so that after this call returns NOTHING is outstanding
// and every GL_COMPLETION_STATUS_KHR reads GL_TRUE - which is
// the observable the extension actually specifies. The pool
// keeps its worker threads (this is not teardown); what changes
// is that AsyncShaderCompileActive() now says no, so
// glCompileShader/glLinkProgram run their bodies inline.
// count == 0xFFFFFFFF "implementation maximum": the pool's full thread count.
// otherwise a concurrency budget, clamped to the thread count - asking for
// more threads than exist cannot conjure any.
//
// A nonzero count is also what LIFTS a previous zero: the suspension lasts exactly until
// the application asks for threads again, and nothing else re-arms it (no implicit
// restore at eglInitialize, at a context switch or at a join). An application that turned
// compiler threads off keeps them off until it says otherwise.
//
// Legal - and a no-op beyond bookkeeping - while MOBILEGL_ASYNC_SHADER_COMPILE is off:
// compilation is already inline, and the call must not fail just because MobileGL had
// nothing to suspend.
void MaxShaderCompilerThreadsKHR_State(GLuint count) {
namespace Async = MG_Util::Async;
if (count == 0) {
MGLOG_D("%s: count = 0; joining all pending shader work and compiling inline", __func__);
Async::SetAsyncShaderCompileSuspended(true);
// Suspend BEFORE joining, not after. The post-condition this call owes the
// application is "nothing is in flight when I return", and only this order
// guarantees it: with the latch already set, anything the join itself causes to
// be compiled runs inline and is therefore already settled when the join ends.
// Joining first would leave a window in which a fresh enqueue is still legal.
if (MG_State::pGLContext) MG_State::pGLContext->JoinAllPendingShaderWork();
return;
}
Async::ShaderCompilePool& pool = Async::ShaderCompilePool::Get();
const Uint threadCount = pool.GetThreadCount();
const Uint requested = count == 0xFFFFFFFFu ? threadCount : std::min<Uint>(count, threadCount);
pool.SetMaxConcurrency(requested);
Async::SetAsyncShaderCompileSuspended(false);
MGLOG_D("%s: count = %u; concurrency = %u of %u threads", __func__, count, requested, threadCount);
}
GLuint CreateProgram_State() {
return MG_State::pGLContext->CreateProgram();
}
@@ -693,6 +742,19 @@ namespace MobileGL::MG_Impl::GLImpl {
break;
}
// GL_KHR_parallel_shader_compile. THIS CASE MUST NOT JOIN - it is the one program
// query whose entire purpose is to answer without waiting, and routing it through
// any of ProgramObject's Artifacts() accessors (the join gate, invariant I5) would
// block the caller and make the extension a lie: an application polling it would
// serialize itself on the very link it is trying to overlap. IsLinkComplete() is the
// node-direct reader that exists for exactly this.
//
// No link at all reads GL_TRUE, which is what the extension requires: the query
// means "is anything still outstanding", not "has this program ever been linked".
case GL_COMPLETION_STATUS_KHR:
*params = programObject->IsLinkComplete() ? GL_TRUE : GL_FALSE;
break;
case GL_PROGRAM_BINARY_LENGTH:
// No program binary format is exposed, so a program never has a retrievable
// binary and its length is zero (ARB_get_program_binary).
@@ -746,6 +808,13 @@ namespace MobileGL::MG_Impl::GLImpl {
case GL_SHADER_SOURCE_LENGTH:
*params = shaderObject->GetShaderSource().empty() ? 0 : (GLint)shaderObject->GetShaderSource().length() + 1;
break;
// GL_KHR_parallel_shader_compile. THIS CASE MUST NOT JOIN - see the identical case in
// GetProgramiv_State. GL_COMPILE_STATUS two cases up deliberately DOES join (it has
// to: it reports the outcome); this one reports whether there is an outcome yet, and
// reading it through Compiled() would defeat the whole extension.
case GL_COMPLETION_STATUS_KHR:
*params = shaderObject->IsCompileComplete() ? GL_TRUE : GL_FALSE;
break;
default:
MG_State::pGLContext->RecordError(
ErrorCode::InvalidEnum,
@@ -1832,6 +1901,16 @@ namespace MobileGL::MG_Impl::GLImpl {
void CompileShader(GLuint shader) {
CompileShader_State(shader);
}
void MaxShaderCompilerThreadsKHR(GLuint count) {
MaxShaderCompilerThreadsKHR_State(count);
}
// GL_ARB_parallel_shader_compile's spelling of the same entry point.
void MaxShaderCompilerThreadsARB(GLuint count) {
MaxShaderCompilerThreadsKHR_State(count);
}
GLuint CreateProgram(void) {
return CreateProgram_State();
}
@@ -42,6 +42,10 @@ namespace MobileGL::MG_Impl::GLImpl {
GLboolean IsProgram(GLuint program);
GLboolean IsShader(GLuint shader);
void LinkProgram(GLuint program);
// GL_KHR_parallel_shader_compile / GL_ARB_parallel_shader_compile. Both names are the
// same entry point; see MaxShaderCompilerThreadsKHR_State for the semantics of count.
void MaxShaderCompilerThreadsKHR(GLuint count);
void MaxShaderCompilerThreadsARB(GLuint count);
void ShaderSource(GLuint shader, GLsizei count, const GLchar* const* string, const GLint* length);
void UseProgram(GLuint program);
void Uniform1f(GLint location, GLfloat v0);
@@ -50,6 +50,7 @@ add_executable(MobileGLIntegrationTest
Scenarios/CrossFrameBufferScenario.cpp
Scenarios/ResidentIndexScenario.cpp
Scenarios/MultiDrawScenario.cpp
Scenarios/AsyncCompileScenario.cpp
)
target_include_directories(MobileGLIntegrationTest PRIVATE
@@ -0,0 +1,467 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/AsyncCompileScenario.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario E - asynchronous shader compilation and GL_KHR_parallel_shader_compile
// on a REAL driver.
//
// WHY THIS EXISTS ALONGSIDE THE UNIT SUITES. MG_Test/Program's async suites already
// drive the same GL entry points, but they stop at the frontend: nothing there ever
// reaches a driver, so nothing there can catch the failure this scenario is built for
// - artifacts produced on a worker thread that the BACKEND then rejects, mis-binds or
// renders differently from the ones the GL thread produced. The frontend cannot tell
// the two apart; a pixel can.
//
// The five things it pins, in order:
//
// (a) 64 heavy compiles are enqueued and polled through GL_COMPLETION_STATUS_KHR.
// At least one must be observed GL_FALSE - i.e. the query really answers while
// work is outstanding rather than silently joining. Skipped, never failed, when
// the machine drained the whole batch before the first poll: a fast box must not
// be able to turn this into a red.
// (b) Forcing the join afterwards produces the right answer for every one of them:
// GL_COMPILE_STATUS true, an empty info log, and a program that links.
// (c) The extension string matches the configuration. This is the half a recorded
// trace can never cover - Iris and Sodium change their submission schedule the
// moment they see the string - so it is asserted against a real backend's real
// GL_EXTENSIONS, through both glGetString and glGetStringi.
// (d) glMaxShaderCompilerThreadsKHR(0) leaves nothing in flight: every subsequent
// GL_COMPLETION_STATUS_KHR reads GL_TRUE immediately, and compilation after it
// is synchronous. That is what the extension requires of a zero count.
// (e) THE ONE THAT NEEDS A GPU: the same frame, drawn with programs compiled and
// linked asynchronously and then with programs compiled and linked inline, must
// come out byte-identical under glReadPixels. Anything the worker thread got
// wrong about the compile environment, the reflection or the SPIR-V shows up
// here as a pixel difference and nowhere else.
//
// Backend selection is the module's usual one process, one backend (MOBILEGL_BACKEND_TYPE),
// so this file runs twice per ctest invocation.
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#include "Config.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
// GL_KHR_parallel_shader_compile. Spelled out rather than relying on the host's
// glext.h: this module is built against whatever GL headers the machine has, and an
// older one has neither token. Both are also GL_*_ARB with identical values.
#ifndef GL_MAX_SHADER_COMPILER_THREADS_KHR
#define GL_MAX_SHADER_COMPILER_THREADS_KHR 0x91B0
#endif
#ifndef GL_COMPLETION_STATUS_KHR
#define GL_COMPLETION_STATUS_KHR 0x91B1
#endif
// The entry point under test, resolved by the linker straight into MobileGL_s like
// every other gl* call in this module. Declared here for the same reason as the
// tokens above.
extern "C" void glMaxShaderCompilerThreadsKHR(GLuint count);
namespace MGITest {
namespace {
using MobileGL::MG_Config::QuirkOverride;
// Same shape as the other scenarios: a two-attribute pass-through, so the only
// thing that can differ between the two compilation modes is the compilation.
constexpr const char* kVertexSource = R"(#version 330 core
in vec2 aPos;
in vec3 aColor;
out vec3 vColor;
void main() {
vColor = aColor;
gl_Position = vec4(aPos, 0.0, 1.0);
}
)";
constexpr const char* kFragmentSource = R"(#version 330 core
in vec3 vColor;
out vec4 oColor;
void main() {
oColor = vec4(vColor, 1.0);
}
)";
// Asymmetric in both axes, so a mode difference that also happens to be a
// symmetry of the image cannot hide (the same reason OrientationScenario draws
// quadrants rather than stripes).
struct Vertex {
float x, y;
float r, g, b;
};
void AppendQuad(std::vector<Vertex>& out, float x0, float x1, float y0, float y1, float r, float g, float b) {
const Vertex bl{x0, y0, r, g, b};
const Vertex br{x1, y0, r, g, b};
const Vertex tr{x1, y1, r, g, b};
const Vertex tl{x0, y1, r, g, b};
out.insert(out.end(), {bl, br, tr, bl, tr, tl});
}
std::vector<Vertex> QuadrantGeometry() {
std::vector<Vertex> vertices;
vertices.reserve(24);
AppendQuad(vertices, -1.0f, 0.0f, -1.0f, 0.0f, 0.0f, 0.0f, 1.0f); // bottom-left: blue
AppendQuad(vertices, 0.0f, 1.0f, -1.0f, 0.0f, 0.0f, 1.0f, 0.0f); // bottom-right: green
AppendQuad(vertices, -1.0f, 0.0f, 0.0f, 1.0f, 1.0f, 0.0f, 0.0f); // top-left: red
AppendQuad(vertices, 0.0f, 1.0f, 0.0f, 1.0f, 1.0f, 1.0f, 1.0f); // top-right: white
return vertices;
}
// Expensive enough that a compile is not instantaneous, and distinct per index so
// the source-hash memo never turns one into a no-op: without both properties the
// pool has no backlog and (a) has nothing to observe.
std::string BulkyFragmentSource(int index) {
std::string source = "#version 330 core\n";
source += "in vec3 vColor;\nout vec4 oColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 320; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " oColor = vec4(vColor * acc, 1.0);\n}\n";
return source;
}
// MOBILEGL_ASYNC_SHADER_COMPILE decides the ambient mode; a scenario that wants
// the other one says so here and gets the ambient one back on scope exit. Forcing
// it in-process is what lets ONE ctest run compare the two modes against each
// other - the whole point of (e).
class AsyncModeScope {
public:
explicit AsyncModeScope(bool async) : m_saved(MobileGL::MG_Config::Features.AsyncShaderCompile) {
MobileGL::MG_Config::Features.AsyncShaderCompile =
async ? QuirkOverride::ForceOn : QuirkOverride::ForceOff;
}
~AsyncModeScope() { MobileGL::MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const QuirkOverride m_saved;
};
// glMaxShaderCompilerThreadsKHR writes process-wide state; a scenario that calls
// it has to put the pool back or it changes how every scenario after it compiles.
class CompilerThreadScope {
public:
CompilerThreadScope() = default;
~CompilerThreadScope() {
MobileGL::MG_Util::Async::SetAsyncShaderCompileSuspended(false);
auto& pool = MobileGL::MG_Util::Async::ShaderCompilePool::Get();
pool.SetMaxConcurrency(pool.GetThreadCount());
}
CompilerThreadScope(const CompilerThreadScope&) = delete;
CompilerThreadScope& operator=(const CompilerThreadScope&) = delete;
};
GLint ShaderCompletion(GLuint shader) {
GLint status = -1;
glGetShaderiv(shader, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint ShaderCompileStatus(GLuint shader) {
GLint status = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
std::string ShaderInfoLog(GLuint shader) {
GLint length = 0;
glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return std::string();
std::vector<char> buffer(static_cast<std::size_t>(length));
GLsizei written = 0;
glGetShaderInfoLog(shader, length, &written, buffer.data());
return std::string(buffer.data(), static_cast<std::size_t>(written));
}
class AsyncCompileScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
const std::vector<Vertex> vertices = QuadrantGeometry();
m_vertexCount = static_cast<int>(vertices.size());
glGenVertexArrays(1, &m_vao);
glBindVertexArray(m_vao);
glGenBuffers(1, &m_vbo);
glBindBuffer(GL_ARRAY_BUFFER, m_vbo);
glBufferData(GL_ARRAY_BUFFER, GLsizeiptr(vertices.size() * sizeof(Vertex)), vertices.data(),
GL_STATIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), reinterpret_cast<void*>(0));
glEnableVertexAttribArray(1);
glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), reinterpret_cast<void*>(8));
glBindVertexArray(0);
ASSERT_EQ(FirstGLError(), GLenum(GL_NO_ERROR)) << "setup left a GL error behind";
}
void TearDown() override {
if (!Ready()) return;
if (m_vbo != 0) glDeleteBuffers(1, &m_vbo);
if (m_vao != 0) glDeleteVertexArrays(1, &m_vao);
}
// A fresh program every time, compiled and linked in whatever mode is in
// force. Reusing one would defeat the comparison: the second mode would just
// read the first mode's artifacts back out of the memo.
GLuint BuildProgram() {
std::string error;
const GLuint program = CompileProgram(kVertexSource, kFragmentSource, &error);
EXPECT_NE(program, 0u) << error;
return program;
}
Image DrawFrameWith(GLuint program) {
BindDefaultFramebuffer();
ClearTo(0.0f, 0.0f, 0.0f, 1.0f);
glDisable(GL_DEPTH_TEST);
glDisable(GL_BLEND);
glUseProgram(program);
glBindVertexArray(m_vao);
glDrawArrays(GL_TRIANGLES, 0, m_vertexCount);
glBindVertexArray(0);
Image image = ReadPixels(Gl().Width(), Gl().Height());
Gl().EndFrame();
return image;
}
// Enqueues `count` distinct heavy compiles and returns their names WITHOUT
// reading anything back, so the pool is left with a real backlog.
std::vector<GLuint> EnqueueBacklog(int count, int seedBase) {
std::vector<GLuint> shaders;
shaders.reserve(static_cast<std::size_t>(count));
m_sources.reserve(m_sources.size() + static_cast<std::size_t>(count));
for (int i = 0; i < count; ++i) {
m_sources.push_back(BulkyFragmentSource(seedBase + i));
const char* text = m_sources.back().c_str();
const GLuint shader = glCreateShader(GL_FRAGMENT_SHADER);
glShaderSource(shader, 1, &text, nullptr);
glCompileShader(shader);
shaders.push_back(shader);
}
return shaders;
}
GLuint m_vao = 0;
GLuint m_vbo = 0;
int m_vertexCount = 0;
// Kept alive for the whole case: glShaderSource copies, but keeping the
// strings makes a failure message able to name the source it came from.
std::vector<std::string> m_sources;
};
// ---- (a) + (b) ------------------------------------------------------------
// A backlog is enqueued, polled without joining, then forced to settle and
// checked for correctness. Both halves in one case on purpose: (b) is only
// interesting for shaders that (a) proved were genuinely still outstanding.
TEST_F(AsyncCompileScenario, CompletionStatusPollingThenForcedJoin) {
if (!Ready()) return;
const AsyncModeScope async(true);
const CompilerThreadScope threads;
// One worker, so the queue behind it is what the poll observes.
glMaxShaderCompilerThreadsKHR(1);
const std::vector<GLuint> shaders = EnqueueBacklog(64, 6000);
int outstanding = 0;
for (const GLuint shader : shaders) {
const GLint completion = ShaderCompletion(shader);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE)
<< "GL_COMPLETION_STATUS_KHR returned " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "this machine drained 64 heavy compiles before the first poll; "
"nothing was outstanding to observe";
}
// (b) Forced join: every one of them is correct, and usable.
for (const GLuint shader : shaders) {
EXPECT_EQ(ShaderCompileStatus(shader), GL_TRUE) << ShaderInfoLog(shader);
EXPECT_TRUE(ShaderInfoLog(shader).empty());
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE) << "GL_COMPILE_STATUS must have joined";
}
// And a link over one of them really produces a usable program on this driver.
const GLuint vs = glCreateShader(GL_VERTEX_SHADER);
glShaderSource(vs, 1, &kVertexSource, nullptr);
glCompileShader(vs);
const GLuint program = glCreateProgram();
glAttachShader(program, vs);
glAttachShader(program, shaders.front());
glBindAttribLocation(program, 0, "aPos");
glBindAttribLocation(program, 1, "aColor");
glLinkProgram(program);
GLint linked = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &linked);
EXPECT_EQ(linked, GL_TRUE);
EXPECT_GE(glGetUniformLocation(program, "uSeed6000"), 0);
glDeleteProgram(program);
glDeleteShader(vs);
for (const GLuint shader : shaders) glDeleteShader(shader);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (c) ------------------------------------------------------------------
// The extension string, read from a real backend that really brought a driver
// up. No mode forcing here: a backend builds its advertised list once, from the
// configuration in force at its first use, so the meaningful assertion is
// against the AMBIENT configuration - which is exactly what makes this case
// worth running in both of the suite's flag states.
TEST_F(AsyncCompileScenario, ExtensionStringMatchesTheConfiguration) {
if (!Ready()) return;
const bool expected = MobileGL::MG_Util::Async::AsyncShaderCompileEnabled();
const char* extensions = reinterpret_cast<const char*>(glGetString(GL_EXTENSIONS));
ASSERT_NE(extensions, nullptr);
const std::string extensionString(extensions);
const bool inString = extensionString.find("GL_KHR_parallel_shader_compile") != std::string::npos;
EXPECT_EQ(inString, expected)
<< "backend " << Gl().BackendName() << " GL_EXTENSIONS = " << extensionString;
// LWJGL builds GLCapabilities from the INDEXED form on a core profile, so the
// two spellings disagreeing would be invisible to the check above and fatal
// to a real application.
GLint count = 0;
glGetIntegerv(GL_NUM_EXTENSIONS, &count);
ASSERT_GT(count, 0);
bool inIndexed = false;
for (GLint i = 0; i < count; ++i) {
const char* name = reinterpret_cast<const char*>(glGetStringi(GL_EXTENSIONS, GLuint(i)));
if (name != nullptr && std::string(name) == "GL_KHR_parallel_shader_compile") inIndexed = true;
}
EXPECT_EQ(inIndexed, expected);
// The companion query, which an application reads right after the string.
GLint maxThreads = -1;
glGetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &maxThreads);
if (expected) {
EXPECT_GE(maxThreads, 1);
} else {
EXPECT_EQ(maxThreads, 0);
}
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (d) ------------------------------------------------------------------
// A zero count must leave nothing in flight and keep it that way.
TEST_F(AsyncCompileScenario, ZeroCompilerThreadsSettlesEverythingImmediately) {
if (!Ready()) return;
const AsyncModeScope async(true);
const CompilerThreadScope threads;
glMaxShaderCompilerThreadsKHR(1);
const std::vector<GLuint> backlog = EnqueueBacklog(48, 6200);
glMaxShaderCompilerThreadsKHR(0);
for (const GLuint shader : backlog) {
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE)
<< "glMaxShaderCompilerThreadsKHR(0) must join everything still in flight";
EXPECT_EQ(ShaderCompileStatus(shader), GL_TRUE) << ShaderInfoLog(shader);
}
// Compilation after the zero count is synchronous too.
const std::vector<GLuint> serial = EnqueueBacklog(6, 6300);
for (const GLuint shader : serial) {
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE) << "a compile after a zero count must be synchronous";
}
for (const GLuint shader : backlog) glDeleteShader(shader);
for (const GLuint shader : serial) glDeleteShader(shader);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (e) ------------------------------------------------------------------
// The one that needs the GPU. Two programs, identical source, one built with
// compilation and linking on worker threads and one built inline; the frames
// they draw must be byte-identical.
//
// Compared through the DEFAULT framebuffer deliberately: that is where the
// backend's orientation and present path live, so the comparison covers the
// whole pipeline rather than the reflection tables alone.
TEST_F(AsyncCompileScenario, AsyncAndSyncProgramsRenderIdenticalFrames) {
if (!Ready()) return;
Image asyncImage;
{
const AsyncModeScope async(true);
const GLuint program = BuildProgram();
ASSERT_NE(program, 0u);
asyncImage = DrawFrameWith(program);
glDeleteProgram(program);
}
Image syncImage;
{
const AsyncModeScope async(false);
const GLuint program = BuildProgram();
ASSERT_NE(program, 0u);
syncImage = DrawFrameWith(program);
glDeleteProgram(program);
}
ASSERT_FALSE(asyncImage.Empty());
ASSERT_FALSE(syncImage.Empty());
// The frame is the expected one in the first place - two identically WRONG
// frames would otherwise pass.
EXPECT_EQ(asyncImage.QuadrantSignature(), "blue,green,red,white")
<< "the asynchronously compiled program did not draw the expected frame";
EXPECT_EQ(asyncImage, syncImage)
<< "asynchronous and synchronous compilation rendered different frames ("
<< asyncImage.ByteDiffCount(syncImage) << " bytes differ); backend " << Gl().BackendName();
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// The same comparison over a batch, which is the shape a shaderpack load has:
// many programs enqueued before any of them is read back, then each one drawn.
// A per-worker state leak (glslang's thread-local pools are the obvious
// candidate) shows up here and not in the single-program case above.
TEST_F(AsyncCompileScenario, ABatchOfAsyncProgramsAllRenderCorrectly) {
if (!Ready()) return;
constexpr int kPrograms = 12;
std::vector<GLuint> programs;
{
const AsyncModeScope async(true);
const CompilerThreadScope threads;
glMaxShaderCompilerThreadsKHR(1);
// Everything enqueued before anything is read: the only shape in which
// more than one job is in flight at a time.
for (int i = 0; i < kPrograms; ++i) {
programs.push_back(BuildProgram());
}
}
for (int i = 0; i < kPrograms; ++i) {
ASSERT_NE(programs[static_cast<std::size_t>(i)], 0u) << "program " << i;
const Image image = DrawFrameWith(programs[static_cast<std::size_t>(i)]);
EXPECT_EQ(image.QuadrantSignature(), "blue,green,red,white") << "program " << i;
}
for (const GLuint program : programs) glDeleteProgram(program);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
} // namespace
} // namespace MGITest
+41 -1
View File
@@ -10,6 +10,7 @@
#include "MG_State/GLState/RenderbufferState/RenderbufferObject.h"
#include "MG_State/EGLState/Core.h"
#include <MG_Backend/BackendObjects.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <Config.h>
@@ -40,6 +41,13 @@ namespace MobileGL::MG_State {
// Error
void GLContext::RecordError(ErrorCode code, UniquePtr<ErrorInfo> info) {
// Invariant I1, mechanically enforced: the GL error state is GL-thread-owned.
// A compile or link body that needs to raise an error must append to its node's
// JobDiagnostics and let the join replay it here (see the P1 design section 6);
// reaching this from a worker would corrupt the sticky-flag set that
// glGetError's ordering depends on.
MOBILEGL_ASSERT(!MG_Util::Async::ShaderCompilePool::IsPoolThread(),
"GLContext::RecordError() called from a shader-compile pool thread");
m_errorState.RecordError(code, Move(info));
}
@@ -349,6 +357,10 @@ namespace MobileGL::MG_State {
return m_programState.GetShaderObject(index);
}
void GLContext::JoinAllPendingShaderWork() {
m_programState.JoinAllPendingWork();
}
void GLContext::UseProgram(Uint program) {
return m_programState.UseProgram(program);
}
@@ -360,11 +372,35 @@ namespace MobileGL::MG_State {
const SharedPtr<ProgramObject>& GLContext::GetProgramForDraw() {
static const SharedPtr<ProgramObject> nullProgram = nullptr;
const auto& currentProgram = m_programState.GetCurrentProgram();
if (currentProgram) return currentProgram;
if (currentProgram) {
// P1 join site J1, plain glUseProgram half. The backends read a program's
// lifetimeId / backendStateVersion / UBO content version to decide whether
// their per-program caches are still valid, and none of those pass through
// ProgramObject's join gate - so a draw could sample a version, join later
// inside the same draw when it finally touched an artifact, and cache under a
// version the publish had already superseded. Settling here means every
// version a backend reads during a draw describes the program it is drawing.
// One null check in steady state.
currentProgram->JoinLink();
return currentProgram;
}
if (m_boundProgramPipeline == 0) return nullProgram;
const auto& pipeline = GetBoundProgramPipeline();
if (!pipeline) return nullProgram;
// P1 join site J1. ComputeDrawProgramSignature() keys the composite cache on each
// stage program's lifetimeId and backendStateVersion - NON-artifact fields, so
// they do not pass through ProgramObject's join gate and a pending link would
// stay pending right through the signature. Since the version is bumped both at
// enqueue and at publish, the signature computed inside a pending window is one
// that will never be produced again: every draw would miss the cache and rebuild
// (and relink) the composite. Join first, so the signature describes settled
// programs. In steady state this is a null check per stage.
for (SizeT stage = 0; stage < static_cast<SizeT>(ShaderStage::ShaderStageCount); ++stage) {
const auto& stageProgram = pipeline->GetStageProgram(static_cast<ShaderStage>(stage));
if (stageProgram) stageProgram->JoinLink();
}
const auto signature = pipeline->ComputeDrawProgramSignature();
if (const auto& cached = pipeline->GetCachedDrawProgram(signature)) return cached;
@@ -392,6 +428,10 @@ namespace MobileGL::MG_State {
// A pipeline with no fragment stage still rasterises, so the default fragment
// shader is wanted here even though the separable stage programs never get one.
composite->Link(true);
// P1 join site J2. The draw that asked for this program is the very next thing to
// happen, so enqueueing the composite's link buys nothing and only moves the wait
// to whichever backend accessor happens to touch its artifacts first.
composite->JoinLink();
pipeline->SetCachedDrawProgram(signature, Move(composite));
return pipeline->GetCachedDrawProgram(signature);
}
+3
View File
@@ -153,6 +153,9 @@ namespace MobileGL {
Bool ValidateShaderName(Uint index) const;
const SharedPtr<ProgramObject>& GetProgramObject(Uint index);
const SharedPtr<ShaderObject>& GetShaderObject(Uint index);
// Settles every compile and link this context still owns; see
// ProgramState::JoinAllPendingWork. Called by glMaxShaderCompilerThreadsKHR(0).
void JoinAllPendingShaderWork();
void UseProgram(Uint program);
const SharedPtr<ProgramObject>& GetCurrentProgram();
// What a draw or dispatch actually executes: the program in use, or - when
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,111 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ProgramLinkTask.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ProgramObject.h>
#include <MG_State/GLState/ProgramState/ShaderCompileTask.h>
#include <MG_Util/Async/JobNode.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
namespace MobileGL::MG_State::GLState {
// One attached shader, as the link sees it: never the ShaderObject, always a snapshot.
//
// The ShaderObject is GL-thread-owned and may be re-sourced, detached or destroyed while
// this link is still queued; everything below is either immutable or independently owned,
// so none of that can reach the worker.
struct LinkShaderInput {
ShaderStage stage = ShaderStage::Unknown;
// For the compile-error diagnostic and the compute local_size check, both of which
// quote the ORIGINAL source rather than the preprocessed one.
SharedPtr<const String> source;
// The authoritative compiled state. Null, or non-Complete, both read as "this shader
// did not compile" - the same verdict ShaderObject's join gate produces.
SharedPtr<const ShaderCompileTask> compiled;
};
// The unit of asynchronous linking: one glLinkProgram's worth of pure CPU work - glslang
// link + mapIO, SPIR-V generation and optimization, the GL-facing reflection surface, the
// global-UBO routing tables, fragment-output validation and transform-feedback
// resolution - with every input it needs snapshotted at enqueue.
//
// Same ownership rule as ShaderCompileTask: the body reads nothing but `in` (all of it
// owned or immutable) and writes nothing but `artifacts`. No GL call, no
// pActiveBackendObject read, no pGLContext->RecordError(); the device limits arrive
// through the CompileEnv snapshot and diagnostics are deferred to the join.
//
// ONE LINK IS ONE HANDLER. RunBody() runs start to finish inside a single pool handler
// and is the only place `artifacts` is written. Do not split it across handlers to
// "pipeline" the reflection half: the intermediates that GlslangToSpv and buildReflection
// share are mutated in a strict order (see the GenerateSpirv-before-DoReflection comment
// in Run()), and a second handler would let a cancel land between them and publish a
// program whose SPIR-V and reflection describe different things.
class ProgramLinkTask final : public MG_Util::Async::JobNode {
public:
// ---- inputs, snapshotted on the GL thread in ProgramObject::Link()'s prologue ----
struct Inputs {
Uint externalIndex = 0; // logs only
Vector<LinkShaderInput> shaders; // already stage-sorted
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
// The four "takes effect at the next link" request maps. Snapshotted rather than
// referenced, which is precisely what makes glBindAttribLocation and friends
// legal to call over a pending link without cancelling it: the pending link keeps
// linking the inputs it was given.
UnorderedMap<String, Uint> explicitAttribLocations; // glBindAttribLocation
UnorderedMap<String, Uint> explicitFragDataLocation; // glBindFragDataLocation
UnorderedMap<String, Uint> explicitFragDataIndex; // glBindFragDataLocationIndexed
Vector<String> requestedXfbVaryings; // glTransformFeedbackVaryings
GLenum requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS;
Int maxFragmentOutputColorNumber = 8; // GL_MAX_DRAW_BUFFERS, stamped in by the entry point
} in;
// ---- output: valid iff IsComplete(), immutable afterwards ----
// Moved (never copied) into the ProgramObject by EnsureLinkJoined().
ProgramObject::LinkArtifacts artifacts;
// Posts this job once every compile in `deps` is terminal - and not one moment
// earlier, so the body never waits on anything (invariant I4: no job body may block
// on another job, or the pool could deadlock with all its workers waiting on each
// other). `deps` is the subset of the snapshot's compile nodes that were still
// in flight; an already-terminal one needs no edge.
//
// GL thread only, and only after the caller has stored a SharedPtr to this node:
// OnDepSettled takes shared_from_this().
void SubmitAfter(const Vector<SharedPtr<ShaderCompileTask>>& deps);
private:
void RunBody() override;
// Runs when one dependency goes terminal - on whichever thread drove it there, which
// is a pool worker for a compile that finished on one. Non-throwing by construction;
// see the definition.
void OnDepSettled();
// ---- the link body, split exactly as ProgramObject::Link() had it ----
// Each returns false to abort the link with `artifacts.infoLog` already set, which is
// GL's definition of a failed link: LINK_STATUS false plus a log, never a GL error.
Bool ConsumeShaders(Vector<SharedPtr<glslang::TShader>>& outShaders);
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
Bool ValidateFragmentOutputLocations();
Bool ResolveTransformFeedbackVaryings();
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
void GenerateSpirv();
void BuildGlobalUboRouting();
// Worker-side MGLOG replacement: appended to diagnostics.logLines and replayed by the
// join, on the GL thread, where a serial implementation would have printed it.
// Logging straight from a worker interleaves mid-line with the GL thread's output and
// lands out of order relative to the glLinkProgram that caused it.
void DeferLog(String line);
// Counts down to zero exactly once. Starts at deps + 1: the extra guard is released
// by SubmitAfter itself, so a dependency that settles while the edges are still being
// registered cannot post the job from under a half-built dependency list.
std::atomic<Int> m_remainingDeps{0};
};
} // namespace MobileGL::MG_State::GLState
File diff suppressed because it is too large Load Diff
@@ -14,9 +14,22 @@
#include <MG_Util/ShaderTranspiler/SpvcSession.h>
namespace MobileGL::MG_State::GLState {
// The link job. Only ever held by SharedPtr here, so a forward declaration is enough -
// ProgramLinkTask.h includes THIS header (it outputs a LinkArtifacts), so including it
// back would be circular. The destructor is therefore out of line.
class ProgramLinkTask;
class ProgramObject {
public:
ProgramObject(Uint externalIndex) : m_externalIndex(externalIndex), m_lifetimeId(AllocateLifetimeId()) {}
// Cancel-not-join, exactly like ~ShaderObject: the link job owns its inputs, so an
// in-flight link whose program just went away is safe to abandon where it stands.
// Nothing can observe its result any more - this object was the only route to it.
// Out of line because ProgramLinkTask is incomplete here.
~ProgramObject();
ProgramObject(const ProgramObject&) = delete;
ProgramObject& operator=(const ProgramObject&) = delete;
bool ShaderIsAttached(const SharedPtr<ShaderObject>& shader);
// GL-visible attachment: in the attach list and not pending detach (glDetachShader
// defers the actual removal to the next link).
@@ -151,14 +164,7 @@ namespace MobileGL::MG_State::GLState {
: -1;
}
Bool IsValidUniformLocation(Int location) const {
if (location < 0 || location > static_cast<Int>(Artifacts().maxUniformLocation)) return false;
if (static_cast<SizeT>(location) >= Artifacts().uniformIndexInTProgram.size()) return false;
const Int uniformIndexInProgram = Artifacts().uniformIndexInTProgram[location];
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
uniformIndexInProgram >= 0 &&
uniformIndexInProgram < static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
}
Bool IsValidUniformLocation(Int location) const { return IsValidUniformLocation(Artifacts(), location); }
GLenum GetUniformType(Uint location) const {
auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
@@ -176,12 +182,7 @@ namespace MobileGL::MG_State::GLState {
// for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
// the artifacts' uniformIndexInTProgram stores).
GLint GetUniformArraySizeByTIndex(Int tIndex) const {
const auto& uniform = Artifacts().program->getUniform(tIndex);
const glslang::TType* type = uniform.getType();
if (type != nullptr && type->isSizedArray()) {
return type->getOuterArraySize();
}
return uniform.size < 1 ? 1 : uniform.size;
return GetUniformArraySizeByTIndex(Artifacts(), tIndex);
}
GLint GetActiveUniformArraySize(Uint index) const {
@@ -371,7 +372,7 @@ namespace MobileGL::MG_State::GLState {
// can skip re-uploading an unchanged UBO on every draw. ~0u is reserved as the
// backends' "never uploaded" sentinel, so skip over it on wrap.
Uint32 GetUBOContentVersion() const { return m_uboContentVersion; }
void MarkUBOContentDirty() {
void MarkUBOContentDirty() const {
if (++m_uboContentVersion == ~0u) m_uboContentVersion = 0;
}
Uint32 GetBackendStateVersion() const { return m_backendStateVersion; }
@@ -439,8 +440,13 @@ namespace MobileGL::MG_State::GLState {
// glProgramBinary always fails here (there is no format it could accept) and the
// spec then requires the program's LINK_STATUS to read FALSE.
void MarkLinkFailedByProgramBinary() {
// Before anything reads m_artifacts: a pending link would otherwise publish its
// (possibly successful) result over the failure this call is required to install
// - and Artifacts() below would be the thing that let it. Cancel-not-join: GL
// gives glProgramBinary no reason to wait for a link it is about to invalidate.
CancelLink();
BumpLinkObservableVersions();
ResetLinkArtifacts();
ResetLinkArtifacts(Artifacts());
Artifacts().infoLog = "No program binary format is supported.";
}
Bool GetValidateStatus() const { return m_validateStatus; }
@@ -633,12 +639,68 @@ namespace MobileGL::MG_State::GLState {
Uint32 xfbPackedStride = 0;
};
// Blocks until a pending link (P1 stage 4 onwards) has published its artifacts.
// Public because a few call sites have to join without reading anything - see the
// explicit-join list in the P1 design. Today there is never a pending link, so this
// is a no-op; it is wired up when glLinkProgram starts enqueueing.
// ---- artifacts-only helpers, shared with ProgramLinkTask ----
// Static and taking the block explicitly, because from stage 4 the link BODY needs
// them while its artifacts still live on the job node, not on any ProgramObject. The
// member overloads above are the same functions read through the join gate.
// Clears every field one link produces, EXCEPT infoLog, linkedFragDataLocation/Index
// and the geometry strip-capture pair. That exception is load-bearing: the callers
// that survive (glProgramBinary's mandated failure, and the link body's own mid-link
// aborts) write infoLog immediately AFTER calling here. Link()'s prologue does not
// use this at all - it assigns a whole default-constructed LinkArtifacts, where the
// ordering is explicit and nothing is exempt.
static void ResetLinkArtifacts(LinkArtifacts& artifacts);
static Bool IsValidUniformLocation(const LinkArtifacts& artifacts, Int location) {
if (location < 0 || location > static_cast<Int>(artifacts.maxUniformLocation)) return false;
if (static_cast<SizeT>(location) >= artifacts.uniformIndexInTProgram.size()) return false;
const Int uniformIndexInProgram = artifacts.uniformIndexInTProgram[location];
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
uniformIndexInProgram >= 0 &&
uniformIndexInProgram < static_cast<Int>(artifacts.tProgramUniformIndexToGl.size());
}
// Number of active array elements (GL_UNIFORM_SIZE / GL_ARRAY_SIZE); 1 for a non-array.
// glslang's TObjectReflection.size only carries the element count for a NON-block array; for
// a block array member it reports 1, so take the count from the TType, which is authoritative
// for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
// the artifacts' uniformIndexInTProgram stores).
static GLint GetUniformArraySizeByTIndex(const LinkArtifacts& artifacts, Int tIndex) {
const auto& uniform = artifacts.program->getUniform(tIndex);
const glslang::TType* type = uniform.getType();
if (type != nullptr && type->isSizedArray()) {
return type->getOuterArraySize();
}
return uniform.size < 1 ? 1 : uniform.size;
}
// Blocks until a pending link has published its artifacts. Public because a few call
// sites have to join without reading anything - see the explicit-join list (J1-J8) in
// the P1 design. GL thread only.
void JoinLink() const { EnsureLinkJoined(); }
// Drops a link that is still in flight, without waiting for it. Called at the points
// where the pending link's result stops being the answer to "what did this program
// link to": a re-link supersedes it, glProgramBinary must force LINK_STATUS false,
// and a destroyed program has no observers left.
//
// Deliberately NOT called by the "takes effect at the next link" setters
// (glBindAttribLocation, glBindFragDataLocation(Indexed), glTransformFeedbackVaryings,
// glProgramParameteri) NOR by glAttachShader/glDetachShader. Every one of those is
// defined by GL to leave the CURRENT link result alone, and the pending link already
// snapshotted its own inputs at enqueue, so it is computing exactly the answer GL
// requires. Cancelling on any of them would make
// glLinkProgram(p); <setter>; glGetProgramiv(p, GL_LINK_STATUS)
// report FALSE for a link that succeeded - and for the attach/detach pair it would
// additionally break glCreateShaderProgramv, which detaches immediately after linking.
void CancelLink();
// MUST NOT JOIN - this is what GL_COMPLETION_STATUS_KHR reads when the extension
// surface lands. "No job at all" counts as complete: there is nothing outstanding to
// wait for.
Bool IsLinkComplete() const { return m_pendingLink == nullptr || IsPendingLinkTerminal(); }
void SetTransformFeedbackVaryings(Vector<String>&& names, GLenum bufferMode) {
m_requestedXfbVaryings = Move(names);
m_requestedXfbBufferMode = bufferMode;
@@ -685,19 +747,22 @@ namespace MobileGL::MG_State::GLState {
private:
// ---- The one and only join gate for link output (P1 invariant I5) ----
// Blocks until a pending link has finished and its LinkArtifacts have been
// published into m_artifacts. Today no link is ever pending - glLinkProgram still
// runs the whole body inline - so this is an unconditional no-op, and the whole
// Artifacts() indirection compiles away. It exists NOW so that the ~120 readers of
// link output are already routed through it when stage 4 makes it block: the edit
// that turns links asynchronous then touches this function and nothing else.
// published into m_artifacts. It exists so that the ~120 readers of link output are
// routed through it by the compiler rather than by review: m_artifacts is private
// and Artifacts() is the only spelling that reaches it.
//
// Defined inline (not in ProgramObject.cpp) on purpose: this is called from every
// Artifacts() read - ~1200 call sites project-wide - and the project never builds
// with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line empty body would leave a
// real cross-TU call at every one of them instead of folding away. Stage 4's
// version, which actually blocks, moves the wait itself out-of-line behind a
// `m_pendingLink` check that stays inline here.
void EnsureLinkJoined() const {}
// The fast path - no pending link - is one predictable branch and stays inline: it
// runs on every Artifacts() read (~1200 call sites project-wide) and the project
// never builds with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line body would be a
// real cross-TU call at every one of them. The blocking half is out of line.
void EnsureLinkJoined() const {
if (m_pendingLink) JoinPendingLink();
}
void JoinPendingLink() const;
// ProgramLinkTask is incomplete here, so IsLinkComplete()'s non-joining peek at the
// node's state goes through this out-of-line helper.
Bool IsPendingLinkTerminal() const;
LinkArtifacts& Artifacts() {
EnsureLinkJoined();
return m_artifacts;
@@ -707,28 +772,10 @@ namespace MobileGL::MG_State::GLState {
return m_artifacts;
}
void ResetLinkArtifacts();
// GL-thread-only companion to ResetLinkArtifacts (see its definition).
void BumpLinkObservableVersions();
// Builds the GL-facing reflection surface from the linked TProgram. Returns false
// (with the artifacts' infoLog set and link artifacts reset) when reflection itself fails or an
// explicit-uniform-location conflict makes the link invalid.
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
// Resolves the requested transform feedback varyings against the linked
// vertex stage; fails the link (GL semantics) on unknown or duplicate
// names or exceeded capture limits.
Bool ResolveTransformFeedbackVaryings();
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
// The former GenerateBinary, split around DoReflection's data dependencies:
// SPIR-V must be generated BEFORE buildReflection touches the linked TProgram (its
// live-variable analysis mutates the intermediates enough to change
// GlslangToSpv output), while the glUniform*-to-global-UBO routing tables are
// sized and keyed by reflection results (maxUniformLocation, uniformLocations)
// and so must run AFTER it.
void GenerateSpirv();
void BuildGlobalUboRouting();
// GL-thread-only companion to ResetLinkArtifacts (see its definition). Const because
// the publish half of the join calls it; see the mutable counters below.
void BumpLinkObservableVersions() const;
void AddDefaultFragmentShaderIfMissing();
Bool ValidateFragmentOutputLocations();
static Uint64 AllocateLifetimeId();
@@ -762,7 +809,10 @@ namespace MobileGL::MG_State::GLState {
Bool m_binaryRetrievableHint = false;
Bool m_separable = false;
Bool m_validateStatus = true;
Uint32 m_backendStateVersion = 0;
// Mutable, like m_artifacts and for the same reason: publishing a pending link is a
// READ-side operation (the first gated getter is what pulls the result in), and the
// publish has to bump these. Still GL-thread-only - a worker never touches them.
mutable Uint32 m_backendStateVersion = 0;
// Backend-owned content-hash memo (see GetBackendHashMemo): valid only while
// m_backendStateVersion matches. Several slots, not one: a backend may resolve the same
@@ -778,12 +828,20 @@ namespace MobileGL::MG_State::GLState {
mutable Array<BackendHashMemoSlot, kBackendHashMemoSlotCount> m_backendHashMemoSlots{};
mutable SizeT m_backendHashMemoNextSlot = 0;
mutable Uint32 m_backendHashMemoVersion = ~0u;
Uint32 m_uboContentVersion = 0;
Uint32 m_linkVersion = 0;
mutable Uint32 m_uboContentVersion = 0;
mutable Uint32 m_linkVersion = 0;
// ---- Link OUTPUT ----
// Written by the link and by the post-link setters GL allows (glUniform1i's sampler
// unit, glUniformBlockBinding). Reachable only through Artifacts(); see LinkArtifacts.
LinkArtifacts m_artifacts;
//
// Mutable because publishing is a READ-side operation: a const getter has to be able
// to settle an outstanding link before answering it.
mutable LinkArtifacts m_artifacts;
// The link job, from enqueue until the first observable read pulls its result. Null
// means m_artifacts is already the answer - which is the state every reader outside
// the pending window sees, and the whole reason the gate above is one branch.
mutable SharedPtr<ProgramLinkTask> m_pendingLink;
};
} // namespace MobileGL::MG_State::GLState
@@ -39,6 +39,14 @@ namespace MobileGL::MG_State::GLState {
void ProgramState::DestroyProgramSlot(const Uint program) {
auto& programObject = m_programObjects[program];
// P1 join site J4/J5 (glDeleteProgram, and the deferred destroy UseProgram performs
// when a deletion-flagged program stops being current). The program's name is about
// to go, so nothing can observe its link any more: cancel-not-join, so a delete never
// blocks the GL thread on a worker. Explicit rather than left to ~ProgramObject,
// because the reset below only destroys the object if this table held the last
// reference - a program still bound as current, or still referenced by a pipeline,
// outlives it, and its link should stop the moment the name does.
programObject->CancelLink();
// Snapshot the attachments: deleting the program is a detach point for shaders
// that were flagged with glDeleteShader while still attached.
const Vector<SharedPtr<ShaderObject>> attachedShaders = programObject->GetAttachedShaders();
@@ -91,6 +99,31 @@ namespace MobileGL::MG_State::GLState {
return m_shaderObjects[shader];
}
void ProgramState::JoinAllPendingWork() {
// Programs first: a link joins the compiles it depends on, so the shader pass that
// follows finds most of them already settled. The reverse order would be correct but
// would wait on each compile twice - once here, once inside the link's own prologue.
//
// A copy of each slot rather than a reference into the vector, and an index rather
// than an iterator: publishing a link replays deferred diagnostics, which reach
// pGLContext->RecordError. That does not touch these tables today, but it is a sink
// that can grow, and a reallocation underneath this loop would be a use-after-free
// that only shows up on the one GL call that walks the whole table. The copy costs a
// refcount bump on a path a mode switch takes at most once.
for (SizeT i = 0; i < m_programObjects.size(); ++i) {
const SharedPtr<ProgramObject> program = m_programObjects[i];
if (program) program->JoinLink();
}
for (SizeT i = 0; i < m_shaderObjects.size(); ++i) {
const SharedPtr<ShaderObject> shader = m_shaderObjects[i];
if (shader) shader->JoinCompile();
}
// The currently-used program is reachable through m_programObjects unless
// glDeleteProgram already freed its slot while it stayed current. Nothing else holds
// a GL-visible name for it, but a draw would still join it, so settle it here too.
if (m_currentProgram) m_currentProgram->JoinLink();
}
void ProgramState::MarkShaderObjectForDeletion(Uint shader) {
if (!CheckIndexAvail(shader, m_shaderObjects)) return;
auto& shaderObject = m_shaderObjects[shader];
@@ -120,6 +153,11 @@ namespace MobileGL::MG_State::GLState {
auto& shaderObject = m_shaderObjects[shader];
if (shaderObject == nullptr || !shaderObject->GetDeleteStatus()) return;
if (ShaderHasGLVisibleAttachment(shaderObject)) return;
// The name is about to go: nothing can observe this shader's compile any more, so a
// job still in flight for it is pure waste. Cancel-not-join - the job owns its
// inputs, so dropping the object out from under it is safe and the GL thread never
// blocks on a delete.
shaderObject->CancelCompile();
shaderObject.reset();
m_programShaderNameGenerator.Delete(shader);
}
@@ -34,6 +34,19 @@ namespace MobileGL::MG_State::GLState {
const SharedPtr<ProgramObject>& GetCurrentProgram() const { return m_currentProgram; }
// Joins every outstanding compile and link this context still owns, publishing each
// one's artifacts through the ordinary gates. The single caller is
// glMaxShaderCompilerThreadsKHR(0): GL_KHR_parallel_shader_compile requires a zero
// count to leave nothing in flight, so that every subsequent
// GL_COMPLETION_STATUS_KHR reads GL_TRUE.
//
// NOT a teardown path and NOT ShaderCompilePool::StopAndDrain(): the pool keeps its
// threads and stays usable, because a later nonzero count has to bring asynchronous
// compilation straight back. Nodes belonging to objects this context has already
// dropped are not joined - nothing can observe them, and waiting on them would make
// a GL call's cost depend on garbage.
void JoinAllPendingWork();
// P0b layer 2. Exposed for tests and diagnostics; the GL frontend never touches it
// directly - shader objects reach it through the pointer they are handed at
// CreateShader().
@@ -0,0 +1,338 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderCompileTask.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ShaderCompileTask.h"
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h>
#include <MG_Util/ShaderTranspiler/Types.h>
#include <glslang/Include/PoolAlloc.h>
#include <charconv>
namespace {
struct ComputeLocalSize {
MobileGL::Uint x = 1;
MobileGL::Uint y = 1;
MobileGL::Uint z = 1;
bool declared = false;
};
static MobileGL::String StripGlslComments(const MobileGL::String& source) {
MobileGL::String result;
result.reserve(source.length());
bool inLineComment = false;
bool inBlockComment = false;
for (MobileGL::SizeT i = 0; i < source.length(); ++i) {
if (inLineComment) {
if (source[i] == '\n') {
inLineComment = false;
result.push_back(source[i]);
} else {
result.push_back(' ');
}
continue;
}
if (inBlockComment) {
if (source[i] == '*' && i + 1 < source.length() && source[i + 1] == '/') {
inBlockComment = false;
result.append(" ");
++i;
} else {
result.push_back(source[i] == '\n' ? '\n' : ' ');
}
continue;
}
if (source[i] == '/' && i + 1 < source.length()) {
if (source[i + 1] == '/') {
inLineComment = true;
result.append(" ");
++i;
continue;
}
if (source[i + 1] == '*') {
inBlockComment = true;
result.append(" ");
++i;
continue;
}
}
result.push_back(source[i]);
}
return result;
}
// Hoisted out of ParseComputeLocalSize: constructing a std::regex costs far more than
// running it over a small source, and it was being rebuilt on every compute compile. A
// const regex carries no mutable state, so sharing one instance across workers is safe.
static const std::regex kComputeLocalSizePattern(R"(local_size_([xyz])\s*=\s*([0-9]+))");
static ComputeLocalSize ParseComputeLocalSize(const MobileGL::String& source) {
ComputeLocalSize localSize;
const MobileGL::String uncommentedSource = StripGlslComments(source);
for (std::sregex_iterator it(uncommentedSource.begin(), uncommentedSource.end(), kComputeLocalSizePattern),
end;
it != end; ++it) {
const char axis = (*it)[1].str()[0];
// The [0-9]+ capture is unbounded, so `local_size_x = 99999999999999999999999`
// is a legal match. std::stoull would throw std::out_of_range on it and let the
// exception escape glCompileShader; std::from_chars reports the overflow instead.
// An overflowing literal saturates to UINT_MAX, which the device-limit check
// below rejects anyway - the same verdict a non-overflowing huge value gets.
const MobileGL::String digits = (*it)[2].str();
unsigned long long value = 0;
const std::from_chars_result parsed =
std::from_chars(digits.data(), digits.data() + digits.size(), value);
const MobileGL::Uint clampedValue = (parsed.ec != std::errc() || value > UINT_MAX)
? UINT_MAX
: static_cast<MobileGL::Uint>(value);
// TODO: Replace this literal layout scanner with parser/AST-backed validation so expressions and
// specialization-id layouts are handled consistently with glslang.
localSize.declared = true;
if (axis == 'x') {
localSize.x = clampedValue;
} else if (axis == 'y') {
localSize.y = clampedValue;
} else {
localSize.z = clampedValue;
}
}
return localSize;
}
// The device limits come from the CompileEnv snapshot, never from a live driver query.
// GL_MAX_COMPUTE_WORK_GROUP_SIZE is a real GLES call on the DirectGLES backend: issued
// off the context thread it would silently no-op and turn a legal local_size_z into
// COMPILE_STATUS=FALSE. CaptureCompileEnv() issues it once, on the GL thread.
static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(
const MobileGL::String& source, const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
const ComputeLocalSize localSize = ParseComputeLocalSize(source);
if (!localSize.declared) return std::nullopt;
if (localSize.x > env.maxComputeWorkGroupSize[0] || localSize.y > env.maxComputeWorkGroupSize[1] ||
localSize.z > env.maxComputeWorkGroupSize[2]) {
return "Compute shader local_size exceeds GL_MAX_COMPUTE_WORK_GROUP_SIZE.";
}
const unsigned long long invocations = static_cast<unsigned long long>(localSize.x) * localSize.y * localSize.z;
if (invocations > env.maxComputeWorkGroupInvocations) {
return "Compute shader local_size product exceeds GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS.";
}
return std::nullopt;
}
// The half of a compile that depends on nothing but the source text, the stage and the
// environment snapshot: preprocessing, the two lexical rejections, and the two lexical
// side-channel extractions. Split out so P0b layer 2 can memoize exactly this and
// nothing else - the glslang parse stays per-object because its TShader is consume-once.
// Deliberately free of any per-object state so the memo is sound.
//
// The compute local-size verdict reads `env` rather than the live backend, and
// env.fingerprint is part of the P0b cache key, so a memo can never be returned against
// limits other than the ones it was computed against.
static MobileGL::MG_State::GLState::ShaderPreprocessResult RunSourceOnlyPipeline(
const MobileGL::ShaderStage stage, const MobileGL::String& source,
const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
using namespace MobileGL;
using namespace MobileGL::MG_Util::ShaderTranspiler;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
MobileGL::MG_State::GLState::ShaderPreprocessResult result;
result.preprocessedSource = source;
PreprocessShaderSource(stage, result.preprocessedSource, env);
if (stage == ShaderStage::Compute) {
if (const std::optional<String> localSizeError =
ValidateComputeLocalSizeLimits(result.preprocessedSource, env)) {
result.outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
result.infoLog = *localSizeError;
return result;
}
}
if (const std::optional<String> reservedError = FindReservedIdentifierViolation(result.preprocessedSource)) {
result.outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
result.infoLog = *reservedError;
return result;
}
// The parse this feeds runs in the link-compatible configuration (Vulkan-client
// env with relaxed rules): the TShader it produces is what glLinkProgram links and
// what the backends' SPIR-V is generated from - there is no second, GL-client
// parse. The GL frontend semantics the relaxed parse cannot provide are restored
// on top: explicit default-block uniform locations through the lexical
// side-channels below, dead-uniform/global-UBO filtering in
// ProgramObject::DoReflection.
result.explicitUniformLocations = ExtractExplicitUniformLocations(result.preprocessedSource);
result.explicitOpaqueBindings = ExtractExplicitOpaqueBindings(result.preprocessedSource);
result.outcome = ShaderPreprocessOutcome::Preprocessed;
return result;
}
} // namespace
namespace MobileGL::MG_State::GLState {
// glslang has no "detach this thread" API in the vendored revision (there is no
// InitThread/DetachThread pair any more; thread attachment is implicit through
// thread_local state, and glslang::InitializeProcess() is process-wide, refcounted and
// mutex-guarded, so it needs no per-worker counterpart). The pool allocator is the part
// that needs undoing; see the declaration in ShaderCompileTask.h.
GlslangThreadAllocatorGuard::~GlslangThreadAllocatorGuard() { glslang::SetThreadPoolAllocator(nullptr); }
// Pure CPU work only. Everything this reads is either an input the node owns or a
// process-wide constant; everything it writes is `artifacts`. Do not add a GL/EGL call,
// a pActiveBackendObject read, or a pGLContext->RecordError() here - the first two are
// what CompileEnv exists to replace, and the third is why the design's section 6
// deferral mechanism (and JobNode's debug assert on it) exists.
void ShaderCompileTask::RunBody() {
// Own the failure rather than letting JobNode's backstop settle the node as
// Cancelled: an abandoned node publishes nothing, so the shader would report
// COMPILE_STATUS false with an EMPTY info log. GL models a failed compile as
// status + log, so turn a throw into exactly that - a completed job whose result
// is "this shader did not compile", with a log the application can read.
// (JobNode still catches: it is the last resort for anything below.)
try {
RunCompilePipeline();
} catch (const std::exception& e) {
artifacts = {};
artifacts.env = env;
artifacts.compileStatus = false;
artifacts.infoLog = std::format("Error: shader compilation failed: {}", e.what());
} catch (...) {
artifacts = {};
artifacts.env = env;
artifacts.compileStatus = false;
artifacts.infoLog = "Error: shader compilation failed: unknown exception";
}
}
void ShaderCompileTask::RunCompilePipeline() {
using namespace MG_Util::ShaderTranspiler;
const GlslangThreadAllocatorGuard glslangGuard;
const CompileEnv& compileEnv = *env;
artifacts.env = env;
// P0b layer 2: another shader object in this context may already have run the
// source-only half over byte-identical text under the same environment.
ShaderPreprocessResultPtr cached =
cache ? cache->Find(stage, sourceHash, *source, compileEnv.fingerprint) : nullptr;
SharedPtr<ShaderPreprocessResult> fresh;
if (!cached) fresh = MakeShared<ShaderPreprocessResult>(RunSourceOnlyPipeline(stage, *source, compileEnv));
const ShaderPreprocessResult& shared = cached ? *cached : *fresh;
const Bool shouldPopulateCache = !cached && cache != nullptr;
if (!shared.Preprocessed()) {
// Rejected lexically, or a glslang failure this context has already seen for
// this exact source (ParseFailed) - either way the parse can be skipped.
artifacts.infoLog = shared.infoLog;
if (shouldPopulateCache) {
cache->Insert(stage, sourceHash, *source, compileEnv.fingerprint, Move(fresh));
}
return;
}
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(stage),
.sourceStr = shared.preprocessedSource,
.flags = 0,
.env = &compileEnv};
auto result = ShaderCompiler::CompileShader(attrib);
if (result) {
artifacts.compileStatus = true;
artifacts.shader = result.value();
// Copy, not move: `shared` may alias a cache entry that has to outlive us, and
// `fresh` is about to be handed to the cache.
artifacts.preprocessedSource = shared.preprocessedSource;
artifacts.explicitUniformLocations = shared.explicitUniformLocations;
artifacts.explicitOpaqueBindings = shared.explicitOpaqueBindings;
artifacts.infoLog.clear();
if (shouldPopulateCache) {
cache->Insert(stage, sourceHash, *source, compileEnv.fingerprint, Move(fresh));
}
} else {
artifacts.infoLog = result.error().log;
// Deferred, not logged here, for two reasons. MGLOG from a pool thread interleaves
// mid-line with the GL thread's own output and lands out of order relative to the
// glCompileShader that caused it; diagnostics.logLines is replayed by the join, on
// the GL thread, exactly where a serial implementation would have printed it.
// And a one-line summary rather than the old full source dump: a shaderpack stage
// is ~100KB, so the dump was the single largest thing this driver ever wrote to
// the log, for every failing shader. The info log is what names the offending
// line; the source is recoverable from the application.
const SizeT firstLineEnd = artifacts.infoLog.find('\n');
diagnostics.logLines.push_back(std::format(
"ShaderCompileTask: shader {} (stage {}) failed to compile; compileStatus = false. "
"Preprocessed source: {} bytes. First log line: {}",
externalIndex, static_cast<Int>(stage), shared.preprocessedSource.length(),
artifacts.infoLog.substr(0, firstLineEnd == String::npos ? artifacts.infoLog.length()
: firstLineEnd)));
if (shouldPopulateCache) {
fresh->outcome = ShaderPreprocessOutcome::ParseFailed;
fresh->infoLog = artifacts.infoLog;
fresh->explicitUniformLocations.clear();
fresh->explicitOpaqueBindings.clear();
cache->Insert(stage, sourceHash, *source, compileEnv.fingerprint, Move(fresh));
}
}
}
SharedPtr<glslang::TShader> ShaderCompileTask::ClaimParsedShader(String& outReparseLog) const {
MOBILEGL_ASSERT(IsComplete(),
"ShaderCompileTask::ClaimParsedShader() on a job that has not completed; its artifacts "
"are still being written");
if (artifacts.shader) {
// The whole race, in one instruction. Acquire-release because the winner is about
// to hand the TShader to glslang's linker on a possibly different thread from the
// one that parsed it - the node's terminal transition already published the
// parse, and this orders the two claimants against each other.
Bool expected = false;
if (m_parseClaimed.compare_exchange_strong(expected, true, std::memory_order_acq_rel,
std::memory_order_acquire)) {
return artifacts.shader;
}
}
// Either another link already consumed the stored parse (and mapIO mutated its
// intermediate), or there never was one. Re-parse the preprocessed source through the
// identical configuration; that costs one glslang parse, which is what GenerateBinary
// used to spend here on EVERY link rather than only on reuse.
//
// The guard is not optional on this path: from stage 4 this runs on a pool worker,
// and TShader::parse would leave that worker's TLS allocator pointing at a pool the
// GL thread is about to free. (ProgramLinkTask::RunBody holds one too; they nest
// harmlessly - both just reset the thread to its own default.)
const GlslangThreadAllocatorGuard glslangGuard;
using namespace MG_Util::ShaderTranspiler;
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(stage),
.sourceStr = artifacts.preprocessedSource,
.flags = 0,
// Re-parse against the SAME environment the original parse used,
// not against whatever the backend reports now.
.env = artifacts.env.get()};
auto result = ShaderCompiler::CompileShader(attrib);
if (!result) {
// Should be unreachable: the same source parsed successfully at Compile().
outReparseLog = result.error().log;
return nullptr;
}
return result.value();
}
} // namespace MobileGL::MG_State::GLState
@@ -0,0 +1,136 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderCompileTask.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_Util/Async/JobNode.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <MG_State/GLState/ProgramState/ShaderPreprocessCache.h>
namespace MobileGL::MG_State::GLState {
// glslang has no "detach this thread" API in the vendored revision, but TShader::parse
// leaves the calling thread's TLS pool allocator pointing at the shader's own pool and
// never restores it. Left there, the next allocation this thread makes - in an unrelated
// job, or in glslang code reached from a different object - would come out of a pool the
// GL thread may already have deleted with the TShader. SetThreadPoolAllocator(nullptr)
// reverts the thread to its own thread_local default and is the documented idiom.
//
// A scope guard, so it also runs when a body throws. Declared here rather than kept
// file-local because stage 4 gave it a second user: ProgramLinkTask's body parses (the
// claim-CAS loser's re-parse), links and emits SPIR-V, all on a pool thread.
struct GlslangThreadAllocatorGuard {
GlslangThreadAllocatorGuard() = default;
~GlslangThreadAllocatorGuard();
GlslangThreadAllocatorGuard(const GlslangThreadAllocatorGuard&) = delete;
GlslangThreadAllocatorGuard& operator=(const GlslangThreadAllocatorGuard&) = delete;
};
// Everything one glCompileShader PRODUCES, in one block.
//
// This is exactly the set a single run of the compile pipeline writes, which is what
// makes "discard the artifacts" a complete invalidation and "move the artifacts" a
// complete publish. It lives on the job node rather than on ShaderObject: a worker fills
// it in, and the GL thread reads it through ShaderObject's join gate.
struct ShaderCompileArtifacts {
// The CompileEnv snapshot this compile ran against. Held so the consume-once
// re-parse in ClaimParsedShader() reproduces the original parse exactly, instead of
// re-reading whatever the backend says now.
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
SharedPtr<glslang::TShader> shader;
// The source the parse actually consumed (after PreprocessShaderSource), kept for
// ClaimParsedShader's re-parse so a later link never depends on the preprocessor
// being deterministic across backend-state changes.
String preprocessedSource;
UnorderedMap<String, Int> explicitUniformLocations;
UnorderedMap<String, Uint> explicitOpaqueBindings;
String infoLog;
Bool compileStatus = false;
};
// The unit of asynchronous shader compilation: one glCompileShader's worth of pure CPU
// work - preprocess, the two lexical rejections, the two lexical extractions, and the
// glslang parse - with every input it needs owned by the node itself.
//
// That ownership is the whole point. The node reads no GL-thread state (the source is a
// SharedPtr<const String> snapshot, the device limits come from the CompileEnv snapshot,
// the P0b cross-object memo is shared-owned and internally locked) and writes nothing
// but its own `artifacts`. So a node whose ShaderObject was re-sourced, deleted, or
// destroyed while it was still running is safe to simply abandon - no wait, no
// synchronization with the GL thread beyond the node's own terminal state.
class ShaderCompileTask final : public MG_Util::Async::JobNode {
public:
ShaderCompileTask(const ShaderStage stage, SharedPtr<const String> source, const Uint64 sourceHash,
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env,
SharedPtr<ShaderPreprocessCache> cache, const Uint externalIndex)
: stage(stage), source(Move(source)), sourceHash(sourceHash), env(Move(env)), cache(Move(cache)),
externalIndex(externalIndex) {}
// ---- inputs: immutable after construction, all owned by the node ----
const ShaderStage stage;
// The exact text at enqueue. ShaderObject compares this pointer against its own
// m_source to decide whether its layer-1 memo is armed, which is why glShaderSource
// only ever swaps the pointer when the text genuinely differs.
const SharedPtr<const String> source;
const Uint64 sourceHash;
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
// P0b layer 2, or null. Null is the "no context" case (the default fragment shader,
// the backends' internal blit/mipmap shaders) and doubles as the marker for
// "compile inline regardless of the async flag" - see ShaderObject::Compile().
const SharedPtr<ShaderPreprocessCache> cache;
const Uint externalIndex; // logs only
// ---- output: valid iff IsComplete(), immutable afterwards ----
ShaderCompileArtifacts artifacts;
// Hands out a link-consumable TShader, exactly once for the stored parse.
//
// glslang's mapIO mutates the TShader's aliased intermediate, so the parse this node
// produced may feed exactly ONE link; every later link (a relink, or the same shader
// attached to a second program) needs a fresh parse. The claim is a CAS on this
// shared node rather than a flag on the ShaderObject because from stage 4 the two
// callers can be two ProgramLinkTasks running on two workers: two programs sharing
// one shader, linked back to back. Copying the parse out and tracking consumed-ness
// per program would let both of them decide they were the first, run mapIO over the
// same intermediate twice, and ship silently corrupt SPIR-V.
//
// The CAS loser re-parses artifacts.preprocessedSource against THIS node's own
// CompileEnv (not against whatever the backend reports now), through the identical
// CompileShader path - so winner and loser produce byte-identical SPIR-V. Callable
// only once IsComplete() and compileStatus are true. Returns null only if that
// re-parse fails, and outReparseLog then carries its diagnostics.
//
// Const because the claim is the node's own synchronization, not a mutation of its
// published artifacts: a claim that is taken and then abandoned (its link was
// cancelled) costs one extra re-parse later and nothing else.
SharedPtr<glslang::TShader> ClaimParsedShader(String& outReparseLog) const;
// Sticky marker for "a ProgramLinkTask has this node in its input snapshot".
//
// It exists to keep a cancel from eating a result someone still needs. A pending link
// holds its dependencies by SharedPtr, so the NODE always outlives the ShaderObject -
// but Cancel() is not about lifetime, it discards the result. The reachable sequence
// is the ordinary one: compile, attach, glLinkProgram (enqueued), glDetachShader,
// glDeleteShader. The detach makes the shader GL-invisible, so the delete frees its
// name, and ReleaseShaderNameIfOrphaned would cancel a compile the enqueued link is
// waiting on - turning a link that must report GL_TRUE into GL_FALSE. Set on the GL
// thread in Link()'s prologue, read on the GL thread by ShaderObject::CancelCompile.
//
// Never cleared: the worst case is one stale node compiling to completion for nobody,
// which is exactly what the pre-stage-3 implementation always did.
void MarkLinkReferenced() { m_linkReferenced.store(true, std::memory_order_release); }
Bool IsLinkReferenced() const { return m_linkReferenced.load(std::memory_order_acquire); }
private:
void RunBody() override;
// The real body; RunBody wraps it so a throw becomes a GL-visible compile failure.
void RunCompilePipeline();
mutable std::atomic<Bool> m_parseClaimed{false};
std::atomic<Bool> m_linkReferenced{false};
};
} // namespace MobileGL::MG_State::GLState
@@ -8,338 +8,126 @@
#include "ShaderObject.h"
#include "ShaderPreprocessCache.h"
#include <MG_Util/ShaderTranspiler/Types.h>
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <MG_Util/Converters/MGToGL/ProgramEnumConverter.h>
#include <MG_Util/ShaderTranspiler/ShaderSourceProcessor.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <MG_Util/ShaderTranspiler/glslang/UniformTraverser.h>
#include <charconv>
namespace {
struct ComputeLocalSize {
MobileGL::Uint x = 1;
MobileGL::Uint y = 1;
MobileGL::Uint z = 1;
bool declared = false;
};
static MobileGL::String StripGlslComments(const MobileGL::String& source) {
MobileGL::String result;
result.reserve(source.length());
bool inLineComment = false;
bool inBlockComment = false;
for (MobileGL::SizeT i = 0; i < source.length(); ++i) {
if (inLineComment) {
if (source[i] == '\n') {
inLineComment = false;
result.push_back(source[i]);
} else {
result.push_back(' ');
}
continue;
}
if (inBlockComment) {
if (source[i] == '*' && i + 1 < source.length() && source[i + 1] == '/') {
inBlockComment = false;
result.append(" ");
++i;
} else {
result.push_back(source[i] == '\n' ? '\n' : ' ');
}
continue;
}
if (source[i] == '/' && i + 1 < source.length()) {
if (source[i + 1] == '/') {
inLineComment = true;
result.append(" ");
++i;
continue;
}
if (source[i + 1] == '*') {
inBlockComment = true;
result.append(" ");
++i;
continue;
}
}
result.push_back(source[i]);
}
return result;
}
// Hoisted out of ParseComputeLocalSize: constructing a std::regex costs far more than
// running it over a small source, and it was being rebuilt on every compute compile. A
// const regex carries no mutable state, so sharing one instance is safe.
static const std::regex kComputeLocalSizePattern(R"(local_size_([xyz])\s*=\s*([0-9]+))");
static ComputeLocalSize ParseComputeLocalSize(const MobileGL::String& source) {
ComputeLocalSize localSize;
const MobileGL::String uncommentedSource = StripGlslComments(source);
for (std::sregex_iterator it(uncommentedSource.begin(), uncommentedSource.end(), kComputeLocalSizePattern),
end;
it != end; ++it) {
const char axis = (*it)[1].str()[0];
// The [0-9]+ capture is unbounded, so `local_size_x = 99999999999999999999999`
// is a legal match. std::stoull would throw std::out_of_range on it and let the
// exception escape glCompileShader; std::from_chars reports the overflow instead.
// An overflowing literal saturates to UINT_MAX, which the device-limit check
// below rejects anyway - the same verdict a non-overflowing huge value gets.
const MobileGL::String digits = (*it)[2].str();
unsigned long long value = 0;
const std::from_chars_result parsed =
std::from_chars(digits.data(), digits.data() + digits.size(), value);
const MobileGL::Uint clampedValue = (parsed.ec != std::errc() || value > UINT_MAX)
? UINT_MAX
: static_cast<MobileGL::Uint>(value);
// TODO: Replace this literal layout scanner with parser/AST-backed validation so expressions and
// specialization-id layouts are handled consistently with glslang.
localSize.declared = true;
if (axis == 'x') {
localSize.x = clampedValue;
} else if (axis == 'y') {
localSize.y = clampedValue;
} else {
localSize.z = clampedValue;
}
}
return localSize;
}
// The device limits come from the CompileEnv snapshot, never from a live driver query.
// GL_MAX_COMPUTE_WORK_GROUP_SIZE is a real GLES call on the DirectGLES backend: issued
// off the context thread it would silently no-op and turn a legal local_size_z into
// COMPILE_STATUS=FALSE. CaptureCompileEnv() issues it once, on the GL thread.
static std::optional<MobileGL::String> ValidateComputeLocalSizeLimits(
const MobileGL::String& source, const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
const ComputeLocalSize localSize = ParseComputeLocalSize(source);
if (!localSize.declared) return std::nullopt;
if (localSize.x > env.maxComputeWorkGroupSize[0] || localSize.y > env.maxComputeWorkGroupSize[1] ||
localSize.z > env.maxComputeWorkGroupSize[2]) {
return "Compute shader local_size exceeds GL_MAX_COMPUTE_WORK_GROUP_SIZE.";
}
const unsigned long long invocations = static_cast<unsigned long long>(localSize.x) * localSize.y * localSize.z;
if (invocations > env.maxComputeWorkGroupInvocations) {
return "Compute shader local_size product exceeds GL_MAX_COMPUTE_WORK_GROUP_INVOCATIONS.";
}
return std::nullopt;
}
// The half of ShaderObject::Compile() that depends on nothing but the source text and
// the stage: preprocessing, the two lexical rejections, and the two lexical
// side-channel extractions. Split out so P0b layer 2 can memoize exactly this and
// nothing else - the glslang parse stays per-object because its TShader is
// consume-once. Deliberately free of any per-object state so the memo is sound.
//
// The former caveat is gone: the compute local-size verdict reads `env` rather than the
// live backend, and env.fingerprint is part of the P0b cache key, so a memo can never be
// returned against limits other than the ones it was computed against.
static MobileGL::MG_State::GLState::ShaderPreprocessResult RunSourceOnlyPipeline(
const MobileGL::ShaderStage stage, const MobileGL::String& source,
const MobileGL::MG_Util::ShaderTranspiler::CompileEnv& env) {
using namespace MobileGL;
using namespace MobileGL::MG_Util::ShaderTranspiler;
using MobileGL::MG_State::GLState::ShaderPreprocessOutcome;
MobileGL::MG_State::GLState::ShaderPreprocessResult result;
result.preprocessedSource = source;
PreprocessShaderSource(stage, result.preprocessedSource, env);
if (stage == ShaderStage::Compute) {
if (const std::optional<String> localSizeError =
ValidateComputeLocalSizeLimits(result.preprocessedSource, env)) {
result.outcome = ShaderPreprocessOutcome::ComputeLocalSizeRejected;
result.infoLog = *localSizeError;
return result;
}
}
if (const std::optional<String> reservedError = FindReservedIdentifierViolation(result.preprocessedSource)) {
result.outcome = ShaderPreprocessOutcome::ReservedIdentifierRejected;
result.infoLog = *reservedError;
return result;
}
// The parse this feeds runs in the link-compatible configuration (Vulkan-client
// env with relaxed rules): the TShader it produces is what glLinkProgram links and
// what the backends' SPIR-V is generated from - there is no second, GL-client
// parse. The GL frontend semantics the relaxed parse cannot provide are restored
// on top: explicit default-block uniform locations through the lexical
// side-channels below, dead-uniform/global-UBO filtering in
// ProgramObject::DoReflection.
result.explicitUniformLocations = ExtractExplicitUniformLocations(result.preprocessedSource);
result.explicitOpaqueBindings = ExtractExplicitOpaqueBindings(result.preprocessedSource);
result.outcome = ShaderPreprocessOutcome::Preprocessed;
return result;
}
}
#include <MG_Util/ShaderTranspiler/ShaderCompiler.h>
#include <MG_Util/ShaderTranspiler/Types.h>
namespace MobileGL::MG_State::GLState {
void ShaderObject::SetShaderSource(const String& source) {
// P0b layer 1. glShaderSource always REPLACES the source, but replacing it with a
// byte-identical one cannot change what a compile would produce: the whole
// pipeline below (preprocess -> lexical checks -> glslang parse) is a pure
// function of (stage, source) plus context-lifetime backend limits. So keeping the
// compiled state is not an optimization that changes observable behaviour - the
// COMPILE_STATUS, the info log and the reflection a caller can query are exactly
// what a real recompile would have rebuilt, byte for byte.
// pipeline (preprocess -> lexical checks -> glslang parse) is a pure function of
// (stage, source, CompileEnv). So keeping the compiled state is not an optimization
// that changes observable behaviour - the COMPILE_STATUS, the info log and the
// reflection a caller can query are exactly what a real recompile would have
// rebuilt, byte for byte. A compile still IN FLIGHT is left running for the same
// reason: it is computing the right answer for text this object still holds.
if (SourceMatchesCompiledState(source)) return;
m_source = source;
// The text genuinely changed, so whatever a running job is computing is now about
// an old source. Drop it where it stands - it owns its own copy of that old string,
// so swapping the pointer below cannot race its storage.
CancelCompile();
m_source = MakeShared<const String>(source);
InvalidateCompiledState();
}
void ShaderObject::SetShaderSource(String&& source) {
if (SourceMatchesCompiledState(source)) return;
m_source = Move(source);
CancelCompile();
m_source = MakeShared<const String>(Move(source));
InvalidateCompiledState();
}
Bool ShaderObject::SourceMatchesCompiledState(const String& candidate) const {
if (!m_hasCompiledState) return false;
if (candidate.length() != m_compiledSourceLength) return false;
if (ShaderPreprocessCache::HashSource(candidate) != m_compiledSourceHash) return false;
// The hash is a fast reject only; confirm against the actual stored text. While
// m_hasCompiledState holds, m_source IS the source that produced the state.
return candidate == m_source;
// The memo is armed exactly while a job exists that was built from the string this
// object still points at - pending or finished, success or failure.
if (!HasMemoizedCompile()) return false;
if (candidate.length() != m_source->length()) return false;
// Never let correctness ride on a hash: the answer is the full text comparison.
// (The stored hash on the node is a cache-lookup accelerator, not a substitute.)
return candidate == *m_source;
}
void ShaderObject::RememberCompiledSource(const Uint64 sourceHash) {
m_hasCompiledState = true;
m_compiledSourceHash = sourceHash;
m_compiledSourceLength = m_source.length();
void ShaderObject::JoinPendingCompile() const {
MOBILEGL_ASSERT(!MG_Util::Async::ShaderCompilePool::IsPoolThread(),
"ShaderObject::EnsureCompileJoined() reached from a pool thread; a job body must never read "
"GL-thread-owned objects");
m_compiled->Wait();
m_compileJoined = true;
// Errors and worker-side log lines are raised HERE, on the GL thread, at the first
// join of the job that produced them - which for a single shader is trivially the
// order a serial implementation would have produced them in.
MG_Util::Async::ApplyDeferredDiagnostics(*m_compiled);
// A node that settled as Cancelled published nothing. Dropping it here is what keeps
// the object's state machine to two reachable cases - "no job" and "a job that
// completed" - so every reader below can treat a live node as authoritative.
if (!m_compiled->IsComplete()) m_compiled.reset();
}
// EnsureCompileJoined() is defined inline in ShaderObject.h (see the comment there for
// why: no LTO, and it is called from every Compiled() read).
void ShaderObject::InvalidateCompiledState() {
// The compile artifacts are exactly what one Compile() writes, so discarding them
// wholesale IS the invalidation. (Kept as an explicit reset rather than a
// default-construct so the intent survives a future field addition.)
Compiled() = CompileArtifacts{};
m_hasCompiledState = false;
m_compiledSourceHash = 0;
m_compiledSourceLength = 0;
// The job node holds exactly what one Compile() produces, so discarding it IS the
// invalidation - and it re-arms nothing, so the next Compile() genuinely recompiles.
m_compiled.reset();
}
void ShaderObject::CancelCompile() {
if (!m_compiled || m_compiled->IsTerminal()) return;
// Cooperative and non-blocking. A node that no worker has picked up settles
// immediately; one that is running is flagged and settles when its body returns,
// writing only into itself the whole time.
//
// Unless a pending LINK is waiting on it. Cancelling is about discarding a result
// nothing can observe any more, and this object is no longer the only route to this
// one: an enqueued ProgramLinkTask holds the node as a dependency, and a cancel would
// turn its link into GL_FALSE. Reached by the ordinary link-then-detach-then-delete
// shader teardown - see ShaderCompileTask::MarkLinkReferenced. Dropping our own
// reference is still right; the link keeps the node alive and finishes it.
if (!m_compiled->IsLinkReferenced()) m_compiled->Cancel();
m_compiled.reset();
}
void ShaderObject::Compile() {
using namespace MG_Util::ShaderTranspiler;
// P0b layer 1: the state this object holds was produced by a previous Compile() of
// the exact source it still holds, so a recompile is a no-op. This covers the
// failure case too - the info log stays queryable because nothing is cleared.
// P0b layer 1, as a tri-state: the memo is "the node in m_compiled was built from
// the string m_source still points at". SetShaderSource only swaps that pointer when
// the text actually differs, so this is a pointer compare, and it covers Pending as
// well as Complete - a second glCompileShader on an in-flight object is a no-op, not
// a duplicate job racing to write the same fields.
//
// shaderConsumedByLink interaction: if the stored TShader already fed a link,
// the no-op leaves preprocessedSource and both side-channel maps intact, which
// is precisely what TakeShaderForLink's on-demand re-parse needs. A real recompile
// would have handed the next link a fresh parse; the no-op hands it a fresh
// re-parse of the identical source instead. Same result, one parse either way.
if (m_hasCompiledState) return;
// The failure case is covered too: the info log stays queryable because nothing is
// cleared. And if the stored TShader already fed a link, the no-op leaves
// preprocessedSource and both side-channel maps intact, which is precisely what
// ClaimParsedShader's on-demand re-parse needs - a real recompile would have handed
// the next link a fresh parse, the no-op hands it a fresh re-parse of the identical
// source instead. Same result, one parse either way.
if (HasMemoizedCompile()) return;
InvalidateCompiledState();
// The compile-environment snapshot is taken HERE, on the GL thread, and handed to
// the job. Everything the pipeline needs to know about the device comes through it,
// never through pActiveBackendObject - that is what makes the body movable.
m_compiled = MakeShared<ShaderCompileTask>(m_stage, m_source, ShaderPreprocessCache::HashSource(*m_source),
MG_Util::ShaderTranspiler::GetCurrentCompileEnv(),
m_preprocessCache, m_externalIndex);
m_compileJoined = false;
const Uint64 sourceHash = ShaderPreprocessCache::HashSource(m_source);
// The compile-environment snapshot, taken here on the GL thread. Everything below
// reads the device through it and never through pActiveBackendObject, which is what
// makes the whole body movable onto a worker in stage 3.
CompileArtifacts& compiled = Compiled();
compiled.env = MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
const MG_Util::ShaderTranspiler::CompileEnv& env = *compiled.env;
// P0b layer 2: another shader object in this context may already have run the
// source-only half over byte-identical text under the same environment.
ShaderPreprocessResultPtr cached =
m_preprocessCache ? m_preprocessCache->Find(m_stage, sourceHash, m_source, env.fingerprint) : nullptr;
SharedPtr<ShaderPreprocessResult> fresh;
if (!cached) fresh = MakeShared<ShaderPreprocessResult>(RunSourceOnlyPipeline(m_stage, m_source, env));
const ShaderPreprocessResult& shared = cached ? *cached : *fresh;
const Bool shouldPopulateCache = !cached && m_preprocessCache != nullptr;
if (!shared.Preprocessed()) {
// Rejected lexically, or a glslang failure this context has already seen for
// this exact source (ParseFailed) - either way the parse can be skipped.
compiled.infoLog = shared.infoLog;
if (shouldPopulateCache) {
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
RememberCompiledSource(sourceHash);
// Two reasons to stay on this thread, one rule. Without the async flag the whole
// path must be byte-identical to the synchronous implementation, and a cache-less
// object is an internal shader that compiles and reads its status in the same
// breath (see the constructor comment) - a job would only add a round trip.
// AsyncShaderCompileActive(), not ...Enabled(): a glMaxShaderCompilerThreadsKHR(0)
// has to put compilation back on this thread even though the extension is still
// advertised, and that is exactly what makes the GL_COMPLETION_STATUS_KHR the
// extension mandates after a zero count (immediately GL_TRUE) fall out for free.
if (!m_preprocessCache || !MG_Util::Async::AsyncShaderCompileActive()) {
m_compiled->RunInline();
// Inline means the node is already terminal, so this join only replays
// diagnostics; it is here so the synchronous and asynchronous paths publish
// through the identical code.
EnsureCompileJoined();
return;
}
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = shared.preprocessedSource,
.flags = 0,
.env = &env};
auto result = ShaderCompiler::CompileShader(attrib);
if (result) {
compiled.compileStatus = true;
compiled.shader = result.value();
// Copy, not move: `shared` may alias a cache entry that has to outlive us, and
// `fresh` is about to be handed to the cache.
compiled.preprocessedSource = shared.preprocessedSource;
compiled.explicitUniformLocations = shared.explicitUniformLocations;
compiled.explicitOpaqueBindings = shared.explicitOpaqueBindings;
compiled.infoLog.clear();
if (shouldPopulateCache) {
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
} else {
compiled.infoLog = result.error().log;
MGLOG_D("ShaderObject::Compile: Shader %d compilation failed.\nSource:\n%s\nInfoLog:\n%s\nSetting "
"compileStatus = false as a result.",
m_externalIndex, shared.preprocessedSource.c_str(), compiled.infoLog.c_str());
if (shouldPopulateCache) {
fresh->outcome = ShaderPreprocessOutcome::ParseFailed;
fresh->infoLog = compiled.infoLog;
fresh->explicitUniformLocations.clear();
fresh->explicitOpaqueBindings.clear();
m_preprocessCache->Insert(m_stage, sourceHash, m_source, env.fingerprint, Move(fresh));
}
}
RememberCompiledSource(sourceHash);
}
SharedPtr<glslang::TShader> ShaderObject::TakeShaderForLink(String& outReparseLog) {
CompileArtifacts& compiled = Compiled();
if (compiled.shader && !compiled.shaderConsumedByLink) {
compiled.shaderConsumedByLink = true;
return compiled.shader;
}
// The stored parse already fed a link, whose mapIO mutated its intermediate.
// Re-parse the preprocessed source through the identical configuration; this
// costs one glslang parse, which is exactly what GenerateBinary used to spend
// here on EVERY link rather than only on reuse.
using namespace MG_Util::ShaderTranspiler;
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
.sourceStr = compiled.preprocessedSource,
.flags = 0,
// Re-parse against the SAME environment the original parse used,
// not against whatever the backend reports now.
.env = compiled.env.get()};
auto result = ShaderCompiler::CompileShader(attrib);
if (!result) {
// Should be unreachable: the same source parsed successfully at Compile().
outReparseLog = result.error().log;
MGLOG_E("ShaderObject::TakeShaderForLink: re-parse of shader %d failed:\n%s", m_externalIndex,
outReparseLog.c_str());
return nullptr;
}
return result.value();
MG_Util::Async::ShaderCompilePool::Get().Post(m_compiled);
}
void ShaderObject::MarkAsDeleted() {
@@ -8,56 +8,71 @@
#pragma once
#include <Includes.h>
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
#include <MG_State/GLState/ProgramState/ShaderStage.h>
#include <MG_State/GLState/ProgramState/ShaderCompileTask.h>
namespace MobileGL {
enum class ShaderStage {
Vertex,
TessControl,
TessEval,
Geometry,
Fragment,
Compute,
ShaderStageCount,
Unknown = -1
};
namespace MG_State::GLState {
// P0b layer 2. Declared, not included: the cache keys on ShaderStage, so including
// its header here would be circular.
class ShaderPreprocessCache;
// The GL-visible shader name. It owns the source text and one compile job node; the
// job node owns everything a compile produces.
//
// Every member below is GL-thread-owned, and every read of worker-produced state
// goes through Compiled(), which joins first. That is invariant I5 of the P1 design:
// because Compiled() is the SOLE accessor of the node's artifacts, the compiler
// enumerates every reader for us and none can be forgotten.
class ShaderObject {
public:
// `preprocessCache` is the owning context's cross-object memo (P0b layer 2);
// null is fully supported and simply means "no sharing" - that is what the
// context-less internal shader objects (the default FS, the blit pipeline) use.
// Shared ownership rather than a raw pointer: once compiles run on a worker the
// job outlives neither the object nor the context deterministically, and the
// cache has to stay alive for whoever is still reading it.
// `preprocessCache` is the owning context's cross-object memo (P0b layer 2).
// Null is fully supported and means two things at once: "no sharing", and
// "compile inline, never on a worker". Those coincide exactly - the only
// cache-less shader objects are the internal ones (ProgramObject's default
// fragment shader, the DirectVulkan blit and depth-mipmap shaders) and every one
// of them compiles and reads its status in the same breath, so a job would only
// add a round trip. Shared ownership rather than a raw pointer: a compile job
// outlives neither the object nor the context deterministically, and the cache
// has to stay alive for whoever is still reading it.
ShaderObject(const ShaderStage stage, Uint externalIndex,
SharedPtr<ShaderPreprocessCache> preprocessCache = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(Move(preprocessCache)) {}
// Cancel-not-join: the node owns its inputs, so an in-flight compile whose
// object just went away is safe to abandon where it stands. Nothing can observe
// its result any more - this object was the only route to it.
~ShaderObject() { CancelCompile(); }
ShaderObject(const ShaderObject&) = delete;
ShaderObject& operator=(const ShaderObject&) = delete;
void SetShaderSource(const String& source);
void SetShaderSource(String&& source);
void Compile();
// Drops a compile that is still in flight, without waiting for it. Called at the
// points where the object's compiled state stops being observable: a real source
// change, and the release of an orphaned shader name.
void CancelCompile();
void MarkAsDeleted();
// Hands out a link-consumable TShader. glslang's mapIO mutates the TShader's
// aliased intermediate, so the parse stored by Compile() may feed exactly one
// link; every later link (relink, or the same shader attached to a second
// program) gets a fresh parse of the stored preprocessed source through the
// byte-identical CompileShader path (including the legacy-460 retry). Only
// callable while GetCompileStatus() is true. Returns null only if that
// re-parse fails - outReparseLog then carries its diagnostics.
SharedPtr<glslang::TShader> TakeShaderForLink(String& outReparseLog);
// The compile job node itself, for ProgramObject::Link()'s input snapshot.
// DELIBERATELY DOES NOT JOIN, and that is the entire point of stage 4: the link
// takes the node as a dependency and is posted only once the node is terminal,
// so glLinkProgram never blocks on glCompileShader. Null means this object has
// never been compiled (or its last compile was abandoned), which the link reads
// as COMPILE_STATUS false - the same verdict the joining path produces.
//
// The caller must MarkLinkReferenced() whatever it keeps: from here on the node's
// result has an observer this object knows nothing about (see the marker's
// comment in ShaderCompileTask.h).
const SharedPtr<ShaderCompileTask>& CompiledNodeForLink() const { return m_compiled; }
Uint GetExternalIndex() const { return m_externalIndex; }
ShaderStage GetShaderStage() const { return m_stage; }
const String& GetShaderSource() const { return m_source; }
// No join: the source is GL-thread-owned, and a worker only ever reads the
// immutable snapshot it was handed at enqueue.
const String& GetShaderSource() const { return *m_source; }
// The snapshot itself, for whoever needs to hand it to a job.
const SharedPtr<const String>& GetShaderSourcePtr() const { return m_source; }
const SharedPtr<glslang::TShader>& GetCompiledShader() const { return Compiled().shader; }
const String& GetInfoLog() const { return Compiled().infoLog; }
const UnorderedMap<String, Uint>& GetUniformLocations() const { return Compiled().uniforms; }
// Explicit layout(location = N) qualifiers on this shader's default-block
// uniforms, captured lexically at Compile() because the relaxed parse drops
// them from reflection (see ExtractExplicitUniformLocations).
@@ -73,96 +88,98 @@ namespace MobileGL {
Bool GetCompileStatus() const { return Compiled().compileStatus; }
Bool GetDeleteStatus() const { return m_deleteStatus; }
// Blocks until a pending compile (P1 stage 3 onwards) has published its
// artifacts. Public for the few sites that must join without reading anything.
// A no-op today - nothing is ever pending.
// Blocks until a pending compile has published its artifacts. Public for the
// sites that must join without reading anything - ProgramObject::Link's
// prologue, which needs every attached shader settled before it runs.
void JoinCompile() const { EnsureCompileJoined(); }
// True while this object holds the outcome (success OR failure) of a previous
// Compile() of exactly the source it currently holds - i.e. while the P0b
// layer-1 memo is armed and a glCompileShader would be a no-op. Diagnostics
// and tests only; nothing in the GL frontend branches on it.
// True while this object holds the outcome (success OR failure) of a Compile()
// of exactly the source it currently holds - i.e. while the P0b layer-1 memo is
// armed and a glCompileShader would be a no-op. Diagnostics and tests only;
// nothing in the GL frontend branches on it.
//
// Deliberately does NOT join: the memo bookkeeping below is GL-thread-owned and
// says nothing about whether a worker has finished, which is exactly the
// property GL_COMPLETION_STATUS_KHR needs when stage 3 lands.
Bool HasMemoizedCompile() const { return m_hasCompiledState; }
// Tri-state, and deliberately NOT joining: an in-flight compile of the current
// source counts as memoized (a second glCompileShader must not enqueue a
// duplicate job), but asking that question must never block.
// A node that settled as Cancelled (the job body threw, or the enqueue failed)
// carries no result, so it must NOT satisfy the memo: otherwise a second
// glCompileShader on the same source enqueues nothing and the eventual join
// reports GL_FALSE forever. The synchronous path retries in exactly this case.
Bool HasMemoizedCompile() const {
return m_compiled != nullptr && m_compiled->source == m_source && !m_compiled->IsCancelled();
}
// MUST NOT JOIN - this is what GL_COMPLETION_STATUS_KHR will read when the
// extension surface lands. "No job at all" counts as complete: there is nothing
// outstanding to wait for.
Bool IsCompileComplete() const { return m_compiled == nullptr || m_compiled->IsTerminal(); }
private:
// ---- P1: everything a compile PRODUCES, in one block ----
//
// Same rule as ProgramObject::LinkArtifacts: this is exactly what
// InvalidateCompiledState() clears, i.e. exactly what one run of Compile()
// writes. Stage 3 lifts this struct wholesale into ShaderCompileTask, where a
// worker fills it in and the GL thread reads it through the same gate.
struct CompileArtifacts {
// The CompileEnv snapshot this compile ran against. Held so the
// consume-once re-parse in TakeShaderForLink() reproduces the original
// parse exactly, instead of re-reading whatever the backend says now.
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
SharedPtr<glslang::TShader> shader;
// The source Compile() actually parsed (after PreprocessShaderSource), kept
// for TakeShaderForLink's re-parse so a later link never depends on the
// preprocessor being deterministic across backend-state changes.
String preprocessedSource;
UnorderedMap<String, Uint> uniforms;
UnorderedMap<String, Int> explicitUniformLocations;
UnorderedMap<String, Uint> explicitOpaqueBindings;
Bool shaderConsumedByLink = false;
String infoLog;
Bool compileStatus = false;
};
// ---- The one and only join gate for compile output (P1 invariant I5) ----
// Blocks until a pending compile has published into m_compiled. Today nothing
// is ever pending - glCompileShader still runs the whole body inline - so this
// is an unconditional no-op. It exists NOW so that every reader of compile
// output is already routed through it when stage 3 makes it block.
// The fast path - no job, or a job whose result this object has already pulled -
// is two predictable branches and stays inline: it runs on every Compiled() read
// and the project never builds with LTO, so an out-of-line body would be a real
// cross-TU call at each of those sites. The blocking half is out of line.
//
// Defined inline (not in ShaderObject.cpp): called from every Compiled() read,
// and the project never builds with LTO, so an out-of-line empty body would be
// a real cross-TU call at each of those call sites instead of folding away.
void EnsureCompileJoined() const {}
CompileArtifacts& Compiled() {
EnsureCompileJoined();
return m_compiled;
// The gate keys on "has this object pulled the job's result yet", NOT on "is the
// job terminal". Those differ in the case that matters: a worker can finish a
// compile before the GL thread ever looks at it, and the pull is where deferred
// diagnostics get replayed and an abandoned node gets dropped. Keying on
// terminality would silently skip both.
void EnsureCompileJoined() const {
if (m_compiled && !m_compileJoined) JoinPendingCompile();
}
const CompileArtifacts& Compiled() const {
void JoinPendingCompile() const;
// The artifacts of a compile that ran to completion. A node that was abandoned
// (cancelled at teardown, or whose body threw) never publishes: JoinPendingCompile
// drops it, so anything reachable here is either Complete or absent, and "absent"
// reads as the never-compiled defaults - COMPILE_STATUS false, empty info log,
// which is exactly what GL requires before the first glCompileShader.
static const ShaderCompileArtifacts& EmptyArtifacts() {
static const ShaderCompileArtifacts empty;
return empty;
}
const ShaderCompileArtifacts& Compiled() const {
EnsureCompileJoined();
return m_compiled;
return m_compiled ? m_compiled->artifacts : EmptyArtifacts();
}
void InvalidateCompiledState();
// ---- P0b layer 1: per-object no-op recompile ----
// True iff `candidate` is byte-identical to the source that produced the
// compiled state this object is currently holding. The stored hash and length
// are only a fast reject; the answer is always confirmed against the full
// stored text, so no behaviour rides on a 64-bit hash.
// True iff `candidate` is byte-identical to the source that produced (or is
// producing) the compiled state this object currently holds.
Bool SourceMatchesCompiledState(const String& candidate) const;
// Arms the layer-1 memo for the source that Compile() just processed.
void RememberCompiledSource(Uint64 sourceHash);
// ---- GL-thread-owned state: never produced by a compile, so it never joins ----
const Uint m_externalIndex = 0;
const ShaderStage m_stage;
// glShaderSource text. A worker only ever reads the snapshot handed to it, so
// GL_SHADER_SOURCE_LENGTH and glGetShaderSource never join.
String m_source;
const Uint m_externalIndex = 0;
// The pre-glShaderSource state, shared by every untouched object rather than
// allocated per glCreateShader.
static const SharedPtr<const String>& EmptySource() {
static const SharedPtr<const String> empty = MakeShared<const String>();
return empty;
}
// glShaderSource text, as an immutable snapshot. Never null. A job holds its own
// SharedPtr to the exact string it was given, so replacing the source under a
// running compile cannot race its storage - and the layer-1 memo collapses to a
// pointer comparison against the job's snapshot, because the setter only swaps
// the pointer when the text genuinely differs.
SharedPtr<const String> m_source = EmptySource();
// P0b layer 2: the owning context's cross-object memo, or null.
// P0b layer 2: the owning context's cross-object memo, or null. Internally
// locked, because several workers hit it at once.
const SharedPtr<ShaderPreprocessCache> m_preprocessCache;
// P0b layer 1. m_hasCompiledState is the invariant "m_source is byte-identical
// to the source that produced the compile artifacts"; it is armed at the end of
// every Compile() and disarmed by InvalidateCompiledState(). Stage 3 replaces
// all three with a pointer compare against the in-flight job's source snapshot.
Bool m_hasCompiledState = false;
Uint64 m_compiledSourceHash = 0;
SizeT m_compiledSourceLength = 0;
Bool m_deleteStatus = false;
// ---- Compile OUTPUT ---- reachable only through Compiled().
CompileArtifacts m_compiled;
// ---- Compile OUTPUT ---- pending OR completed; reachable only through Compiled().
// Mutable because the join is a read-side operation: a const getter has to be
// able to settle an outstanding job before answering.
mutable SharedPtr<ShaderCompileTask> m_compiled;
// Exactly-once latch for the pull above. Armed with every new job node, set by
// the one join that consumes it.
mutable Bool m_compileJoined = false;
};
} // namespace MG_State::GLState
} // namespace MobileGL
@@ -10,7 +10,9 @@
#include <Includes.h>
#include <list>
#include <mutex>
#include <MG_State/GLState/ProgramState/ShaderObject.h>
// Deliberately NOT ShaderObject.h: ShaderCompileTask.h needs this header, and ShaderObject.h
// needs ShaderCompileTask.h. Only ShaderStage was ever used from there.
#include <MG_State/GLState/ProgramState/ShaderStage.h>
namespace MobileGL::MG_State::GLState {
// Where the shared, source-only half of ShaderObject::Compile() stopped. The two
@@ -0,0 +1,25 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderStage.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
namespace MobileGL {
// Split out of ShaderObject.h so the compile pipeline's headers form a DAG:
// ShaderStage.h <- ShaderPreprocessCache.h <- ShaderCompileTask.h <- ShaderObject.h.
// Every existing includer of ShaderObject.h still sees this type unchanged.
enum class ShaderStage {
Vertex,
TessControl,
TessEval,
Geometry,
Fragment,
Compute,
ShaderStageCount,
Unknown = -1
};
} // namespace MobileGL
@@ -0,0 +1,571 @@
// MobileGL - MobileGL/MG_Test/Program/AsyncCompileTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 3: glCompileShader enqueues, and every observable read joins.
//
// Every test here drives the real GL entry points and flips
// MG_Config::Features.AsyncShaderCompile itself rather than reading the environment. That is
// what lets one binary assert the property that actually matters - the async path and the
// synchronous path are indistinguishable through the GL surface - and it makes the file
// behave identically whether or not the suite was launched with
// MOBILEGL_ASYNC_SHADER_COMPILE=1.
#include <gtest/gtest.h>
#include <chrono>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
// Restores whatever the environment asked for when the test ends, so a case that forces
// one mode cannot leak into the next.
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async)
: m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform mat4 uModel;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = uModel * vec4(aPos, 1.0);
}
)";
const char* kFs = R"(#version 460
in vec4 vColor;
layout(location = 0) out vec4 fragColor;
uniform float uAlpha;
void main() { fragColor = vec4(vColor.rgb, vColor.a * uAlpha); }
)";
// Fails in glslang, not in the lexical pre-checks: that routes through the same
// ParseFailed path a real broken shaderpack source takes.
const char* kBrokenFs = R"(#version 460
layout(location = 0) out vec4 fragColor;
void main() { fragColor = thisIdentifierWasNeverDeclared; }
)";
// Rejected by the lexical reserved-identifier scan, before glslang is ever reached - the
// other half of the "compile failed" surface, and the one that never allocates a parse.
const char* kReservedIdentifierFs = R"(#version 460
layout(location = 0) out vec4 fragColor;
float gl_NotAllowedToDeclareThis = 1.0;
void main() { fragColor = vec4(gl_NotAllowedToDeclareThis); }
)";
// Big enough that a compile is not instantaneous, so the pool actually has a backlog to
// observe. Templated on an index so every instance is a distinct source (no P0b hit).
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 220; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
return shader;
}
GLint QueryCompileStatus(const GLuint shader) {
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
String QueryShaderInfoLog(const GLuint shader) {
GLint length = 0;
GetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return String();
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetShaderInfoLog(shader, length, &written, buffer.data());
return String(buffer.data(), static_cast<size_t>(written));
}
GLint QueryLinkStatus(const GLuint program) {
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
return status;
}
// The non-joining view of the object, i.e. what GL_COMPLETION_STATUS_KHR will report.
Bool CompileIsSettled(const GLuint shader) {
const auto& object = MG_State::pGLContext->GetShaderObject(shader);
return object == nullptr || object->IsCompileComplete();
}
Bool HasMemoizedCompile(const GLuint shader) {
const auto& object = MG_State::pGLContext->GetShaderObject(shader);
return object != nullptr && object->HasMemoizedCompile();
}
// Enqueues `count` distinct heavy compiles and returns their names WITHOUT reading
// anything back, so the pool is left with a real backlog for the caller to race against.
Vector<GLuint> SaturatePool(const int count, Vector<String>& sourceStorage) {
Vector<GLuint> shaders;
shaders.reserve(static_cast<SizeT>(count));
sourceStorage.reserve(sourceStorage.size() + static_cast<SizeT>(count));
for (int i = 0; i < count; ++i) {
sourceStorage.push_back(MakeBulkySource(1000 + i));
const char* text = sourceStorage.back().c_str();
const GLuint shader = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(shader, 1, &text, nullptr);
CompileShader(shader);
shaders.push_back(shader);
}
return shaders;
}
class AsyncCompileTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
};
} // namespace
// ---------------------------------------------------------------------------------------
// Correctness through the full GL surface
// ---------------------------------------------------------------------------------------
// N shaders compiled with the flag on: every status, every info log and every link has to
// come out the same as the synchronous path produces.
TEST_F(AsyncCompileTest, ManyShadersCompileAndLinkCorrectlyWithAsyncOn) {
const AsyncModeScope async(true);
ASSERT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
constexpr int kCount = 24;
Vector<GLuint> vertexShaders;
Vector<GLuint> fragmentShaders;
Vector<String> sources;
sources.reserve(kCount);
// Enqueue everything first, read nothing: this is the shape a shaderpack load has, and
// the only shape where the pool has more than one job in flight at a time.
for (int i = 0; i < kCount; ++i) {
vertexShaders.push_back(MakeShader(GL_VERTEX_SHADER, kVs));
sources.push_back(MakeBulkySource(i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
fragmentShaders.push_back(fs);
CompileShader(vertexShaders.back());
}
for (int i = 0; i < kCount; ++i) {
EXPECT_EQ(QueryCompileStatus(vertexShaders[i]), GL_TRUE) << QueryShaderInfoLog(vertexShaders[i]);
EXPECT_EQ(QueryCompileStatus(fragmentShaders[i]), GL_TRUE) << QueryShaderInfoLog(fragmentShaders[i]);
EXPECT_TRUE(QueryShaderInfoLog(vertexShaders[i]).empty());
EXPECT_TRUE(QueryShaderInfoLog(fragmentShaders[i]).empty());
}
// And the artifacts are actually usable: link, and reflect a uniform out of each stage.
for (int i = 0; i < kCount; ++i) {
const GLuint program = CreateProgram();
AttachShader(program, vertexShaders[i]);
AttachShader(program, fragmentShaders[i]);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "program " << i;
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
EXPECT_GE(GetUniformLocation(program, ("uSeed" + std::to_string(i)).c_str()), 0);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glCompileShader must return before the work is done. Timing-based assertions flake, so
// this observes the state machine instead: with a saturated pool at least one of the just
// -enqueued shaders has to be unsettled at the moment we ask. Skipped rather than failed if
// the machine drained the whole batch first - it can then never be a false red.
TEST_F(AsyncCompileTest, CompileShaderReturnsBeforeTheWorkIsDone) {
const AsyncModeScope async(true);
Vector<String> sources;
const Vector<GLuint> shaders = SaturatePool(64, sources);
int unsettled = 0;
for (const GLuint shader : shaders) {
if (!CompileIsSettled(shader)) ++unsettled;
}
if (unsettled == 0) {
GTEST_SKIP() << "the pool drained 64 compiles before the first observation; nothing to prove here";
}
// Whatever was outstanding still has to produce the right answer once asked.
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE) << QueryShaderInfoLog(shader);
EXPECT_TRUE(CompileIsSettled(shader)) << "reading COMPILE_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The synchronous path must stay synchronous: with the flag off, a compile is finished by
// the time glCompileShader returns. This is the guard that keeps the default shippable.
TEST_F(AsyncCompileTest, CompileIsFullySynchronousWithAsyncOff) {
const AsyncModeScope async(false);
ASSERT_FALSE(MG_Util::Async::AsyncShaderCompileEnabled());
Vector<String> sources;
const Vector<GLuint> shaders = SaturatePool(8, sources);
for (const GLuint shader : shaders) {
EXPECT_TRUE(CompileIsSettled(shader));
EXPECT_TRUE(HasMemoizedCompile(shader));
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE) << QueryShaderInfoLog(shader);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Diagnostics: the failing paths must read identically in both modes
// ---------------------------------------------------------------------------------------
// A compile failure is reported through COMPILE_STATUS and the info log, never through
// glGetError - that is exactly why moving the work off-thread is legal. Both failure
// classes are covered: the glslang parse failure and the lexical reserved-identifier
// rejection (which never reaches glslang at all).
TEST_F(AsyncCompileTest, FailingCompileLogIsByteIdenticalAcrossModes) {
for (const char* source : {kBrokenFs, kReservedIdentifierFs}) {
String syncLog;
{
const AsyncModeScope async(false);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, source);
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_FALSE);
syncLog = QueryShaderInfoLog(fs);
EXPECT_FALSE(syncLog.empty());
// GL defines compile FAILURE as a status plus a log, not as a GL error.
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
{
const AsyncModeScope async(true);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, source);
CompileShader(fs);
EXPECT_EQ(QueryCompileStatus(fs), GL_FALSE);
EXPECT_EQ(QueryShaderInfoLog(fs), syncLog);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
}
// A link whose vertex shader failed to compile has to reproduce that shader's log verbatim
// inside the program info log, whichever thread produced it.
TEST_F(AsyncCompileTest, LinkDiagnosticsQuoteTheAsyncCompileLog) {
const AsyncModeScope async(true);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
CompileShader(vs);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
// No status read between the enqueue and the link: the link's own prologue is what has
// to join the two compiles.
LinkProgram(program);
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE);
GLint length = 0;
GetProgramiv(program, GL_INFO_LOG_LENGTH, &length);
ASSERT_GT(length, 1);
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetProgramInfoLog(program, length, &written, buffer.data());
const String programLog(buffer.data(), static_cast<size_t>(written));
const String shaderLog = QueryShaderInfoLog(fs);
ASSERT_FALSE(shaderLog.empty());
EXPECT_NE(programLog.find(shaderLog), String::npos)
<< "program log:\n" << programLog << "\nshader log:\n" << shaderLog;
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Mutation over an in-flight compile
// ---------------------------------------------------------------------------------------
// glShaderSource with DIFFERENT text over a pending compile: the running job is abandoned
// and the next compile reflects the new source. The re-source happens with the pool
// saturated, so the job it replaces is very likely still queued or running.
TEST_F(AsyncCompileTest, ShaderSourceOverAPendingCompileCancelsAndTheNewSourceWins) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String firstSource = MakeBulkySource(7001);
const char* firstText = firstSource.c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &firstText, nullptr);
CompileShader(fs);
// Replace the text while that compile is (very probably) still outstanding. This must
// not wait, must not corrupt the abandoned job's view of the old string, and must
// disarm the layer-1 memo.
const String secondSource = MakeBulkySource(7002);
const char* secondText = secondSource.c_str();
ShaderSource(fs, 1, &secondText, nullptr);
EXPECT_FALSE(HasMemoizedCompile(fs)) << "a real source change must invalidate the compiled state";
EXPECT_EQ(QueryCompileStatus(fs), GL_FALSE) << "the replaced compile must not publish";
CompileShader(fs);
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE);
// The SECOND source's uniform is the one that exists.
EXPECT_GE(GetUniformLocation(program, "uSeed7002"), 0);
EXPECT_EQ(GetUniformLocation(program, "uSeed7001"), -1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glShaderSource with byte-identical text over a pending compile is a no-op: the job stays,
// the memo stays armed, and the result is still the right one.
TEST_F(AsyncCompileTest, IdenticalShaderSourceOverAPendingCompileKeepsTheJob) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(7100);
const char* text = source.c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
ShaderSource(fs, 1, &text, nullptr);
EXPECT_TRUE(HasMemoizedCompile(fs)) << "identical re-source must not disturb an in-flight compile";
EXPECT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// A second glCompileShader on a pending object must be a no-op, not a duplicate job racing
// the first one to write the same fields. Observed through the object identity of the node:
// HasMemoizedCompile stays true across the second call, and the result is still correct.
TEST_F(AsyncCompileTest, RepeatedCompileShaderOnAPendingObjectEnqueuesOneJob) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(7200);
const char* text = source.c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
// A copy, not the slot reference: creating another shader can reallocate the table.
const SharedPtr<MG_State::GLState::ShaderObject> object = MG_State::pGLContext->GetShaderObject(fs);
ASSERT_NE(object, nullptr);
EXPECT_FALSE(object->HasMemoizedCompile());
CompileShader(fs);
EXPECT_TRUE(object->HasMemoizedCompile());
for (int i = 0; i < 8; ++i) {
CompileShader(fs);
EXPECT_TRUE(object->HasMemoizedCompile());
}
EXPECT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glDeleteShader on an unattached object with a compile still in flight. The name goes away
// immediately - no wait for a worker - and the abandoned job must neither crash nor keep the
// object alive in a way anything can observe.
TEST_F(AsyncCompileTest, DeleteShaderWhileACompileIsPending) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
Vector<GLuint> doomed;
Vector<String> sources;
for (int i = 0; i < 16; ++i) {
sources.push_back(MakeBulkySource(7300 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
doomed.push_back(fs);
}
for (const GLuint fs : doomed) {
DeleteShader(fs);
EXPECT_EQ(IsShader(fs), GL_FALSE) << "an unattached deleted shader's name goes immediately";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
// The context still works afterwards - the abandoned jobs did not take the pool, the
// preprocess cache or the glslang process state down with them.
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
CompileShader(vs);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glDeleteShader on a shader still ATTACHED to a program only flags it: the pending compile
// has to survive, because the link that follows still needs its artifacts.
TEST_F(AsyncCompileTest, DeleteShaderWhileAttachedKeepsThePendingCompileAlive) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
CompileShader(vs);
CompileShader(fs);
DeleteShader(vs);
DeleteShader(fs);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
EXPECT_GE(GetUniformLocation(program, "uAlpha"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Stress
// ---------------------------------------------------------------------------------------
// The adversarial interleaving: enqueue, query, re-source, re-enqueue, delete, all with the
// pool busy. Nothing here asserts timing - what it hunts for is a missed join or a use of an
// abandoned node, both of which surface as a wrong status, a wrong log, or a crash.
TEST_F(AsyncCompileTest, StressCompileQueryResourceDeleteInterleaved) {
const AsyncModeScope async(true);
constexpr int kRounds = 6;
constexpr int kPerRound = 12;
for (int round = 0; round < kRounds; ++round) {
Vector<String> sources;
Vector<GLuint> shaders;
sources.reserve(kPerRound * 2);
for (int i = 0; i < kPerRound; ++i) {
sources.push_back(MakeBulkySource(round * 1000 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
// Immediately query a PREVIOUS one while this one is still outstanding: the
// join has to settle exactly the object asked about and no other.
if (i > 0) {
const GLuint earlier = shaders[static_cast<SizeT>(i - 1)];
EXPECT_EQ(QueryCompileStatus(earlier), GL_TRUE) << QueryShaderInfoLog(earlier);
}
}
// Re-source half of them mid-flight, then recompile.
for (int i = 0; i < kPerRound; i += 2) {
sources.push_back(MakeBulkySource(round * 1000 + 500 + i));
const char* text = sources.back().c_str();
ShaderSource(shaders[static_cast<SizeT>(i)], 1, &text, nullptr);
CompileShader(shaders[static_cast<SizeT>(i)]);
}
for (int i = 0; i < kPerRound; ++i) {
const GLuint shader = shaders[static_cast<SizeT>(i)];
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE) << QueryShaderInfoLog(shader);
const String expectedUniform =
"uSeed" + std::to_string(round * 1000 + (i % 2 == 0 ? 500 + i : i));
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, shader);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "round " << round << " shader " << i;
EXPECT_GE(GetUniformLocation(program, expectedUniform.c_str()), 0)
<< "round " << round << " shader " << i << " expected " << expectedUniform;
DeleteProgram(program);
DeleteShader(vs);
}
for (const GLuint shader : shaders) {
DeleteShader(shader);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// The P0b cross-object memo is hit from several workers at once here: 8 objects share each
// of 6 distinct sources, all enqueued before anything is read. Every object must still end
// up with its own parse and its own correct reflection - a torn cache entry or an entry
// evicted from under a reader shows up as a link failure or a missing uniform.
TEST_F(AsyncCompileTest, ConcurrentCompilesShareThePreprocessCacheSafely) {
const AsyncModeScope async(true);
constexpr int kDistinct = 6;
constexpr int kDuplicates = 8;
Vector<String> sources;
sources.reserve(kDistinct);
for (int i = 0; i < kDistinct; ++i) {
sources.push_back(MakeBulkySource(8100 + i));
}
Vector<GLuint> shaders;
for (int duplicate = 0; duplicate < kDuplicates; ++duplicate) {
for (int i = 0; i < kDistinct; ++i) {
const char* text = sources[static_cast<SizeT>(i)].c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
}
for (SizeT s = 0; s < shaders.size(); ++s) {
const GLuint fs = shaders[s];
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "shader index " << s;
const String uniform = "uSeed" + std::to_string(8100 + static_cast<int>(s % kDistinct));
EXPECT_GE(GetUniformLocation(program, uniform.c_str()), 0) << uniform;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
+749
View File
@@ -0,0 +1,749 @@
// MobileGL - MobileGL/MG_Test/Program/AsyncLinkTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 4: glLinkProgram enqueues a ProgramLinkTask behind its shaders' compiles, and
// every observable read of link output joins.
//
// Like AsyncCompileTest, every case here drives the real GL entry points and flips
// MG_Config::Features.AsyncShaderCompile itself rather than reading the environment - so one
// binary can assert the property that actually matters (the async and synchronous paths are
// indistinguishable through the GL surface) regardless of how the suite was launched.
#include <gtest/gtest.h>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_Impl/GLImpl/Program/GL_ProgramPipeline.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async) : m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform mat4 uModel;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = uModel * vec4(aPos, 1.0);
}
)";
const char* kFs = R"(#version 460
in vec4 vColor;
layout(location = 0) out vec4 fragColor;
uniform float uAlpha;
void main() { fragColor = vec4(vColor.rgb, vColor.a * uAlpha); }
)";
// A vertex shader that captures something transform feedback can name.
const char* kXfbVs = R"(#version 460
layout(location = 0) in vec3 aPos;
out vec3 vWorld;
void main() {
vWorld = aPos * 2.0;
gl_Position = vec4(aPos, 1.0);
}
)";
const char* kBrokenFs = R"(#version 460
layout(location = 0) out vec4 fragColor;
void main() { fragColor = thisIdentifierWasNeverDeclared; }
)";
// Big enough that neither the compile nor the link is instantaneous, so the pool has a
// real backlog to race against. Templated on an index so every instance is distinct
// source text (no P0b memo hit).
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 220; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
CompileShader(shader);
return shader;
}
GLint QueryLinkStatus(const GLuint program) {
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
return status;
}
String QueryProgramInfoLog(const GLuint program) {
GLint length = 0;
GetProgramiv(program, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return String();
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetProgramInfoLog(program, length, &written, buffer.data());
return String(buffer.data(), static_cast<size_t>(written));
}
// The non-joining view of the program, i.e. what GL_COMPLETION_STATUS_KHR will report.
Bool LinkIsSettled(const GLuint program) {
const auto& object = MG_State::pGLContext->GetProgramObject(program);
return object == nullptr || object->IsLinkComplete();
}
// Enqueues `count` distinct heavy compiles without reading anything back, so the pool is
// left with a real backlog for the caller to race against.
Vector<GLuint> SaturatePool(const int count, Vector<String>& sourceStorage) {
Vector<GLuint> shaders;
shaders.reserve(static_cast<SizeT>(count));
sourceStorage.reserve(sourceStorage.size() + static_cast<SizeT>(count));
for (int i = 0; i < count; ++i) {
sourceStorage.push_back(MakeBulkySource(20000 + i));
const char* text = sourceStorage.back().c_str();
const GLuint shader = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(shader, 1, &text, nullptr);
CompileShader(shader);
shaders.push_back(shader);
}
return shaders;
}
// Content hash of a linked program's generated SPIR-V, through the state layer (there is
// no GL query for it). Joins, like every other artifact read.
Vector<Uint64> SpirvDigest(const GLuint program) {
const auto& object = MG_State::pGLContext->GetProgramObject(program);
Vector<Uint64> digest;
if (!object) return digest;
for (const auto& module : object->GetGeneratedSpirv()) {
Uint64 hash = 1469598103934665603ull;
for (const unsigned word : module) {
hash = (hash ^ static_cast<Uint64>(word)) * 1099511628211ull;
}
digest.push_back(hash);
}
return digest;
}
class AsyncLinkTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
};
} // namespace
// ---------------------------------------------------------------------------------------
// The consume-once claim
// ---------------------------------------------------------------------------------------
// The stage-4 headline risk: two programs share one shader and are linked back to back, so
// two ProgramLinkTasks race for that shader's single glslang parse. Exactly one may win the
// claim; the loser must re-parse the same preprocessed source against the same CompileEnv.
// If either half of that is wrong the two programs get DIFFERENT SPIR-V for the same shader,
// which is the silent-corruption class this whole mechanism exists to prevent.
TEST_F(AsyncLinkTest, TwoProgramsSharingAShaderGenerateIdenticalSpirv) {
for (const Bool async : {false, true}) {
const AsyncModeScope scope(async);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
// Both links enqueued before either result is read: with the flag on this is the
// window in which two workers can hold the same node at once.
const GLuint programA = CreateProgram();
AttachShader(programA, vs);
AttachShader(programA, fs);
LinkProgram(programA);
const GLuint programB = CreateProgram();
AttachShader(programB, vs);
AttachShader(programB, fs);
LinkProgram(programB);
ASSERT_EQ(QueryLinkStatus(programA), GL_TRUE) << QueryProgramInfoLog(programA);
ASSERT_EQ(QueryLinkStatus(programB), GL_TRUE) << QueryProgramInfoLog(programB);
const Vector<Uint64> digestA = SpirvDigest(programA);
const Vector<Uint64> digestB = SpirvDigest(programB);
ASSERT_EQ(digestA.size(), 2u) << "async=" << async;
EXPECT_EQ(digestA, digestB)
<< "the claim winner and the re-parsing loser must produce identical SPIR-V (async=" << async << ")";
// And the two programs really are usable independently.
EXPECT_GE(GetUniformLocation(programA, "uColor"), 0);
EXPECT_GE(GetUniformLocation(programB, "uColor"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// The same property many ways at once, with the pool loaded: N programs over the SAME shader
// pair, all enqueued before anything is read, so one claim winner is racing N-1 re-parsers.
// Every program must come out byte-identical.
//
// The shader pair has to be identical across the programs for this to mean anything: glslang
// links the stages together, so a stage's SPIR-V is legitimately a function of the WHOLE
// program (mapIO's cross-stage location assignment, live-variable analysis). Comparing one
// shared vertex shader across programs with different fragment stages would compare things
// that are allowed to differ.
TEST_F(AsyncLinkTest, ManyProgramsSharingOneShaderPairAgreeOnTheirSpirv) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
constexpr int kPrograms = 12;
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
Vector<GLuint> programs;
for (int i = 0; i < kPrograms; ++i) {
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
programs.push_back(program);
}
Vector<Uint64> reference;
for (int i = 0; i < kPrograms; ++i) {
const GLuint program = programs[static_cast<SizeT>(i)];
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "program " << i << ": " << QueryProgramInfoLog(program);
const Vector<Uint64> digest = SpirvDigest(program);
ASSERT_EQ(digest.size(), 2u);
if (i == 0) {
reference = digest;
} else {
EXPECT_EQ(digest, reference) << "SPIR-V differs in program " << i;
}
EXPECT_GE(GetUniformLocation(program, "uColor"), 0) << "program " << i;
EXPECT_GE(GetUniformLocation(program, "uAlpha"), 0) << "program " << i;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Mutation over a pending link (the cancel matrix)
// ---------------------------------------------------------------------------------------
// The last link wins. A re-link over a pending one cancels it and enqueues afresh; the
// result the application eventually reads must be the SECOND link's.
TEST_F(AsyncLinkTest, RelinkOverAPendingLinkPublishesTheSecondLink) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const String firstSource = MakeBulkySource(7001);
const char* firstText = firstSource.c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &firstText, nullptr);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
// Swap the fragment shader's source and relink, all without ever reading the first
// link's status - so the first link is very probably still queued or running.
const String secondSource = MakeBulkySource(7002);
const char* secondText = secondSource.c_str();
ShaderSource(fs, 1, &secondText, nullptr);
CompileShader(fs);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_GE(GetUniformLocation(program, "uSeed7002"), 0);
EXPECT_EQ(GetUniformLocation(program, "uSeed7001"), -1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The take-effect-at-next-link setters must NOT disturb a pending link: the pending link
// snapshotted its own inputs at enqueue, so
// glLinkProgram; glTransformFeedbackVaryings; glGetProgramiv(LINK_STATUS)
// has to report the FIRST link - which captured nothing.
TEST_F(AsyncLinkTest, TransformFeedbackVaryingsOverAPendingLinkReportsTheFirstLink) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kXfbVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
const char* varyings[] = {"vWorld"};
TransformFeedbackVaryings(program, 1, varyings, GL_INTERLEAVED_ATTRIBS);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
GLint captured = -1;
GetProgramiv(program, GL_TRANSFORM_FEEDBACK_VARYINGS, &captured);
EXPECT_EQ(captured, 0) << "the pending link must publish the request set it snapshotted, not a later one";
// And the request does take effect at the NEXT link.
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
GetProgramiv(program, GL_TRANSFORM_FEEDBACK_VARYINGS, &captured);
EXPECT_EQ(captured, 1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glBindAttribLocation is the same family and must likewise leave a pending link alone.
TEST_F(AsyncLinkTest, BindAttribLocationOverAPendingLinkDoesNotDisturbIt) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
BindAttribLocation(program, 5, "aPos");
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_EQ(GetAttribLocation(program, "aPos"), 0) << "the first link's layout(location = 0) must survive";
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glAttachShader after glLinkProgram is defined to leave the current link status alone (it
// takes effect at the next link). It must therefore NOT cancel a pending link - the failure
// mode being guarded here is a program that linked fine reporting GL_FALSE.
TEST_F(AsyncLinkTest, AttachShaderOverAPendingLinkKeepsTheLinkResult) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
// A second, unrelated fragment shader attached over the pending link. (Attaching two
// shaders of one stage is legal; only the next link would have to reconcile them.)
const GLuint extraFs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
AttachShader(program, extraFs);
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_GE(GetUniformLocation(program, "uAlpha"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The link-then-detach-then-delete teardown every LWJGL/Blaze3D-shaped app performs. The
// detach makes the shader GL-invisible, so glDeleteShader frees its name and would otherwise
// cancel a compile the enqueued link is still waiting on - flipping a link that must report
// GL_TRUE to GL_FALSE. Runs with the pool saturated so the compiles really are outstanding.
TEST_F(AsyncLinkTest, DetachAndDeleteShadersOverAPendingLinkKeepsTheLinkResult) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(7400);
const char* text = source.c_str();
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
DetachShader(program, vs);
DetachShader(program, fs);
DeleteShader(vs);
DeleteShader(fs);
EXPECT_EQ(IsShader(vs), GL_FALSE);
EXPECT_EQ(IsShader(fs), GL_FALSE);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_GE(GetUniformLocation(program, "uSeed7400"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glCreateShaderProgramv is specified as create-source-compile-create-attach-LINK-detach, so
// it is the in-tree caller that exercises the detach-immediately-after-link ordering. It
// self-joins through its status queries (design join site J7) and needs no edit of its own -
// this is the guard that says so.
TEST_F(AsyncLinkTest, CreateShaderProgramvLinksUnderAsync) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const char* sources[] = {kVs};
const GLuint program = CreateShaderProgramv(GL_VERTEX_SHADER, 1, sources);
ASSERT_NE(program, 0u);
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glProgramBinary over a pending link: no format is supported, so the spec requires
// LINK_STATUS to read FALSE afterwards. The pending link must not publish over that.
TEST_F(AsyncLinkTest, ProgramBinaryOverAPendingLinkForcesLinkFalse) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
const GLuint dummy = 0;
ProgramBinary(program, 0, &dummy, static_cast<GLsizei>(sizeof(dummy)));
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE) << "glProgramBinary must win over the pending link";
EXPECT_FALSE(QueryProgramInfoLog(program).empty());
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glDeleteProgram over a pending link. The name goes away immediately - no wait for a worker
// - and the abandoned job must neither crash nor keep anything observable alive.
TEST_F(AsyncLinkTest, DeleteProgramWhileALinkIsPending) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
Vector<GLuint> doomed;
Vector<String> sources;
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
for (int i = 0; i < 16; ++i) {
sources.push_back(MakeBulkySource(7500 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
doomed.push_back(program);
}
for (const GLuint program : doomed) {
DeleteProgram(program);
EXPECT_EQ(IsProgram(program), GL_FALSE) << "an unused deleted program's name goes immediately";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
// The context still works afterwards: the abandoned links did not take the pool, the
// preprocess cache or the glslang process state down with them.
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// The join gates
// ---------------------------------------------------------------------------------------
// glLinkProgram must return before the work is done, and the first observable read must
// join. Observed through the state machine rather than through timing, so it can never be a
// false red: with a saturated pool at least one of the just-enqueued links has to be
// unsettled at the moment we ask; skipped if the machine drained everything first.
TEST_F(AsyncLinkTest, LinkProgramReturnsBeforeTheWorkIsDone) {
const AsyncModeScope async(true);
constexpr int kPrograms = 32;
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
Vector<GLuint> programs;
Vector<String> sources;
for (int i = 0; i < kPrograms; ++i) {
sources.push_back(MakeBulkySource(7600 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
programs.push_back(program);
}
int unsettled = 0;
for (const GLuint program : programs) {
if (!LinkIsSettled(program)) ++unsettled;
}
if (unsettled == 0) {
GTEST_SKIP() << "the pool drained every link before the first observation; nothing to prove here";
}
for (const GLuint program : programs) {
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_TRUE(LinkIsSettled(program)) << "reading LINK_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// With the flag off, a link is finished by the time glLinkProgram returns. This is the guard
// that keeps the default shippable.
TEST_F(AsyncLinkTest, LinkIsFullySynchronousWithAsyncOff) {
const AsyncModeScope async(false);
ASSERT_FALSE(MG_Util::Async::AsyncShaderCompileEnabled());
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
EXPECT_TRUE(LinkIsSettled(program));
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// P1 join site J1: the composite draw program for a pipeline is cached against a signature
// built from each stage program's lifetime id and backend state version - NON-artifact
// fields, which do not pass through the join gate. GetProgramForDraw has to settle the stage
// programs first, or the signature describes a link generation that no longer exists and the
// composite is rebuilt on every draw.
TEST_F(AsyncLinkTest, DrawThroughAPipelineWithAPendingStageProgramJoinsFirst) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
// Built by hand rather than through glCreateShaderProgramv: that entry point detaches the
// shader immediately after linking, so the next link would remove it and leave the stage
// program with nothing attached to composite from.
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint vsProgram = CreateProgram();
ProgramParameteri(vsProgram, GL_PROGRAM_SEPARABLE, GL_TRUE);
AttachShader(vsProgram, vs);
LinkProgram(vsProgram);
ASSERT_EQ(QueryLinkStatus(vsProgram), GL_TRUE) << QueryProgramInfoLog(vsProgram);
GLuint pipeline = 0;
GenProgramPipelines(1, &pipeline);
ASSERT_NE(pipeline, 0u);
// Bind before UseProgramStages: glGenProgramPipelines only reserves the name, and the
// first bind is what turns it into an object glUseProgramStages can find.
BindProgramPipeline(pipeline);
UseProgramStages(pipeline, GL_VERTEX_SHADER_BIT, vsProgram);
ASSERT_EQ(GetError(), GL_NO_ERROR);
// Re-link the stage program and immediately ask for the draw program, without reading
// the link's status in between: the pending link is what J1 has to settle.
LinkProgram(vsProgram);
const SharedPtr<MG_State::GLState::ProgramObject> drawProgram = MG_State::pGLContext->GetProgramForDraw();
ASSERT_NE(drawProgram, nullptr);
EXPECT_TRUE(LinkIsSettled(vsProgram)) << "GetProgramForDraw must have joined the stage program";
EXPECT_TRUE(drawProgram->GetLinkStatus()) << drawProgram->GetInfoLog();
// Asking again with nothing changed must hit the composite cache, which is only possible
// if the signature was computed against settled programs both times.
const SharedPtr<MG_State::GLState::ProgramObject> again = MG_State::pGLContext->GetProgramForDraw();
EXPECT_EQ(again.get(), drawProgram.get()) << "the composite draw program must be cached across draws";
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Diagnostics
// ---------------------------------------------------------------------------------------
// A link whose fragment shader failed to compile has to reproduce that shader's log verbatim
// inside the program info log, whichever thread produced it - and the failure must be
// reported as LINK_STATUS plus a log, never as a GL error.
TEST_F(AsyncLinkTest, FailingLinkLogIsIdenticalAcrossModes) {
String syncLog;
{
const AsyncModeScope scope(false);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
ASSERT_EQ(QueryLinkStatus(program), GL_FALSE);
syncLog = QueryProgramInfoLog(program);
EXPECT_FALSE(syncLog.empty());
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
{
const AsyncModeScope scope(true);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE);
EXPECT_EQ(QueryProgramInfoLog(program), syncLog);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// A program with nothing attached fails in the GL-thread prologue, before any job exists.
// That path has to reach the same info log in both modes.
TEST_F(AsyncLinkTest, LinkWithNoShadersFailsIdenticallyInBothModes) {
String syncLog;
for (const Bool async : {false, true}) {
const AsyncModeScope scope(async);
const GLuint program = CreateProgram();
LinkProgram(program);
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE);
const String log = QueryProgramInfoLog(program);
EXPECT_FALSE(log.empty());
if (!async) {
syncLog = log;
} else {
EXPECT_EQ(log, syncLog);
}
EXPECT_TRUE(LinkIsSettled(program)) << "a prologue failure leaves no job pending";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// End to end
// ---------------------------------------------------------------------------------------
// The shape a shaderpack load actually has: compile N shaders, link M programs, read
// NOTHING until the end, then query everything. This is the only shape in which the pool has
// many compiles and many links in flight simultaneously, with the link jobs chained behind
// compile jobs that are themselves still queued.
TEST_F(AsyncLinkTest, PackShapedBurstCompilesLinksAndQueriesEverything) {
const AsyncModeScope async(true);
constexpr int kShaders = 24;
constexpr int kPrograms = 24;
Vector<String> sources;
Vector<GLuint> vertexShaders;
Vector<GLuint> fragmentShaders;
for (int i = 0; i < kShaders; ++i) {
vertexShaders.push_back(MakeShader(GL_VERTEX_SHADER, kVs));
sources.push_back(MakeBulkySource(8000 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
fragmentShaders.push_back(fs);
}
Vector<GLuint> programs;
for (int i = 0; i < kPrograms; ++i) {
const GLuint program = CreateProgram();
AttachShader(program, vertexShaders[static_cast<SizeT>(i % kShaders)]);
AttachShader(program, fragmentShaders[static_cast<SizeT>(i % kShaders)]);
LinkProgram(program);
programs.push_back(program);
}
for (int i = 0; i < kPrograms; ++i) {
const GLuint program = programs[static_cast<SizeT>(i)];
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "program " << i << ": " << QueryProgramInfoLog(program);
EXPECT_GE(GetUniformLocation(program, "uColor"), 0) << "program " << i;
EXPECT_GE(GetUniformLocation(program, ("uSeed" + std::to_string(8000 + i % kShaders)).c_str()), 0)
<< "program " << i;
EXPECT_EQ(GetAttribLocation(program, "aPos"), 0) << "program " << i;
EXPECT_EQ(SpirvDigest(program).size(), 2u) << "program " << i;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The adversarial interleaving: link, query a previous one, re-source, re-link, delete, all
// with the pool busy. Nothing here asserts timing - what it hunts for is a missed join, a
// consumed-twice parse, or a use of an abandoned node, all of which surface as a wrong
// status, a missing uniform, or a crash.
TEST_F(AsyncLinkTest, StressLinkQueryRelinkDeleteInterleaved) {
const AsyncModeScope async(true);
constexpr int kRounds = 5;
constexpr int kPerRound = 10;
for (int round = 0; round < kRounds; ++round) {
Vector<String> sources;
Vector<GLuint> programs;
Vector<GLuint> fragmentShaders;
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
for (int i = 0; i < kPerRound; ++i) {
sources.push_back(MakeBulkySource(round * 1000 + 300 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
fragmentShaders.push_back(fs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
programs.push_back(program);
// Query a PREVIOUS program while this one is still outstanding: the join has to
// settle exactly the program asked about and no other.
if (i > 0) {
const GLuint earlier = programs[static_cast<SizeT>(i - 1)];
EXPECT_EQ(QueryLinkStatus(earlier), GL_TRUE) << QueryProgramInfoLog(earlier);
}
}
// Re-source half of them mid-flight and relink over the pending link.
for (int i = 0; i < kPerRound; i += 2) {
sources.push_back(MakeBulkySource(round * 1000 + 700 + i));
const char* text = sources.back().c_str();
ShaderSource(fragmentShaders[static_cast<SizeT>(i)], 1, &text, nullptr);
CompileShader(fragmentShaders[static_cast<SizeT>(i)]);
LinkProgram(programs[static_cast<SizeT>(i)]);
}
for (int i = 0; i < kPerRound; ++i) {
const GLuint program = programs[static_cast<SizeT>(i)];
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE)
<< "round " << round << " program " << i << ": " << QueryProgramInfoLog(program);
const String expected =
"uSeed" + std::to_string(round * 1000 + (i % 2 == 0 ? 700 + i : 300 + i));
EXPECT_GE(GetUniformLocation(program, expected.c_str()), 0)
<< "round " << round << " program " << i << " expected " << expected;
DeleteProgram(program);
}
for (const GLuint fs : fragmentShaders) DeleteShader(fs);
DeleteShader(vs);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
@@ -0,0 +1,133 @@
// MobileGL - MobileGL/MG_Test/Program/AsyncTeardownTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 4, item S6: MobileGL::Destroy() with compile AND link jobs still in flight.
//
// This is the one cancellation path in the whole design that WAITS, and the order it waits
// in is load-bearing: in-flight jobs own their own inputs and are safe against everything
// teardown does EXCEPT glslang's process globals and the TShader/TProgram objects hanging off
// pGLContext - both of which DestroyImpl is about to free. StopAndDrain() therefore runs
// first, before pGLContext.reset() and before glslang::FinalizeProcess().
//
// ITS OWN BINARY, deliberately. ShaderCompilePool::StopAndDrain() is a one-way latch: from
// the first eglTerminate onwards every job in the process runs inline on the calling thread.
// Sharing a binary with AsyncCompileTest/AsyncLinkTest would silently turn every case
// declared after this one synchronous, and they would keep passing while testing nothing.
#include <gtest/gtest.h>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = vec4(aPos, 1.0);
}
)";
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 220; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
CompileShader(shader);
return shader;
}
} // namespace
// Fills the pool with compiles, chains links behind them, and tears the library down without
// reading a single result. Nothing here can assert on the jobs' outcomes - by design there is
// no one left to ask - so what it asserts is that teardown COMPLETES: it must not hang
// (StopAndDrain joining a worker that is itself waiting on something), must not crash (a
// worker inside glslang while FinalizeProcess frees its symbol tables, or a link job reading
// a shader node the GL thread has dropped), and must leave the process able to come back up.
TEST(AsyncTeardown, DestroyWithCompilesAndLinksInFlight) {
// After Initialize(), not before: MG_ConfigLoader::Init() re-reads the whole feature
// block from the environment and would overwrite the override.
MobileGL::Initialize();
MG_Config::Features.AsyncShaderCompile = MG_Config::QuirkOverride::ForceOn;
ASSERT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
constexpr int kCount = 64;
Vector<String> sources;
Vector<GLuint> shaders;
Vector<GLuint> programs;
sources.reserve(kCount);
// Bare compiles first, so the pool has a backlog the links below will queue behind.
for (int i = 0; i < kCount; ++i) {
sources.push_back(MakeBulkySource(30000 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
// Then links, each chained behind a compile that is very probably still outstanding: at
// the moment Destroy() runs there are queued compiles, running compiles, links waiting on
// a dependency edge, and links already handed to the pool.
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
for (int i = 0; i < kCount; ++i) {
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, shaders[static_cast<SizeT>(i)]);
LinkProgram(program);
programs.push_back(program);
}
// No status read anywhere above - the jobs are genuinely in flight.
MobileGL::Destroy();
// Back up again. The pool stays stopped for the rest of the process (a one-way latch), so
// this second life is synchronous - which is exactly the documented behaviour, and it has
// to still be a WORKING one.
MobileGL::Initialize();
const GLuint vs2 = MakeShader(GL_VERTEX_SHADER, kVs);
const char* fsSource = R"(#version 460
in vec4 vColor;
layout(location = 0) out vec4 fragColor;
void main() { fragColor = vColor; }
)";
const GLuint fs2 = MakeShader(GL_FRAGMENT_SHADER, fsSource);
const GLuint program = CreateProgram();
AttachShader(program, vs2);
AttachShader(program, fs2);
LinkProgram(program);
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
EXPECT_EQ(status, GL_TRUE) << "the library must be usable after a teardown that drained jobs in flight";
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
+74
View File
@@ -28,6 +28,73 @@ add_executable(
ProgramTest.cpp
)
add_executable(
AsyncCompileTest
AsyncCompileTest.cpp
)
target_include_directories(AsyncCompileTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
AsyncCompileTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
add_executable(
AsyncLinkTest
AsyncLinkTest.cpp
)
target_include_directories(AsyncLinkTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
AsyncLinkTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
add_executable(
ParallelShaderCompileTest
ParallelShaderCompileTest.cpp
)
target_include_directories(ParallelShaderCompileTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
ParallelShaderCompileTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
# Its own binary on purpose: this one calls MobileGL::Destroy(), and ShaderCompilePool's
# stop is a one-way latch for the whole process - every case declared after it in the same
# binary would silently run its compiles and links inline.
add_executable(
AsyncTeardownTest
AsyncTeardownTest.cpp
)
target_include_directories(AsyncTeardownTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
AsyncTeardownTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
target_include_directories(ProgramTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
@@ -42,3 +109,10 @@ target_link_libraries(
include(GoogleTest)
gtest_discover_tests(ProgramUtilTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
gtest_discover_tests(ProgramTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
# Heavier than the rest of the unit suite by design: several cases deliberately saturate the
# compile pool so there is something in flight to race against.
gtest_discover_tests(AsyncCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncLinkTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
# Same reason: the GL_COMPLETION_STATUS_KHR cases saturate a one-worker pool on purpose.
gtest_discover_tests(ParallelShaderCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncTeardownTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
@@ -0,0 +1,508 @@
// MobileGL - MobileGL/MG_Test/Program/ParallelShaderCompileTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 5: the GL_KHR_parallel_shader_compile application surface.
//
// Four things are under test, and they are the four an application actually touches:
// * GL_COMPLETION_STATUS_KHR on shaders and programs, which MUST NOT JOIN - the whole
// point of the query is to answer while the work is still outstanding;
// * glMaxShaderCompilerThreadsKHR / ...ARB, including the count == 0 mode switch the
// extension mandates and what lifts it again;
// * GL_MAX_SHADER_COMPILER_THREADS_KHR;
// * the extension string itself, which must appear if and only if asynchronous
// compilation is enabled - the kill switch has to revert the application-visible
// behaviour change, not only the threading.
//
// Like the other async suites, every case drives the real GL entry points and flips
// MG_Config::Features.AsyncShaderCompile itself, so the file behaves identically whether or
// not the suite was launched with MOBILEGL_ASYNC_SHADER_COMPILE=1.
#include <gtest/gtest.h>
#include <algorithm>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Backend/BackendObjects.h"
#include "MG_Backend/DirectGLES/BackendObject_DirectGLES.h"
#include "MG_Backend/DirectVulkan/BackendObject_DirectVulkan.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async) : m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
// glMaxShaderCompilerThreadsKHR writes PROCESS-wide state (the pool's concurrency budget
// and the suspension latch), so a case that touches it has to put both back or it
// poisons every case declared after it in this binary.
class CompilerThreadScope {
public:
CompilerThreadScope() = default;
~CompilerThreadScope() {
MG_Util::Async::SetAsyncShaderCompileSuspended(false);
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(
MG_Util::Async::ShaderCompilePool::Get().GetThreadCount());
}
CompilerThreadScope(const CompilerThreadScope&) = delete;
CompilerThreadScope& operator=(const CompilerThreadScope&) = delete;
};
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = vec4(aPos, 1.0);
}
)";
// Deliberately expensive, and distinct per index so the source-hash memo never turns a
// second instance into a no-op: a saturated pool is the only way to observe an
// outstanding job without asserting on timing.
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 320; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
return shader;
}
GLint QueryShaderCompletion(const GLuint shader) {
GLint status = -1;
GetShaderiv(shader, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint QueryProgramCompletion(const GLuint program) {
GLint status = -1;
GetProgramiv(program, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint QueryCompileStatus(const GLuint shader) {
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
GLint QueryLinkStatus(const GLuint program) {
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
return status;
}
// Enqueues `count` distinct heavy compiles and returns their names without reading
// anything back, leaving the pool with a real backlog.
Vector<GLuint> EnqueueBacklog(const int count, const int seedBase, Vector<String>& sourceStorage) {
Vector<GLuint> shaders;
shaders.reserve(static_cast<SizeT>(count));
for (int i = 0; i < count; ++i) {
sourceStorage.push_back(MakeBulkySource(seedBase + i));
const char* text = sourceStorage.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
return shaders;
}
Bool Advertises(const Vector<GLExtension>& extensions, const GLExtension wanted) {
return std::find(extensions.begin(), extensions.end(), wanted) != extensions.end();
}
class ParallelShaderCompileTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
};
} // namespace
// ---------------------------------------------------------------------------------------
// GL_COMPLETION_STATUS_KHR must not join
// ---------------------------------------------------------------------------------------
// The load-bearing case of the whole stage. A single-worker pool is saturated with heavy
// compiles, so jobs are demonstrably still queued; GL_COMPLETION_STATUS_KHR then has to
// report GL_FALSE for at least one of them *and leave it outstanding*. If the query joined -
// which is what happens if it is ever routed through the ordinary Compiled() gate - it could
// only ever return GL_TRUE, and the extension would be a lie that costs an application the
// exact stall it added the polling loop to avoid.
//
// Skipped rather than failed when the machine drained the backlog first, so it can never be
// a false red on a fast box.
TEST_F(ParallelShaderCompileTest, ShaderCompletionStatusReportsFalseWithoutJoining) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
// One worker: the queue behind it is the thing being observed.
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(64, 4000, sources);
int outstanding = 0;
for (const GLuint shader : shaders) {
const GLint completion = QueryShaderCompletion(shader);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE) << "completion = " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "the pool drained 64 heavy compiles before the first query; nothing outstanding to observe";
}
// Asking again must still not have settled anything: the query is a peek, so a second
// one cannot have made progress happen. (A joining implementation would report every
// shader complete by now.)
int stillOutstanding = 0;
for (const GLuint shader : shaders) {
if (QueryShaderCompletion(shader) == GL_FALSE) ++stillOutstanding;
}
EXPECT_GT(stillOutstanding, 0) << "GL_COMPLETION_STATUS_KHR joined - every shader settled just by being asked";
// And once the real (joining) query is used, everything is complete and correct.
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE) << "GL_COMPILE_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The program half: a link enqueued behind a saturated pool cannot be complete either, and
// asking must not drag it forward.
TEST_F(ParallelShaderCompileTest, ProgramCompletionStatusReportsFalseWithoutJoining) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
EnqueueBacklog(48, 4200, sources);
Vector<GLuint> programs;
for (int i = 0; i < 8; ++i) {
sources.push_back(MakeBulkySource(4400 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
programs.push_back(program);
}
int outstanding = 0;
for (const GLuint program : programs) {
const GLint completion = QueryProgramCompletion(program);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE) << "completion = " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "the pool drained the whole backlog before the first query; nothing outstanding to observe";
}
for (const GLuint program : programs) {
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE) << "GL_LINK_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// "Nothing outstanding" is the answer for an object that was never compiled or linked at
// all: the query asks whether work is pending, not whether work ever happened.
TEST_F(ParallelShaderCompileTest, CompletionStatusIsTrueForUntouchedObjects) {
const AsyncModeScope async(true);
const GLuint shader = MakeShader(GL_FRAGMENT_SHADER, "#version 460\nvoid main() {}\n");
const GLuint program = CreateProgram();
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// With the flag off nothing is ever in flight, so the query is constant GL_TRUE - and, just
// as importantly, still a recognized pname rather than a GL_INVALID_ENUM.
TEST_F(ParallelShaderCompileTest, CompletionStatusIsAlwaysTrueWithAsyncOff) {
const AsyncModeScope async(false);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(8, 4600, sources);
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE);
}
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, shaders.front());
LinkProgram(program);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The pname is new; the rejection of everything else must be untouched.
TEST_F(ParallelShaderCompileTest, UnknownPnamesStillRaiseInvalidEnum) {
const GLuint shader = MakeShader(GL_FRAGMENT_SHADER, "#version 460\nvoid main() {}\n");
const GLuint program = CreateProgram();
GLint value = 0;
GetShaderiv(shader, GL_TEXTURE_2D, &value);
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
GetProgramiv(program, GL_TEXTURE_2D, &value);
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
}
// ---------------------------------------------------------------------------------------
// glMaxShaderCompilerThreadsKHR / ...ARB
// ---------------------------------------------------------------------------------------
// count == 0 is the mode switch the extension defines: no compiler threads. Two obligations
// follow, and both are asserted here - everything already in flight is settled by the time
// the call returns (so every GL_COMPLETION_STATUS_KHR reads GL_TRUE straight away), and
// compilation that happens AFTERWARDS is synchronous too.
TEST_F(ParallelShaderCompileTest, ZeroCompilerThreadsJoinsEverythingAndCompilesInline) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> backlog = EnqueueBacklog(48, 4800, sources);
MaxShaderCompilerThreadsKHR(0);
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
EXPECT_FALSE(MG_Util::Async::AsyncShaderCompileActive());
// The configuration flag itself is untouched: the extension is still advertised, the
// application just asked for serial compilation.
EXPECT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
for (const GLuint shader : backlog) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE)
<< "glMaxShaderCompilerThreadsKHR(0) must leave nothing in flight";
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
}
// Anything compiled from here on is finished before its glCompileShader returns.
Vector<String> serialSources;
const Vector<GLuint> serial = EnqueueBacklog(6, 4900, serialSources);
for (const GLuint shader : serial) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE) << "a compile after a zero count must be synchronous";
}
// Links too, not just compiles.
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, serial.front());
LinkProgram(program);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE) << "a link after a zero count must be synchronous";
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ...and a later NONZERO count is what lifts it. Nothing else does: not a new context, not a
// join, not eglInitialize. That is the documented contract, so it gets an assertion.
TEST_F(ParallelShaderCompileTest, NonzeroCompilerThreadsRestoresAsynchronousCompilation) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(0);
ASSERT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
// Re-initializing must NOT quietly re-arm it - the application asked for serial
// compilation and has not taken that back.
MobileGL::Initialize();
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
MaxShaderCompilerThreadsKHR(4);
EXPECT_FALSE(MG_Util::Async::IsAsyncShaderCompileSuspended());
EXPECT_TRUE(MG_Util::Async::AsyncShaderCompileActive());
// And work really is being enqueued again: with the budget back at one worker a heavy
// backlog leaves something outstanding (skip-not-fail if the box drained it first).
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(64, 5000, sources);
const Bool anyOutstanding = std::any_of(shaders.begin(), shaders.end(), [](const GLuint shader) {
return QueryShaderCompletion(shader) == GL_FALSE;
});
if (!anyOutstanding) {
GTEST_SKIP() << "the pool drained the backlog before the first query; asynchrony not observable here";
}
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The three count cases map onto the pool's concurrency budget: a request above the thread
// count cannot conjure threads, 0xFFFFFFFF means "implementation maximum", and an ordinary
// value is taken as given (clamped to at least one).
TEST_F(ParallelShaderCompileTest, CompilerThreadCountIsClampedToTheThreadCount) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
auto& pool = MG_Util::Async::ShaderCompilePool::Get();
const Uint threadCount = pool.GetThreadCount();
ASSERT_GE(threadCount, 1u);
MaxShaderCompilerThreadsKHR(1);
EXPECT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsKHR(threadCount + 1000);
EXPECT_EQ(pool.GetMaxConcurrency(), threadCount) << "asking for more threads than exist cannot create any";
MaxShaderCompilerThreadsKHR(1);
ASSERT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsKHR(0xFFFFFFFFu);
EXPECT_EQ(pool.GetMaxConcurrency(), threadCount) << "0xFFFFFFFF is the implementation maximum";
// The ARB spelling is the same entry point, not a second piece of state.
MaxShaderCompilerThreadsARB(1);
EXPECT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsARB(0);
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
MaxShaderCompilerThreadsKHR(threadCount);
EXPECT_FALSE(MG_Util::Async::IsAsyncShaderCompileSuspended())
<< "the KHR and ARB names must share one piece of state";
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// A zero count with the feature switched off is legal and does nothing observable: there is
// nothing to suspend, and the call must not fail just because MobileGL never had threads.
TEST_F(ParallelShaderCompileTest, CompilerThreadCallsAreHarmlessWithAsyncOff) {
const AsyncModeScope async(false);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(0);
MaxShaderCompilerThreadsKHR(8);
MaxShaderCompilerThreadsARB(0xFFFFFFFFu);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// GL_MAX_SHADER_COMPILER_THREADS_KHR
// ---------------------------------------------------------------------------------------
TEST_F(ParallelShaderCompileTest, MaxShaderCompilerThreadsGetter) {
{
const AsyncModeScope async(true);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, static_cast<GLint>(MG_Util::Async::ShaderCompilePool::Get().GetThreadCount()));
EXPECT_GE(value, 1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
{
// No compiler threads exist in this configuration, and the extension is not
// advertised either, so zero is the honest answer.
const AsyncModeScope async(false);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// The reported maximum is the pool's THREAD count, not its current concurrency budget: an
// application that lowered the budget still wants to know what the implementation can do.
TEST_F(ParallelShaderCompileTest, MaxShaderCompilerThreadsIgnoresTheCurrentBudget) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
const Uint threadCount = MG_Util::Async::ShaderCompilePool::Get().GetThreadCount();
MaxShaderCompilerThreadsKHR(1);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, static_cast<GLint>(threadCount));
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// The extension string
// ---------------------------------------------------------------------------------------
// The advertisement is the riskiest half of P1 - a recorded trace cannot cover it, because
// Iris and Sodium change their submission schedule the moment they see the string - so
// MOBILEGL_ASYNC_SHADER_COMPILE=0 has to withdraw it. Asserted on both backends' own
// BuildAdvertisedExtensions, which is the single source of truth each of them (and the
// driver POST) builds the list from.
TEST_F(ParallelShaderCompileTest, BothBackendsAdvertiseTheExtensionIffAsyncIsEnabled) {
{
const AsyncModeScope async(true);
EXPECT_TRUE(Advertises(MG_Backend::DirectGLES::BuildAdvertisedExtensions(false, false),
E_GL_KHR_parallel_shader_compile));
EXPECT_TRUE(Advertises(MG_Backend::DirectVulkan::BuildAdvertisedExtensions(false, false, false),
E_GL_KHR_parallel_shader_compile));
}
{
const AsyncModeScope async(false);
EXPECT_FALSE(Advertises(MG_Backend::DirectGLES::BuildAdvertisedExtensions(false, false),
E_GL_KHR_parallel_shader_compile))
<< "MOBILEGL_ASYNC_SHADER_COMPILE=0 must withdraw the extension, not only the threading";
EXPECT_FALSE(Advertises(MG_Backend::DirectVulkan::BuildAdvertisedExtensions(false, false, false),
E_GL_KHR_parallel_shader_compile))
<< "MOBILEGL_ASYNC_SHADER_COMPILE=0 must withdraw the extension, not only the threading";
}
}
// The same fact through the GL surface an application actually reads. No flag flipping here:
// a backend's advertised list is built once, at its first use, from the configuration that
// was in force then - so this case asserts against the AMBIENT configuration, which is
// exactly what makes it meaningful in both of the suite's two runs (with and without
// MOBILEGL_ASYNC_SHADER_COMPILE=1 exported).
TEST_F(ParallelShaderCompileTest, GLExtensionStringTracksTheAmbientConfiguration) {
UniquePtr<MG_Backend::BackendObject> previousBackend = Move(MG_Backend::pActiveBackendObject);
MG_Backend::pActiveBackendObject = MakeUnique<MG_Backend::DirectGLES::BackendObject_DirectGLES>();
const char* extensions = reinterpret_cast<const char*>(GetString(GL_EXTENSIONS));
ASSERT_NE(extensions, nullptr);
const String extensionString(extensions);
const Bool advertised = extensionString.find("GL_KHR_parallel_shader_compile") != String::npos;
EXPECT_EQ(advertised, MG_Util::Async::AsyncShaderCompileEnabled()) << "GL_EXTENSIONS = " << extensionString;
// glGetStringi must agree with the monolithic string - LWJGL builds GLCapabilities from
// the indexed form on a core profile.
GLint count = 0;
GetIntegerv(GL_NUM_EXTENSIONS, &count);
ASSERT_GT(count, 0);
Bool foundIndexed = false;
for (GLint i = 0; i < count; ++i) {
const char* name = reinterpret_cast<const char*>(GetStringi(GL_EXTENSIONS, static_cast<GLuint>(i)));
if (name != nullptr && std::string(name) == "GL_KHR_parallel_shader_compile") foundIndexed = true;
}
EXPECT_EQ(foundIndexed, advertised);
MG_Backend::pActiveBackendObject = Move(previousBackend);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
+2 -2
View File
@@ -2934,7 +2934,7 @@ TEST_F(ProgramTest, RecompileWithIdenticalSourceKeepsCompiledStateAndStillLinks)
EXPECT_TRUE(ShaderHasMemoizedCompile(vs));
// A first link consumes the stored TShader; the redundant recompile below must not
// disturb the preprocessed source that TakeShaderForLink re-parses from.
// disturb the preprocessed source that ClaimParsedShader re-parses from.
GLuint firstProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(firstProgram, "uColor"), 0);
@@ -2959,7 +2959,7 @@ TEST_F(ProgramTest, RecompileWithIdenticalSourceKeepsCompiledStateAndStillLinks)
EXPECT_EQ(String(sourceBuffer.data(), static_cast<size_t>(written)), String(kP0bVs));
// A second program built from the same, redundantly recompiled shaders links and
// reflects - i.e. TakeShaderForLink's re-parse path survived the no-op.
// reflects - i.e. ClaimParsedShader's re-parse path survived the no-op.
GLuint secondProgram = LinkVsFs(vs, fs, GL_TRUE);
EXPECT_GE(GetUniformLocation(secondProgram, "uColor"), 0);
EXPECT_GE(GetUniformLocation(secondProgram, "uModel"), 0);
+17 -4
View File
@@ -135,12 +135,25 @@ TEST(ShaderCompilePoolLifecycle, DetectedThreadCountIsPositive) {
EXPECT_GE(DetectShaderCompileThreadCount(), 1u);
}
TEST(ShaderCompilePoolLifecycle, AsyncIsOffByDefaultInThisStage) {
// Stage 1 ships the machinery wired to nothing. If this ever fails without the default
// constant having been deliberately flipped, something enabled async by accident.
EXPECT_EQ(MG_Config::Features.AsyncShaderCompile, MG_Config::QuirkOverride::Auto);
TEST(ShaderCompilePoolLifecycle, AsyncIsOffByDefaultAndTheOverrideDecidesEitherWay) {
// The shipped default is still off, and an unset MOBILEGL_ASYNC_SHADER_COMPILE resolves
// to it. If the first expectation ever fails without the constant having been
// deliberately flipped, something enabled async by accident.
//
// Driven through Features rather than read from it: from stage 3 on, the whole suite is
// also run with MOBILEGL_ASYNC_SHADER_COMPILE=1 exported, so a test that simply asserted
// "the resolved answer is false" would either fail there or - worse - silently pass in a
// binary that never loaded the config and prove nothing at all.
EXPECT_FALSE(kAsyncShaderCompileDefault);
const MG_Config::QuirkOverride saved = MG_Config::Features.AsyncShaderCompile;
MG_Config::Features.AsyncShaderCompile = MG_Config::QuirkOverride::Auto;
EXPECT_EQ(AsyncShaderCompileEnabled(), kAsyncShaderCompileDefault);
MG_Config::Features.AsyncShaderCompile = MG_Config::QuirkOverride::ForceOn;
EXPECT_TRUE(AsyncShaderCompileEnabled());
MG_Config::Features.AsyncShaderCompile = MG_Config::QuirkOverride::ForceOff;
EXPECT_FALSE(AsyncShaderCompileEnabled());
MG_Config::Features.AsyncShaderCompile = saved;
}
// ---------------------------------------------------------------------------------------
+64 -3
View File
@@ -9,11 +9,37 @@
#include "JobNode.h"
#include "ShaderCompilePool.h"
#include <MG_State/GLState/Core.h>
namespace MobileGL::MG_Util::Async {
namespace {
Bool IsTerminalState(const JobState state) {
return state == JobState::Complete || state == JobState::Cancelled;
}
// Job BODIES have been contained since stage 1 (JobNode::Run); continuations were
// not, and stage 4 introduces the first real ones. A continuation runs on whichever
// thread drove the node terminal - for a compile that finished on a worker, that is
// inside an Asio handler, where an escaping exception means thread_pool::run()
// rethrows and the process terminates. It would also skip every continuation after
// it in the list, stranding unrelated dependents.
//
// Containing it here is a backstop, not the contract: a continuation cannot be
// repaired from the outside (the dispatcher has no idea what the callback was for),
// so the registrar still owns "this cannot fail". See JobNode::OnTerminal.
void RunContinuation(const std::function<void()>& continuation) {
if (!continuation) return;
try {
continuation();
} catch (const std::exception& e) {
MGLOG_E("JobNode: a terminal continuation threw (%s); it has been contained, but whatever it "
"was going to do did not happen",
e.what());
} catch (...) {
MGLOG_E("JobNode: a terminal continuation threw a non-std exception; it has been contained, "
"but whatever it was going to do did not happen");
}
}
} // namespace
Bool JobNode::IsTerminal() const { return IsTerminalState(m_state.load(std::memory_order_acquire)); }
@@ -45,9 +71,10 @@ namespace MobileGL::MG_Util::Async {
m_cv.notify_all();
// Run continuations OUTSIDE the lock: a continuation is free to call back into this
// node (IsComplete, State) and, in the link-dependency case, to post the dependent
// job to the pool from whichever thread drove this node terminal.
// job to the pool from whichever thread drove this node terminal. Individually
// contained, so one broken dependent cannot strand the rest of the list.
for (auto& continuation : continuations) {
if (continuation) continuation();
RunContinuation(continuation);
}
return true;
}
@@ -120,6 +147,40 @@ namespace MobileGL::MG_Util::Async {
return;
}
}
fn();
// Already terminal: the caller's thread runs it, through the same guard the deferred
// path uses. OnTerminal is reached from Link()'s GL-thread prologue as well as from a
// worker, and glLinkProgram is not a place an exception may escape from either.
RunContinuation(fn);
}
void ApplyDeferredDiagnostics(JobNode& node) {
MOBILEGL_ASSERT(!ShaderCompilePool::IsPoolThread(),
"ApplyDeferredDiagnostics() called from a pool thread; deferred diagnostics exist precisely "
"so that a worker never touches the GL error state");
MOBILEGL_ASSERT(node.IsTerminal(),
"ApplyDeferredDiagnostics() called on a job that has not settled; its diagnostics are still "
"being written");
if (!node.diagnostics.logLines.empty()) {
Vector<String> lines;
lines.swap(node.diagnostics.logLines);
for (const String& line : lines) {
MGLOG_W("%s", line.c_str());
}
}
if (node.diagnostics.errors.empty()) return;
Vector<DeferredError> errors;
errors.swap(node.diagnostics.errors);
// Ascending sequence == job-enqueue order == the order a serial implementation would
// have recorded them in, which is what decides WHICH payload the application sees:
// MobileGL implements GL's sticky-flag semantics, so a repeat of an already-pending
// code is discarded and only the first occurrence of each code survives.
std::sort(errors.begin(), errors.end(),
[](const DeferredError& a, const DeferredError& b) { return a.sequence < b.sequence; });
if (!MG_State::pGLContext) return;
for (DeferredError& error : errors) {
MG_State::pGLContext->RecordError(error.code, Move(error.info));
}
}
} // namespace MobileGL::MG_Util::Async
+27 -1
View File
@@ -52,7 +52,12 @@ namespace MobileGL::MG_Util::Async {
// Running -> Cancelled (cancelled mid-run, or RunBody() threw)
// Complete and Cancelled are terminal and the node is immutable afterwards, so every
// reader that observed IsTerminal() may read the outputs without further synchronization.
class JobNode {
//
// enable_shared_from_this because a dependency edge outlives its registrar: a node that
// posts itself from another node's continuation (ProgramLinkTask::OnDepSettled) has to
// hand the pool a strong reference from inside itself. Every JobNode is therefore created
// through MakeShared - a stack-allocated one may not use SubmitAfter-style chaining.
class JobNode : public std::enable_shared_from_this<JobNode> {
public:
JobNode() = default;
virtual ~JobNode() = default;
@@ -88,6 +93,16 @@ namespace MobileGL::MG_Util::Async {
// `fn` runs on the calling thread before OnTerminal returns. Exactly-once in both
// directions: the callback is either handed to the finishing thread or run inline,
// never both.
//
// A continuation must not throw. It is dispatched from whichever thread drove this
// node terminal, which on the pool side is an Asio handler - an exception escaping
// one propagates out of thread_pool::run() and terminates the process. The dispatcher
// contains a throw anyway (see RunContinuation) so that one broken continuation
// cannot strand the others, but the continuation itself is where the guarantee
// belongs: whoever registers one owns the "and it cannot fail" argument, because the
// dispatcher can only log, never repair. ProgramLinkTask::OnDepSettled is the worked
// example - it catches internally and cancels itself, because a link that is never
// posted is a joiner blocked forever.
void OnTerminal(std::function<void()> fn);
// Runs the body on the calling thread. The synchronous path (async disabled,
@@ -113,4 +128,15 @@ namespace MobileGL::MG_Util::Async {
std::atomic<Bool> m_cancelled{false};
Vector<std::function<void()>> m_continuations;
};
// Replays a settled node's worker-side diagnostics on the calling thread: log lines
// first, in the order the body produced them, then any deferred GL error in ascending
// `sequence`. GL thread only - it is the join that calls this, which is exactly the
// point at which a deferred error becomes indistinguishable from one a serial
// implementation would have raised inside glCompileShader/glLinkProgram (an application
// cannot observe a pending job's effects by any other route).
//
// Drains what it replays, so calling it twice on one node is a no-op the second time.
// Must be called with the node terminal.
void ApplyDeferredDiagnostics(JobNode& node);
} // namespace MobileGL::MG_Util::Async
+131 -19
View File
@@ -36,11 +36,29 @@ namespace MobileGL::MG_Util::Async {
// first pool use, so it is guaranteed to run before any static destructor.
Bool g_processTeardown = false;
std::once_flag g_teardownSentinelOnce;
// The process-wide pool from Get(), for the atexit handler to stop. Never the
// stack-allocated pools a test builds - those join themselves in their destructor.
std::atomic<ShaderCompilePool*> g_processPool{nullptr};
Bool InProcessTeardown() { return g_processTeardown; }
void EnsureProcessTeardownSentinel() {
std::call_once(g_teardownSentinelOnce, [] { std::atexit(+[] { g_processTeardown = true; }); });
std::call_once(g_teardownSentinelOnce, [] {
std::atexit(+[] {
g_processTeardown = true;
// Latching the flag is not enough: a worker that is ALREADY inside
// glslang has to be out of it before static destruction reaches
// glslang's process globals, the SPIRV-Tools tables, or anything else a
// job body touches. This is the same wait Init.cpp's DestroyImpl does -
// it just also has to happen for a process that exits without ever
// calling eglTerminate, which is the norm for a test binary and legal
// for an application. Registered here, during main, so it runs before
// the destructors of statics constructed at load time.
if (ShaderCompilePool* pool = g_processPool.load(std::memory_order_acquire)) {
pool->StopAndDrain();
}
});
});
}
Uint64 ReadCpuMaxFrequencyKHz(const Uint cpu) {
@@ -89,6 +107,25 @@ namespace MobileGL::MG_Util::Async {
return kAsyncShaderCompileDefault;
}
namespace {
// Written only by glMaxShaderCompilerThreadsKHR/ARB, i.e. only on the GL thread, but
// read by every enqueue decision, so it is atomic rather than plain: a worker never
// reads it, but a second GL thread in another context shares this process-wide pool.
std::atomic<Bool> g_asyncSuspendedByApplication{false};
} // namespace
void SetAsyncShaderCompileSuspended(const Bool suspended) {
g_asyncSuspendedByApplication.store(suspended, std::memory_order_release);
}
Bool IsAsyncShaderCompileSuspended() {
return g_asyncSuspendedByApplication.load(std::memory_order_acquire);
}
Bool AsyncShaderCompileActive() {
return AsyncShaderCompileEnabled() && !IsAsyncShaderCompileSuspended();
}
Uint DetectShaderCompileThreadCount() {
if (const Uint32 configured = MG_Config::Features.AsyncShaderCompileThreads; configured > 0) {
// An explicit request is honoured as given - it is the escape hatch for measuring
@@ -118,12 +155,30 @@ namespace MobileGL::MG_Util::Async {
// out by a concurrent StopAndDrain between the decision and the dispatch: asio::post
// only enqueues, it never runs the handler on the calling thread, so it cannot
// re-enter this mutex.
void DispatchLocked() {
//
// A node asio::post fails to hand off is appended to `toCancel` instead of being
// Cancel()'d here: Cancel() runs the node's OnTerminal continuations inline (stage 4
// added ProgramLinkTask::OnDepSettled as a real one), and a continuation is free to
// call ShaderCompilePool::Post() again. Every caller of DispatchLocked holds `mutex`
// (a plain, non-recursive std::mutex) - Cancel()'ing in here would let that
// re-entrant Post() deadlock on the very lock this frame already owns. The caller
// drains `toCancel` after releasing the lock.
void DispatchLocked(Vector<SharedPtr<JobNode>>& toCancel) {
while (!queue.empty() && inFlight < maxConcurrency && !stopped.load(std::memory_order_acquire)) {
SharedPtr<JobNode> node = Move(queue.front());
// Copy rather than move into the handler: if asio::post throws (it allocates)
// the local SharedPtr is still valid, so the node can be settled instead of
// being stranded Pending in a queue nothing will dispatch from again - a
// joiner would block on it forever. Reclaiming the slot matters just as much:
// a leaked `inFlight` shrinks the pool's concurrency budget permanently.
SharedPtr<JobNode> node = queue.front();
queue.pop_front();
++inFlight;
asio::post(*pool, [this, node = Move(node)]() mutable { RunOnWorker(Move(node)); });
try {
asio::post(*pool, [this, node]() mutable { RunOnWorker(Move(node)); });
} catch (...) {
--inFlight;
toCancel.push_back(Move(node));
}
}
}
@@ -131,14 +186,23 @@ namespace MobileGL::MG_Util::Async {
tl_isPoolThread = true;
// A node that was already handed to Asio when StopAndDrain ran still arrives
// here; cancelling it first turns the dispatch into a state transition instead of
// a full compile, so the drain's join() returns promptly.
// a full compile, so the drain's join() returns promptly. This Cancel() runs
// before `mutex` is ever taken in this frame, so it is not subject to the
// re-entrancy hazard DispatchLocked's comment describes.
if (stopped.load(std::memory_order_acquire)) node->Cancel();
node->Run();
node.reset();
const std::lock_guard<std::mutex> lock(mutex);
--inFlight;
DispatchLocked();
Vector<SharedPtr<JobNode>> toCancel;
{
const std::lock_guard<std::mutex> lock(mutex);
--inFlight;
DispatchLocked(toCancel);
}
// Outside the lock: see DispatchLocked's comment.
for (const auto& n : toCancel) {
if (n) n->Cancel();
}
}
};
@@ -147,10 +211,17 @@ namespace MobileGL::MG_Util::Async {
ShaderCompilePool::~ShaderCompilePool() { StopAndDrain(); }
ShaderCompilePool& ShaderCompilePool::Get() {
// Leak-at-exit, like the other MobileGL singletons: a process that exits without
// eglTerminate hands the threads to the OS rather than joining them from a static
// destructor, where the rest of the library may already be gone.
static ShaderCompilePool* pool = new ShaderCompilePool(DetectShaderCompileThreadCount());
// Leak-at-exit, like the other MobileGL singletons: the object itself is never
// destroyed, so no static destructor can race a late entry point for it. Its THREADS
// are a different matter and are stopped explicitly - by Init.cpp's DestroyImpl on
// the normal path, and by the atexit sentinel below for a process that exits without
// ever calling eglTerminate.
static ShaderCompilePool* pool = [] {
auto* created = new ShaderCompilePool(DetectShaderCompileThreadCount());
g_processPool.store(created, std::memory_order_release);
EnsureProcessTeardownSentinel();
return created;
}();
return *pool;
}
@@ -164,30 +235,71 @@ namespace MobileGL::MG_Util::Async {
}
void ShaderCompilePool::SetMaxConcurrency(const Uint n) {
const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
// Raising the budget releases whatever the old one was holding back.
if (m_impl->pool) m_impl->DispatchLocked();
Vector<SharedPtr<JobNode>> toCancel;
{
const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
// Raising the budget releases whatever the old one was holding back.
if (m_impl->pool) m_impl->DispatchLocked(toCancel);
}
// Outside the lock: see DispatchLocked's comment.
for (const auto& n2 : toCancel) {
if (n2) n2->Cancel();
}
}
void ShaderCompilePool::Post(SharedPtr<JobNode> node) {
if (!node) return;
EnsureProcessTeardownSentinel();
{
// Enqueueing can throw: the thread_pool construction and asio::post both allocate,
// and under memory pressure a throw here would escape glCompileShader leaving the
// node Pending with nothing left to dispatch it - the first observable read would
// then block the GL thread forever. Settle the node instead: a cancelled node is a
// state every joiner already handles.
//
// `node` is still valid in the catch for every throw this try can produce. The
// thread_pool construction runs before the move; deque::push_back is strongly
// exception-safe and SharedPtr's move constructor is noexcept, so a throwing
// push_back never consumed it; and DispatchLocked contains its own asio::post
// failures rather than propagating them (see above). Keep it that way.
Bool enqueued = false;
Vector<SharedPtr<JobNode>> toCancel;
try {
const std::lock_guard<std::mutex> lock(m_impl->mutex);
if (!m_impl->stopped.load(std::memory_order_acquire) && !InProcessTeardown()) {
if (!m_impl->pool) m_impl->pool = MakeUnique<asio::thread_pool>(m_impl->threadCount);
m_impl->queue.push_back(Move(node));
m_impl->DispatchLocked();
return;
m_impl->DispatchLocked(toCancel);
enqueued = true;
}
} catch (...) {
MGLOG_E("ShaderCompilePool::Post: enqueue failed; cancelling the job so its joiner "
"cannot block forever");
if (node) node->Cancel();
return;
}
// Outside the lock: see DispatchLocked's comment - a Cancel() here may run a
// continuation (e.g. ProgramLinkTask::OnDepSettled) that calls back into Post().
for (const auto& n : toCancel) {
if (n) n->Cancel();
}
if (enqueued) return;
// A stopped pool is a synchronous pool, not a black hole: the node still runs, just
// on the caller's thread. Everything downstream already handles "terminal by the time
// Post returns", because that is exactly what the inline path looks like. Run it
// outside the lock - a body, or a continuation it releases, is free to Post again.
//
// Say so once. StopAndDrain is a one-way latch (see its tail), so from the first
// eglTerminate onwards EVERY compile in this process silently runs on the GL thread;
// without this line the only symptom is that asynchronous compilation stopped helping,
// with nothing in the log to point at. Once, not per node: a pack load posts hundreds.
static std::atomic<Bool> warnedStopped{false};
if (!warnedStopped.exchange(true, std::memory_order_relaxed)) {
MGLOG_W("ShaderCompilePool::Post: the pool is stopped (eglTerminate, or process exit); shader "
"compilation runs inline on the calling thread until MobileGL is re-initialized");
}
node->RunInline();
}
+29 -3
View File
@@ -27,11 +27,37 @@ namespace MobileGL::MG_Util::Async {
inline constexpr Bool kAsyncShaderCompileDefault = false;
// MOBILEGL_ASYNC_SHADER_COMPILE forces the answer either way; unset keeps the built-in
// default above. Falsy is a complete kill switch: it reverts the threading *and* (from
// the extension stage on) withdraws GL_KHR_parallel_shader_compile, so the application
// behaviour change goes with it.
// default above. Falsy is a complete kill switch: it reverts the threading *and*
// withdraws GL_KHR_parallel_shader_compile, so the application behaviour change goes
// with it.
//
// This is the pure CONFIGURATION answer, and it is deliberately not affected by
// glMaxShaderCompilerThreadsKHR: it is what decides whether the extension is advertised
// at all, and an application that switched threading off through the extension has not
// made the extension go away. Code deciding whether to enqueue asks
// AsyncShaderCompileActive() instead.
Bool AsyncShaderCompileEnabled();
// ---- GL_KHR_parallel_shader_compile: glMaxShaderCompilerThreadsKHR(count) ----
// The extension defines count == 0 as "no compiler threads": compilation must happen on
// the application's thread. That is a mode switch, not a concurrency budget of one, so it
// is a latch of its own rather than SetMaxConcurrency(1) - a budget of one would still
// move the work off-thread and still report GL_COMPLETION_STATUS_KHR = GL_FALSE, both of
// which the extension forbids after a zero count.
//
// The latch is process-wide, matching the pool it suspends. It is released by the next
// nonzero glMaxShaderCompilerThreadsKHR/ARB, which is the only thing that releases it:
// no implicit re-arm on eglInitialize, on a context switch or at any join, because an
// application that asked for serial compilation gets to keep it until it asks otherwise.
void SetAsyncShaderCompileSuspended(Bool suspended);
Bool IsAsyncShaderCompileSuspended();
// What every enqueue site branches on: the configuration flag AND the absence of a
// glMaxShaderCompilerThreadsKHR(0). False makes glCompileShader/glLinkProgram run their
// bodies inline, exactly as the flag-off path does, which is what makes a subsequent
// GL_COMPLETION_STATUS_KHR read immediately GL_TRUE.
Bool AsyncShaderCompileActive();
// min(4, big cores), where a big core is one whose cpufreq ceiling is within 15% of the
// machine maximum; the whole CPU count where that sysfs tree is absent. Clamped to [1, 4]
// because peak RSS scales as workers x largest glslang arena, and four
@@ -858,6 +858,9 @@ namespace MobileGL::MG_Util::BackendLoader {
if (std::strcmp(extension, "GL_EXT_disjoint_timer_query") == 0) {
caps.SupportsDisjointTimerQuery = true;
}
if (std::strcmp(extension, "GL_KHR_parallel_shader_compile") == 0) {
caps.SupportsParallelShaderCompile = true;
}
if (std::strcmp(extension, "GL_EXT_blend_func_extended") == 0) {
caps.SupportsDualSourceBlend = true;
}
@@ -1220,6 +1223,8 @@ namespace MobileGL::MG_Util::BackendLoader {
caps.IsAngleLlvmpipeRenderer && MG_Config::Features.AvoidSamplerMipmapMinFilter;
MGLOG_I(" GL_EXT_disjoint_timer_query supported: %s",
caps.SupportsDisjointTimerQuery ? "true" : "false");
MGLOG_I(" GL_KHR_parallel_shader_compile supported: %s",
caps.SupportsParallelShaderCompile ? "true" : "false");
MGLOG_I(" ANGLE renderer: %s", caps.IsAngleRenderer ? "true" : "false");
MGLOG_I(" ANGLE llvmpipe renderer: %s", caps.IsAngleLlvmpipeRenderer ? "true" : "false");
MGLOG_I(" Avoid sampler mipmap min filter: %s",
@@ -1053,6 +1053,16 @@ namespace MobileGL {
Bool SupportsBaseInstance = false;
// GL_EXT_disjoint_timer_query is present in the extension string.
Bool SupportsDisjointTimerQuery = false;
// GL_KHR_parallel_shader_compile is present in the HOST driver's extension string,
// i.e. the device driver can compile its own (ESSL) shaders on its own threads.
//
// Purely informational today, and NOT what gates MobileGL's advertisement of the
// same extension: MobileGL's parallelism is its own compile pool turning GLSL into
// SPIR-V, which is where the shaderpack time goes, and it works on a driver that
// has never heard of the extension. This flag becomes load-bearing only if the
// driver-side glCompileShader of the translated ESSL is parallelised too, at which
// point it decides whether that half can overlap.
Bool SupportsParallelShaderCompile = false;
// glPolygonModeNV/ANGLE loaded (GL_NV_polygon_mode / GL_ANGLE_polygon_mode). GLES core
// has no glPolygonMode, so without this the mode stays FILL.
Bool SupportsPolygonMode = false;
+47
View File
@@ -17,6 +17,7 @@
// MG_State code: it runs standalone, before MG_State::Init().
#include <MG_State/GLState/VertexArrayState/VertexArrayObject.h>
#include <MG_Util/Converters/MGToStr/GLExtensionConverter.h>
#include <MG_Util/Async/ShaderCompilePool.h>
#include <chrono>
#include <thread>
@@ -122,6 +123,37 @@ namespace MobileGL::MG_Util::SelfTest {
return result;
}
// ---- Asynchronous shader compilation ------------------------------------
// MobileGL's OWN capability row, appended for both backends: nothing about it comes
// from the device driver, so it is the same fact on Espryt and on Magma. The POST
// rule ("every new capability gets a row") applies to frontend capabilities too -
// and this one especially, because it is the capability that changes what
// applications DO, not just what they can do: with the extension advertised, Iris
// and Sodium batch their pipeline compiles and poll GL_COMPLETION_STATUS_KHR.
//
// PASS when it is on (the intended configuration once the default flips), INFO when
// it is off - "off" is a supported configuration, not a degradation, so it must not
// colour the verdict. Either way the row names MOBILEGL_ASYNC_SHADER_COMPILE, so a
// user reading a POST page can tell which side of the switch they are on and how to
// change it.
void AppendAsyncShaderCompileRow(ReportBuilder& builder) {
constexpr const char* rowName = "Asynchronous shader compilation";
if (!MG_Util::Async::AsyncShaderCompileEnabled()) {
builder.Info(rowName,
"off; glCompileShader and glLinkProgram run on the calling thread and "
"GL_KHR_parallel_shader_compile is not advertised (set environment variable "
"MOBILEGL_ASYNC_SHADER_COMPILE=1 to enable it)");
return;
}
const Uint threads = MG_Util::Async::DetectShaderCompileThreadCount();
builder.Pass(rowName,
format("on with {} compiler thread{}; GL_KHR_parallel_shader_compile is advertised "
"and GL_MAX_SHADER_COMPILER_THREADS_KHR = {} (set environment variable "
"MOBILEGL_ASYNC_SHADER_COMPILE=0 to disable it, or "
"MOBILEGL_ASYNC_SHADER_COMPILE_THREADS=n to change the count)",
threads, threads == 1 ? "" : "s", threads));
}
// Appends the four "MobileGL reported ..." rows for one backend section.
// GL_VENDOR and GL_VERSION only depend on the backend's static identity, so
// they are always concrete; GL_RENDERER and GL_EXTENSIONS need data from the
@@ -130,6 +162,9 @@ namespace MobileGL::MG_Util::SelfTest {
const Optional<String>& backendApiVersionString,
const Optional<String>& advertisedExtensions) {
static const String Unavailable = "unavailable (backend probe failed)";
// Frontend capability, not a probe result, so it is appended on every path -
// including one where the device probe failed outright.
AppendAsyncShaderCompileRow(builder);
builder.MobileGLReported("MobileGL reported GL_VENDOR", BuildReportedGLVendor(identity));
builder.MobileGLReported("MobileGL reported GL_VERSION", BuildReportedGLVersion(identity));
builder.MobileGLReported("MobileGL reported GL_RENDERER",
@@ -416,6 +451,18 @@ namespace MobileGL::MG_Util::SelfTest {
"not supported; 16-bit normalized texture formats need emulation");
}
// INFO, never WARN: this is the HOST driver's ability to compile its own ESSL on
// its own threads, and MobileGL's asynchronous compilation does not depend on it
// in the slightest - the pool parallelises GLSL -> SPIR-V -> ESSL translation,
// which is where a shaderpack load actually spends its time, and it does that on
// a driver that has never heard of the extension. The row exists so that the day
// the driver-side half is overlapped too, the POST already says which devices can.
builder.Info("Driver GL_KHR_parallel_shader_compile",
caps.SupportsParallelShaderCompile
? "supported; the device driver can also compile the translated ESSL off-thread"
: "not supported; the device driver compiles the translated ESSL on the calling "
"thread (MobileGL's own compile pool is unaffected)");
builder.Info("Indirect gl_InstanceID semantics",
caps.IndirectDrawInstanceIdIncludesBaseInstance
? "includes baseInstance (ANGLE-style; MobileGL's shader rewrite keeps gl_InstanceID "
@@ -249,6 +249,56 @@ namespace MobileGL {
return retryResult;
}
// Namespace-level rather than a function-local static, because it has to be
// CLEARABLE: what PrewarmBuiltins latches is not a property of this process, it is
// a property of the built-in symbol tables glslang currently holds, and
// glslang::FinalizeProcess() deletes those. A function-local latch survived the
// teardown that invalidated it, so an Initialize -> Destroy -> Initialize cycle
// came back up with the tables gone and the prewarm skipped - which is exactly the
// serialized-first-parse stall this function exists to prevent, only now
// unfixable for the rest of the process. Reset it from DestroyImpl.
namespace {
Bool g_builtinsPrewarmed = false;
} // namespace
void ShaderCompiler::ResetPrewarmLatch() { g_builtinsPrewarmed = false; }
void ShaderCompiler::PrewarmBuiltins() {
if (g_builtinsPrewarmed) return;
g_builtinsPrewarmed = true;
// One vertex and one fragment shader is enough: the built-in table is cached
// per (version, spvVersion, profile, source), not per stage language, and
// both configurations CompileShader can reach - the declared-460 path and
// the retargeted-legacy path - resolve to the same combination here because
// ParseShaderSource always passes 460/ECoreProfile as the default. Parsing
// both anyway costs microseconds and keeps this honest if that ever changes.
static constexpr const char* kPrewarmVertexSource =
"#version 460\nvoid main() { gl_Position = vec4(0.0); }\n";
static constexpr const char* kPrewarmFragmentSource =
"#version 460\nlayout(location = 0) out vec4 c;\nvoid main() { c = vec4(0.0); }\n";
static constexpr const char* kPrewarmLegacyVertexSource =
"#version 330 core\nvoid main() { gl_Position = vec4(0.0); }\n";
const CompileEnv& env = *GetDefaultCompileEnv();
for (const auto& [type, source] :
{std::pair{GL_VERTEX_SHADER, kPrewarmVertexSource},
std::pair{GL_FRAGMENT_SHADER, kPrewarmFragmentSource},
std::pair{GL_VERTEX_SHADER, kPrewarmLegacyVertexSource}}) {
ShaderAttrib attrib{.shaderType = static_cast<GLenum>(type),
.sourceStr = source,
.flags = 0,
.env = &env};
// The result is deliberately discarded: the value is the symbol table
// glslang cached as a side effect. A failure here is not fatal - it just
// means the first real compile pays for the table, exactly as before.
(void)CompileShader(attrib);
}
// The parses above left this thread's glslang allocator pointing at the last
// TShader's pool, and that TShader is about to be destroyed with it.
glslang::SetThreadPoolAllocator(nullptr);
}
Result<SharedPtr<glslang::TProgram>> ShaderCompiler::LinkProgram(const ProgramAttrib& attrib) {
SharedPtr<glslang::TProgram> program = MakeShared<glslang::TProgram>();
for (auto& s : attrib.shaders) {
@@ -77,6 +77,30 @@ namespace MobileGL {
static bool UseUnformattedFloatStorageImagesForVulkan(
const Vector<Uint32>& inputBinary, Vector<uint32_t>& outputBinary);
static Result<String> DecompileShader(SpvcSession& session);
// Parses one trivial shader in each configuration the production path can
// reach, on the calling thread, so the built-in symbol tables those
// configurations need are already cached before any worker asks for one.
//
// Why it matters: glslang builds a built-in TSymbolTable per distinct
// (version, spvVersion, profile, source) combination, and does it under a
// process-wide lock held for the whole build. Without this, the first
// parallel compiles of a shaderpack load all pile up behind that lock and
// show no speedup at all - which is easy to misread as asynchronous
// compilation not working. Call once, from the GL thread, right after
// glslang::InitializeProcess(). Idempotent and cheap on repeat.
//
// Only worth its cost when compiles can actually run in parallel, so the GL
// frontend calls it only when asynchronous compilation is enabled: a
// synchronous build would pay for three throwaway parses at every
// eglInitialize to prewarm tables the first real compile builds anyway.
static void PrewarmBuiltins();
// Clears the "already prewarmed" latch. MUST be called wherever
// glslang::FinalizeProcess() is, and for the same reason: finalizing deletes
// the cached built-in tables the latch is asserting the existence of. Without
// it, the second eglInitialize of a process comes back up unwarmed and with
// no way left to warm it.
static void ResetPrewarmLatch();
};
} // namespace ShaderTranspiler
} // namespace MG_Util