mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-09-07 19:58:32 +09:00
[Feat] (MG_State, MG_Util): async program linking on the job graph (P1 stage 4)
glLinkProgram with the flag on snapshots its inputs in a GL-thread prologue (stage-sorted shaders with their compile nodes taken without joining, env, explicit locations/fragdata/xfb, draw-buffer count), then runs the whole link body - glslang link/mapIO, SPIR-V, reflection, routing tables - as a ProgramLinkTask that auto-posts when its last compile dependency settles (+1-guarded countdown; no worker ever waits on another job). The publish is one move of the LinkArtifacts block at the join, with the second version bump so nothing memoized during the pending window survives. The consume-once TShader claim moved onto the shared compile node as a CAS: two link jobs racing for one shader resolve to winner-takes-the-parse, loser re-parses the preprocessed source against the node's own env - identical SPIR-V pinned by test for 2 and for 12 sharing programs. Two deliberate corrections to the design's cancel matrix, both test-proven: attach/detach do NOT cancel a pending link (the snapshot isolates it, and glCreateShaderProgramv's link-then-detach would otherwise discard its own result before anyone read it); and a compile node a pending link depends on is pinned against the orphan-name sweep - the ordinary LWJGL teardown compile/attach/link/detach/delete used to cancel the dependency and turn a must-pass link into GL_FALSE. Continuations are now throw-contained per-item (a stage-3 leftover made load-bearing by the first real continuation), and the review's deadlock find is fixed: the dispatch loop no longer cancels a node while holding the pool mutex, since that cancel can run OnDepSettled -> Post -> same mutex. Explicit joins: the draw path (GetProgramForDraw, both the pipeline stage loop and the plain-UseProgram half) and the composite-link site; destroy paths cancel-not-join; COMPLETION_STATUS readers stay non-joining. Gates: 506/506 unit both flag states; AsyncCompile/AsyncLink/AsyncTeardown suites x10 repeats clean both states (teardown with 128 jobs in flight, then re-Initialize); full NVIDIA DirectGLES retrace flag on twice - result sets identical to flag off, zero new deltas. Compile-phase prefix-diff, flag on vs off: complementary-reimagined 5.21s -> 2.16s, BSL 1.72s -> 0.90s - past the design's final acceptance targets before the KHR extension is even advertised. Default remains OFF until stage 5+7.
This commit is contained in:
BZLZHH
@@ -294,6 +294,7 @@ set(SOURCE_FILES
|
||||
MobileGL/MG_State/GLState/TextureState/TextureUnit.cpp
|
||||
MobileGL/MG_State/GLState/TextureState/TextureState.cpp
|
||||
MobileGL/MG_State/GLState/ProgramState/ProgramObject.cpp
|
||||
MobileGL/MG_State/GLState/ProgramState/ProgramLinkTask.cpp
|
||||
MobileGL/MG_State/GLState/ProgramState/ShaderCompileTask.cpp
|
||||
MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp
|
||||
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
|
||||
|
||||
@@ -368,11 +368,35 @@ namespace MobileGL::MG_State {
|
||||
const SharedPtr<ProgramObject>& GLContext::GetProgramForDraw() {
|
||||
static const SharedPtr<ProgramObject> nullProgram = nullptr;
|
||||
const auto& currentProgram = m_programState.GetCurrentProgram();
|
||||
if (currentProgram) return currentProgram;
|
||||
if (currentProgram) {
|
||||
// P1 join site J1, plain glUseProgram half. The backends read a program's
|
||||
// lifetimeId / backendStateVersion / UBO content version to decide whether
|
||||
// their per-program caches are still valid, and none of those pass through
|
||||
// ProgramObject's join gate - so a draw could sample a version, join later
|
||||
// inside the same draw when it finally touched an artifact, and cache under a
|
||||
// version the publish had already superseded. Settling here means every
|
||||
// version a backend reads during a draw describes the program it is drawing.
|
||||
// One null check in steady state.
|
||||
currentProgram->JoinLink();
|
||||
return currentProgram;
|
||||
}
|
||||
if (m_boundProgramPipeline == 0) return nullProgram;
|
||||
const auto& pipeline = GetBoundProgramPipeline();
|
||||
if (!pipeline) return nullProgram;
|
||||
|
||||
// P1 join site J1. ComputeDrawProgramSignature() keys the composite cache on each
|
||||
// stage program's lifetimeId and backendStateVersion - NON-artifact fields, so
|
||||
// they do not pass through ProgramObject's join gate and a pending link would
|
||||
// stay pending right through the signature. Since the version is bumped both at
|
||||
// enqueue and at publish, the signature computed inside a pending window is one
|
||||
// that will never be produced again: every draw would miss the cache and rebuild
|
||||
// (and relink) the composite. Join first, so the signature describes settled
|
||||
// programs. In steady state this is a null check per stage.
|
||||
for (SizeT stage = 0; stage < static_cast<SizeT>(ShaderStage::ShaderStageCount); ++stage) {
|
||||
const auto& stageProgram = pipeline->GetStageProgram(static_cast<ShaderStage>(stage));
|
||||
if (stageProgram) stageProgram->JoinLink();
|
||||
}
|
||||
|
||||
const auto signature = pipeline->ComputeDrawProgramSignature();
|
||||
if (const auto& cached = pipeline->GetCachedDrawProgram(signature)) return cached;
|
||||
|
||||
@@ -400,6 +424,10 @@ namespace MobileGL::MG_State {
|
||||
// A pipeline with no fragment stage still rasterises, so the default fragment
|
||||
// shader is wanted here even though the separable stage programs never get one.
|
||||
composite->Link(true);
|
||||
// P1 join site J2. The draw that asked for this program is the very next thing to
|
||||
// happen, so enqueueing the composite's link buys nothing and only moves the wait
|
||||
// to whichever backend accessor happens to touch its artifacts first.
|
||||
composite->JoinLink();
|
||||
pipeline->SetCachedDrawProgram(signature, Move(composite));
|
||||
return pipeline->GetCachedDrawProgram(signature);
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,111 @@
|
||||
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ProgramLinkTask.h
|
||||
// Copyright (c) 2025-2026 MobileGL-Dev
|
||||
// Licensed under the GNU Lesser General Public License v3.0:
|
||||
// https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
// https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
// SPDX-License-Identifier: LGPL-3.0-only
|
||||
// End of Source File Header
|
||||
|
||||
#pragma once
|
||||
#include <Includes.h>
|
||||
#include <MG_State/GLState/ProgramState/ProgramObject.h>
|
||||
#include <MG_State/GLState/ProgramState/ShaderCompileTask.h>
|
||||
#include <MG_Util/Async/JobNode.h>
|
||||
#include <MG_Util/ShaderTranspiler/CompileEnv.h>
|
||||
|
||||
namespace MobileGL::MG_State::GLState {
|
||||
// One attached shader, as the link sees it: never the ShaderObject, always a snapshot.
|
||||
//
|
||||
// The ShaderObject is GL-thread-owned and may be re-sourced, detached or destroyed while
|
||||
// this link is still queued; everything below is either immutable or independently owned,
|
||||
// so none of that can reach the worker.
|
||||
struct LinkShaderInput {
|
||||
ShaderStage stage = ShaderStage::Unknown;
|
||||
// For the compile-error diagnostic and the compute local_size check, both of which
|
||||
// quote the ORIGINAL source rather than the preprocessed one.
|
||||
SharedPtr<const String> source;
|
||||
// The authoritative compiled state. Null, or non-Complete, both read as "this shader
|
||||
// did not compile" - the same verdict ShaderObject's join gate produces.
|
||||
SharedPtr<const ShaderCompileTask> compiled;
|
||||
};
|
||||
|
||||
// The unit of asynchronous linking: one glLinkProgram's worth of pure CPU work - glslang
|
||||
// link + mapIO, SPIR-V generation and optimization, the GL-facing reflection surface, the
|
||||
// global-UBO routing tables, fragment-output validation and transform-feedback
|
||||
// resolution - with every input it needs snapshotted at enqueue.
|
||||
//
|
||||
// Same ownership rule as ShaderCompileTask: the body reads nothing but `in` (all of it
|
||||
// owned or immutable) and writes nothing but `artifacts`. No GL call, no
|
||||
// pActiveBackendObject read, no pGLContext->RecordError(); the device limits arrive
|
||||
// through the CompileEnv snapshot and diagnostics are deferred to the join.
|
||||
//
|
||||
// ONE LINK IS ONE HANDLER. RunBody() runs start to finish inside a single pool handler
|
||||
// and is the only place `artifacts` is written. Do not split it across handlers to
|
||||
// "pipeline" the reflection half: the intermediates that GlslangToSpv and buildReflection
|
||||
// share are mutated in a strict order (see the GenerateSpirv-before-DoReflection comment
|
||||
// in Run()), and a second handler would let a cancel land between them and publish a
|
||||
// program whose SPIR-V and reflection describe different things.
|
||||
class ProgramLinkTask final : public MG_Util::Async::JobNode {
|
||||
public:
|
||||
// ---- inputs, snapshotted on the GL thread in ProgramObject::Link()'s prologue ----
|
||||
struct Inputs {
|
||||
Uint externalIndex = 0; // logs only
|
||||
Vector<LinkShaderInput> shaders; // already stage-sorted
|
||||
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
|
||||
// The four "takes effect at the next link" request maps. Snapshotted rather than
|
||||
// referenced, which is precisely what makes glBindAttribLocation and friends
|
||||
// legal to call over a pending link without cancelling it: the pending link keeps
|
||||
// linking the inputs it was given.
|
||||
UnorderedMap<String, Uint> explicitAttribLocations; // glBindAttribLocation
|
||||
UnorderedMap<String, Uint> explicitFragDataLocation; // glBindFragDataLocation
|
||||
UnorderedMap<String, Uint> explicitFragDataIndex; // glBindFragDataLocationIndexed
|
||||
Vector<String> requestedXfbVaryings; // glTransformFeedbackVaryings
|
||||
GLenum requestedXfbBufferMode = GL_INTERLEAVED_ATTRIBS;
|
||||
Int maxFragmentOutputColorNumber = 8; // GL_MAX_DRAW_BUFFERS, stamped in by the entry point
|
||||
} in;
|
||||
|
||||
// ---- output: valid iff IsComplete(), immutable afterwards ----
|
||||
// Moved (never copied) into the ProgramObject by EnsureLinkJoined().
|
||||
ProgramObject::LinkArtifacts artifacts;
|
||||
|
||||
// Posts this job once every compile in `deps` is terminal - and not one moment
|
||||
// earlier, so the body never waits on anything (invariant I4: no job body may block
|
||||
// on another job, or the pool could deadlock with all its workers waiting on each
|
||||
// other). `deps` is the subset of the snapshot's compile nodes that were still
|
||||
// in flight; an already-terminal one needs no edge.
|
||||
//
|
||||
// GL thread only, and only after the caller has stored a SharedPtr to this node:
|
||||
// OnDepSettled takes shared_from_this().
|
||||
void SubmitAfter(const Vector<SharedPtr<ShaderCompileTask>>& deps);
|
||||
|
||||
private:
|
||||
void RunBody() override;
|
||||
|
||||
// Runs when one dependency goes terminal - on whichever thread drove it there, which
|
||||
// is a pool worker for a compile that finished on one. Non-throwing by construction;
|
||||
// see the definition.
|
||||
void OnDepSettled();
|
||||
|
||||
// ---- the link body, split exactly as ProgramObject::Link() had it ----
|
||||
// Each returns false to abort the link with `artifacts.infoLog` already set, which is
|
||||
// GL's definition of a failed link: LINK_STATUS false plus a log, never a GL error.
|
||||
Bool ConsumeShaders(Vector<SharedPtr<glslang::TShader>>& outShaders);
|
||||
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
|
||||
Bool ValidateFragmentOutputLocations();
|
||||
Bool ResolveTransformFeedbackVaryings();
|
||||
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
|
||||
void GenerateSpirv();
|
||||
void BuildGlobalUboRouting();
|
||||
|
||||
// Worker-side MGLOG replacement: appended to diagnostics.logLines and replayed by the
|
||||
// join, on the GL thread, where a serial implementation would have printed it.
|
||||
// Logging straight from a worker interleaves mid-line with the GL thread's output and
|
||||
// lands out of order relative to the glLinkProgram that caused it.
|
||||
void DeferLog(String line);
|
||||
|
||||
// Counts down to zero exactly once. Starts at deps + 1: the extra guard is released
|
||||
// by SubmitAfter itself, so a dependency that settles while the edges are still being
|
||||
// registered cannot post the job from under a half-built dependency list.
|
||||
std::atomic<Int> m_remainingDeps{0};
|
||||
};
|
||||
} // namespace MobileGL::MG_State::GLState
|
||||
File diff suppressed because it is too large
Load Diff
@@ -14,9 +14,22 @@
|
||||
#include <MG_Util/ShaderTranspiler/SpvcSession.h>
|
||||
|
||||
namespace MobileGL::MG_State::GLState {
|
||||
// The link job. Only ever held by SharedPtr here, so a forward declaration is enough -
|
||||
// ProgramLinkTask.h includes THIS header (it outputs a LinkArtifacts), so including it
|
||||
// back would be circular. The destructor is therefore out of line.
|
||||
class ProgramLinkTask;
|
||||
|
||||
class ProgramObject {
|
||||
public:
|
||||
ProgramObject(Uint externalIndex) : m_externalIndex(externalIndex), m_lifetimeId(AllocateLifetimeId()) {}
|
||||
// Cancel-not-join, exactly like ~ShaderObject: the link job owns its inputs, so an
|
||||
// in-flight link whose program just went away is safe to abandon where it stands.
|
||||
// Nothing can observe its result any more - this object was the only route to it.
|
||||
// Out of line because ProgramLinkTask is incomplete here.
|
||||
~ProgramObject();
|
||||
ProgramObject(const ProgramObject&) = delete;
|
||||
ProgramObject& operator=(const ProgramObject&) = delete;
|
||||
|
||||
bool ShaderIsAttached(const SharedPtr<ShaderObject>& shader);
|
||||
// GL-visible attachment: in the attach list and not pending detach (glDetachShader
|
||||
// defers the actual removal to the next link).
|
||||
@@ -151,14 +164,7 @@ namespace MobileGL::MG_State::GLState {
|
||||
: -1;
|
||||
}
|
||||
|
||||
Bool IsValidUniformLocation(Int location) const {
|
||||
if (location < 0 || location > static_cast<Int>(Artifacts().maxUniformLocation)) return false;
|
||||
if (static_cast<SizeT>(location) >= Artifacts().uniformIndexInTProgram.size()) return false;
|
||||
const Int uniformIndexInProgram = Artifacts().uniformIndexInTProgram[location];
|
||||
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
|
||||
uniformIndexInProgram >= 0 &&
|
||||
uniformIndexInProgram < static_cast<Int>(Artifacts().tProgramUniformIndexToGl.size());
|
||||
}
|
||||
Bool IsValidUniformLocation(Int location) const { return IsValidUniformLocation(Artifacts(), location); }
|
||||
|
||||
GLenum GetUniformType(Uint location) const {
|
||||
auto& uniform = Artifacts().program->getUniform(Artifacts().uniformIndexInTProgram[location]);
|
||||
@@ -176,12 +182,7 @@ namespace MobileGL::MG_State::GLState {
|
||||
// for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
|
||||
// the artifacts' uniformIndexInTProgram stores).
|
||||
GLint GetUniformArraySizeByTIndex(Int tIndex) const {
|
||||
const auto& uniform = Artifacts().program->getUniform(tIndex);
|
||||
const glslang::TType* type = uniform.getType();
|
||||
if (type != nullptr && type->isSizedArray()) {
|
||||
return type->getOuterArraySize();
|
||||
}
|
||||
return uniform.size < 1 ? 1 : uniform.size;
|
||||
return GetUniformArraySizeByTIndex(Artifacts(), tIndex);
|
||||
}
|
||||
|
||||
GLint GetActiveUniformArraySize(Uint index) const {
|
||||
@@ -371,7 +372,7 @@ namespace MobileGL::MG_State::GLState {
|
||||
// can skip re-uploading an unchanged UBO on every draw. ~0u is reserved as the
|
||||
// backends' "never uploaded" sentinel, so skip over it on wrap.
|
||||
Uint32 GetUBOContentVersion() const { return m_uboContentVersion; }
|
||||
void MarkUBOContentDirty() {
|
||||
void MarkUBOContentDirty() const {
|
||||
if (++m_uboContentVersion == ~0u) m_uboContentVersion = 0;
|
||||
}
|
||||
Uint32 GetBackendStateVersion() const { return m_backendStateVersion; }
|
||||
@@ -439,8 +440,13 @@ namespace MobileGL::MG_State::GLState {
|
||||
// glProgramBinary always fails here (there is no format it could accept) and the
|
||||
// spec then requires the program's LINK_STATUS to read FALSE.
|
||||
void MarkLinkFailedByProgramBinary() {
|
||||
// Before anything reads m_artifacts: a pending link would otherwise publish its
|
||||
// (possibly successful) result over the failure this call is required to install
|
||||
// - and Artifacts() below would be the thing that let it. Cancel-not-join: GL
|
||||
// gives glProgramBinary no reason to wait for a link it is about to invalidate.
|
||||
CancelLink();
|
||||
BumpLinkObservableVersions();
|
||||
ResetLinkArtifacts();
|
||||
ResetLinkArtifacts(Artifacts());
|
||||
Artifacts().infoLog = "No program binary format is supported.";
|
||||
}
|
||||
Bool GetValidateStatus() const { return m_validateStatus; }
|
||||
@@ -633,12 +639,68 @@ namespace MobileGL::MG_State::GLState {
|
||||
Uint32 xfbPackedStride = 0;
|
||||
};
|
||||
|
||||
// Blocks until a pending link (P1 stage 4 onwards) has published its artifacts.
|
||||
// Public because a few call sites have to join without reading anything - see the
|
||||
// explicit-join list in the P1 design. Today there is never a pending link, so this
|
||||
// is a no-op; it is wired up when glLinkProgram starts enqueueing.
|
||||
// ---- artifacts-only helpers, shared with ProgramLinkTask ----
|
||||
// Static and taking the block explicitly, because from stage 4 the link BODY needs
|
||||
// them while its artifacts still live on the job node, not on any ProgramObject. The
|
||||
// member overloads above are the same functions read through the join gate.
|
||||
|
||||
// Clears every field one link produces, EXCEPT infoLog, linkedFragDataLocation/Index
|
||||
// and the geometry strip-capture pair. That exception is load-bearing: the callers
|
||||
// that survive (glProgramBinary's mandated failure, and the link body's own mid-link
|
||||
// aborts) write infoLog immediately AFTER calling here. Link()'s prologue does not
|
||||
// use this at all - it assigns a whole default-constructed LinkArtifacts, where the
|
||||
// ordering is explicit and nothing is exempt.
|
||||
static void ResetLinkArtifacts(LinkArtifacts& artifacts);
|
||||
|
||||
static Bool IsValidUniformLocation(const LinkArtifacts& artifacts, Int location) {
|
||||
if (location < 0 || location > static_cast<Int>(artifacts.maxUniformLocation)) return false;
|
||||
if (static_cast<SizeT>(location) >= artifacts.uniformIndexInTProgram.size()) return false;
|
||||
const Int uniformIndexInProgram = artifacts.uniformIndexInTProgram[location];
|
||||
return uniformIndexInProgram != glslang::TQualifier::layoutLocationEnd &&
|
||||
uniformIndexInProgram >= 0 &&
|
||||
uniformIndexInProgram < static_cast<Int>(artifacts.tProgramUniformIndexToGl.size());
|
||||
}
|
||||
|
||||
// Number of active array elements (GL_UNIFORM_SIZE / GL_ARRAY_SIZE); 1 for a non-array.
|
||||
// glslang's TObjectReflection.size only carries the element count for a NON-block array; for
|
||||
// a block array member it reports 1, so take the count from the TType, which is authoritative
|
||||
// for both. GL 3.3 core uniforms are always sized. Takes a TProgram uniform index (the space
|
||||
// the artifacts' uniformIndexInTProgram stores).
|
||||
static GLint GetUniformArraySizeByTIndex(const LinkArtifacts& artifacts, Int tIndex) {
|
||||
const auto& uniform = artifacts.program->getUniform(tIndex);
|
||||
const glslang::TType* type = uniform.getType();
|
||||
if (type != nullptr && type->isSizedArray()) {
|
||||
return type->getOuterArraySize();
|
||||
}
|
||||
return uniform.size < 1 ? 1 : uniform.size;
|
||||
}
|
||||
|
||||
// Blocks until a pending link has published its artifacts. Public because a few call
|
||||
// sites have to join without reading anything - see the explicit-join list (J1-J8) in
|
||||
// the P1 design. GL thread only.
|
||||
void JoinLink() const { EnsureLinkJoined(); }
|
||||
|
||||
// Drops a link that is still in flight, without waiting for it. Called at the points
|
||||
// where the pending link's result stops being the answer to "what did this program
|
||||
// link to": a re-link supersedes it, glProgramBinary must force LINK_STATUS false,
|
||||
// and a destroyed program has no observers left.
|
||||
//
|
||||
// Deliberately NOT called by the "takes effect at the next link" setters
|
||||
// (glBindAttribLocation, glBindFragDataLocation(Indexed), glTransformFeedbackVaryings,
|
||||
// glProgramParameteri) NOR by glAttachShader/glDetachShader. Every one of those is
|
||||
// defined by GL to leave the CURRENT link result alone, and the pending link already
|
||||
// snapshotted its own inputs at enqueue, so it is computing exactly the answer GL
|
||||
// requires. Cancelling on any of them would make
|
||||
// glLinkProgram(p); <setter>; glGetProgramiv(p, GL_LINK_STATUS)
|
||||
// report FALSE for a link that succeeded - and for the attach/detach pair it would
|
||||
// additionally break glCreateShaderProgramv, which detaches immediately after linking.
|
||||
void CancelLink();
|
||||
|
||||
// MUST NOT JOIN - this is what GL_COMPLETION_STATUS_KHR reads when the extension
|
||||
// surface lands. "No job at all" counts as complete: there is nothing outstanding to
|
||||
// wait for.
|
||||
Bool IsLinkComplete() const { return m_pendingLink == nullptr || IsPendingLinkTerminal(); }
|
||||
|
||||
void SetTransformFeedbackVaryings(Vector<String>&& names, GLenum bufferMode) {
|
||||
m_requestedXfbVaryings = Move(names);
|
||||
m_requestedXfbBufferMode = bufferMode;
|
||||
@@ -685,19 +747,22 @@ namespace MobileGL::MG_State::GLState {
|
||||
private:
|
||||
// ---- The one and only join gate for link output (P1 invariant I5) ----
|
||||
// Blocks until a pending link has finished and its LinkArtifacts have been
|
||||
// published into m_artifacts. Today no link is ever pending - glLinkProgram still
|
||||
// runs the whole body inline - so this is an unconditional no-op, and the whole
|
||||
// Artifacts() indirection compiles away. It exists NOW so that the ~120 readers of
|
||||
// link output are already routed through it when stage 4 makes it block: the edit
|
||||
// that turns links asynchronous then touches this function and nothing else.
|
||||
// published into m_artifacts. It exists so that the ~120 readers of link output are
|
||||
// routed through it by the compiler rather than by review: m_artifacts is private
|
||||
// and Artifacts() is the only spelling that reaches it.
|
||||
//
|
||||
// Defined inline (not in ProgramObject.cpp) on purpose: this is called from every
|
||||
// Artifacts() read - ~1200 call sites project-wide - and the project never builds
|
||||
// with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line empty body would leave a
|
||||
// real cross-TU call at every one of them instead of folding away. Stage 4's
|
||||
// version, which actually blocks, moves the wait itself out-of-line behind a
|
||||
// `m_pendingLink` check that stays inline here.
|
||||
void EnsureLinkJoined() const {}
|
||||
// The fast path - no pending link - is one predictable branch and stays inline: it
|
||||
// runs on every Artifacts() read (~1200 call sites project-wide) and the project
|
||||
// never builds with LTO (MOBILEGL_ENABLE_LTO=OFF), so an out-of-line body would be a
|
||||
// real cross-TU call at every one of them. The blocking half is out of line.
|
||||
void EnsureLinkJoined() const {
|
||||
if (m_pendingLink) JoinPendingLink();
|
||||
}
|
||||
void JoinPendingLink() const;
|
||||
// ProgramLinkTask is incomplete here, so IsLinkComplete()'s non-joining peek at the
|
||||
// node's state goes through this out-of-line helper.
|
||||
Bool IsPendingLinkTerminal() const;
|
||||
|
||||
LinkArtifacts& Artifacts() {
|
||||
EnsureLinkJoined();
|
||||
return m_artifacts;
|
||||
@@ -707,28 +772,10 @@ namespace MobileGL::MG_State::GLState {
|
||||
return m_artifacts;
|
||||
}
|
||||
|
||||
void ResetLinkArtifacts();
|
||||
// GL-thread-only companion to ResetLinkArtifacts (see its definition).
|
||||
void BumpLinkObservableVersions();
|
||||
// Builds the GL-facing reflection surface from the linked TProgram. Returns false
|
||||
// (with the artifacts' infoLog set and link artifacts reset) when reflection itself fails or an
|
||||
// explicit-uniform-location conflict makes the link invalid.
|
||||
Bool DoReflection(const MG_Util::ShaderTranspiler::CompileEnv& env);
|
||||
// Resolves the requested transform feedback varyings against the linked
|
||||
// vertex stage; fails the link (GL semantics) on unknown or duplicate
|
||||
// names or exceeded capture limits.
|
||||
Bool ResolveTransformFeedbackVaryings();
|
||||
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
|
||||
// The former GenerateBinary, split around DoReflection's data dependencies:
|
||||
// SPIR-V must be generated BEFORE buildReflection touches the linked TProgram (its
|
||||
// live-variable analysis mutates the intermediates enough to change
|
||||
// GlslangToSpv output), while the glUniform*-to-global-UBO routing tables are
|
||||
// sized and keyed by reflection results (maxUniformLocation, uniformLocations)
|
||||
// and so must run AFTER it.
|
||||
void GenerateSpirv();
|
||||
void BuildGlobalUboRouting();
|
||||
// GL-thread-only companion to ResetLinkArtifacts (see its definition). Const because
|
||||
// the publish half of the join calls it; see the mutable counters below.
|
||||
void BumpLinkObservableVersions() const;
|
||||
void AddDefaultFragmentShaderIfMissing();
|
||||
Bool ValidateFragmentOutputLocations();
|
||||
|
||||
static Uint64 AllocateLifetimeId();
|
||||
|
||||
@@ -762,7 +809,10 @@ namespace MobileGL::MG_State::GLState {
|
||||
Bool m_binaryRetrievableHint = false;
|
||||
Bool m_separable = false;
|
||||
Bool m_validateStatus = true;
|
||||
Uint32 m_backendStateVersion = 0;
|
||||
// Mutable, like m_artifacts and for the same reason: publishing a pending link is a
|
||||
// READ-side operation (the first gated getter is what pulls the result in), and the
|
||||
// publish has to bump these. Still GL-thread-only - a worker never touches them.
|
||||
mutable Uint32 m_backendStateVersion = 0;
|
||||
|
||||
// Backend-owned content-hash memo (see GetBackendHashMemo): valid only while
|
||||
// m_backendStateVersion matches. Several slots, not one: a backend may resolve the same
|
||||
@@ -778,12 +828,20 @@ namespace MobileGL::MG_State::GLState {
|
||||
mutable Array<BackendHashMemoSlot, kBackendHashMemoSlotCount> m_backendHashMemoSlots{};
|
||||
mutable SizeT m_backendHashMemoNextSlot = 0;
|
||||
mutable Uint32 m_backendHashMemoVersion = ~0u;
|
||||
Uint32 m_uboContentVersion = 0;
|
||||
Uint32 m_linkVersion = 0;
|
||||
mutable Uint32 m_uboContentVersion = 0;
|
||||
mutable Uint32 m_linkVersion = 0;
|
||||
|
||||
// ---- Link OUTPUT ----
|
||||
// Written by the link and by the post-link setters GL allows (glUniform1i's sampler
|
||||
// unit, glUniformBlockBinding). Reachable only through Artifacts(); see LinkArtifacts.
|
||||
LinkArtifacts m_artifacts;
|
||||
//
|
||||
// Mutable because publishing is a READ-side operation: a const getter has to be able
|
||||
// to settle an outstanding link before answering it.
|
||||
mutable LinkArtifacts m_artifacts;
|
||||
|
||||
// The link job, from enqueue until the first observable read pulls its result. Null
|
||||
// means m_artifacts is already the answer - which is the state every reader outside
|
||||
// the pending window sees, and the whole reason the gate above is one branch.
|
||||
mutable SharedPtr<ProgramLinkTask> m_pendingLink;
|
||||
};
|
||||
} // namespace MobileGL::MG_State::GLState
|
||||
|
||||
@@ -39,6 +39,14 @@ namespace MobileGL::MG_State::GLState {
|
||||
|
||||
void ProgramState::DestroyProgramSlot(const Uint program) {
|
||||
auto& programObject = m_programObjects[program];
|
||||
// P1 join site J4/J5 (glDeleteProgram, and the deferred destroy UseProgram performs
|
||||
// when a deletion-flagged program stops being current). The program's name is about
|
||||
// to go, so nothing can observe its link any more: cancel-not-join, so a delete never
|
||||
// blocks the GL thread on a worker. Explicit rather than left to ~ProgramObject,
|
||||
// because the reset below only destroys the object if this table held the last
|
||||
// reference - a program still bound as current, or still referenced by a pipeline,
|
||||
// outlives it, and its link should stop the moment the name does.
|
||||
programObject->CancelLink();
|
||||
// Snapshot the attachments: deleting the program is a detach point for shaders
|
||||
// that were flagged with glDeleteShader while still attached.
|
||||
const Vector<SharedPtr<ShaderObject>> attachedShaders = programObject->GetAttachedShaders();
|
||||
|
||||
@@ -185,24 +185,16 @@ namespace {
|
||||
return result;
|
||||
}
|
||||
|
||||
// glslang has no "detach this thread" API in the vendored revision (there is no
|
||||
// InitThread/DetachThread pair any more; thread attachment is implicit through
|
||||
// thread_local state, and glslang::InitializeProcess() is process-wide, refcounted and
|
||||
// mutex-guarded, so it needs no per-worker counterpart).
|
||||
//
|
||||
// What DOES need undoing is the thread pool allocator: TShader::parse sets the calling
|
||||
// thread's TLS allocator to the shader's own pool and never restores it. Left pointing
|
||||
// there, the next allocation this worker makes - in an unrelated job, or in glslang code
|
||||
// reached from a different object - would come out of a pool the GL thread may already
|
||||
// have deleted with the TShader. SetThreadPoolAllocator(nullptr) reverts the thread to
|
||||
// its own thread_local default and is the documented idiom. A scope guard, so it also
|
||||
// runs when a body throws.
|
||||
struct GlslangThreadAllocatorGuard {
|
||||
~GlslangThreadAllocatorGuard() { glslang::SetThreadPoolAllocator(nullptr); }
|
||||
};
|
||||
} // namespace
|
||||
|
||||
namespace MobileGL::MG_State::GLState {
|
||||
// glslang has no "detach this thread" API in the vendored revision (there is no
|
||||
// InitThread/DetachThread pair any more; thread attachment is implicit through
|
||||
// thread_local state, and glslang::InitializeProcess() is process-wide, refcounted and
|
||||
// mutex-guarded, so it needs no per-worker counterpart). The pool allocator is the part
|
||||
// that needs undoing; see the declaration in ShaderCompileTask.h.
|
||||
GlslangThreadAllocatorGuard::~GlslangThreadAllocatorGuard() { glslang::SetThreadPoolAllocator(nullptr); }
|
||||
|
||||
// Pure CPU work only. Everything this reads is either an input the node owns or a
|
||||
// process-wide constant; everything it writes is `artifacts`. Do not add a GL/EGL call,
|
||||
// a pActiveBackendObject read, or a pGLContext->RecordError() here - the first two are
|
||||
@@ -300,4 +292,47 @@ namespace MobileGL::MG_State::GLState {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
SharedPtr<glslang::TShader> ShaderCompileTask::ClaimParsedShader(String& outReparseLog) const {
|
||||
MOBILEGL_ASSERT(IsComplete(),
|
||||
"ShaderCompileTask::ClaimParsedShader() on a job that has not completed; its artifacts "
|
||||
"are still being written");
|
||||
|
||||
if (artifacts.shader) {
|
||||
// The whole race, in one instruction. Acquire-release because the winner is about
|
||||
// to hand the TShader to glslang's linker on a possibly different thread from the
|
||||
// one that parsed it - the node's terminal transition already published the
|
||||
// parse, and this orders the two claimants against each other.
|
||||
Bool expected = false;
|
||||
if (m_parseClaimed.compare_exchange_strong(expected, true, std::memory_order_acq_rel,
|
||||
std::memory_order_acquire)) {
|
||||
return artifacts.shader;
|
||||
}
|
||||
}
|
||||
|
||||
// Either another link already consumed the stored parse (and mapIO mutated its
|
||||
// intermediate), or there never was one. Re-parse the preprocessed source through the
|
||||
// identical configuration; that costs one glslang parse, which is what GenerateBinary
|
||||
// used to spend here on EVERY link rather than only on reuse.
|
||||
//
|
||||
// The guard is not optional on this path: from stage 4 this runs on a pool worker,
|
||||
// and TShader::parse would leave that worker's TLS allocator pointing at a pool the
|
||||
// GL thread is about to free. (ProgramLinkTask::RunBody holds one too; they nest
|
||||
// harmlessly - both just reset the thread to its own default.)
|
||||
const GlslangThreadAllocatorGuard glslangGuard;
|
||||
using namespace MG_Util::ShaderTranspiler;
|
||||
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(stage),
|
||||
.sourceStr = artifacts.preprocessedSource,
|
||||
.flags = 0,
|
||||
// Re-parse against the SAME environment the original parse used,
|
||||
// not against whatever the backend reports now.
|
||||
.env = artifacts.env.get()};
|
||||
auto result = ShaderCompiler::CompileShader(attrib);
|
||||
if (!result) {
|
||||
// Should be unreachable: the same source parsed successfully at Compile().
|
||||
outReparseLog = result.error().log;
|
||||
return nullptr;
|
||||
}
|
||||
return result.value();
|
||||
}
|
||||
} // namespace MobileGL::MG_State::GLState
|
||||
|
||||
@@ -13,6 +13,23 @@
|
||||
#include <MG_State/GLState/ProgramState/ShaderPreprocessCache.h>
|
||||
|
||||
namespace MobileGL::MG_State::GLState {
|
||||
// glslang has no "detach this thread" API in the vendored revision, but TShader::parse
|
||||
// leaves the calling thread's TLS pool allocator pointing at the shader's own pool and
|
||||
// never restores it. Left there, the next allocation this thread makes - in an unrelated
|
||||
// job, or in glslang code reached from a different object - would come out of a pool the
|
||||
// GL thread may already have deleted with the TShader. SetThreadPoolAllocator(nullptr)
|
||||
// reverts the thread to its own thread_local default and is the documented idiom.
|
||||
//
|
||||
// A scope guard, so it also runs when a body throws. Declared here rather than kept
|
||||
// file-local because stage 4 gave it a second user: ProgramLinkTask's body parses (the
|
||||
// claim-CAS loser's re-parse), links and emits SPIR-V, all on a pool thread.
|
||||
struct GlslangThreadAllocatorGuard {
|
||||
GlslangThreadAllocatorGuard() = default;
|
||||
~GlslangThreadAllocatorGuard();
|
||||
GlslangThreadAllocatorGuard(const GlslangThreadAllocatorGuard&) = delete;
|
||||
GlslangThreadAllocatorGuard& operator=(const GlslangThreadAllocatorGuard&) = delete;
|
||||
};
|
||||
|
||||
// Everything one glCompileShader PRODUCES, in one block.
|
||||
//
|
||||
// This is exactly the set a single run of the compile pipeline writes, which is what
|
||||
@@ -21,21 +38,16 @@ namespace MobileGL::MG_State::GLState {
|
||||
// it in, and the GL thread reads it through ShaderObject's join gate.
|
||||
struct ShaderCompileArtifacts {
|
||||
// The CompileEnv snapshot this compile ran against. Held so the consume-once
|
||||
// re-parse in TakeShaderForLink() reproduces the original parse exactly, instead of
|
||||
// re-parse in ClaimParsedShader() reproduces the original parse exactly, instead of
|
||||
// re-reading whatever the backend says now.
|
||||
SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env;
|
||||
SharedPtr<glslang::TShader> shader;
|
||||
// The source the parse actually consumed (after PreprocessShaderSource), kept for
|
||||
// TakeShaderForLink's re-parse so a later link never depends on the preprocessor
|
||||
// ClaimParsedShader's re-parse so a later link never depends on the preprocessor
|
||||
// being deterministic across backend-state changes.
|
||||
String preprocessedSource;
|
||||
UnorderedMap<String, Int> explicitUniformLocations;
|
||||
UnorderedMap<String, Uint> explicitOpaqueBindings;
|
||||
// GL-thread-owned, and the one field here a worker never touches: TakeShaderForLink
|
||||
// flips it after the join. Stage 4 replaces it with an atomic claim on this node,
|
||||
// because two ProgramLinkTasks for two programs sharing this shader can then race
|
||||
// for the parse on two workers.
|
||||
Bool shaderConsumedByLink = false;
|
||||
String infoLog;
|
||||
Bool compileStatus = false;
|
||||
};
|
||||
@@ -75,9 +87,50 @@ namespace MobileGL::MG_State::GLState {
|
||||
// ---- output: valid iff IsComplete(), immutable afterwards ----
|
||||
ShaderCompileArtifacts artifacts;
|
||||
|
||||
// Hands out a link-consumable TShader, exactly once for the stored parse.
|
||||
//
|
||||
// glslang's mapIO mutates the TShader's aliased intermediate, so the parse this node
|
||||
// produced may feed exactly ONE link; every later link (a relink, or the same shader
|
||||
// attached to a second program) needs a fresh parse. The claim is a CAS on this
|
||||
// shared node rather than a flag on the ShaderObject because from stage 4 the two
|
||||
// callers can be two ProgramLinkTasks running on two workers: two programs sharing
|
||||
// one shader, linked back to back. Copying the parse out and tracking consumed-ness
|
||||
// per program would let both of them decide they were the first, run mapIO over the
|
||||
// same intermediate twice, and ship silently corrupt SPIR-V.
|
||||
//
|
||||
// The CAS loser re-parses artifacts.preprocessedSource against THIS node's own
|
||||
// CompileEnv (not against whatever the backend reports now), through the identical
|
||||
// CompileShader path - so winner and loser produce byte-identical SPIR-V. Callable
|
||||
// only once IsComplete() and compileStatus are true. Returns null only if that
|
||||
// re-parse fails, and outReparseLog then carries its diagnostics.
|
||||
//
|
||||
// Const because the claim is the node's own synchronization, not a mutation of its
|
||||
// published artifacts: a claim that is taken and then abandoned (its link was
|
||||
// cancelled) costs one extra re-parse later and nothing else.
|
||||
SharedPtr<glslang::TShader> ClaimParsedShader(String& outReparseLog) const;
|
||||
|
||||
// Sticky marker for "a ProgramLinkTask has this node in its input snapshot".
|
||||
//
|
||||
// It exists to keep a cancel from eating a result someone still needs. A pending link
|
||||
// holds its dependencies by SharedPtr, so the NODE always outlives the ShaderObject -
|
||||
// but Cancel() is not about lifetime, it discards the result. The reachable sequence
|
||||
// is the ordinary one: compile, attach, glLinkProgram (enqueued), glDetachShader,
|
||||
// glDeleteShader. The detach makes the shader GL-invisible, so the delete frees its
|
||||
// name, and ReleaseShaderNameIfOrphaned would cancel a compile the enqueued link is
|
||||
// waiting on - turning a link that must report GL_TRUE into GL_FALSE. Set on the GL
|
||||
// thread in Link()'s prologue, read on the GL thread by ShaderObject::CancelCompile.
|
||||
//
|
||||
// Never cleared: the worst case is one stale node compiling to completion for nobody,
|
||||
// which is exactly what the pre-stage-3 implementation always did.
|
||||
void MarkLinkReferenced() { m_linkReferenced.store(true, std::memory_order_release); }
|
||||
Bool IsLinkReferenced() const { return m_linkReferenced.load(std::memory_order_acquire); }
|
||||
|
||||
private:
|
||||
void RunBody() override;
|
||||
// The real body; RunBody wraps it so a throw becomes a GL-visible compile failure.
|
||||
void RunCompilePipeline();
|
||||
|
||||
mutable std::atomic<Bool> m_parseClaimed{false};
|
||||
std::atomic<Bool> m_linkReferenced{false};
|
||||
};
|
||||
} // namespace MobileGL::MG_State::GLState
|
||||
|
||||
@@ -77,7 +77,14 @@ namespace MobileGL::MG_State::GLState {
|
||||
// Cooperative and non-blocking. A node that no worker has picked up settles
|
||||
// immediately; one that is running is flagged and settles when its body returns,
|
||||
// writing only into itself the whole time.
|
||||
m_compiled->Cancel();
|
||||
//
|
||||
// Unless a pending LINK is waiting on it. Cancelling is about discarding a result
|
||||
// nothing can observe any more, and this object is no longer the only route to this
|
||||
// one: an enqueued ProgramLinkTask holds the node as a dependency, and a cancel would
|
||||
// turn its link into GL_FALSE. Reached by the ordinary link-then-detach-then-delete
|
||||
// shader teardown - see ShaderCompileTask::MarkLinkReferenced. Dropping our own
|
||||
// reference is still right; the link keeps the node alive and finishes it.
|
||||
if (!m_compiled->IsLinkReferenced()) m_compiled->Cancel();
|
||||
m_compiled.reset();
|
||||
}
|
||||
|
||||
@@ -91,7 +98,7 @@ namespace MobileGL::MG_State::GLState {
|
||||
// The failure case is covered too: the info log stays queryable because nothing is
|
||||
// cleared. And if the stored TShader already fed a link, the no-op leaves
|
||||
// preprocessedSource and both side-channel maps intact, which is precisely what
|
||||
// TakeShaderForLink's on-demand re-parse needs - a real recompile would have handed
|
||||
// ClaimParsedShader's on-demand re-parse needs - a real recompile would have handed
|
||||
// the next link a fresh parse, the no-op hands it a fresh re-parse of the identical
|
||||
// source instead. Same result, one parse either way.
|
||||
if (HasMemoizedCompile()) return;
|
||||
@@ -119,43 +126,6 @@ namespace MobileGL::MG_State::GLState {
|
||||
MG_Util::Async::ShaderCompilePool::Get().Post(m_compiled);
|
||||
}
|
||||
|
||||
SharedPtr<glslang::TShader> ShaderObject::TakeShaderForLink(String& outReparseLog) {
|
||||
EnsureCompileJoined();
|
||||
// Unreachable through the GL frontend: callers gate on GetCompileStatus().
|
||||
if (!m_compiled) return nullptr;
|
||||
|
||||
// Mutating the node's artifacts from here is legal precisely because the node is
|
||||
// terminal by now: no worker will ever touch it again, and this thread is the GL
|
||||
// thread. Stage 4, where two link JOBS can reach the same node concurrently,
|
||||
// replaces this flag with an atomic claim on the node.
|
||||
ShaderCompileArtifacts& compiled = m_compiled->artifacts;
|
||||
if (compiled.shader && !compiled.shaderConsumedByLink) {
|
||||
compiled.shaderConsumedByLink = true;
|
||||
return compiled.shader;
|
||||
}
|
||||
|
||||
// The stored parse already fed a link, whose mapIO mutated its intermediate.
|
||||
// Re-parse the preprocessed source through the identical configuration; this
|
||||
// costs one glslang parse, which is exactly what GenerateBinary used to spend
|
||||
// here on EVERY link rather than only on reuse.
|
||||
using namespace MG_Util::ShaderTranspiler;
|
||||
ShaderAttrib attrib{.shaderType = MG_Util::ConvertShaderStageToGLEnum(m_stage),
|
||||
.sourceStr = compiled.preprocessedSource,
|
||||
.flags = 0,
|
||||
// Re-parse against the SAME environment the original parse used,
|
||||
// not against whatever the backend reports now.
|
||||
.env = compiled.env.get()};
|
||||
auto result = ShaderCompiler::CompileShader(attrib);
|
||||
if (!result) {
|
||||
// Should be unreachable: the same source parsed successfully at Compile().
|
||||
outReparseLog = result.error().log;
|
||||
MGLOG_E("ShaderObject::TakeShaderForLink: re-parse of shader %d failed:\n%s", m_externalIndex,
|
||||
outReparseLog.c_str());
|
||||
return nullptr;
|
||||
}
|
||||
return result.value();
|
||||
}
|
||||
|
||||
void ShaderObject::MarkAsDeleted() {
|
||||
m_deleteStatus = true;
|
||||
}
|
||||
|
||||
@@ -51,14 +51,17 @@ namespace MobileGL {
|
||||
void CancelCompile();
|
||||
void MarkAsDeleted();
|
||||
|
||||
// Hands out a link-consumable TShader. glslang's mapIO mutates the TShader's
|
||||
// aliased intermediate, so the parse stored by Compile() may feed exactly one
|
||||
// link; every later link (relink, or the same shader attached to a second
|
||||
// program) gets a fresh parse of the stored preprocessed source through the
|
||||
// byte-identical CompileShader path (including the legacy-460 retry). Only
|
||||
// callable while GetCompileStatus() is true. Returns null only if that
|
||||
// re-parse fails - outReparseLog then carries its diagnostics.
|
||||
SharedPtr<glslang::TShader> TakeShaderForLink(String& outReparseLog);
|
||||
// The compile job node itself, for ProgramObject::Link()'s input snapshot.
|
||||
// DELIBERATELY DOES NOT JOIN, and that is the entire point of stage 4: the link
|
||||
// takes the node as a dependency and is posted only once the node is terminal,
|
||||
// so glLinkProgram never blocks on glCompileShader. Null means this object has
|
||||
// never been compiled (or its last compile was abandoned), which the link reads
|
||||
// as COMPILE_STATUS false - the same verdict the joining path produces.
|
||||
//
|
||||
// The caller must MarkLinkReferenced() whatever it keeps: from here on the node's
|
||||
// result has an observer this object knows nothing about (see the marker's
|
||||
// comment in ShaderCompileTask.h).
|
||||
const SharedPtr<ShaderCompileTask>& CompiledNodeForLink() const { return m_compiled; }
|
||||
|
||||
Uint GetExternalIndex() const { return m_externalIndex; }
|
||||
ShaderStage GetShaderStage() const { return m_stage; }
|
||||
|
||||
@@ -0,0 +1,749 @@
|
||||
// MobileGL - MobileGL/MG_Test/Program/AsyncLinkTest.cpp
|
||||
// Copyright (c) 2025-2026 MobileGL-Dev
|
||||
// Licensed under the GNU Lesser General Public License v3.0:
|
||||
// https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
// https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
// SPDX-License-Identifier: LGPL-3.0-only
|
||||
// End of Source File Header
|
||||
|
||||
// P1 stage 4: glLinkProgram enqueues a ProgramLinkTask behind its shaders' compiles, and
|
||||
// every observable read of link output joins.
|
||||
//
|
||||
// Like AsyncCompileTest, every case here drives the real GL entry points and flips
|
||||
// MG_Config::Features.AsyncShaderCompile itself rather than reading the environment - so one
|
||||
// binary can assert the property that actually matters (the async and synchronous paths are
|
||||
// indistinguishable through the GL surface) regardless of how the suite was launched.
|
||||
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "Config.h"
|
||||
#include "Includes.h"
|
||||
#include "Init.h"
|
||||
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
|
||||
#include "MG_Impl/GLImpl/Program/GL_Program.h"
|
||||
#include "MG_Impl/GLImpl/Program/GL_ProgramPipeline.h"
|
||||
#include "MG_State/GLState/Core.h"
|
||||
#include "MG_Util/Async/ShaderCompilePool.h"
|
||||
|
||||
using namespace MobileGL;
|
||||
using namespace MobileGL::MG_Impl::GLImpl;
|
||||
|
||||
namespace {
|
||||
class AsyncModeScope {
|
||||
public:
|
||||
explicit AsyncModeScope(const Bool async) : m_saved(MG_Config::Features.AsyncShaderCompile) {
|
||||
MG_Config::Features.AsyncShaderCompile =
|
||||
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
|
||||
}
|
||||
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
|
||||
AsyncModeScope(const AsyncModeScope&) = delete;
|
||||
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
|
||||
|
||||
private:
|
||||
const MG_Config::QuirkOverride m_saved;
|
||||
};
|
||||
|
||||
const char* kVs = R"(#version 460
|
||||
layout(location = 0) in vec3 aPos;
|
||||
uniform mat4 uModel;
|
||||
uniform vec4 uColor;
|
||||
out vec4 vColor;
|
||||
void main() {
|
||||
vColor = uColor;
|
||||
gl_Position = uModel * vec4(aPos, 1.0);
|
||||
}
|
||||
)";
|
||||
|
||||
const char* kFs = R"(#version 460
|
||||
in vec4 vColor;
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
uniform float uAlpha;
|
||||
void main() { fragColor = vec4(vColor.rgb, vColor.a * uAlpha); }
|
||||
)";
|
||||
|
||||
// A vertex shader that captures something transform feedback can name.
|
||||
const char* kXfbVs = R"(#version 460
|
||||
layout(location = 0) in vec3 aPos;
|
||||
out vec3 vWorld;
|
||||
void main() {
|
||||
vWorld = aPos * 2.0;
|
||||
gl_Position = vec4(aPos, 1.0);
|
||||
}
|
||||
)";
|
||||
|
||||
const char* kBrokenFs = R"(#version 460
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
void main() { fragColor = thisIdentifierWasNeverDeclared; }
|
||||
)";
|
||||
|
||||
// Big enough that neither the compile nor the link is instantaneous, so the pool has a
|
||||
// real backlog to race against. Templated on an index so every instance is distinct
|
||||
// source text (no P0b memo hit).
|
||||
String MakeBulkySource(const int index) {
|
||||
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
|
||||
source += "uniform float uSeed" + std::to_string(index) + ";\n";
|
||||
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
|
||||
for (int i = 0; i < 220; ++i) {
|
||||
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
|
||||
}
|
||||
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
|
||||
return source;
|
||||
}
|
||||
|
||||
GLuint MakeShader(const GLenum type, const char* source) {
|
||||
const GLuint shader = CreateShader(type);
|
||||
ShaderSource(shader, 1, &source, nullptr);
|
||||
CompileShader(shader);
|
||||
return shader;
|
||||
}
|
||||
|
||||
GLint QueryLinkStatus(const GLuint program) {
|
||||
GLint status = GL_FALSE;
|
||||
GetProgramiv(program, GL_LINK_STATUS, &status);
|
||||
return status;
|
||||
}
|
||||
|
||||
String QueryProgramInfoLog(const GLuint program) {
|
||||
GLint length = 0;
|
||||
GetProgramiv(program, GL_INFO_LOG_LENGTH, &length);
|
||||
if (length <= 0) return String();
|
||||
std::vector<GLchar> buffer(static_cast<size_t>(length));
|
||||
GLsizei written = 0;
|
||||
GetProgramInfoLog(program, length, &written, buffer.data());
|
||||
return String(buffer.data(), static_cast<size_t>(written));
|
||||
}
|
||||
|
||||
// The non-joining view of the program, i.e. what GL_COMPLETION_STATUS_KHR will report.
|
||||
Bool LinkIsSettled(const GLuint program) {
|
||||
const auto& object = MG_State::pGLContext->GetProgramObject(program);
|
||||
return object == nullptr || object->IsLinkComplete();
|
||||
}
|
||||
|
||||
// Enqueues `count` distinct heavy compiles without reading anything back, so the pool is
|
||||
// left with a real backlog for the caller to race against.
|
||||
Vector<GLuint> SaturatePool(const int count, Vector<String>& sourceStorage) {
|
||||
Vector<GLuint> shaders;
|
||||
shaders.reserve(static_cast<SizeT>(count));
|
||||
sourceStorage.reserve(sourceStorage.size() + static_cast<SizeT>(count));
|
||||
for (int i = 0; i < count; ++i) {
|
||||
sourceStorage.push_back(MakeBulkySource(20000 + i));
|
||||
const char* text = sourceStorage.back().c_str();
|
||||
const GLuint shader = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(shader, 1, &text, nullptr);
|
||||
CompileShader(shader);
|
||||
shaders.push_back(shader);
|
||||
}
|
||||
return shaders;
|
||||
}
|
||||
|
||||
// Content hash of a linked program's generated SPIR-V, through the state layer (there is
|
||||
// no GL query for it). Joins, like every other artifact read.
|
||||
Vector<Uint64> SpirvDigest(const GLuint program) {
|
||||
const auto& object = MG_State::pGLContext->GetProgramObject(program);
|
||||
Vector<Uint64> digest;
|
||||
if (!object) return digest;
|
||||
for (const auto& module : object->GetGeneratedSpirv()) {
|
||||
Uint64 hash = 1469598103934665603ull;
|
||||
for (const unsigned word : module) {
|
||||
hash = (hash ^ static_cast<Uint64>(word)) * 1099511628211ull;
|
||||
}
|
||||
digest.push_back(hash);
|
||||
}
|
||||
return digest;
|
||||
}
|
||||
|
||||
class AsyncLinkTest : public ::testing::Test {
|
||||
protected:
|
||||
void SetUp() override { MobileGL::Initialize(); }
|
||||
};
|
||||
} // namespace
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// The consume-once claim
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
// The stage-4 headline risk: two programs share one shader and are linked back to back, so
|
||||
// two ProgramLinkTasks race for that shader's single glslang parse. Exactly one may win the
|
||||
// claim; the loser must re-parse the same preprocessed source against the same CompileEnv.
|
||||
// If either half of that is wrong the two programs get DIFFERENT SPIR-V for the same shader,
|
||||
// which is the silent-corruption class this whole mechanism exists to prevent.
|
||||
TEST_F(AsyncLinkTest, TwoProgramsSharingAShaderGenerateIdenticalSpirv) {
|
||||
for (const Bool async : {false, true}) {
|
||||
const AsyncModeScope scope(async);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
|
||||
// Both links enqueued before either result is read: with the flag on this is the
|
||||
// window in which two workers can hold the same node at once.
|
||||
const GLuint programA = CreateProgram();
|
||||
AttachShader(programA, vs);
|
||||
AttachShader(programA, fs);
|
||||
LinkProgram(programA);
|
||||
|
||||
const GLuint programB = CreateProgram();
|
||||
AttachShader(programB, vs);
|
||||
AttachShader(programB, fs);
|
||||
LinkProgram(programB);
|
||||
|
||||
ASSERT_EQ(QueryLinkStatus(programA), GL_TRUE) << QueryProgramInfoLog(programA);
|
||||
ASSERT_EQ(QueryLinkStatus(programB), GL_TRUE) << QueryProgramInfoLog(programB);
|
||||
|
||||
const Vector<Uint64> digestA = SpirvDigest(programA);
|
||||
const Vector<Uint64> digestB = SpirvDigest(programB);
|
||||
ASSERT_EQ(digestA.size(), 2u) << "async=" << async;
|
||||
EXPECT_EQ(digestA, digestB)
|
||||
<< "the claim winner and the re-parsing loser must produce identical SPIR-V (async=" << async << ")";
|
||||
|
||||
// And the two programs really are usable independently.
|
||||
EXPECT_GE(GetUniformLocation(programA, "uColor"), 0);
|
||||
EXPECT_GE(GetUniformLocation(programB, "uColor"), 0);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
}
|
||||
|
||||
// The same property many ways at once, with the pool loaded: N programs over the SAME shader
|
||||
// pair, all enqueued before anything is read, so one claim winner is racing N-1 re-parsers.
|
||||
// Every program must come out byte-identical.
|
||||
//
|
||||
// The shader pair has to be identical across the programs for this to mean anything: glslang
|
||||
// links the stages together, so a stage's SPIR-V is legitimately a function of the WHOLE
|
||||
// program (mapIO's cross-stage location assignment, live-variable analysis). Comparing one
|
||||
// shared vertex shader across programs with different fragment stages would compare things
|
||||
// that are allowed to differ.
|
||||
TEST_F(AsyncLinkTest, ManyProgramsSharingOneShaderPairAgreeOnTheirSpirv) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
constexpr int kPrograms = 12;
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
|
||||
Vector<GLuint> programs;
|
||||
for (int i = 0; i < kPrograms; ++i) {
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
programs.push_back(program);
|
||||
}
|
||||
|
||||
Vector<Uint64> reference;
|
||||
for (int i = 0; i < kPrograms; ++i) {
|
||||
const GLuint program = programs[static_cast<SizeT>(i)];
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "program " << i << ": " << QueryProgramInfoLog(program);
|
||||
const Vector<Uint64> digest = SpirvDigest(program);
|
||||
ASSERT_EQ(digest.size(), 2u);
|
||||
if (i == 0) {
|
||||
reference = digest;
|
||||
} else {
|
||||
EXPECT_EQ(digest, reference) << "SPIR-V differs in program " << i;
|
||||
}
|
||||
EXPECT_GE(GetUniformLocation(program, "uColor"), 0) << "program " << i;
|
||||
EXPECT_GE(GetUniformLocation(program, "uAlpha"), 0) << "program " << i;
|
||||
}
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// Mutation over a pending link (the cancel matrix)
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
// The last link wins. A re-link over a pending one cancels it and enqueues afresh; the
|
||||
// result the application eventually reads must be the SECOND link's.
|
||||
TEST_F(AsyncLinkTest, RelinkOverAPendingLinkPublishesTheSecondLink) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const String firstSource = MakeBulkySource(7001);
|
||||
const char* firstText = firstSource.c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &firstText, nullptr);
|
||||
CompileShader(fs);
|
||||
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
// Swap the fragment shader's source and relink, all without ever reading the first
|
||||
// link's status - so the first link is very probably still queued or running.
|
||||
const String secondSource = MakeBulkySource(7002);
|
||||
const char* secondText = secondSource.c_str();
|
||||
ShaderSource(fs, 1, &secondText, nullptr);
|
||||
CompileShader(fs);
|
||||
LinkProgram(program);
|
||||
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_GE(GetUniformLocation(program, "uSeed7002"), 0);
|
||||
EXPECT_EQ(GetUniformLocation(program, "uSeed7001"), -1);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// The take-effect-at-next-link setters must NOT disturb a pending link: the pending link
|
||||
// snapshotted its own inputs at enqueue, so
|
||||
// glLinkProgram; glTransformFeedbackVaryings; glGetProgramiv(LINK_STATUS)
|
||||
// has to report the FIRST link - which captured nothing.
|
||||
TEST_F(AsyncLinkTest, TransformFeedbackVaryingsOverAPendingLinkReportsTheFirstLink) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kXfbVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
const char* varyings[] = {"vWorld"};
|
||||
TransformFeedbackVaryings(program, 1, varyings, GL_INTERLEAVED_ATTRIBS);
|
||||
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
GLint captured = -1;
|
||||
GetProgramiv(program, GL_TRANSFORM_FEEDBACK_VARYINGS, &captured);
|
||||
EXPECT_EQ(captured, 0) << "the pending link must publish the request set it snapshotted, not a later one";
|
||||
|
||||
// And the request does take effect at the NEXT link.
|
||||
LinkProgram(program);
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
GetProgramiv(program, GL_TRANSFORM_FEEDBACK_VARYINGS, &captured);
|
||||
EXPECT_EQ(captured, 1);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// glBindAttribLocation is the same family and must likewise leave a pending link alone.
|
||||
TEST_F(AsyncLinkTest, BindAttribLocationOverAPendingLinkDoesNotDisturbIt) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
BindAttribLocation(program, 5, "aPos");
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_EQ(GetAttribLocation(program, "aPos"), 0) << "the first link's layout(location = 0) must survive";
|
||||
|
||||
LinkProgram(program);
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// glAttachShader after glLinkProgram is defined to leave the current link status alone (it
|
||||
// takes effect at the next link). It must therefore NOT cancel a pending link - the failure
|
||||
// mode being guarded here is a program that linked fine reporting GL_FALSE.
|
||||
TEST_F(AsyncLinkTest, AttachShaderOverAPendingLinkKeepsTheLinkResult) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
// A second, unrelated fragment shader attached over the pending link. (Attaching two
|
||||
// shaders of one stage is legal; only the next link would have to reconcile them.)
|
||||
const GLuint extraFs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
|
||||
AttachShader(program, extraFs);
|
||||
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_GE(GetUniformLocation(program, "uAlpha"), 0);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// The link-then-detach-then-delete teardown every LWJGL/Blaze3D-shaped app performs. The
|
||||
// detach makes the shader GL-invisible, so glDeleteShader frees its name and would otherwise
|
||||
// cancel a compile the enqueued link is still waiting on - flipping a link that must report
|
||||
// GL_TRUE to GL_FALSE. Runs with the pool saturated so the compiles really are outstanding.
|
||||
TEST_F(AsyncLinkTest, DetachAndDeleteShadersOverAPendingLinkKeepsTheLinkResult) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const String source = MakeBulkySource(7400);
|
||||
const char* text = source.c_str();
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
DetachShader(program, vs);
|
||||
DetachShader(program, fs);
|
||||
DeleteShader(vs);
|
||||
DeleteShader(fs);
|
||||
EXPECT_EQ(IsShader(vs), GL_FALSE);
|
||||
EXPECT_EQ(IsShader(fs), GL_FALSE);
|
||||
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_GE(GetUniformLocation(program, "uSeed7400"), 0);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// glCreateShaderProgramv is specified as create-source-compile-create-attach-LINK-detach, so
|
||||
// it is the in-tree caller that exercises the detach-immediately-after-link ordering. It
|
||||
// self-joins through its status queries (design join site J7) and needs no edit of its own -
|
||||
// this is the guard that says so.
|
||||
TEST_F(AsyncLinkTest, CreateShaderProgramvLinksUnderAsync) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const char* sources[] = {kVs};
|
||||
const GLuint program = CreateShaderProgramv(GL_VERTEX_SHADER, 1, sources);
|
||||
ASSERT_NE(program, 0u);
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// glProgramBinary over a pending link: no format is supported, so the spec requires
|
||||
// LINK_STATUS to read FALSE afterwards. The pending link must not publish over that.
|
||||
TEST_F(AsyncLinkTest, ProgramBinaryOverAPendingLinkForcesLinkFalse) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
|
||||
const GLuint dummy = 0;
|
||||
ProgramBinary(program, 0, &dummy, static_cast<GLsizei>(sizeof(dummy)));
|
||||
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
|
||||
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE) << "glProgramBinary must win over the pending link";
|
||||
EXPECT_FALSE(QueryProgramInfoLog(program).empty());
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// glDeleteProgram over a pending link. The name goes away immediately - no wait for a worker
|
||||
// - and the abandoned job must neither crash nor keep anything observable alive.
|
||||
TEST_F(AsyncLinkTest, DeleteProgramWhileALinkIsPending) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
Vector<GLuint> doomed;
|
||||
Vector<String> sources;
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
for (int i = 0; i < 16; ++i) {
|
||||
sources.push_back(MakeBulkySource(7500 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
doomed.push_back(program);
|
||||
}
|
||||
for (const GLuint program : doomed) {
|
||||
DeleteProgram(program);
|
||||
EXPECT_EQ(IsProgram(program), GL_FALSE) << "an unused deleted program's name goes immediately";
|
||||
}
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
|
||||
// The context still works afterwards: the abandoned links did not take the pool, the
|
||||
// preprocess cache or the glslang process state down with them.
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// The join gates
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
// glLinkProgram must return before the work is done, and the first observable read must
|
||||
// join. Observed through the state machine rather than through timing, so it can never be a
|
||||
// false red: with a saturated pool at least one of the just-enqueued links has to be
|
||||
// unsettled at the moment we ask; skipped if the machine drained everything first.
|
||||
TEST_F(AsyncLinkTest, LinkProgramReturnsBeforeTheWorkIsDone) {
|
||||
const AsyncModeScope async(true);
|
||||
constexpr int kPrograms = 32;
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
Vector<GLuint> programs;
|
||||
Vector<String> sources;
|
||||
for (int i = 0; i < kPrograms; ++i) {
|
||||
sources.push_back(MakeBulkySource(7600 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
programs.push_back(program);
|
||||
}
|
||||
|
||||
int unsettled = 0;
|
||||
for (const GLuint program : programs) {
|
||||
if (!LinkIsSettled(program)) ++unsettled;
|
||||
}
|
||||
if (unsettled == 0) {
|
||||
GTEST_SKIP() << "the pool drained every link before the first observation; nothing to prove here";
|
||||
}
|
||||
|
||||
for (const GLuint program : programs) {
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_TRUE(LinkIsSettled(program)) << "reading LINK_STATUS must have joined";
|
||||
}
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// With the flag off, a link is finished by the time glLinkProgram returns. This is the guard
|
||||
// that keeps the default shippable.
|
||||
TEST_F(AsyncLinkTest, LinkIsFullySynchronousWithAsyncOff) {
|
||||
const AsyncModeScope async(false);
|
||||
ASSERT_FALSE(MG_Util::Async::AsyncShaderCompileEnabled());
|
||||
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
EXPECT_TRUE(LinkIsSettled(program));
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE) << QueryProgramInfoLog(program);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// P1 join site J1: the composite draw program for a pipeline is cached against a signature
|
||||
// built from each stage program's lifetime id and backend state version - NON-artifact
|
||||
// fields, which do not pass through the join gate. GetProgramForDraw has to settle the stage
|
||||
// programs first, or the signature describes a link generation that no longer exists and the
|
||||
// composite is rebuilt on every draw.
|
||||
TEST_F(AsyncLinkTest, DrawThroughAPipelineWithAPendingStageProgramJoinsFirst) {
|
||||
const AsyncModeScope async(true);
|
||||
Vector<String> backlog;
|
||||
SaturatePool(48, backlog);
|
||||
|
||||
// Built by hand rather than through glCreateShaderProgramv: that entry point detaches the
|
||||
// shader immediately after linking, so the next link would remove it and leave the stage
|
||||
// program with nothing attached to composite from.
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint vsProgram = CreateProgram();
|
||||
ProgramParameteri(vsProgram, GL_PROGRAM_SEPARABLE, GL_TRUE);
|
||||
AttachShader(vsProgram, vs);
|
||||
LinkProgram(vsProgram);
|
||||
ASSERT_EQ(QueryLinkStatus(vsProgram), GL_TRUE) << QueryProgramInfoLog(vsProgram);
|
||||
|
||||
GLuint pipeline = 0;
|
||||
GenProgramPipelines(1, &pipeline);
|
||||
ASSERT_NE(pipeline, 0u);
|
||||
// Bind before UseProgramStages: glGenProgramPipelines only reserves the name, and the
|
||||
// first bind is what turns it into an object glUseProgramStages can find.
|
||||
BindProgramPipeline(pipeline);
|
||||
UseProgramStages(pipeline, GL_VERTEX_SHADER_BIT, vsProgram);
|
||||
ASSERT_EQ(GetError(), GL_NO_ERROR);
|
||||
|
||||
// Re-link the stage program and immediately ask for the draw program, without reading
|
||||
// the link's status in between: the pending link is what J1 has to settle.
|
||||
LinkProgram(vsProgram);
|
||||
const SharedPtr<MG_State::GLState::ProgramObject> drawProgram = MG_State::pGLContext->GetProgramForDraw();
|
||||
ASSERT_NE(drawProgram, nullptr);
|
||||
EXPECT_TRUE(LinkIsSettled(vsProgram)) << "GetProgramForDraw must have joined the stage program";
|
||||
EXPECT_TRUE(drawProgram->GetLinkStatus()) << drawProgram->GetInfoLog();
|
||||
|
||||
// Asking again with nothing changed must hit the composite cache, which is only possible
|
||||
// if the signature was computed against settled programs both times.
|
||||
const SharedPtr<MG_State::GLState::ProgramObject> again = MG_State::pGLContext->GetProgramForDraw();
|
||||
EXPECT_EQ(again.get(), drawProgram.get()) << "the composite draw program must be cached across draws";
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// Diagnostics
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
// A link whose fragment shader failed to compile has to reproduce that shader's log verbatim
|
||||
// inside the program info log, whichever thread produced it - and the failure must be
|
||||
// reported as LINK_STATUS plus a log, never as a GL error.
|
||||
TEST_F(AsyncLinkTest, FailingLinkLogIsIdenticalAcrossModes) {
|
||||
String syncLog;
|
||||
{
|
||||
const AsyncModeScope scope(false);
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_FALSE);
|
||||
syncLog = QueryProgramInfoLog(program);
|
||||
EXPECT_FALSE(syncLog.empty());
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
{
|
||||
const AsyncModeScope scope(true);
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const GLuint fs = MakeShader(GL_FRAGMENT_SHADER, kBrokenFs);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE);
|
||||
EXPECT_EQ(QueryProgramInfoLog(program), syncLog);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
}
|
||||
|
||||
// A program with nothing attached fails in the GL-thread prologue, before any job exists.
|
||||
// That path has to reach the same info log in both modes.
|
||||
TEST_F(AsyncLinkTest, LinkWithNoShadersFailsIdenticallyInBothModes) {
|
||||
String syncLog;
|
||||
for (const Bool async : {false, true}) {
|
||||
const AsyncModeScope scope(async);
|
||||
const GLuint program = CreateProgram();
|
||||
LinkProgram(program);
|
||||
EXPECT_EQ(QueryLinkStatus(program), GL_FALSE);
|
||||
const String log = QueryProgramInfoLog(program);
|
||||
EXPECT_FALSE(log.empty());
|
||||
if (!async) {
|
||||
syncLog = log;
|
||||
} else {
|
||||
EXPECT_EQ(log, syncLog);
|
||||
}
|
||||
EXPECT_TRUE(LinkIsSettled(program)) << "a prologue failure leaves no job pending";
|
||||
}
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------------------
|
||||
// End to end
|
||||
// ---------------------------------------------------------------------------------------
|
||||
|
||||
// The shape a shaderpack load actually has: compile N shaders, link M programs, read
|
||||
// NOTHING until the end, then query everything. This is the only shape in which the pool has
|
||||
// many compiles and many links in flight simultaneously, with the link jobs chained behind
|
||||
// compile jobs that are themselves still queued.
|
||||
TEST_F(AsyncLinkTest, PackShapedBurstCompilesLinksAndQueriesEverything) {
|
||||
const AsyncModeScope async(true);
|
||||
constexpr int kShaders = 24;
|
||||
constexpr int kPrograms = 24;
|
||||
|
||||
Vector<String> sources;
|
||||
Vector<GLuint> vertexShaders;
|
||||
Vector<GLuint> fragmentShaders;
|
||||
for (int i = 0; i < kShaders; ++i) {
|
||||
vertexShaders.push_back(MakeShader(GL_VERTEX_SHADER, kVs));
|
||||
sources.push_back(MakeBulkySource(8000 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
fragmentShaders.push_back(fs);
|
||||
}
|
||||
|
||||
Vector<GLuint> programs;
|
||||
for (int i = 0; i < kPrograms; ++i) {
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vertexShaders[static_cast<SizeT>(i % kShaders)]);
|
||||
AttachShader(program, fragmentShaders[static_cast<SizeT>(i % kShaders)]);
|
||||
LinkProgram(program);
|
||||
programs.push_back(program);
|
||||
}
|
||||
|
||||
for (int i = 0; i < kPrograms; ++i) {
|
||||
const GLuint program = programs[static_cast<SizeT>(i)];
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "program " << i << ": " << QueryProgramInfoLog(program);
|
||||
EXPECT_GE(GetUniformLocation(program, "uColor"), 0) << "program " << i;
|
||||
EXPECT_GE(GetUniformLocation(program, ("uSeed" + std::to_string(8000 + i % kShaders)).c_str()), 0)
|
||||
<< "program " << i;
|
||||
EXPECT_EQ(GetAttribLocation(program, "aPos"), 0) << "program " << i;
|
||||
EXPECT_EQ(SpirvDigest(program).size(), 2u) << "program " << i;
|
||||
}
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
|
||||
// The adversarial interleaving: link, query a previous one, re-source, re-link, delete, all
|
||||
// with the pool busy. Nothing here asserts timing - what it hunts for is a missed join, a
|
||||
// consumed-twice parse, or a use of an abandoned node, all of which surface as a wrong
|
||||
// status, a missing uniform, or a crash.
|
||||
TEST_F(AsyncLinkTest, StressLinkQueryRelinkDeleteInterleaved) {
|
||||
const AsyncModeScope async(true);
|
||||
constexpr int kRounds = 5;
|
||||
constexpr int kPerRound = 10;
|
||||
|
||||
for (int round = 0; round < kRounds; ++round) {
|
||||
Vector<String> sources;
|
||||
Vector<GLuint> programs;
|
||||
Vector<GLuint> fragmentShaders;
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
|
||||
for (int i = 0; i < kPerRound; ++i) {
|
||||
sources.push_back(MakeBulkySource(round * 1000 + 300 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
fragmentShaders.push_back(fs);
|
||||
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, fs);
|
||||
LinkProgram(program);
|
||||
programs.push_back(program);
|
||||
|
||||
// Query a PREVIOUS program while this one is still outstanding: the join has to
|
||||
// settle exactly the program asked about and no other.
|
||||
if (i > 0) {
|
||||
const GLuint earlier = programs[static_cast<SizeT>(i - 1)];
|
||||
EXPECT_EQ(QueryLinkStatus(earlier), GL_TRUE) << QueryProgramInfoLog(earlier);
|
||||
}
|
||||
}
|
||||
|
||||
// Re-source half of them mid-flight and relink over the pending link.
|
||||
for (int i = 0; i < kPerRound; i += 2) {
|
||||
sources.push_back(MakeBulkySource(round * 1000 + 700 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
ShaderSource(fragmentShaders[static_cast<SizeT>(i)], 1, &text, nullptr);
|
||||
CompileShader(fragmentShaders[static_cast<SizeT>(i)]);
|
||||
LinkProgram(programs[static_cast<SizeT>(i)]);
|
||||
}
|
||||
|
||||
for (int i = 0; i < kPerRound; ++i) {
|
||||
const GLuint program = programs[static_cast<SizeT>(i)];
|
||||
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE)
|
||||
<< "round " << round << " program " << i << ": " << QueryProgramInfoLog(program);
|
||||
const String expected =
|
||||
"uSeed" + std::to_string(round * 1000 + (i % 2 == 0 ? 700 + i : 300 + i));
|
||||
EXPECT_GE(GetUniformLocation(program, expected.c_str()), 0)
|
||||
<< "round " << round << " program " << i << " expected " << expected;
|
||||
DeleteProgram(program);
|
||||
}
|
||||
for (const GLuint fs : fragmentShaders) DeleteShader(fs);
|
||||
DeleteShader(vs);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
// MobileGL - MobileGL/MG_Test/Program/AsyncTeardownTest.cpp
|
||||
// Copyright (c) 2025-2026 MobileGL-Dev
|
||||
// Licensed under the GNU Lesser General Public License v3.0:
|
||||
// https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
// https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
// SPDX-License-Identifier: LGPL-3.0-only
|
||||
// End of Source File Header
|
||||
|
||||
// P1 stage 4, item S6: MobileGL::Destroy() with compile AND link jobs still in flight.
|
||||
//
|
||||
// This is the one cancellation path in the whole design that WAITS, and the order it waits
|
||||
// in is load-bearing: in-flight jobs own their own inputs and are safe against everything
|
||||
// teardown does EXCEPT glslang's process globals and the TShader/TProgram objects hanging off
|
||||
// pGLContext - both of which DestroyImpl is about to free. StopAndDrain() therefore runs
|
||||
// first, before pGLContext.reset() and before glslang::FinalizeProcess().
|
||||
//
|
||||
// ITS OWN BINARY, deliberately. ShaderCompilePool::StopAndDrain() is a one-way latch: from
|
||||
// the first eglTerminate onwards every job in the process runs inline on the calling thread.
|
||||
// Sharing a binary with AsyncCompileTest/AsyncLinkTest would silently turn every case
|
||||
// declared after this one synchronous, and they would keep passing while testing nothing.
|
||||
|
||||
#include <gtest/gtest.h>
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "Config.h"
|
||||
#include "Includes.h"
|
||||
#include "Init.h"
|
||||
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
|
||||
#include "MG_Impl/GLImpl/Program/GL_Program.h"
|
||||
#include "MG_State/GLState/Core.h"
|
||||
#include "MG_Util/Async/ShaderCompilePool.h"
|
||||
|
||||
using namespace MobileGL;
|
||||
using namespace MobileGL::MG_Impl::GLImpl;
|
||||
|
||||
namespace {
|
||||
const char* kVs = R"(#version 460
|
||||
layout(location = 0) in vec3 aPos;
|
||||
uniform vec4 uColor;
|
||||
out vec4 vColor;
|
||||
void main() {
|
||||
vColor = uColor;
|
||||
gl_Position = vec4(aPos, 1.0);
|
||||
}
|
||||
)";
|
||||
|
||||
String MakeBulkySource(const int index) {
|
||||
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
|
||||
source += "uniform float uSeed" + std::to_string(index) + ";\n";
|
||||
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
|
||||
for (int i = 0; i < 220; ++i) {
|
||||
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
|
||||
}
|
||||
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
|
||||
return source;
|
||||
}
|
||||
|
||||
GLuint MakeShader(const GLenum type, const char* source) {
|
||||
const GLuint shader = CreateShader(type);
|
||||
ShaderSource(shader, 1, &source, nullptr);
|
||||
CompileShader(shader);
|
||||
return shader;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
// Fills the pool with compiles, chains links behind them, and tears the library down without
|
||||
// reading a single result. Nothing here can assert on the jobs' outcomes - by design there is
|
||||
// no one left to ask - so what it asserts is that teardown COMPLETES: it must not hang
|
||||
// (StopAndDrain joining a worker that is itself waiting on something), must not crash (a
|
||||
// worker inside glslang while FinalizeProcess frees its symbol tables, or a link job reading
|
||||
// a shader node the GL thread has dropped), and must leave the process able to come back up.
|
||||
TEST(AsyncTeardown, DestroyWithCompilesAndLinksInFlight) {
|
||||
// After Initialize(), not before: MG_ConfigLoader::Init() re-reads the whole feature
|
||||
// block from the environment and would overwrite the override.
|
||||
MobileGL::Initialize();
|
||||
MG_Config::Features.AsyncShaderCompile = MG_Config::QuirkOverride::ForceOn;
|
||||
ASSERT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
|
||||
|
||||
constexpr int kCount = 64;
|
||||
Vector<String> sources;
|
||||
Vector<GLuint> shaders;
|
||||
Vector<GLuint> programs;
|
||||
sources.reserve(kCount);
|
||||
|
||||
// Bare compiles first, so the pool has a backlog the links below will queue behind.
|
||||
for (int i = 0; i < kCount; ++i) {
|
||||
sources.push_back(MakeBulkySource(30000 + i));
|
||||
const char* text = sources.back().c_str();
|
||||
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
|
||||
ShaderSource(fs, 1, &text, nullptr);
|
||||
CompileShader(fs);
|
||||
shaders.push_back(fs);
|
||||
}
|
||||
|
||||
// Then links, each chained behind a compile that is very probably still outstanding: at
|
||||
// the moment Destroy() runs there are queued compiles, running compiles, links waiting on
|
||||
// a dependency edge, and links already handed to the pool.
|
||||
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
for (int i = 0; i < kCount; ++i) {
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs);
|
||||
AttachShader(program, shaders[static_cast<SizeT>(i)]);
|
||||
LinkProgram(program);
|
||||
programs.push_back(program);
|
||||
}
|
||||
|
||||
// No status read anywhere above - the jobs are genuinely in flight.
|
||||
MobileGL::Destroy();
|
||||
|
||||
// Back up again. The pool stays stopped for the rest of the process (a one-way latch), so
|
||||
// this second life is synchronous - which is exactly the documented behaviour, and it has
|
||||
// to still be a WORKING one.
|
||||
MobileGL::Initialize();
|
||||
const GLuint vs2 = MakeShader(GL_VERTEX_SHADER, kVs);
|
||||
const char* fsSource = R"(#version 460
|
||||
in vec4 vColor;
|
||||
layout(location = 0) out vec4 fragColor;
|
||||
void main() { fragColor = vColor; }
|
||||
)";
|
||||
const GLuint fs2 = MakeShader(GL_FRAGMENT_SHADER, fsSource);
|
||||
const GLuint program = CreateProgram();
|
||||
AttachShader(program, vs2);
|
||||
AttachShader(program, fs2);
|
||||
LinkProgram(program);
|
||||
|
||||
GLint status = GL_FALSE;
|
||||
GetProgramiv(program, GL_LINK_STATUS, &status);
|
||||
EXPECT_EQ(status, GL_TRUE) << "the library must be usable after a teardown that drained jobs in flight";
|
||||
EXPECT_GE(GetUniformLocation(program, "uColor"), 0);
|
||||
EXPECT_EQ(GetError(), GL_NO_ERROR);
|
||||
}
|
||||
@@ -44,6 +44,41 @@ target_link_libraries(
|
||||
${LINK_LIBRARIES}
|
||||
)
|
||||
|
||||
add_executable(
|
||||
AsyncLinkTest
|
||||
AsyncLinkTest.cpp
|
||||
)
|
||||
|
||||
target_include_directories(AsyncLinkTest PRIVATE
|
||||
${MGL_ROOT}/include
|
||||
${MGL_ROOT}/MobileGL
|
||||
)
|
||||
|
||||
target_link_libraries(
|
||||
AsyncLinkTest PRIVATE
|
||||
GTest::gtest_main
|
||||
${LINK_LIBRARIES}
|
||||
)
|
||||
|
||||
# Its own binary on purpose: this one calls MobileGL::Destroy(), and ShaderCompilePool's
|
||||
# stop is a one-way latch for the whole process - every case declared after it in the same
|
||||
# binary would silently run its compiles and links inline.
|
||||
add_executable(
|
||||
AsyncTeardownTest
|
||||
AsyncTeardownTest.cpp
|
||||
)
|
||||
|
||||
target_include_directories(AsyncTeardownTest PRIVATE
|
||||
${MGL_ROOT}/include
|
||||
${MGL_ROOT}/MobileGL
|
||||
)
|
||||
|
||||
target_link_libraries(
|
||||
AsyncTeardownTest PRIVATE
|
||||
GTest::gtest_main
|
||||
${LINK_LIBRARIES}
|
||||
)
|
||||
|
||||
target_include_directories(ProgramTest PRIVATE
|
||||
${MGL_ROOT}/include
|
||||
${MGL_ROOT}/MobileGL
|
||||
@@ -61,3 +96,5 @@ gtest_discover_tests(ProgramTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
|
||||
# Heavier than the rest of the unit suite by design: several cases deliberately saturate the
|
||||
# compile pool so there is something in flight to race against.
|
||||
gtest_discover_tests(AsyncCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
|
||||
gtest_discover_tests(AsyncLinkTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
|
||||
gtest_discover_tests(AsyncTeardownTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
|
||||
|
||||
@@ -2934,7 +2934,7 @@ TEST_F(ProgramTest, RecompileWithIdenticalSourceKeepsCompiledStateAndStillLinks)
|
||||
EXPECT_TRUE(ShaderHasMemoizedCompile(vs));
|
||||
|
||||
// A first link consumes the stored TShader; the redundant recompile below must not
|
||||
// disturb the preprocessed source that TakeShaderForLink re-parses from.
|
||||
// disturb the preprocessed source that ClaimParsedShader re-parses from.
|
||||
GLuint firstProgram = LinkVsFs(vs, fs, GL_TRUE);
|
||||
EXPECT_GE(GetUniformLocation(firstProgram, "uColor"), 0);
|
||||
|
||||
@@ -2959,7 +2959,7 @@ TEST_F(ProgramTest, RecompileWithIdenticalSourceKeepsCompiledStateAndStillLinks)
|
||||
EXPECT_EQ(String(sourceBuffer.data(), static_cast<size_t>(written)), String(kP0bVs));
|
||||
|
||||
// A second program built from the same, redundantly recompiled shaders links and
|
||||
// reflects - i.e. TakeShaderForLink's re-parse path survived the no-op.
|
||||
// reflects - i.e. ClaimParsedShader's re-parse path survived the no-op.
|
||||
GLuint secondProgram = LinkVsFs(vs, fs, GL_TRUE);
|
||||
EXPECT_GE(GetUniformLocation(secondProgram, "uColor"), 0);
|
||||
EXPECT_GE(GetUniformLocation(secondProgram, "uModel"), 0);
|
||||
|
||||
@@ -16,6 +16,30 @@ namespace MobileGL::MG_Util::Async {
|
||||
Bool IsTerminalState(const JobState state) {
|
||||
return state == JobState::Complete || state == JobState::Cancelled;
|
||||
}
|
||||
|
||||
// Job BODIES have been contained since stage 1 (JobNode::Run); continuations were
|
||||
// not, and stage 4 introduces the first real ones. A continuation runs on whichever
|
||||
// thread drove the node terminal - for a compile that finished on a worker, that is
|
||||
// inside an Asio handler, where an escaping exception means thread_pool::run()
|
||||
// rethrows and the process terminates. It would also skip every continuation after
|
||||
// it in the list, stranding unrelated dependents.
|
||||
//
|
||||
// Containing it here is a backstop, not the contract: a continuation cannot be
|
||||
// repaired from the outside (the dispatcher has no idea what the callback was for),
|
||||
// so the registrar still owns "this cannot fail". See JobNode::OnTerminal.
|
||||
void RunContinuation(const std::function<void()>& continuation) {
|
||||
if (!continuation) return;
|
||||
try {
|
||||
continuation();
|
||||
} catch (const std::exception& e) {
|
||||
MGLOG_E("JobNode: a terminal continuation threw (%s); it has been contained, but whatever it "
|
||||
"was going to do did not happen",
|
||||
e.what());
|
||||
} catch (...) {
|
||||
MGLOG_E("JobNode: a terminal continuation threw a non-std exception; it has been contained, "
|
||||
"but whatever it was going to do did not happen");
|
||||
}
|
||||
}
|
||||
} // namespace
|
||||
|
||||
Bool JobNode::IsTerminal() const { return IsTerminalState(m_state.load(std::memory_order_acquire)); }
|
||||
@@ -47,9 +71,10 @@ namespace MobileGL::MG_Util::Async {
|
||||
m_cv.notify_all();
|
||||
// Run continuations OUTSIDE the lock: a continuation is free to call back into this
|
||||
// node (IsComplete, State) and, in the link-dependency case, to post the dependent
|
||||
// job to the pool from whichever thread drove this node terminal.
|
||||
// job to the pool from whichever thread drove this node terminal. Individually
|
||||
// contained, so one broken dependent cannot strand the rest of the list.
|
||||
for (auto& continuation : continuations) {
|
||||
if (continuation) continuation();
|
||||
RunContinuation(continuation);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
@@ -122,7 +147,10 @@ namespace MobileGL::MG_Util::Async {
|
||||
return;
|
||||
}
|
||||
}
|
||||
fn();
|
||||
// Already terminal: the caller's thread runs it, through the same guard the deferred
|
||||
// path uses. OnTerminal is reached from Link()'s GL-thread prologue as well as from a
|
||||
// worker, and glLinkProgram is not a place an exception may escape from either.
|
||||
RunContinuation(fn);
|
||||
}
|
||||
|
||||
void ApplyDeferredDiagnostics(JobNode& node) {
|
||||
|
||||
@@ -52,7 +52,12 @@ namespace MobileGL::MG_Util::Async {
|
||||
// Running -> Cancelled (cancelled mid-run, or RunBody() threw)
|
||||
// Complete and Cancelled are terminal and the node is immutable afterwards, so every
|
||||
// reader that observed IsTerminal() may read the outputs without further synchronization.
|
||||
class JobNode {
|
||||
//
|
||||
// enable_shared_from_this because a dependency edge outlives its registrar: a node that
|
||||
// posts itself from another node's continuation (ProgramLinkTask::OnDepSettled) has to
|
||||
// hand the pool a strong reference from inside itself. Every JobNode is therefore created
|
||||
// through MakeShared - a stack-allocated one may not use SubmitAfter-style chaining.
|
||||
class JobNode : public std::enable_shared_from_this<JobNode> {
|
||||
public:
|
||||
JobNode() = default;
|
||||
virtual ~JobNode() = default;
|
||||
@@ -88,6 +93,16 @@ namespace MobileGL::MG_Util::Async {
|
||||
// `fn` runs on the calling thread before OnTerminal returns. Exactly-once in both
|
||||
// directions: the callback is either handed to the finishing thread or run inline,
|
||||
// never both.
|
||||
//
|
||||
// A continuation must not throw. It is dispatched from whichever thread drove this
|
||||
// node terminal, which on the pool side is an Asio handler - an exception escaping
|
||||
// one propagates out of thread_pool::run() and terminates the process. The dispatcher
|
||||
// contains a throw anyway (see RunContinuation) so that one broken continuation
|
||||
// cannot strand the others, but the continuation itself is where the guarantee
|
||||
// belongs: whoever registers one owns the "and it cannot fail" argument, because the
|
||||
// dispatcher can only log, never repair. ProgramLinkTask::OnDepSettled is the worked
|
||||
// example - it catches internally and cancels itself, because a link that is never
|
||||
// posted is a joiner blocked forever.
|
||||
void OnTerminal(std::function<void()> fn);
|
||||
|
||||
// Runs the body on the calling thread. The synchronous path (async disabled,
|
||||
|
||||
@@ -136,7 +136,15 @@ namespace MobileGL::MG_Util::Async {
|
||||
// out by a concurrent StopAndDrain between the decision and the dispatch: asio::post
|
||||
// only enqueues, it never runs the handler on the calling thread, so it cannot
|
||||
// re-enter this mutex.
|
||||
void DispatchLocked() {
|
||||
//
|
||||
// A node asio::post fails to hand off is appended to `toCancel` instead of being
|
||||
// Cancel()'d here: Cancel() runs the node's OnTerminal continuations inline (stage 4
|
||||
// added ProgramLinkTask::OnDepSettled as a real one), and a continuation is free to
|
||||
// call ShaderCompilePool::Post() again. Every caller of DispatchLocked holds `mutex`
|
||||
// (a plain, non-recursive std::mutex) - Cancel()'ing in here would let that
|
||||
// re-entrant Post() deadlock on the very lock this frame already owns. The caller
|
||||
// drains `toCancel` after releasing the lock.
|
||||
void DispatchLocked(Vector<SharedPtr<JobNode>>& toCancel) {
|
||||
while (!queue.empty() && inFlight < maxConcurrency && !stopped.load(std::memory_order_acquire)) {
|
||||
// Copy rather than move into the handler: if asio::post throws (it allocates)
|
||||
// the local SharedPtr is still valid, so the node can be settled instead of
|
||||
@@ -150,7 +158,7 @@ namespace MobileGL::MG_Util::Async {
|
||||
asio::post(*pool, [this, node]() mutable { RunOnWorker(Move(node)); });
|
||||
} catch (...) {
|
||||
--inFlight;
|
||||
node->Cancel();
|
||||
toCancel.push_back(Move(node));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -159,14 +167,23 @@ namespace MobileGL::MG_Util::Async {
|
||||
tl_isPoolThread = true;
|
||||
// A node that was already handed to Asio when StopAndDrain ran still arrives
|
||||
// here; cancelling it first turns the dispatch into a state transition instead of
|
||||
// a full compile, so the drain's join() returns promptly.
|
||||
// a full compile, so the drain's join() returns promptly. This Cancel() runs
|
||||
// before `mutex` is ever taken in this frame, so it is not subject to the
|
||||
// re-entrancy hazard DispatchLocked's comment describes.
|
||||
if (stopped.load(std::memory_order_acquire)) node->Cancel();
|
||||
node->Run();
|
||||
node.reset();
|
||||
|
||||
const std::lock_guard<std::mutex> lock(mutex);
|
||||
--inFlight;
|
||||
DispatchLocked();
|
||||
Vector<SharedPtr<JobNode>> toCancel;
|
||||
{
|
||||
const std::lock_guard<std::mutex> lock(mutex);
|
||||
--inFlight;
|
||||
DispatchLocked(toCancel);
|
||||
}
|
||||
// Outside the lock: see DispatchLocked's comment.
|
||||
for (const auto& n : toCancel) {
|
||||
if (n) n->Cancel();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
@@ -199,10 +216,17 @@ namespace MobileGL::MG_Util::Async {
|
||||
}
|
||||
|
||||
void ShaderCompilePool::SetMaxConcurrency(const Uint n) {
|
||||
const std::lock_guard<std::mutex> lock(m_impl->mutex);
|
||||
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
|
||||
// Raising the budget releases whatever the old one was holding back.
|
||||
if (m_impl->pool) m_impl->DispatchLocked();
|
||||
Vector<SharedPtr<JobNode>> toCancel;
|
||||
{
|
||||
const std::lock_guard<std::mutex> lock(m_impl->mutex);
|
||||
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
|
||||
// Raising the budget releases whatever the old one was holding back.
|
||||
if (m_impl->pool) m_impl->DispatchLocked(toCancel);
|
||||
}
|
||||
// Outside the lock: see DispatchLocked's comment.
|
||||
for (const auto& n2 : toCancel) {
|
||||
if (n2) n2->Cancel();
|
||||
}
|
||||
}
|
||||
|
||||
void ShaderCompilePool::Post(SharedPtr<JobNode> node) {
|
||||
@@ -220,13 +244,15 @@ namespace MobileGL::MG_Util::Async {
|
||||
// exception-safe and SharedPtr's move constructor is noexcept, so a throwing
|
||||
// push_back never consumed it; and DispatchLocked contains its own asio::post
|
||||
// failures rather than propagating them (see above). Keep it that way.
|
||||
Bool enqueued = false;
|
||||
Vector<SharedPtr<JobNode>> toCancel;
|
||||
try {
|
||||
const std::lock_guard<std::mutex> lock(m_impl->mutex);
|
||||
if (!m_impl->stopped.load(std::memory_order_acquire) && !InProcessTeardown()) {
|
||||
if (!m_impl->pool) m_impl->pool = MakeUnique<asio::thread_pool>(m_impl->threadCount);
|
||||
m_impl->queue.push_back(Move(node));
|
||||
m_impl->DispatchLocked();
|
||||
return;
|
||||
m_impl->DispatchLocked(toCancel);
|
||||
enqueued = true;
|
||||
}
|
||||
} catch (...) {
|
||||
MGLOG_E("ShaderCompilePool::Post: enqueue failed; cancelling the job so its joiner "
|
||||
@@ -234,6 +260,12 @@ namespace MobileGL::MG_Util::Async {
|
||||
if (node) node->Cancel();
|
||||
return;
|
||||
}
|
||||
// Outside the lock: see DispatchLocked's comment - a Cancel() here may run a
|
||||
// continuation (e.g. ProgramLinkTask::OnDepSettled) that calls back into Post().
|
||||
for (const auto& n : toCancel) {
|
||||
if (n) n->Cancel();
|
||||
}
|
||||
if (enqueued) return;
|
||||
|
||||
// A stopped pool is a synchronous pool, not a black hole: the node still runs, just
|
||||
// on the caller's thread. Everything downstream already handles "terminal by the time
|
||||
|
||||
Reference in New Issue
Block a user