Compare commits

..
2 Commits
Author SHA1 Message Date
BZLZHH c6299f754f [Fix] (MG_Backend/DirectVulkan, MG_State): key per-object memos on lifetime ids, not heap addresses
A destroyed VertexArrayObject's heap address is handed straight back by the
next allocation of its size, and so is a destroyed BufferObject's. DirectVulkan
keyed its per-VAO draw memo on the VAO POINTER and folded the bound buffer's
ADDRESS into the content hash that validates the memoised bindings, so a
delete/recreate pair under a byte-identical attribute layout reproduced both
the key and its validating hash at once. The successor VAO then inherited the
dead one's resolved bindings and the draw fetched from a destroyed VkBuffer.
Both stated defences failed together, because both reduce to the content hash
and the hash's buffer-identity component was itself a recycled address.

VertexArrayObject and BufferObject now carry a globally-unique, never-reused
GetLifetimeId() - the same contract as ProgramObject's, minted from an atomic
starting at 1 so a zero-initialised slot can never name a live object.
VaoDrawMemo matches on (address, lifetime id) and stores the id on recycle,
SetupDrawSnapshot's "the VAO did not move" test compares the id alongside the
config version, and VertexInputStateFactory::ComputeHash hashes the bound
buffer's id instead of its pointer (0 for client memory).

Proven: the use-after-free reproduces at 100% incidence headless on lavapipe,
including a SEGV whose backtrace is the driver dereferencing a destroyed vertex
buffer inside lvp_queue_submit, and it is gone with the fix. New coverage -
MG_Test/State/ObjectLifetimeIdTest (deterministic, GPU-free, no context: it
waits for the real allocator to repeat an address and asserts the id differs,
and skips loudly rather than passing quietly if it never gets the chance), and
MG_IntegrationTest XfbAfterClipDistanceScenario, registered for DirectGLES,
DirectVulkan, and a third DirectVulkan run with async shader compilation pinned
on because that is a second allocation pattern. Gates: 553/553 unit green at
async=0 and async=1; the scenario 5/5 headless at both flag states; 71/72 CI
trace-replay fixtures over both backends, the one failure a pre-existing
lavapipe crash proven not a regression (identical SIGSEGV at the identical
call number under the pre-fix library).

Pending NVIDIA/X11 confirmation: the KHR-GL{32,40} transform_feedback failures
that opened this investigation never reproduced on lavapipe - the -2/-101
pre-fill signature appears in zero pre-fix runs there - so whether this clears
them is UNPROVEN and must be re-measured on the NVIDIA rig against a freshly
re-run pre-fix baseline. The residual suspect is deliberately untouched here:
m_xfbCounterSlotByObject keys its counter slot on the raw GL transform-feedback
name, so a recycled name whose generation check happens to pass would RESUME
instead of BEGIN. That path was never exercised on lavapipe and is neither
confirmed nor exonerated.
2026-08-08 23:56:53 -04:00
BZLZHH dcf918b9ee [Perf] (MG_State): adopt in-flight compile jobs across shader objects (P1 stage 6)
~21% of a shaderpack's glCompileShader calls hand different shader objects
byte-identical source; the P0b cache only helps after one finishes, so under
async two workers would run the whole pipeline twice. Now the GL thread
consults a per-context (stage, hash, length, envFingerprint) -> weak-node
map at enqueue and ADOPTS the in-flight (or completed) node instead of
posting a duplicate - a hit is honored only after a full byte comparison
(the hash never decides), a cancel-requested or settled-cancelled node is
never adopted, and no worker ever waits.

Sharing a node makes the unconditional cancel wrong, so release is now
adopter-counted: a plain GL-thread Int (every mutation site is a GL entry
point; the single-threadedness argument and the terminal-early-out that
keeps the count exact are in the header), and the cancel fires only at
count zero AND with no pending link pinning the node (the stage-4
MarkLinkReferenced precedence). Adoption also re-points the object's source
at the node's snapshot so the layer-1 memo's pointer compare stays armed -
without that, an adopter's next glCompileShader would re-enqueue the very
duplicate this stage removes. Both guards are negative-control-proven: each
removed guard fails exactly its own tests. Count discipline was proven with
a temporary hard-abort on underflow/leak across the full suite and retrace
corpus - zero hits.

18 new tests (13 GL-surface incl. shared-node re-source/delete/orphan-sweep
isolation, shared failure logs, 48-over-6 stress with a deterministic
adoption count, flag-off and KHR-suspended zero-adoption guards; 5 direct
map cases incl. fingerprint mismatch and cancelled/expired pruning).
Gates: 538/538 unit both flag states, async suites x5 no flakes, NVIDIA
DirectGLES retrace identical sets both states. Timing: 2-worker
(Android-shaped) 1-3% faster consistently on complementary and BSL;
4-worker unchanged - the win this stage exists for lands where CPU is
scarce.
2026-08-08 20:17:51 -04:00
28 changed files with 3325 additions and 99 deletions
+1
View File
@@ -298,6 +298,7 @@ set(SOURCE_FILES
MobileGL/MG_State/GLState/ProgramState/ShaderCompileTask.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderObject.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderPreprocessCache.cpp
MobileGL/MG_State/GLState/ProgramState/ShaderCompileAdoptionMap.cpp
MobileGL/MG_State/GLState/ProgramState/ProgramState.cpp
MobileGL/MG_State/GLState/RenderState/RenderState.cpp
MobileGL/MG_State/GLState/FramebufferState/FramebufferObject.cpp
@@ -33,14 +33,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
XXHASH_VERIFY(XXH64_update(m_hashState, &attr.IsBgra, sizeof(attr.IsBgra)));
XXHASH_VERIFY(XXH64_update(m_hashState, &attr.Divisor, sizeof(attr.Divisor)));
// The buffer's heap address is an identity component of the key: a freed
// buffer's reused address can alias an old cache entry, but only under a
// byte-identical attribute layout - and the entry payload is a pure function
// of the hashed inputs, with the draw path re-resolving bindingBufferKeys
// against the live VAO attribute pointers, so an aliased hit returns exactly
// what a rebuild would. Address drift only grows the map; the OnFrameBoundary
// aging sweep bounds that.
const SizeT bufferKey = reinterpret_cast<SizeT>(attr.Buffer.get());
// The bound buffer's IDENTITY is a component of the key, and it has to be the
// buffer's never-reused lifetime id - NOT its heap address, which this used to
// hash. An address is recycled by the allocator, so a deleted-and-recreated
// buffer reproduces it; combined with a byte-identical attribute layout that
// reproduces the WHOLE content hash, and the hash is what
// TryBindResolvedVertexBindings accepts as proof that a memoised binding still
// reads the buffer it was resolved from. It did not: a destroyed buffer's GPU
// slice was bound for its successor's draw, which is how a transform-feedback
// capture came back holding a dead VAO's vertex data (0,0,0,1 - the previous
// test's positions) instead of its own.
// Zero for client memory (no buffer), which is a distinct identity of its own.
const Uint64 bufferKey = attr.Buffer ? attr.Buffer->GetLifetimeId() : 0;
XXHASH_VERIFY(XXH64_update(m_hashState, &bufferKey, sizeof(bufferKey)));
}
@@ -28,11 +28,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
struct BackendVertexInputState {
HashType hash = 0;
// Hash of the resolved Vulkan vertex layout only (bindings, attributes,
// unsupported mask) - NO buffer identities. `hash` mixes buffer heap
// addresses so per-chunk VBOs mint a fresh identity per buffer; keying
// pipelines on that minted one VkPipeline per chunk section for an
// identical layout, defeating pipeline reuse and the per-draw memo.
// Pipelines depend only on the layout, so they key on this instead.
// unsupported mask) - NO buffer identities. `hash` mixes each bound
// buffer's never-reused LIFETIME ID, so per-chunk VBOs mint a fresh
// identity per buffer; keying pipelines on that minted one VkPipeline per
// chunk section for an identical layout, defeating pipeline reuse and the
// per-draw memo. Pipelines depend only on the layout, so they key on this
// instead.
HashType layoutHash = 0;
// Frame boundary of the last cache hit; entries idle past the
// OnFrameBoundary retirement age are evicted (CPU heap only).
@@ -86,8 +87,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const MG_State::GLState::VertexArrayObject& vao, HashType hash);
const BackendVertexInputState& GetOrCreateVertexInputState(const MG_State::GLState::VertexArrayObject& vao);
// Frame boundary hook: ages the cache and evicts entries not hit for many
// frames. The key mixes buffer heap addresses, so buffer/VAO churn keeps
// minting fresh keys; without eviction the map grows for the whole session.
// frames. The key mixes each bound buffer's never-reused lifetime id, so
// buffer/VAO churn keeps minting fresh keys - and does so by construction,
// not by luck: a recreated buffer can no longer land back on its dead
// predecessor's key. Without eviction the map grows for the whole session.
// Entries hold no Vulkan handles (pipeline creation copies the descriptions)
// and the draw path's entry reference never spans a frame boundary, so
// eviction here needs no GPU-idle proof. Self-gated: one counter bump and
@@ -3201,12 +3201,17 @@ void main() {
// carry the most entropy of a multiply.
const Uint64 mixed = static_cast<Uint64>(reinterpret_cast<SizeT>(vao) >> 4) * 0x9E3779B97F4A7C15ull;
const Uint32 index = static_cast<Uint32>(mixed >> 32) & (kVaoDrawMemoSlotCount - 1);
// The address still picks the slot (it is what the caller has in hand), but it is
// the lifetime id that decides whether the slot is THIS object's: an address on
// its own is recycled, and a slot matched on a recycled address hands the new VAO
// the dead one's resolved bindings.
const Uint64 lifetimeId = vao->GetLifetimeId();
VaoDrawMemo& first = m_vaoDrawMemoTable[index];
if (first.vaoKey == vao) {
if (first.vaoKey == vao && first.vaoLifetimeId == lifetimeId) {
return &first;
}
VaoDrawMemo& second = m_vaoDrawMemoTable[index ^ 1u];
if (second.vaoKey == vao) {
if (second.vaoKey == vao && second.vaoLifetimeId == lifetimeId) {
return &second;
}
// Miss: recycle a slot. Prefer an empty one; otherwise evict the entry whose
@@ -3217,6 +3222,7 @@ void main() {
victim = &second;
}
victim->vaoKey = vao;
victim->vaoLifetimeId = lifetimeId;
victim->contentHash = 0;
victim->layoutFactsValid = false;
// Unmatchable until a resolve completes (same rule as before: a bailed-out
@@ -4475,9 +4481,10 @@ void main() {
// vertex-input hash (VAO layout), render-pass hash (render targets + the draw-buffer/format
// driven blend & write-mask gating), and the pipeline-state value hash (all fixed-function state).
// Reset per-frame and on pipeline destruction so a memoized handle can never dangle.
// The identity hash mixes buffer heap addresses (per-chunk VBOs mint a new
// one per buffer); the memo and the pipeline payload key on the resolved
// LAYOUT hash instead, so draws over identical layouts share one pipeline.
// The identity hash mixes each bound buffer's never-reused lifetime id
// (per-chunk VBOs mint a new one per buffer); the memo and the pipeline
// payload key on the resolved LAYOUT hash instead, so draws over identical
// layouts share one pipeline.
// The one-arg fetch rides the VAO's state-pointer memo (no hash, no map).
auto& vis = m_vertexInputStateFactory->GetOrCreateVertexInputState(vao);
const Uint64 vertexLayoutHash = vis.layoutHash;
@@ -5265,7 +5272,8 @@ void main() {
// path, re-resolving descriptors and texture layouts nothing invalidated.
const auto& vao = *MG_State::pGLContext->GetBoundVertexArray();
const Bool vaoMoved =
static_cast<const void*>(&vao) != snap.vao || vao.GetConfigVersion() != snap.vaoConfigVersion;
static_cast<const void*>(&vao) != snap.vao || vao.GetLifetimeId() != snap.vaoLifetimeId ||
vao.GetConfigVersion() != snap.vaoConfigVersion;
const auto& drawFbo =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Draw).GetBoundObject();
if (static_cast<const void*>(drawFbo.get()) != snap.drawFbo ||
@@ -5334,9 +5342,11 @@ void main() {
// VAO's content-hash memo. The hash memo shares the cache line this compare
// chain already loaded (the config version), and the table slot is compact
// and hot - unlike the VAO's aux-memo words, which start a second cold line
// of every object in a VAO-cycling frame. The facts are pure functions of
// the content hash, so a slot whose contentHash equals the live memoised
// hash serves them for ANY VAO object, recycled addresses included.
// of every object in a VAO-cycling frame. The slot only ever answers for
// THIS object: LookupVaoDrawMemo matches (address, lifetime id), so a slot
// a destroyed VAO left behind at a recycled address misses and the facts
// are re-resolved. The contentHash compare is the second gate on top of
// that identity check, catching a reconfiguration of the same live object.
Uint64 auxMasks = 0;
Bool factsKnown = false;
Uint64 contentHash = 0;
@@ -5515,6 +5525,7 @@ void main() {
snap.renderStateVersion = renderStateVersion;
snap.bindGeneration = bindGeneration;
snap.vao = static_cast<const void*>(&vao);
snap.vaoLifetimeId = vao.GetLifetimeId();
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.vaoLayoutHash = vaoLayoutHash;
snap.pipeline = pipeline;
@@ -5933,6 +5944,7 @@ void main() {
snap.programLifetimeId = program.GetLifetimeId();
snap.programVersion = program.GetBackendStateVersion();
snap.vao = &vao;
snap.vaoLifetimeId = vao.GetLifetimeId();
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.drawFbo = drawFbo.get();
snap.fboVersion = drawFbo->GetObjectVersion();
@@ -758,6 +758,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 programLifetimeId = 0;
Uint32 programVersion = 0;
const void* vao = nullptr;
// Same rule as VaoDrawMemo::vaoLifetimeId: (address, config version) is not an
// identity, because a recycled address can arrive carrying a config version
// the dead VAO also had (two mutations to configure one attribute is the
// common shape), and "the VAO did not move" would then skip the layout
// re-resolve for a different VAO.
Uint64 vaoLifetimeId = 0;
Uint32 vaoConfigVersion = 0;
const void* drawFbo = nullptr;
Uint16 fboVersion = 0;
@@ -987,19 +993,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const MG_State::GLState::BufferObject* buffers[kMaxBindings] = {};
Uint64 sliceEpochs[kMaxBindings] = {};
};
// One direct-mapped slot of the per-VAO draw-memo table below. The key is a
// lookup hint only - a slot is never dereferenced through vaoKey; every fact it
// carries is validated against live state before use:
// One direct-mapped slot of the per-VAO draw-memo table below. A slot belongs to
// the object whose (vaoKey, vaoLifetimeId) pair it carries: the address alone
// only picks the slot, and the never-reused lifetime id is what proves the slot
// is THIS VAO's, so the successor allocated onto a destroyed VAO's address
// always misses. That identity check is load-bearing and the content-hash
// validations below do NOT stand in for it - a recycled address under a
// byte-identical configuration reproduces the content hash exactly, which is
// how a destroyed VAO's resolved bindings were once handed to its successor's
// draw. The slot is still never dereferenced through vaoKey, and every fact it
// carries is still validated against live state before use:
// - layoutHash/layoutAuxMasks are valid only while contentHash equals the LIVE
// VAO's own hash memo (which the VAO's config version guards), so a config
// change, a buffer rebind, or a recycled VAO address with a different
// configuration all miss. A recycled address with a byte-identical
// configuration AND identical bound buffers reproduces the content hash, and
// then the facts are correct by construction (they are a pure function of it).
// change or a buffer rebind misses even for the same object.
// - bindings revalidates per draw exactly as before (frame serial, content
// hash, per-binding live buffer pointers and slice epochs).
struct alignas(64) VaoDrawMemo {
const MG_State::GLState::VertexArrayObject* vaoKey = nullptr;
// The VAO's never-reused lifetime id, checked alongside vaoKey. The pointer
// ALONE is not an identity: a deleted VAO's heap address is handed straight
// back by the next glGenVertexArrays-shaped allocation, and the successor then
// matched this slot and inherited the dead object's memos. Both stated
// defences failed with it, because both reduce to the content hash and the
// content hash's buffer-identity component was itself a recycled heap address.
Uint64 vaoLifetimeId = 0;
// The VAO content hash (VertexInputStateFactory::GetOrComputeHash) the two
// layout facts below were derived from; 0 while nothing valid is stored.
Uint64 contentHash = 0;
@@ -51,6 +51,7 @@ add_executable(MobileGLIntegrationTest
Scenarios/ResidentIndexScenario.cpp
Scenarios/MultiDrawScenario.cpp
Scenarios/AsyncCompileScenario.cpp
Scenarios/XfbAfterClipDistanceScenario.cpp
)
target_include_directories(MobileGLIntegrationTest PRIVATE
@@ -217,6 +218,8 @@ mgl_itest_join_environment(MGL_ITEST_GLES_ENVIRONMENT
"MOBILEGL_BACKEND_TYPE=DirectGLES" ${MGL_ITEST_COMMON_ENV})
mgl_itest_join_environment(MGL_ITEST_VULKAN_ENVIRONMENT
"MOBILEGL_BACKEND_TYPE=DirectVulkan" ${MGL_ITEST_VULKAN_ENV})
mgl_itest_join_environment(MGL_ITEST_VULKAN_ASYNC_ENVIRONMENT
"MOBILEGL_BACKEND_TYPE=DirectVulkan" "MOBILEGL_ASYNC_SHADER_COMPILE=1" ${MGL_ITEST_VULKAN_ENV})
# TIMEOUT on every entry: a GPU test that wedges must fail the run, not hang it.
set(MGL_ITEST_TIMEOUT 120)
@@ -243,3 +246,24 @@ gtest_discover_tests(MobileGLIntegrationTest
TIMEOUT ${MGL_ITEST_TIMEOUT}
ENVIRONMENT "${MGL_ITEST_VULKAN_ENVIRONMENT}"
)
# A third registration, of ONE scenario, with asynchronous shader compilation
# pinned on. Not a second code path in the renderer: a second ALLOCATION pattern.
# The async pipeline's job objects change which of the freed blocks the capture
# phase is handed, and that is what decides whether the destroyed-VAO address is
# reached at all - on the ablated (pre-fix) tree async=1 reproduced 3 runs out of
# 3 where the ambient default reproduced 2 of 3. Pinning it here means the
# high-signal configuration runs whatever the shipped default becomes, instead of
# the suite quietly weakening the day that default flips. It must be process-wide
# (the ENVIRONMENT property), not an in-process scope: the compile pool and its
# threads are stood up at initialization, and their allocations are half the
# point. DirectVulkan only - the memo this pins is DirectVulkan's.
gtest_discover_tests(MobileGLIntegrationTest
TEST_PREFIX "DirectVulkan.AsyncCompile."
TEST_FILTER "XfbAfterClipDistanceScenario.*"
DISCOVERY_TIMEOUT 30
PROPERTIES
LABELS integration-gpu
TIMEOUT ${MGL_ITEST_TIMEOUT}
ENVIRONMENT "${MGL_ITEST_VULKAN_ASYNC_ENVIRONMENT}"
)
@@ -0,0 +1,584 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/XfbAfterClipDistanceScenario.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario F - a draw must never read a destroyed object's memoised state.
//
// Distilled from the order-triggered CTS failure: on DirectVulkan, once
// KHR-GLxx.clip_distance.functional had run in the same process, every later
// transform_feedback CAPTURE case failed. It looked like a transform feedback
// bug and is not one. The capture works; the DRAW being captured fetched its
// vertices from the WRONG BUFFER - the one the clip workload had just deleted.
//
// The mechanism, and why the sequence matters. DirectVulkan memoises a VAO's
// resolved Vulkan vertex bindings in a table keyed on the VertexArrayObject's
// heap ADDRESS, validated by a content hash that folds in the bound
// BufferObject's heap ADDRESS. Both are recycled by the allocator, so when the
// workload's VAO and vertex buffer are destroyed and the capture phase's own
// VAO and vertex buffer are allocated onto their addresses under a
// byte-identical attribute layout (one vec4 float array at location 0 - which
// is what both phases use), the key matches, the hash matches, and the memo
// hands the new draw the dead buffer's GPU slice. Nothing about transform
// feedback is involved: capture just makes the wrong vertices legible, because
// the captured record IS the vertex data. The fix gives VertexArrayObject and
// BufferObject never-reused lifetime ids and keys the memo on those.
//
// MOBILEGL_ASYNC_SHADER_COMPILE is not part of the defect. It shifts the
// allocation pattern, so it changes WHICH stop points below land on a recycled
// address - which is why the CTS saw ~100% incidence with it on and ~2% with it
// off, and why the sweep case matters more than any single stop point.
//
// The shapes are the two CTS cases verbatim in structure:
// * the workload is glcClipDistance.cpp FunctionalTest's inner loop (a program
// per (redeclaration, clip count), glEnable(GL_CLIP_DISTANCEi), an FBO per
// primitive type, a draw and a readback), including its early-return
// behaviour: on failure the test returns WITHOUT running its "clip clean"
// loop, so GL_CLIP_DISTANCE0..N-1 stay enabled for the rest of the process.
// That leftover enable state is NOT the carrier (one of the cases below pins
// that); the object churn is.
// * the victim is gl3cTransformFeedback3Tests.cpp's skip_components: a
// gl_SkipComponents capture layout under GL_RASTERIZER_DISCARD, read back
// out of a buffer pre-filled with -1-i so that "captured nothing" is
// distinguishable from "captured the wrong thing".
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
#ifndef GL_CLIP_DISTANCE0
#define GL_CLIP_DISTANCE0 0x3000
#endif
namespace MGITest {
namespace {
GLuint CompileShader(GLenum type, const std::string& source, std::string* log) {
const GLuint shader = glCreateShader(type);
const char* text = source.c_str();
glShaderSource(shader, 1, &text, nullptr);
glCompileShader(shader);
GLint status = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &status);
if (status == GL_FALSE) {
GLint length = 0;
glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
std::vector<char> buffer(static_cast<std::size_t>(length) + 1, '\0');
glGetShaderInfoLog(shader, length + 1, nullptr, buffer.data());
if (log != nullptr) *log = buffer.data();
glDeleteShader(shader);
return 0;
}
return shader;
}
// Links a vertex/fragment pair, optionally declaring transform feedback
// varyings first (glTransformFeedbackVaryings takes effect at the next link,
// exactly as the CTS uses it).
GLuint BuildProgram(const std::string& vertexSource, const std::string& fragmentSource,
const std::vector<const char*>& xfbVaryings, GLenum bufferMode, std::string* log) {
const GLuint vertexShader = CompileShader(GL_VERTEX_SHADER, vertexSource, log);
if (vertexShader == 0) return 0;
const GLuint fragmentShader = CompileShader(GL_FRAGMENT_SHADER, fragmentSource, log);
if (fragmentShader == 0) {
glDeleteShader(vertexShader);
return 0;
}
const GLuint program = glCreateProgram();
glAttachShader(program, vertexShader);
glAttachShader(program, fragmentShader);
if (!xfbVaryings.empty()) {
glTransformFeedbackVaryings(program, static_cast<GLsizei>(xfbVaryings.size()), xfbVaryings.data(),
bufferMode);
}
glLinkProgram(program);
glDeleteShader(vertexShader);
glDeleteShader(fragmentShader);
GLint status = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &status);
if (status == GL_FALSE) {
GLint length = 0;
glGetProgramiv(program, GL_INFO_LOG_LENGTH, &length);
std::vector<char> buffer(static_cast<std::size_t>(length) + 1, '\0');
glGetProgramInfoLog(program, length + 1, nullptr, buffer.data());
if (log != nullptr) *log = buffer.data();
glDeleteProgram(program);
return 0;
}
return program;
}
// ---------------------------------------------------------------- poison
// glcClipDistance.cpp FunctionalTest::m_vertex_shader_code with the same
// three substitutions (redeclaration, clip function, array setter).
std::string ClipVertexSource(bool redeclaration, unsigned clipCount, unsigned clipFunction,
unsigned vertexCount) {
const std::string count = std::to_string(clipCount);
std::string source = "#version 400 core\n\n";
if (redeclaration) {
source += "out float gl_ClipDistance[" + count + "];\n";
}
source += "\n";
switch (clipFunction) {
case 0:
source += "float f(int i)\n{\n return 0.0;\n}\n";
break;
case 1:
source += "float f(int i)\n{\n return 0.25 + 0.75 * (float(i) + 1.0) * (float(gl_VertexID) + 1.0)"
" / (float(" + count + ") * float(" + std::to_string(vertexCount) + "));\n}\n";
break;
default:
source += "float f(int i)\n{\n return - 0.25 - 0.75 * (float(i) + 1.0) * (float(gl_VertexID) + 1.0)"
" / (float(" + count + ") * float(" + std::to_string(vertexCount) + "));\n}\n";
break;
}
source += "\nin vec4 position;\n\nvoid main()\n{\n";
if (redeclaration) {
// Dynamic array setter.
source += " for(int i = 0; i < " + count + "; i++)\n {\n"
" gl_ClipDistance[i] = f(i);\n }\n";
} else {
// Static array setter, at the highest index this iteration enables.
const std::string index = std::to_string(clipCount - 1);
source += " gl_ClipDistance[" + index + "] = f(" + index + ");\n";
}
source += "\n gl_Position = position;\n}\n";
return source;
}
const char* kClipFragmentSource = R"(#version 400 core
out vec4 color;
void main()
{
color = vec4(1.0, 0.0, 0.0, 1.0);
}
)";
// How far into FunctionalTest's loop nest to get before bailing out the way
// the CTS does on a failed check: return immediately, skipping the "clip
// clean" loop that would have disabled GL_CLIP_DISTANCEi again.
struct ClipStopPoint {
unsigned primitiveIndex = 0; // 0 = POINTS, 1 = LINES, 2 = TRIANGLES
unsigned clipFunction = 0;
bool redeclaration = false;
unsigned clipCount = 1; // 1..8, the iteration that "fails"
};
// Runs FunctionalTest's loop nest up to and including `stop`, then returns
// leaving exactly the state the CTS leaves behind on a failure.
void RunClipDistanceWorkload(const ClipStopPoint& stop) {
static const GLenum kPrimitiveTypes[] = {GL_POINTS, GL_LINES, GL_TRIANGLES};
static const GLsizei kPrimitiveIndices[] = {1, 2, 3};
static const float kPositions[3][12] = {
{0.0f, 0.0f, 0.0f, 1.0f},
{-1.0f, 0.0f, 0.0f, 1.0f, 1.0f, 0.0f, 0.0f, 1.0f},
{-1.0f, -1.0f, 0.0f, 1.0f, 1.0f, -1.0f, 0.0f, 1.0f, 0.0f, 1.0f, 0.0f, 1.0f},
};
for (unsigned primitiveIndex = 0; primitiveIndex <= stop.primitiveIndex; ++primitiveIndex) {
const GLenum primitiveType = kPrimitiveTypes[primitiveIndex];
const GLsizei vertexCount = kPrimitiveIndices[primitiveIndex];
const GLsizei framebufferSize = (primitiveType == GL_POINTS) ? 1 : 32;
GLuint colorBuffer = 0;
GLuint framebuffer = 0;
glGenRenderbuffers(1, &colorBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, colorBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8, framebufferSize, framebufferSize);
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, colorBuffer);
glViewport(0, 0, framebufferSize, framebufferSize);
const unsigned lastFunction =
(primitiveIndex == stop.primitiveIndex) ? stop.clipFunction : 2u;
for (unsigned clipFunction = 0; clipFunction <= lastFunction; ++clipFunction) {
const bool atStopFunction =
primitiveIndex == stop.primitiveIndex && clipFunction == stop.clipFunction;
for (unsigned redeclaration = 0; redeclaration < 2; ++redeclaration) {
const bool atStopRedeclaration =
atStopFunction && (redeclaration != 0) == stop.redeclaration;
const unsigned lastCount = atStopRedeclaration ? stop.clipCount : 8u;
for (unsigned clipCount = 1; clipCount <= lastCount; ++clipCount) {
std::string log;
const GLuint program =
BuildProgram(ClipVertexSource(redeclaration != 0, clipCount, clipFunction,
static_cast<unsigned>(vertexCount)),
kClipFragmentSource, {}, GL_INTERLEAVED_ATTRIBS, &log);
if (program == 0) continue;
glUseProgram(program);
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
glEnable(GL_CLIP_DISTANCE0 + clipCount - 1);
GLuint vao = 0;
GLuint vbo = 0;
glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER,
static_cast<GLsizeiptr>(sizeof(float) * 4 * vertexCount),
kPositions[primitiveIndex], GL_STATIC_DRAW);
const GLint location = glGetAttribLocation(program, "position");
if (location >= 0) {
glEnableVertexAttribArray(static_cast<GLuint>(location));
glVertexAttribPointer(static_cast<GLuint>(location), 4, GL_FLOAT, GL_FALSE, 0,
nullptr);
}
glDrawArrays(primitiveType, 0, vertexCount);
std::vector<unsigned char> pixels(
static_cast<std::size_t>(framebufferSize) * framebufferSize * 4, 0);
glReadPixels(0, 0, framebufferSize, framebufferSize, GL_RGBA, GL_UNSIGNED_BYTE,
pixels.data());
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArray(0);
glUseProgram(0);
// MGL_REPRO_KEEPCLIPOBJ leaks the per-iteration objects so
// no GL name and no heap address can be recycled into the
// capture phase.
// Deleting all three is load-bearing, not tidiness: the defect
// this scenario pins needs the VAO's AND its vertex buffer's heap
// addresses to be freed here so the capture phase's own objects
// can be handed the same ones back.
glDeleteBuffers(1, &vbo);
glDeleteVertexArrays(1, &vao);
glDeleteProgram(program);
if (atStopRedeclaration && clipCount == stop.clipCount) {
// The CTS's early return: the "clip clean" loop below
// never runs, so the enables survive.
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &framebuffer);
glDeleteRenderbuffers(1, &colorBuffer);
return;
}
}
for (unsigned i = 0; i < 8; ++i) {
glDisable(GL_CLIP_DISTANCE0 + i);
}
}
}
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &framebuffer);
glDeleteRenderbuffers(1, &colorBuffer);
}
}
// ---------------------------------------------------------------- victim
// gl3cTransformFeedback3Tests.cpp TransformFeedbackBaseTestCase::m_shader_vert.
const char* kXfbVertexSource = R"(#version 400 core
in vec4 vertex;
out vec4 value1;
out vec4 value2;
out vec4 value3;
out vec4 value4;
void main (void)
{
vec4 temp = vertex;
gl_Position = temp;
value1 = abs(temp) * 1.0;
value2 = abs(temp) * 2.0;
value3 = abs(temp) * 3.0;
value4 = abs(temp) * 4.0;
}
)";
const char* kXfbFragmentSource = R"(#version 400 core
out vec4 color;
void main (void)
{
color = vec4(0.0, 0.0, 0.0, 1.0);
}
)";
// The skip_components capture layout, verbatim.
std::vector<const char*> SkipComponentsVaryings() {
return {"gl_SkipComponents1", "value1", "gl_SkipComponents2", "gl_SkipComponents1", "value2",
"gl_SkipComponents3", "gl_SkipComponents2", "value3", "gl_SkipComponents4", "value4"};
}
constexpr unsigned kSkipComponentCount = 4 * 4 + (1 + 2 + 3 + 4 + 1 + 2); // 16 values + 13 skipped
constexpr unsigned kSkipVertexCount = 6;
// Runs skip_components and reports what came back. `outCaptured` is the raw
// readback so a failure can say whether anything was written at all.
void RunSkipComponentsCapture(std::vector<float>& outCaptured, std::string* buildLog) {
outCaptured.clear();
const GLuint program = BuildProgram(kXfbVertexSource, kXfbFragmentSource, SkipComponentsVaryings(),
GL_INTERLEAVED_ATTRIBS, buildLog);
ASSERT_NE(program, 0u) << "skip_components program failed to link: " << (buildLog ? *buildLog : "");
glUseProgram(program);
const std::vector<float> vertices = {
-1.0f, -1.0f, -1.0f, 1.0f, 1.0f, -1.0f, -2.0f, 1.0f, -1.0f, 1.0f, -3.0f, 1.0f,
1.0f, 1.0f, 4.0f, 1.0f, -1.0f, 1.0f, 5.0f, 1.0f, 1.0f, -1.0f, 6.0f, 1.0f,
};
GLuint vao = 0;
GLuint vbo = 0;
glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, static_cast<GLsizeiptr>(sizeof(float) * vertices.size()), vertices.data(),
GL_STATIC_DRAW);
const GLint location = glGetAttribLocation(program, "vertex");
if (location >= 0) {
glEnableVertexAttribArray(static_cast<GLuint>(location));
glVertexAttribPointer(static_cast<GLuint>(location), 4, GL_FLOAT, GL_FALSE, 0, nullptr);
}
const unsigned floatCount = kSkipVertexCount * kSkipComponentCount;
const GLsizeiptr byteSize = static_cast<GLsizeiptr>(sizeof(float) * floatCount);
GLuint captureBuffer = 0;
glGenBuffers(1, &captureBuffer);
glBindBuffer(GL_ARRAY_BUFFER, captureBuffer);
glBufferData(GL_ARRAY_BUFFER, byteSize, nullptr, GL_STATIC_READ);
glBindBufferBase(GL_TRANSFORM_FEEDBACK_BUFFER, 0, captureBuffer);
glBindBuffer(GL_ARRAY_BUFFER, 0);
// The pre-fill that makes "nothing was captured" recognisable.
std::vector<float> prefill(floatCount);
for (unsigned i = 0; i < floatCount; ++i) {
prefill[i] = -1.0f - static_cast<float>(i);
}
glBindBuffer(GL_ARRAY_BUFFER, captureBuffer);
glBufferData(GL_ARRAY_BUFFER, byteSize, prefill.data(), GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
glEnable(GL_RASTERIZER_DISCARD);
glClearColor(0.1f, 0.0f, 0.5f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glBindBufferBase(GL_TRANSFORM_FEEDBACK_BUFFER, 0, captureBuffer);
glBeginTransformFeedback(GL_TRIANGLES);
glDrawArrays(GL_TRIANGLES, 0, static_cast<GLsizei>(kSkipVertexCount));
glEndTransformFeedback();
glDisable(GL_RASTERIZER_DISCARD);
outCaptured.resize(floatCount);
glBindBufferRange(GL_TRANSFORM_FEEDBACK_BUFFER, 0, captureBuffer, 0, byteSize);
const void* mapped = glMapBufferRange(GL_TRANSFORM_FEEDBACK_BUFFER, 0, byteSize, GL_MAP_READ_BIT);
if (mapped != nullptr) {
std::memcpy(outCaptured.data(), mapped, static_cast<std::size_t>(byteSize));
glUnmapBuffer(GL_TRANSFORM_FEEDBACK_BUFFER);
}
glDisableVertexAttribArray(0);
glBindBuffer(GL_ARRAY_BUFFER, 0);
glDeleteBuffers(1, &vbo);
glDeleteBuffers(1, &captureBuffer);
glBindVertexArray(0);
glDeleteVertexArrays(1, &vao);
glUseProgram(0);
glDeleteProgram(program);
glBindBufferBase(GL_TRANSFORM_FEEDBACK_BUFFER, 0, 0);
}
// skip_components' expected buffer: the 13 skipped components keep their
// pre-fill, the 16 captured ones carry |vertex| * n.
std::vector<float> SkipComponentsExpected() {
const std::vector<float> vertices = {
-1.0f, -1.0f, -1.0f, 1.0f, 1.0f, -1.0f, -2.0f, 1.0f, -1.0f, 1.0f, -3.0f, 1.0f,
1.0f, 1.0f, 4.0f, 1.0f, -1.0f, 1.0f, 5.0f, 1.0f, 1.0f, -1.0f, 6.0f, 1.0f,
};
const unsigned floatCount = kSkipVertexCount * kSkipComponentCount;
std::vector<float> expected(floatCount);
for (unsigned i = 0; i < floatCount; ++i) {
expected[i] = -1.0f - static_cast<float>(i);
}
// Record layout, in floats:
// [0] skip1
// [1..4] value1
// [5..7] skip2 + skip1
// [8..11] value2
// [12..16] skip3 + skip2
// [17..20] value3
// [21..24] skip4
// [25..28] value4
static const unsigned kValueOffsets[4] = {1, 8, 17, 25};
for (unsigned v = 0; v < kSkipVertexCount; ++v) {
const unsigned base = v * kSkipComponentCount;
for (unsigned value = 0; value < 4; ++value) {
for (unsigned component = 0; component < 4; ++component) {
const float source = vertices[v * 4 + component];
expected[base + kValueOffsets[value] + component] =
std::fabs(source) * static_cast<float>(value + 1);
}
}
}
return expected;
}
// Reports the first mismatch, and whether the readback is byte-for-byte the
// pre-fill (i.e. the capture never happened).
::testing::AssertionResult CheckSkipComponents(const std::vector<float>& captured) {
const std::vector<float> expected = SkipComponentsExpected();
if (captured.size() != expected.size()) {
return ::testing::AssertionFailure()
<< "readback size " << captured.size() << " != " << expected.size();
}
bool anyWritten = false;
for (std::size_t i = 0; i < captured.size(); ++i) {
if (captured[i] != -1.0f - static_cast<float>(i)) {
anyWritten = true;
break;
}
}
for (std::size_t i = 0; i < expected.size(); ++i) {
if (std::fabs(captured[i] - expected[i]) > 0.0125f) {
return ::testing::AssertionFailure()
<< "capture mismatch at index " << i << ": got " << captured[i] << ", expected "
<< expected[i] << (anyWritten ? "" : " (the whole buffer is still the pre-fill: "
"NOTHING was captured)");
}
}
return ::testing::AssertionSuccess();
}
// The harness turns "no context came up" into a clean skip, and a skip is
// indistinguishable from a pass in a ctest summary. For this scenario that
// is a hole rather than a courtesy: the defect it pins is DirectVulkan's
// alone, and DirectVulkan now comes up headless on any machine at all - a
// surfaceless EGL platform over a software ICD (lavapipe) is enough. So
// "DirectVulkan did not initialise" here means the run is MISCONFIGURED,
// not that the machine has no GPU, and it must not report green.
//
// Local on purpose: the harness-wide skip semantics are deliberate
// (ScenarioFixture.h states the reasoning), and MOBILEGL_ITEST_REQUIRE_GPU
// is the harness-wide lever for the same intent - but that lever also
// demands a HARDWARE renderer, which is exactly what a lavapipe-only box
// cannot offer. This overrides nothing else: only this scenario, only for
// the backend that can regress, and only for the unusable-harness case.
class XfbAfterClipDistanceScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
// Ready() is false on the base's skip path AND on its REQUIRE_GPU
// failure path; the second one has already failed, so leave it alone
// rather than burying its reason under a second message.
if (Ready() || HasFatalFailure()) return;
if (Gl().BackendName() == "DirectVulkan") {
FAIL() << "DirectVulkan could not be brought up, so the regression this scenario guards - a "
"draw served a destroyed VAO's memoised vertex bindings - was never exercised, and "
"that must be a failure rather than a silent skip. Headless bring-up needs only a "
"Vulkan ICD and a surfaceless EGL platform (a software ICD such as lavapipe "
"qualifies: VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json with "
"EGL_PLATFORM=surfaceless). Harness reason: "
<< Gl().SkipReason();
}
}
};
// Control: the capture on its own must work.
TEST_F(XfbAfterClipDistanceScenario, SkipComponentsCaptureAlone) {
if (!Ready()) return;
std::vector<float> captured;
std::string log;
RunSkipComponentsCapture(captured, &log);
EXPECT_TRUE(CheckSkipComponents(captured));
}
// Bisection step 1: only the leftover GL_CLIP_DISTANCEi enables.
TEST_F(XfbAfterClipDistanceScenario, SkipComponentsCaptureAfterClipDistanceEnables) {
if (!Ready()) return;
for (unsigned i = 0; i < 8; ++i) {
glEnable(GL_CLIP_DISTANCE0 + i);
}
std::vector<float> captured;
std::string log;
RunSkipComponentsCapture(captured, &log);
for (unsigned i = 0; i < 8; ++i) {
glDisable(GL_CLIP_DISTANCE0 + i);
}
EXPECT_TRUE(CheckSkipComponents(captured));
}
// Bisection step 2: the whole clip_distance.functional workload, stopped
// where the CTS stopped in the runs that went on to break the capture.
TEST_F(XfbAfterClipDistanceScenario, SkipComponentsCaptureAfterClipDistanceWorkloadLines8) {
if (!Ready()) return;
RunClipDistanceWorkload({.primitiveIndex = 1, .clipFunction = 0, .redeclaration = false, .clipCount = 8});
std::vector<float> captured;
std::string log;
RunSkipComponentsCapture(captured, &log);
for (unsigned i = 0; i < 8; ++i) {
glDisable(GL_CLIP_DISTANCE0 + i);
}
EXPECT_TRUE(CheckSkipComponents(captured));
}
TEST_F(XfbAfterClipDistanceScenario, SkipComponentsCaptureAfterClipDistanceWorkloadPoints1) {
if (!Ready()) return;
RunClipDistanceWorkload({.primitiveIndex = 0, .clipFunction = 0, .redeclaration = true, .clipCount = 1});
std::vector<float> captured;
std::string log;
RunSkipComponentsCapture(captured, &log);
for (unsigned i = 0; i < 8; ++i) {
glDisable(GL_CLIP_DISTANCE0 + i);
}
EXPECT_TRUE(CheckSkipComponents(captured));
}
// A single stop point is not a regression test for this defect: whether the
// capture phase's VAO and vertex buffer land on the addresses the workload just
// freed is a function of how much the workload allocated, so the two cases above
// pin two draws of a lottery. Sweep the grid instead - before the fix, roughly a
// third of these stop points came back holding the workload's vertex data.
TEST_F(XfbAfterClipDistanceScenario, SkipComponentsCaptureSurvivesEveryClipWorkloadStopPoint) {
if (!Ready()) return;
for (unsigned primitiveIndex = 0; primitiveIndex < 3; ++primitiveIndex) {
for (unsigned redeclaration = 0; redeclaration < 2; ++redeclaration) {
for (const unsigned clipCount : {1u, 4u, 8u}) {
RunClipDistanceWorkload({.primitiveIndex = primitiveIndex,
.clipFunction = 0,
.redeclaration = redeclaration != 0,
.clipCount = clipCount});
std::vector<float> captured;
std::string log;
RunSkipComponentsCapture(captured, &log);
for (unsigned i = 0; i < 8; ++i) {
glDisable(GL_CLIP_DISTANCE0 + i);
}
EXPECT_TRUE(CheckSkipComponents(captured))
<< " (stop point: primitive " << primitiveIndex << ", redeclaration " << redeclaration
<< ", clip count " << clipCount << ")";
}
}
}
}
} // namespace
} // namespace MGITest
@@ -8,9 +8,17 @@
#include "BufferObject.h"
#include <atomic>
namespace MobileGL::MG_State::GLState {
namespace {
const BufferBackendOps* g_bufferBackendOps = nullptr;
// Starts at 1 so a zero-initialized cache slot can never carry a live buffer's id.
std::atomic<Uint64> g_nextBufferLifetimeId{1};
}
Uint64 BufferObject::AllocateLifetimeId() {
return g_nextBufferLifetimeId.fetch_add(1, std::memory_order_relaxed);
}
void SetBufferBackendOps(const BufferBackendOps* ops) {
@@ -185,6 +185,13 @@ namespace MobileGL {
Flags<BufferMappingAccessBit> GetMappingAccess() const;
GLbitfield GetStorageFlags() const;
Uint GetExternalIndex() const;
// Globally-unique, never-reused id for THIS object's lifetime - same contract
// and same motivation as ProgramObject::GetLifetimeId() and
// VertexArrayObject::GetLifetimeId(). A backend that folds a buffer's IDENTITY
// into a cache key must use this, never the GL name (LIFO-recycled by
// glGenBuffers) and never the heap address (recycled by the allocator): both
// let a deleted-and-recreated buffer answer to a dead one's cache entry.
Uint64 GetLifetimeId() const { return m_lifetimeId; }
// Monotonic counter bumped on every shadow mutation; backends use it to
// validate cached transient slices.
Uint64 GetChangeSerial() const;
@@ -207,7 +214,10 @@ namespace MobileGL {
// SubData transfer to sync the backend's separate GPU copy.
void NotifyContentWrite(SizeT offset, SizeT size);
static Uint64 AllocateLifetimeId();
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = AllocateLifetimeId();
SizeT m_size = 0;
BufferUsage m_usage = BufferUsage::StaticDraw;
// Owns the buffer's bytes (CPU shadow or backend persistent GPU map) and
+5
View File
@@ -156,6 +156,11 @@ namespace MobileGL {
// Settles every compile and link this context still owns; see
// ProgramState::JoinAllPendingWork. Called by glMaxShaderCompilerThreadsKHR(0).
void JoinAllPendingShaderWork();
// P1 stage 6: the per-context index of adoptable compile nodes, for its
// adoption counter. Diagnostics and tests only - no GL entry point reads it.
ShaderCompileAdoptionMap& GetShaderCompileAdoptionMap() {
return m_programState.GetShaderCompileAdoptionMap();
}
void UseProgram(Uint program);
const SharedPtr<ProgramObject>& GetCurrentProgram();
// What a draw or dispatch actually executes: the program in use, or - when
@@ -87,7 +87,8 @@ namespace MobileGL::MG_State::GLState {
Uint shaderId = 0;
m_programShaderNameGenerator.Generate(1, &shaderId);
EnsureIndexAvail(shaderId, m_shaderObjects);
auto shaderObject = MakeShared<ShaderObject>(stage, shaderId, m_shaderPreprocessCache);
auto shaderObject =
MakeShared<ShaderObject>(stage, shaderId, m_shaderPreprocessCache, m_shaderCompileAdoptionMap);
if (shaderObject == nullptr) return 0;
m_shaderObjects[shaderId] = shaderObject;
return shaderId;
@@ -153,11 +154,13 @@ namespace MobileGL::MG_State::GLState {
auto& shaderObject = m_shaderObjects[shader];
if (shaderObject == nullptr || !shaderObject->GetDeleteStatus()) return;
if (ShaderHasGLVisibleAttachment(shaderObject)) return;
// The name is about to go: nothing can observe this shader's compile any more, so a
// job still in flight for it is pure waste. Cancel-not-join - the job owns its
// inputs, so dropping the object out from under it is safe and the GL thread never
// blocks on a delete.
shaderObject->CancelCompile();
// The name is about to go, so nothing can observe this shader's compile through THIS
// object any more and a job still in flight for it is pure waste - unless another
// shader object adopted the same node (stage 6) or a pending link pinned it, which is
// exactly what ReleaseCompileNode weighs before it cancels anything. Cancel-not-join
// either way: the job owns its inputs, so dropping the object out from under it is
// safe and the GL thread never blocks on a delete.
shaderObject->ReleaseCompileNode();
shaderObject.reset();
m_programShaderNameGenerator.Delete(shader);
}
@@ -10,6 +10,7 @@
#include <Includes.h>
#include <MG_Util/Miscellany/IndexGenerator.h>
#include "ProgramObject.h"
#include "ShaderCompileAdoptionMap.h"
#include "ShaderPreprocessCache.h"
namespace MobileGL::MG_State::GLState {
@@ -52,6 +53,11 @@ namespace MobileGL::MG_State::GLState {
// CreateShader().
ShaderPreprocessCache& GetShaderPreprocessCache() { return *m_shaderPreprocessCache; }
// P1 stage 6, same deal: exposed for tests and diagnostics only. Its adoption counter
// is the one number that says how many glCompileShader calls this context turned into
// no work at all; nothing in the GL frontend branches on it.
ShaderCompileAdoptionMap& GetShaderCompileAdoptionMap() { return *m_shaderCompileAdoptionMap; }
private:
Bool ShaderHasGLVisibleAttachment(const SharedPtr<ShaderObject>& shaderObject) const;
// Frees the name slot and releases orphaned attached shaders; the immediate half
@@ -82,6 +88,11 @@ namespace MobileGL::MG_State::GLState {
// in-flight compile job may outlive the context). The FIRST-member declaration is
// kept anyway - it costs nothing and documents the intent.
SharedPtr<ShaderPreprocessCache> m_shaderPreprocessCache = MakeShared<ShaderPreprocessCache>();
// P1 stage 6: the GL-thread-only index of adoptable compile nodes. Shared ownership
// for the same reason as the cache above - a ShaderObject held by a ProgramObject can
// outlive these tables, and its destructor releases a node - though unlike the cache
// no worker ever sees this one, which is why it carries no lock.
SharedPtr<ShaderCompileAdoptionMap> m_shaderCompileAdoptionMap = MakeShared<ShaderCompileAdoptionMap>();
Vector<SharedPtr<ProgramObject>> m_programObjects;
Vector<SharedPtr<ShaderObject>> m_shaderObjects;
@@ -0,0 +1,90 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderCompileAdoptionMap.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#include "ShaderCompileAdoptionMap.h"
#include "ShaderCompileTask.h"
namespace MobileGL::MG_State::GLState {
SharedPtr<ShaderCompileTask> ShaderCompileAdoptionMap::FindAdoptable(const ShaderStage stage,
const Uint64 sourceHash, const String& source,
const Uint64 envFingerprint) {
const ShaderSourceKey key{.stage = stage,
.sourceHash = sourceHash,
.sourceLength = source.length(),
.envFingerprint = envFingerprint};
const auto it = m_entries.find(key);
if (it == m_entries.end()) return nullptr;
SharedPtr<ShaderCompileTask> node = it->second.lock();
// Expired (every shader object that held it has released it), settled as Cancelled
// (the enqueue lost a race with teardown, or the body threw), or CANCELLATION
// REQUESTED but not yet settled (a releaser fired Cancel() while a worker was still
// inside RunBody(), so the node is stuck at Running until the body returns - see
// JobNode::Run: once m_cancelled is set, the node is DOOMED to end up Cancelled no
// matter how the body finishes, it just has not gotten there yet). All three can
// never publish artifacts a caller may rely on, so all three are misses. Only the
// first two are dead weight worth pruning from the index here - a cancellation-
// requested-but-still-running node is still reachable from its own (about to
// release) ShaderObject and will get pruned once it actually settles, so leave the
// entry alone and just refuse to hand this node out.
if (!node || node->IsCancelled()) {
m_entries.erase(it);
return nullptr;
}
if (node->IsCancellationRequested()) return nullptr;
// Never let correctness ride on a 64-bit hash. Lengths already matched (they are part
// of the key), so this is a plain memcmp - and it is the ONLY thing that authorizes
// two GL shader names to share one compile.
if (*node->source != source) return nullptr;
++m_adoptionCount;
return node;
}
void ShaderCompileAdoptionMap::Register(const SharedPtr<ShaderCompileTask>& node) {
if (!node) return;
SweepIfCrowded();
// operator[] rather than a find/insert pair: an existing entry for this key is either
// a re-registration of the same source (the previous node expired or was cancelled)
// or an astronomically rare hash collision. The newcomer wins in both cases.
m_entries[ShaderSourceKey{.stage = node->stage,
.sourceHash = node->sourceHash,
.sourceLength = node->source->length(),
.envFingerprint = node->env->fingerprint}] = node;
}
void ShaderCompileAdoptionMap::Clear() {
m_entries.clear();
m_sweepThreshold = kMinSweepThreshold;
}
void ShaderCompileAdoptionMap::SweepIfCrowded() {
if (m_entries.size() < m_sweepThreshold) return;
// Collect first, erase after: FastSTL::unordered_map is open-addressed, so erasing
// through an iterator that the same loop is still advancing is not worth reasoning
// about on a path this cold.
Vector<ShaderSourceKey> dead;
for (const auto& entry : m_entries) {
const SharedPtr<ShaderCompileTask> node = entry.second.lock();
if (!node || node->IsCancelled()) dead.push_back(entry.first);
}
for (const ShaderSourceKey& key : dead) {
m_entries.erase(key);
}
// Amortization: after a sweep the map holds exactly the nodes still reachable from
// some shader object, so letting it double before the next sweep makes the whole
// scheme O(1) per Register() while keeping the map O(live nodes).
m_sweepThreshold = std::max(kMinSweepThreshold, m_entries.size() * 2);
}
} // namespace MobileGL::MG_State::GLState
@@ -0,0 +1,97 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderCompileAdoptionMap.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderSourceKey.h>
namespace MobileGL::MG_State::GLState {
class ShaderCompileTask;
// P1 stage 6: the per-context index of compile job nodes that a NEW shader object may
// adopt instead of enqueueing a duplicate of.
//
// Why it is not the P0b preprocess cache. That cache only helps once a compile has
// FINISHED - it memoizes the source-only half of the pipeline, and a worker consults it
// from inside the job body. Under asynchronous compilation the dominant shape is
// different: a shaderpack load hands N different shader objects byte-identical source
// within the same GL-thread burst (measured across bsl/complementary/bliss, ~21% of all
// Compile() calls are such cross-object duplicates), and all N are enqueued before any of
// them completes. Every one of those workers then misses the cache, runs the whole
// pipeline, and races the others to insert the same entry. This map closes that window on
// the GL thread, at enqueue: the second object through takes the FIRST object's node.
//
// What "adopt" means: the two shader objects end up holding the same SharedPtr in their
// m_compiled. They are two distinct GL names with two distinct info-log/COMPILE_STATUS
// queries, but both queries read one set of artifacts - which is exactly right, because
// the pipeline is a pure function of the key below and the full source text. Nothing is
// copied and no worker ever waits (P1 invariant I4 is untouched: this only ever REMOVES
// work from the pool). The single consume-once resource, the glslang parse, is already
// guarded for sharing by ShaderCompileTask::ClaimParsedShader's CAS, which stage 4 built
// for exactly this shape - one node, several links.
//
// ---- Threading: GL thread only, and therefore lock-free ----
// Every entry point below is reached from glCompileShader (ShaderObject::Compile) and
// from nowhere else. That is one GL entry point on the application's context thread, so
// the map needs no mutex, unlike the preprocess cache which several workers hit at once.
// The weak pointers are the ONLY thing this class stores, precisely so it can never keep
// a node - or the artifacts a node owns - alive past its last real holder.
//
// ---- Lifetime and pruning ----
// WeakPtr, never SharedPtr: the map is an index, not an owner. An entry whose node has
// been released by every shader object simply expires, and a node that was CANCELLED
// carries no result at all, so both are treated as misses and pruned where they are
// found. Pruning is otherwise amortized: Register() sweeps the whole map whenever it has
// grown past twice its size at the last sweep, which bounds the map at O(live nodes)
// without a per-call cost.
class ShaderCompileAdoptionMap {
public:
// Never sweep below this: a shaderpack burst is a few hundred distinct sources, and
// an entry is a key plus a weak pointer.
static constexpr SizeT kMinSweepThreshold = 256;
// The adoptable node for this exact source under this exact environment, or null.
//
// A hit is honored only after the FULL source text has been compared byte for byte
// against the candidate node's own snapshot: the hash in the key is a lookup
// accelerator, never the answer (ShaderSourceKey). A node that has settled as
// Cancelled is never handed out - it published nothing, so adopting it would give the
// new object a compile that can never report anything but GL_FALSE. Nor is a node
// whose cancellation has merely been REQUESTED but not yet settled (still Running,
// with IsCancellationRequested() true): JobNode::Run forces such a node to Cancelled
// the moment its body returns regardless of how the body finished, so it is already
// doomed and handing it out would just move the same GL_FALSE-with-no-log outcome to
// a second, unrelated shader object.
//
// A COMPLETED node is adoptable, and deliberately so: the new object gets the right
// answer for zero work, which is the same deal the P0b cache offers one layer down.
SharedPtr<ShaderCompileTask> FindAdoptable(ShaderStage stage, Uint64 sourceHash, const String& source,
Uint64 envFingerprint);
// Indexes `node` as the adoptable one for its key. A key already present is
// overwritten: the newcomer is at least as fresh as whatever was there, and one entry
// per key keeps this a plain map.
void Register(const SharedPtr<ShaderCompileTask>& node);
void Clear();
// ---- diagnostics only; nothing in the GL frontend branches on these ----
// Monotonic count of nodes handed out by FindAdoptable, i.e. of glCompileShader calls
// that did NOT enqueue a job because an equivalent one already existed. Tests read it
// as a delta across a burst.
Uint64 GetAdoptionCount() const { return m_adoptionCount; }
SizeT GetEntryCount() const { return m_entries.size(); }
private:
void SweepIfCrowded();
UnorderedMap<ShaderSourceKey, WeakPtr<ShaderCompileTask>, ShaderSourceKeyHasher> m_entries;
SizeT m_sweepThreshold = kMinSweepThreshold;
Uint64 m_adoptionCount = 0;
};
} // namespace MobileGL::MG_State::GLState
@@ -118,13 +118,61 @@ namespace MobileGL::MG_State::GLState {
// glDeleteShader. The detach makes the shader GL-invisible, so the delete frees its
// name, and ReleaseShaderNameIfOrphaned would cancel a compile the enqueued link is
// waiting on - turning a link that must report GL_TRUE into GL_FALSE. Set on the GL
// thread in Link()'s prologue, read on the GL thread by ShaderObject::CancelCompile.
// thread in Link()'s prologue, read on the GL thread by
// ShaderObject::ReleaseCompileNode - which from stage 6 weighs it together with the
// adopter count below, because a node can now have both kinds of observer at once.
//
// Never cleared: the worst case is one stale node compiling to completion for nobody,
// which is exactly what the pre-stage-3 implementation always did.
void MarkLinkReferenced() { m_linkReferenced.store(true, std::memory_order_release); }
Bool IsLinkReferenced() const { return m_linkReferenced.load(std::memory_order_acquire); }
// ---- P1 stage 6: the adopter count ----
// How many live ShaderObjects currently hold this node in their m_compiled.
//
// It exists because stage 6 lets a node be SHARED: before it, a node had exactly one
// shader object, so "this object stopped caring" and "nothing can observe this
// result" were the same statement and ShaderObject::CancelCompile could cancel
// unconditionally. Once two GL shader names hold one node, that cancel would kill the
// other one's pending compile - a compile that must still report GL_TRUE. So a cancel
// is now authorized by TWO conditions, both checked by the releaser:
// * this release brings the count to zero (no shader object is left), AND
// * IsLinkReferenced() is false (no enqueued link took the node into its snapshot).
// The second is the stage-4 pin, unchanged; the first is what stage 6 adds.
//
// ---- Why a plain Int and not an atomic ----
// Every mutation is made from ShaderObject, and every ShaderObject mutation site is a
// GL entry point on the application's context thread: glCompileShader (adopt/create),
// glShaderSource with different text, glDeleteShader's orphan sweep, and
// ~ShaderObject. All of them are the SAME thread, so the count is never concurrently
// mutated and an atomic would only buy an unneeded lock prefix on the hottest compile
// path. Workers cannot touch it by construction: a job body's entire contract (see
// this class's header comment) is that it reads only the node's inputs and writes only
// `artifacts`, and a plain Int here makes that contract grep-checkable in a way an
// atomic would quietly hide.
//
// The CANCEL that the count authorizes still races the worker, and deliberately so -
// that is the settled cancel-not-join semantics from stage 3: JobNode::Cancel is
// cooperative and non-blocking, a node already running settles as Cancelled when its
// body returns, and a node that has already gone terminal ignores the request.
// Nothing about that changes here.
//
// Exactness under that race: ShaderObject::ReleaseCompileNode returns EARLY, without
// decrementing and without dropping its reference, when the node is already terminal
// (there is nothing left to stop). Terminality is sticky, so if a releaser observes a
// node as NON-terminal then no holder has ever taken that early return on it, and the
// count it reads is exactly the number of holders. If the worker finishes in the
// window between that observation and the Cancel(), the Cancel is a no-op on a
// terminal node - and the count was zero, so there was no other holder to harm.
void AddAdopter() { ++m_adopters; }
void ReleaseAdopter() {
MOBILEGL_ASSERT(m_adopters > 0,
"ShaderCompileTask adopter count underflow; a ShaderObject released a node it did not "
"hold (every release must pair with exactly one AddAdopter)");
--m_adopters;
}
Int AdopterCount() const { return m_adopters; }
private:
void RunBody() override;
// The real body; RunBody wraps it so a throw becomes a GL-visible compile failure.
@@ -132,5 +180,7 @@ namespace MobileGL::MG_State::GLState {
mutable std::atomic<Bool> m_parseClaimed{false};
std::atomic<Bool> m_linkReferenced{false};
// GL-thread-owned; see AddAdopter above for why this is not an atomic.
Int m_adopters = 0;
};
} // namespace MobileGL::MG_State::GLState
@@ -26,16 +26,18 @@ namespace MobileGL::MG_State::GLState {
// reason: it is computing the right answer for text this object still holds.
if (SourceMatchesCompiledState(source)) return;
// The text genuinely changed, so whatever a running job is computing is now about
// an old source. Drop it where it stands - it owns its own copy of that old string,
// so swapping the pointer below cannot race its storage.
CancelCompile();
// an old source. Give up our claim on it - it owns its own copy of that old string,
// so swapping the pointer below cannot race its storage. Note "our claim", not "the
// job": another shader object may have adopted the same node and still be waiting for
// exactly this answer, which is what ReleaseCompileNode's count discipline protects.
ReleaseCompileNode();
m_source = MakeShared<const String>(source);
InvalidateCompiledState();
}
void ShaderObject::SetShaderSource(String&& source) {
if (SourceMatchesCompiledState(source)) return;
CancelCompile();
ReleaseCompileNode();
m_source = MakeShared<const String>(Move(source));
InvalidateCompiledState();
}
@@ -59,33 +61,75 @@ namespace MobileGL::MG_State::GLState {
// Errors and worker-side log lines are raised HERE, on the GL thread, at the first
// join of the job that produced them - which for a single shader is trivially the
// order a serial implementation would have produced them in.
//
// ApplyDeferredDiagnostics DRAINS, so a node shared by several shader objects
// (stage 6) replays its worker-side log line exactly once, at whichever object joins
// first. That is the honest report - one compile ran - and it is log text only: the
// GL-observable half of a failure, COMPILE_STATUS and the info log, lives in
// `artifacts` and every sharer reads the identical copy of it.
MG_Util::Async::ApplyDeferredDiagnostics(*m_compiled);
// A node that settled as Cancelled published nothing. Dropping it here is what keeps
// the object's state machine to two reachable cases - "no job" and "a job that
// completed" - so every reader below can treat a live node as authoritative.
if (!m_compiled->IsComplete()) m_compiled.reset();
//
// Through DropCompileNode, not a bare reset: this object is letting the node go, so
// its adopter slot has to go with it. A node shared with another object stays alive
// and gets dropped once more when that object joins - once per holder, never twice
// for the same one, because DropCompileNode is null-guarded.
if (!m_compiled->IsComplete()) DropCompileNode();
}
void ShaderObject::AdoptCompileNode(SharedPtr<ShaderCompileTask> node) const {
// Never overwrite a hold without giving its slot back first.
DropCompileNode();
m_compiled = Move(node);
m_compiled->AddAdopter();
// Re-arm the join gate: whether this node was just created or just adopted from
// another object, THIS object has not pulled its result yet. (An adopted node may
// already be terminal - the join then only replays what is left of its diagnostics.)
m_compileJoined = false;
}
void ShaderObject::DropCompileNode() const {
if (!m_compiled) return;
m_compiled->ReleaseAdopter();
m_compiled.reset();
}
void ShaderObject::InvalidateCompiledState() {
// The job node holds exactly what one Compile() produces, so discarding it IS the
// invalidation - and it re-arms nothing, so the next Compile() genuinely recompiles.
m_compiled.reset();
DropCompileNode();
}
void ShaderObject::CancelCompile() {
if (!m_compiled || m_compiled->IsTerminal()) return;
// Cooperative and non-blocking. A node that no worker has picked up settles
// immediately; one that is running is flagged and settles when its body returns,
// writing only into itself the whole time.
void ShaderObject::ReleaseCompileNode() {
if (!m_compiled) return;
// Already terminal: there is nothing left to stop, so this is not a release at all -
// the node and this object's claim on it both stay. That early return is older than
// stage 6 and it is load-bearing: ProgramState::ReleaseShaderNameIfOrphaned calls
// this on a shader whose name is going away but whose object a ProgramObject may
// still hold, and dropping a COMPLETED compile there would turn that program's link
// into GL_FALSE.
if (m_compiled->IsTerminal()) return;
// Two independent claimants have to be checked before a cancel, and this object is
// authorized to cancel only if BOTH say the result has become unobservable.
//
// Unless a pending LINK is waiting on it. Cancelling is about discarding a result
// nothing can observe any more, and this object is no longer the only route to this
// one: an enqueued ProgramLinkTask holds the node as a dependency, and a cancel would
// turn its link into GL_FALSE. Reached by the ordinary link-then-detach-then-delete
// shader teardown - see ShaderCompileTask::MarkLinkReferenced. Dropping our own
// reference is still right; the link keeps the node alive and finishes it.
if (!m_compiled->IsLinkReferenced()) m_compiled->Cancel();
m_compiled.reset();
// 1. Other shader objects. From stage 6 a node can be SHARED by several GL shader
// names that were handed byte-identical source; cancelling here would turn a
// compile they must still see as GL_TRUE into GL_FALSE. Only the releaser that
// takes the count to zero - i.e. the last holder - may cancel. See
// ShaderCompileTask::AddAdopter for why a plain Int is sound here and for the
// exactness argument under the worker race.
// 2. A pending LINK. An enqueued ProgramLinkTask holds the node in its input snapshot
// and a cancel would turn its link into GL_FALSE; reached by the ordinary
// link-then-detach-then-delete shader teardown. See MarkLinkReferenced. Never
// cleared, so this is a one-way pin.
//
// The cancel itself is cooperative and non-blocking, exactly as before: a node no
// worker has picked up settles immediately, a running one is flagged and settles when
// its body returns, writing only into itself the whole time.
if (m_compiled->AdopterCount() == 1 && !m_compiled->IsLinkReferenced()) m_compiled->Cancel();
DropCompileNode();
}
void ShaderObject::Compile() {
@@ -103,14 +147,6 @@ namespace MobileGL::MG_State::GLState {
// source instead. Same result, one parse either way.
if (HasMemoizedCompile()) return;
// The compile-environment snapshot is taken HERE, on the GL thread, and handed to
// the job. Everything the pipeline needs to know about the device comes through it,
// never through pActiveBackendObject - that is what makes the body movable.
m_compiled = MakeShared<ShaderCompileTask>(m_stage, m_source, ShaderPreprocessCache::HashSource(*m_source),
MG_Util::ShaderTranspiler::GetCurrentCompileEnv(),
m_preprocessCache, m_externalIndex);
m_compileJoined = false;
// Two reasons to stay on this thread, one rule. Without the async flag the whole
// path must be byte-identical to the synchronous implementation, and a cache-less
// object is an internal shader that compiles and reads its status in the same
@@ -119,7 +155,48 @@ namespace MobileGL::MG_State::GLState {
// has to put compilation back on this thread even though the extension is still
// advertised, and that is exactly what makes the GL_COMPLETION_STATUS_KHR the
// extension mandates after a zero count (immediately GL_TRUE) fall out for free.
if (!m_preprocessCache || !MG_Util::Async::AsyncShaderCompileActive()) {
//
// Hoisted above the node construction because stage 6 keys off it too: this same
// answer decides whether the adoption map is consulted at all, so a
// glMaxShaderCompilerThreadsKHR(0) and a flag-off build both bypass sharing exactly
// as they bypass the pool, and their behaviour stays byte-identical to pre-stage-6.
const Bool runOnPool = m_preprocessCache && MG_Util::Async::AsyncShaderCompileActive();
// The compile-environment snapshot is taken HERE, on the GL thread, and handed to
// the job. Everything the pipeline needs to know about the device comes through it,
// never through pActiveBackendObject - that is what makes the body movable.
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv> env =
MG_Util::ShaderTranspiler::GetCurrentCompileEnv();
const Uint64 sourceHash = ShaderPreprocessCache::HashSource(*m_source);
// ---- P1 stage 6: adopt an equivalent compile instead of enqueueing a duplicate ----
// ~21% of all glCompileShader calls in the shaderpack corpus are a DIFFERENT shader
// object handed byte-identical source. P0b's memo only pays off once one of them has
// finished; under async they are all enqueued in the same burst, so without this each
// one runs the whole pipeline on its own worker. The map hands back the node the
// first of them created - in flight or already complete - and this object simply
// holds it too.
if (runOnPool && m_adoptionMap) {
if (SharedPtr<ShaderCompileTask> shared =
m_adoptionMap->FindAdoptable(m_stage, sourceHash, *m_source, env->fingerprint)) {
// Take the node's own source snapshot as ours. FindAdoptable just compared
// the two strings in full, so this changes nothing observable - but it is not
// optional: the layer-1 memo (HasMemoizedCompile) is a POINTER comparison
// against the node's snapshot, so leaving our own equal-but-distinct copy in
// place would make the very next glCompileShader on this object decide it had
// no memo and enqueue the duplicate this whole stage exists to avoid - and
// would make an identical glShaderSource re-source cancel a shared compile.
// It also collapses N copies of a ~100 KB shaderpack stage into one.
m_source = shared->source;
AdoptCompileNode(Move(shared));
return;
}
}
AdoptCompileNode(MakeShared<ShaderCompileTask>(m_stage, m_source, sourceHash, env, m_preprocessCache,
m_externalIndex));
if (!runOnPool) {
m_compiled->RunInline();
// Inline means the node is already terminal, so this join only replays
// diagnostics; it is here so the synchronous and asynchronous paths publish
@@ -127,6 +204,10 @@ namespace MobileGL::MG_State::GLState {
EnsureCompileJoined();
return;
}
// Registered BEFORE the post, so the very next glCompileShader in this burst can
// adopt it however fast a worker picks it up. Registration is an index entry only -
// the map holds a WeakPtr and never keeps a node alive.
if (m_adoptionMap) m_adoptionMap->Register(m_compiled);
MG_Util::Async::ShaderCompilePool::Get().Post(m_compiled);
}
@@ -10,6 +10,7 @@
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderStage.h>
#include <MG_State/GLState/ProgramState/ShaderCompileTask.h>
#include <MG_State/GLState/ProgramState/ShaderCompileAdoptionMap.h>
namespace MobileGL {
namespace MG_State::GLState {
@@ -31,13 +32,32 @@ namespace MobileGL {
// add a round trip. Shared ownership rather than a raw pointer: a compile job
// outlives neither the object nor the context deterministically, and the cache
// has to stay alive for whoever is still reading it.
//
// `adoptionMap` is the same context's stage-6 index of adoptable compile nodes.
// It is non-null exactly when `preprocessCache` is (ProgramState hands both out
// together, and nobody else hands out either), which is what makes "no cache"
// keep meaning "compile inline, share nothing": an internal shader object has
// neither, so it neither adopts nor registers and its path is byte-identical to
// the pre-stage-6 one. GL-thread-only, so unlike the cache it carries no lock -
// shared ownership only because a ShaderObject may outlive the context's tables.
ShaderObject(const ShaderStage stage, Uint externalIndex,
SharedPtr<ShaderPreprocessCache> preprocessCache = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(Move(preprocessCache)) {}
SharedPtr<ShaderPreprocessCache> preprocessCache = nullptr,
SharedPtr<ShaderCompileAdoptionMap> adoptionMap = nullptr)
: m_stage(stage), m_externalIndex(externalIndex), m_preprocessCache(Move(preprocessCache)),
m_adoptionMap(Move(adoptionMap)) {}
// Cancel-not-join: the node owns its inputs, so an in-flight compile whose
// object just went away is safe to abandon where it stands. Nothing can observe
// its result any more - this object was the only route to it.
~ShaderObject() { CancelCompile(); }
// its result any more - unless another shader object adopted the same node, or a
// link pinned it, which is precisely what ReleaseCompileNode() checks.
~ShaderObject() {
ReleaseCompileNode();
// ReleaseCompileNode KEEPS a node that has already gone terminal - there is
// nothing left to stop, so it is not a release at all. This object is going
// away regardless, so hand the adopter slot back here. That is what keeps
// ShaderCompileTask::AdopterCount() exactly "how many live ShaderObjects hold
// this node" instead of merely an upper bound.
DropCompileNode();
}
ShaderObject(const ShaderObject&) = delete;
ShaderObject& operator=(const ShaderObject&) = delete;
@@ -45,10 +65,17 @@ namespace MobileGL {
void SetShaderSource(const String& source);
void SetShaderSource(String&& source);
void Compile();
// Drops a compile that is still in flight, without waiting for it. Called at the
// points where the object's compiled state stops being observable: a real source
// change, and the release of an orphaned shader name.
void CancelCompile();
// Gives up this object's claim on its compile node, cancelling the node only if
// this object was its LAST claimant. Called at the points where the object's
// compiled state stops being observable through THIS name: a real source change,
// and the release of an orphaned shader name.
//
// Named for what it does rather than for what it used to do: before stage 6 a
// node had exactly one shader object, so giving up the claim and cancelling the
// compile were the same act and this was CancelCompile(). They are not the same
// act any more - see ShaderCompileTask::AddAdopter for the count discipline and
// its single-threadedness argument. Never waits, in either case.
void ReleaseCompileNode();
void MarkAsDeleted();
// The compile job node itself, for ProgramObject::Link()'s input snapshot.
@@ -146,6 +173,16 @@ namespace MobileGL {
}
void InvalidateCompiledState();
// ---- the ONLY two writers of m_compiled (P1 stage 6) ----
// Every adopter-count mutation lives in these two, which is what makes "exactly
// one AddAdopter per hold, exactly one ReleaseAdopter per hold" auditable rather
// than something review has to re-derive at each call site. DropCompileNode is
// null-guarded, so calling it on an object that already let go is a no-op and a
// double release is unrepresentable.
void AdoptCompileNode(SharedPtr<ShaderCompileTask> node) const;
void DropCompileNode() const;
// ---- P0b layer 1: per-object no-op recompile ----
// True iff `candidate` is byte-identical to the source that produced (or is
// producing) the compiled state this object currently holds.
@@ -165,17 +202,31 @@ namespace MobileGL {
// running compile cannot race its storage - and the layer-1 memo collapses to a
// pointer comparison against the job's snapshot, because the setter only swaps
// the pointer when the text genuinely differs.
//
// Not necessarily unique to this object from stage 6 on: adopting a node also
// takes that node's source snapshot (see Compile()), so N shader objects sharing
// one compile share one copy of the text. The string is immutable and shared-
// owned, so that is invisible to every reader.
SharedPtr<const String> m_source = EmptySource();
// P0b layer 2: the owning context's cross-object memo, or null. Internally
// locked, because several workers hit it at once.
const SharedPtr<ShaderPreprocessCache> m_preprocessCache;
// P1 stage 6: the owning context's index of adoptable compile nodes, or null.
// Touched only from Compile(), i.e. only on the GL thread, so it carries no lock.
const SharedPtr<ShaderCompileAdoptionMap> m_adoptionMap;
Bool m_deleteStatus = false;
// ---- Compile OUTPUT ---- pending OR completed; reachable only through Compiled().
// Mutable because the join is a read-side operation: a const getter has to be
// able to settle an outstanding job before answering.
//
// SHARED from stage 6 on: several shader objects holding byte-identical source
// under the same CompileEnv point at one node. Every read below still goes
// through the same join gate, and a second joiner finds the node already
// terminal, so nothing about the read path changes - only the release path does
// (ReleaseCompileNode).
mutable SharedPtr<ShaderCompileTask> m_compiled;
// Exactly-once latch for the pull above. Armed with every new job node, set by
// the one join that consumes it.
@@ -13,6 +13,7 @@
// Deliberately NOT ShaderObject.h: ShaderCompileTask.h needs this header, and ShaderObject.h
// needs ShaderCompileTask.h. Only ShaderStage was ever used from there.
#include <MG_State/GLState/ProgramState/ShaderStage.h>
#include <MG_State/GLState/ProgramState/ShaderSourceKey.h>
namespace MobileGL::MG_State::GLState {
// Where the shared, source-only half of ShaderObject::Compile() stopped. The two
@@ -112,29 +113,10 @@ namespace MobileGL::MG_State::GLState {
}
private:
struct Key {
ShaderStage stage = ShaderStage::Unknown;
Uint64 sourceHash = 0;
SizeT sourceLength = 0;
Uint64 envFingerprint = 0;
Bool operator==(const Key& other) const {
return stage == other.stage && sourceHash == other.sourceHash &&
sourceLength == other.sourceLength && envFingerprint == other.envFingerprint;
}
};
struct KeyHasher {
SizeT operator()(const Key& key) const {
// The source hash already spreads well; fold the two discriminators in so
// that same-hash-different-stage/length keys land in different buckets.
Uint64 mixed = key.sourceHash;
mixed ^= static_cast<Uint64>(key.sourceLength) + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
mixed ^= static_cast<Uint64>(static_cast<Int>(key.stage)) * 0xff51afd7ed558ccdull;
mixed ^= key.envFingerprint + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
return static_cast<SizeT>(mixed);
}
};
// Shared with ShaderCompileAdoptionMap so the two per-context memos cannot key
// themselves on different notions of "the same compile" - see ShaderSourceKey.h.
using Key = ShaderSourceKey;
using KeyHasher = ShaderSourceKeyHasher;
struct Entry {
Key key;
@@ -0,0 +1,51 @@
// MobileGL - MobileGL/MG_State/GLState/ProgramState/ShaderSourceKey.h
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
#pragma once
#include <Includes.h>
#include <MG_State/GLState/ProgramState/ShaderStage.h>
namespace MobileGL::MG_State::GLState {
// The identity of "one glCompileShader's worth of input" - the tuple that decides
// whether two compiles must produce byte-identical results. Shared by the two
// per-context memos keyed on it, so that neither can drift from the other:
// * P0b's ShaderPreprocessCache, which memoizes the source-only half of a compile;
// * P1 stage 6's ShaderCompileAdoptionMap, which shares the job NODE itself.
//
// The 64-bit source hash is a LOOKUP ACCELERATOR ONLY. Every user of this key confirms
// a candidate hit with a full byte comparison of the stored source before honoring it,
// so a hash collision degrades to a miss and never to a wrong answer. That rule is not
// negotiable - see the memo-hazard notes on ShaderPreprocessCache.
//
// envFingerprint is part of the identity because the pipeline's compute local-size
// verdict is computed against CompileEnv's device limits: a memo must never be handed
// back under an environment other than the one it was computed against.
struct ShaderSourceKey {
ShaderStage stage = ShaderStage::Unknown;
Uint64 sourceHash = 0;
SizeT sourceLength = 0;
Uint64 envFingerprint = 0;
Bool operator==(const ShaderSourceKey& other) const {
return stage == other.stage && sourceHash == other.sourceHash &&
sourceLength == other.sourceLength && envFingerprint == other.envFingerprint;
}
};
struct ShaderSourceKeyHasher {
SizeT operator()(const ShaderSourceKey& key) const {
// The source hash already spreads well; fold the three discriminators in so
// that same-hash-different-stage/length/env keys land in different buckets.
Uint64 mixed = key.sourceHash;
mixed ^= static_cast<Uint64>(key.sourceLength) + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
mixed ^= static_cast<Uint64>(static_cast<Int>(key.stage)) * 0xff51afd7ed558ccdull;
mixed ^= key.envFingerprint + 0x9e3779b97f4a7c15ull + (mixed << 6) + (mixed >> 2);
return static_cast<SizeT>(mixed);
}
};
} // namespace MobileGL::MG_State::GLState
@@ -8,7 +8,18 @@
#include "VertexArrayObject.h"
#include <atomic>
namespace MobileGL::MG_State::GLState {
// Starts at 1 so a zero-initialized memo slot can never carry a live object's id.
// Atomic because VAOs are GL-thread-only today but the counter costs nothing to
// make safe, and a duplicate id would resurrect exactly the bug it exists to kill.
static std::atomic<Uint64> s_nextVertexArrayLifetimeId{1};
Uint64 VertexArrayObject::AllocateLifetimeId() {
return s_nextVertexArrayLifetimeId.fetch_add(1, std::memory_order_relaxed);
}
VertexArrayObject::VertexArrayObject(Uint externIndex) : m_externalIndex(externIndex) {
for (int index = 0; index < MAX_VERTEX_ATTRIBS; ++index) {
auto& attr = m_attributes[index];
@@ -84,6 +84,18 @@ namespace MobileGL {
Uint GetExternalIndex() const;
// Globally-unique, never-reused id for THIS object's lifetime - the same
// contract as ProgramObject::GetLifetimeId(), and needed for the same
// reason. Neither the GL name (freed to a LIFO list and handed straight
// back by the next glGenVertexArrays) nor the heap address (freed to the
// allocator and handed straight back by the next allocation of this size)
// can tell a deleted-and-recreated VAO from the original, so a backend
// memo keyed on either one silently inherits the dead object's contents.
// That is not hypothetical: it is what let a transform-feedback capture
// fetch a destroyed VAO's vertex buffer slice (see the VaoDrawMemo key in
// DirectVulkan's VulkanRenderer).
Uint64 GetLifetimeId() const { return m_lifetimeId; }
void SetAttributeDivisor(Uint index, Uint divisor);
Uint GetAttributeDivisor(Uint index) const;
@@ -185,7 +197,10 @@ namespace MobileGL {
return mapping;
}
static Uint64 AllocateLifetimeId();
const Uint m_externalIndex = 0;
const Uint64 m_lifetimeId = AllocateLifetimeId();
Array<VertexAttribute, MAX_VERTEX_ATTRIBS> m_attributes;
Array<VertexAttributeVersion, MAX_VERTEX_ATTRIBS> m_attributeVersions;
BindingSlot<BufferObject> m_indexBufferBindingSlot;
+3
View File
@@ -66,6 +66,9 @@ gtest_discover_tests(SanityTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
add_subdirectory(BackendLoader)
add_subdirectory(Buffer)
# The heap-address-is-not-an-identity invariant the backends' per-object memos
# rest on. No GL context, no driver: it only needs the allocator.
add_subdirectory(State)
add_subdirectory(EGLState)
add_subdirectory(Framebuffer)
add_subdirectory(Texture)
+40
View File
@@ -60,6 +60,22 @@ target_link_libraries(
${LINK_LIBRARIES}
)
add_executable(
ShaderCompileAdoptionTest
ShaderCompileAdoptionTest.cpp
)
target_include_directories(ShaderCompileAdoptionTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
ShaderCompileAdoptionTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
add_executable(
ParallelShaderCompileTest
ParallelShaderCompileTest.cpp
@@ -76,6 +92,25 @@ target_link_libraries(
${LINK_LIBRARIES}
)
# Its own binary so the "fresh process" isolation level in it is really available
# through --gtest_filter, and so its 60 A->B link pairs cannot perturb another
# suite's per-context caches.
add_executable(
XfbFrontendOrderInvarianceTest
XfbFrontendOrderInvarianceTest.cpp
)
target_include_directories(XfbFrontendOrderInvarianceTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
XfbFrontendOrderInvarianceTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
# Its own binary on purpose: this one calls MobileGL::Destroy(), and ShaderCompilePool's
# stop is a one-way latch for the whole process - every case declared after it in the same
# binary would silently run its compiles and links inline.
@@ -113,6 +148,11 @@ gtest_discover_tests(ProgramTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
# compile pool so there is something in flight to race against.
gtest_discover_tests(AsyncCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncLinkTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
# Same reason: the stage-6 cases keep a backlog in flight so a release really can race a
# worker, and the 48-object stress links every one of them.
gtest_discover_tests(ShaderCompileAdoptionTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
# Same reason: the GL_COMPLETION_STATUS_KHR cases saturate a one-worker pool on purpose.
gtest_discover_tests(ParallelShaderCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncTeardownTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
# Same reason again: several cases leave A links outstanding while B compiles and links.
gtest_discover_tests(XfbFrontendOrderInvarianceTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
+14 -2
View File
@@ -20,6 +20,7 @@
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_State/GLState/ProgramState/ShaderPreprocessCache.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#include "MG_Util/ShaderTranspiler/ShaderCompiler.h"
using namespace MobileGL;
@@ -3006,8 +3007,19 @@ TEST_F(ProgramTest, TwoShaderObjectsWithIdenticalSourceLinkIndependently) {
ASSERT_NE(objectA, nullptr);
ASSERT_NE(objectB, nullptr);
EXPECT_EQ(objectA->GetShaderSource(), objectB->GetShaderSource());
// Independent parses despite the shared preprocess.
EXPECT_NE(objectA->GetCompiledShader(), objectB->GetCompiledShader());
// P0b's layer 2 shares the PREPROCESS and never the parse: glslang's TShader is
// consume-once, so a memo hit still has to parse for itself.
//
// P1 stage 6 shares something stronger when it is active - the whole compile JOB, and
// therefore the single parse that job produced - and that sharing is made safe by
// ShaderCompileTask::ClaimParsedShader's CAS instead, exactly as it already was for one
// shader object attached to two programs. ShaderCompileAdoptionTest is where that is
// pinned down (it links both objects and compares the generated SPIR-V). So the
// one-parse-per-object assertion belongs to the non-adopting path; the two independent
// LINKS below are what both modes have to agree on, and they are the point of this case.
if (!MG_Util::Async::AsyncShaderCompileActive()) {
EXPECT_NE(objectA->GetCompiledShader(), objectB->GetCompiledShader());
}
EXPECT_NE(objectA->GetCompiledShader(), nullptr);
EXPECT_NE(objectB->GetCompiledShader(), nullptr);
@@ -0,0 +1,910 @@
// MobileGL - MobileGL/MG_Test/Program/ShaderCompileAdoptionTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 6: two shader objects handed byte-identical source share ONE compile job.
//
// The property under test is a conjunction, and every case here attacks one half of it:
// * the sharing itself - one job, one node, both GL names reporting the same answer, and
// two programs linking that one node to byte-identical SPIR-V;
// * that sharing did not make a cancel dangerous. Before this stage a node had exactly one
// shader object, so "this object stopped caring" and "nothing can observe this result"
// were the same statement and CancelCompile() cancelled unconditionally. They are not the
// same statement any more, and the four mutation paths that used to reach that cancel -
// re-source, delete, the orphan-name sweep, the destructor - are each covered below with
// a second object still holding the node.
//
// Like the other async suites, every case flips MG_Config::Features.AsyncShaderCompile itself
// and drives the real GL entry points, so the file behaves identically whether or not the
// suite was launched with MOBILEGL_ASYNC_SHADER_COMPILE=1.
//
// Adoption is decided ON THE GL THREAD, before anything is posted, so the counter assertions
// here are deterministic rather than timing-dependent: whether the first object's compile has
// already finished changes nothing about whether the second one adopts it.
#include <gtest/gtest.h>
#include <chrono>
#include <string>
#include <thread>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_State/GLState/ProgramState/ShaderCompileAdoptionMap.h"
#include "MG_State/GLState/ProgramState/ShaderCompileTask.h"
#include "MG_State/GLState/ProgramState/ShaderPreprocessCache.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#include "MG_Util/ShaderTranspiler/CompileEnv.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
using MobileGL::MG_State::GLState::ShaderCompileAdoptionMap;
using MobileGL::MG_State::GLState::ShaderCompileTask;
using MobileGL::MG_State::GLState::ShaderObject;
using MobileGL::MG_State::GLState::ShaderPreprocessCache;
namespace {
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async) : m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
// The suspension latch and the concurrency budget are PROCESS-wide, so a case that
// touches either has to put both back or it poisons every case after it in this binary.
class CompilerThreadScope {
public:
CompilerThreadScope() = default;
~CompilerThreadScope() {
MG_Util::Async::SetAsyncShaderCompileSuspended(false);
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(
MG_Util::Async::ShaderCompilePool::Get().GetThreadCount());
}
CompilerThreadScope(const CompilerThreadScope&) = delete;
CompilerThreadScope& operator=(const CompilerThreadScope&) = delete;
};
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform mat4 uModel;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = uModel * vec4(aPos, 1.0);
}
)";
// Fails inside glslang rather than in the lexical pre-checks, so it exercises the same
// ParseFailed path a real broken shaderpack source takes.
const char* kBrokenFs = R"(#version 460
layout(location = 0) out vec4 fragColor;
void main() { fragColor = thisIdentifierWasNeverDeclared; }
)";
// Big enough that a compile is not instantaneous, so a duplicate really would cost
// something. Templated on an index so every instance is a distinct source.
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 220; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
// Heavy enough that a spinning GL thread can reliably observe the compile Running on a
// single-worker pool, for RunningCancelRequestedNodeIsNotAdopted below - MakeBulkySource
// is tuned for "not instantaneous", this one is tuned for "actually spin-observable".
String MakeVeryHeavySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 4000; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
Uint64 AdoptionCount() {
return MG_State::pGLContext->GetShaderCompileAdoptionMap().GetAdoptionCount();
}
// A copy of the slot, never the reference: creating another shader can reallocate the
// context's object table.
SharedPtr<ShaderObject> Object(const GLuint shader) {
return MG_State::pGLContext->GetShaderObject(shader);
}
// The node identity, WITHOUT joining - this is what "they share one job" means, and
// asking must not settle anything.
const ShaderCompileTask* NodeOf(const GLuint shader) {
const SharedPtr<ShaderObject> object = Object(shader);
return object ? object->CompiledNodeForLink().get() : nullptr;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
return shader;
}
GLuint MakeAndCompile(const GLenum type, const char* source) {
const GLuint shader = MakeShader(type, source);
CompileShader(shader);
return shader;
}
GLint QueryCompileStatus(const GLuint shader) {
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
String QueryShaderInfoLog(const GLuint shader) {
GLint length = 0;
GetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return String();
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetShaderInfoLog(shader, length, &written, buffer.data());
return String(buffer.data(), static_cast<size_t>(written));
}
GLint QueryLinkStatus(const GLuint program) {
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
return status;
}
// Content hash of a linked program's generated SPIR-V, through the state layer (there is
// no GL query for it). This is what catches a mis-shared parse: if the claim CAS on a
// SHARED node let two links both run mapIO over the same intermediate, the two programs
// would disagree here.
Vector<Uint64> SpirvDigest(const GLuint program) {
const auto& object = MG_State::pGLContext->GetProgramObject(program);
Vector<Uint64> digest;
if (!object) return digest;
for (const auto& module : object->GetGeneratedSpirv()) {
Uint64 hash = 1469598103934665603ull;
for (const unsigned word : module) {
hash = (hash ^ static_cast<Uint64>(word)) * 1099511628211ull;
}
digest.push_back(hash);
}
return digest;
}
// Enqueues `count` distinct heavy compiles and reads nothing back, so the pool is left
// with a real backlog for the caller's mutations to race against.
void SaturatePool(const int count, Vector<String>& sourceStorage) {
sourceStorage.reserve(sourceStorage.size() + static_cast<SizeT>(count));
for (int i = 0; i < count; ++i) {
sourceStorage.push_back(MakeBulkySource(90000 + i));
const char* text = sourceStorage.back().c_str();
const GLuint shader = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(shader, 1, &text, nullptr);
CompileShader(shader);
}
}
// Links `shader` against a freshly compiled vertex stage and returns the program.
GLuint LinkWith(const GLuint shader) {
const GLuint vs = MakeAndCompile(GL_VERTEX_SHADER, kVs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, shader);
LinkProgram(program);
return program;
}
class ShaderCompileAdoptionTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
};
} // namespace
// ---------------------------------------------------------------------------------------
// The sharing itself
// ---------------------------------------------------------------------------------------
// The headline: two GL shader names, byte-identical source, exactly one job. Both names must
// answer every query correctly, and the ONE parse they share must link into two separate
// programs with byte-identical SPIR-V - which is the stage-4 claim CAS being exercised on a
// shared node for the first time.
TEST_F(ShaderCompileAdoptionTest, TwoObjectsWithIdenticalSourceShareOneCompileJob) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(32, backlog);
const String source = MakeBulkySource(100);
const char* text = source.c_str();
const Uint64 before = AdoptionCount();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
EXPECT_EQ(AdoptionCount() - before, 1u) << "the second glCompileShader must not enqueue a duplicate";
ASSERT_NE(NodeOf(a), nullptr);
EXPECT_EQ(NodeOf(a), NodeOf(b)) << "both objects must hold the very same job node";
// Both names still answer for themselves.
EXPECT_EQ(QueryCompileStatus(a), GL_TRUE) << QueryShaderInfoLog(a);
EXPECT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
EXPECT_EQ(QueryShaderInfoLog(a), QueryShaderInfoLog(b));
EXPECT_TRUE(QueryShaderInfoLog(a).empty());
// One node, two links: exactly one of them wins ClaimParsedShader, the other re-parses,
// and the two must agree bit for bit.
const GLuint programA = LinkWith(a);
const GLuint programB = LinkWith(b);
ASSERT_EQ(QueryLinkStatus(programA), GL_TRUE);
ASSERT_EQ(QueryLinkStatus(programB), GL_TRUE);
const Vector<Uint64> digestA = SpirvDigest(programA);
const Vector<Uint64> digestB = SpirvDigest(programB);
ASSERT_EQ(digestA.size(), 2u);
EXPECT_EQ(digestA, digestB) << "a shared node linked twice produced different SPIR-V";
EXPECT_GE(GetUniformLocation(programA, "uSeed100"), 0);
EXPECT_GE(GetUniformLocation(programB, "uSeed100"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Adoption must also re-arm the adopter's layer-1 memo. It is a POINTER comparison against
// the node's own source snapshot, so an adopter that kept its own equal-but-distinct copy
// would decide on the very next glCompileShader that it had no memo and enqueue the exact
// duplicate this stage exists to remove - and an identical glShaderSource would cancel a
// compile another object is still waiting on.
TEST_F(ShaderCompileAdoptionTest, AdoptingAlsoArmsTheLayerOneMemo) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(32, backlog);
const String source = MakeBulkySource(110);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
const SharedPtr<ShaderObject> objectB = Object(b);
ASSERT_NE(objectB, nullptr);
EXPECT_TRUE(objectB->HasMemoizedCompile()) << "an adopted node must satisfy the layer-1 memo";
const ShaderCompileTask* shared = NodeOf(b);
const Uint64 before = AdoptionCount();
for (int i = 0; i < 4; ++i) {
CompileShader(b);
EXPECT_EQ(NodeOf(b), shared) << "a repeat glCompileShader on an adopter must be a no-op";
}
// A byte-identical re-source is a no-op too, so it must not disturb the shared node.
ShaderSource(b, 1, &text, nullptr);
EXPECT_EQ(NodeOf(b), shared);
EXPECT_EQ(AdoptionCount(), before) << "no-op calls must not even reach the adoption map";
EXPECT_EQ(QueryCompileStatus(a), GL_TRUE) << QueryShaderInfoLog(a);
EXPECT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Different source, and same source in a different STAGE, are different keys. This is the
// guard against the map ever handing out a node that does not belong to the caller.
TEST_F(ShaderCompileAdoptionTest, DifferentSourceOrStageIsNotAdopted) {
const AsyncModeScope async(true);
const String first = MakeBulkySource(120);
const String second = MakeBulkySource(121);
const char* firstText = first.c_str();
const char* secondText = second.c_str();
const Uint64 before = AdoptionCount();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &firstText, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &secondText, nullptr);
CompileShader(b);
EXPECT_EQ(AdoptionCount(), before) << "different text must not adopt";
EXPECT_NE(NodeOf(a), NodeOf(b));
// The same text in two stages: the vertex/fragment pair below shares no node either,
// because the stage is part of the key.
const GLuint vsA = MakeAndCompile(GL_VERTEX_SHADER, kVs);
const GLuint vsB = MakeAndCompile(GL_VERTEX_SHADER, kVs);
EXPECT_EQ(NodeOf(vsA), NodeOf(vsB)) << "same stage, same text: must share";
EXPECT_NE(NodeOf(vsA), NodeOf(a));
EXPECT_EQ(QueryCompileStatus(a), GL_TRUE) << QueryShaderInfoLog(a);
EXPECT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// A failed compile is shared exactly like a successful one, and both names must report the
// identical status and the identical log - the info log lives in the node's artifacts, so
// this is also the guard that a second joiner is not left with an empty one.
TEST_F(ShaderCompileAdoptionTest, AdoptedFailingCompileReportsTheIdenticalLogToBothObjects) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(32, backlog);
const Uint64 before = AdoptionCount();
const GLuint a = MakeAndCompile(GL_FRAGMENT_SHADER, kBrokenFs);
const GLuint b = MakeAndCompile(GL_FRAGMENT_SHADER, kBrokenFs);
EXPECT_EQ(AdoptionCount() - before, 1u);
EXPECT_EQ(NodeOf(a), NodeOf(b));
EXPECT_EQ(QueryCompileStatus(a), GL_FALSE);
EXPECT_EQ(QueryCompileStatus(b), GL_FALSE);
const String logA = QueryShaderInfoLog(a);
EXPECT_FALSE(logA.empty());
EXPECT_EQ(QueryShaderInfoLog(b), logA);
// GL models a failed compile as status + log, never as a GL error - which is what makes
// moving the work off-thread (and sharing it) legal at all.
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// The four release paths, each with a second object still holding the node
// ---------------------------------------------------------------------------------------
// glShaderSource with DIFFERENT text on one sharer. Its release must NOT cancel the node the
// other one is still waiting on; the re-sourced object gets a fresh compile of its own.
TEST_F(ShaderCompileAdoptionTest, ResourcingOneSharerLeavesTheOtherIntact) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String shared = MakeBulkySource(200);
const char* sharedText = shared.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &sharedText, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &sharedText, nullptr);
CompileShader(b);
const ShaderCompileTask* sharedNode = NodeOf(b);
ASSERT_NE(sharedNode, nullptr);
ASSERT_EQ(NodeOf(a), sharedNode);
// Replace A's text while the shared compile is very probably still outstanding.
const String replacement = MakeBulkySource(201);
const char* replacementText = replacement.c_str();
ShaderSource(a, 1, &replacementText, nullptr);
EXPECT_EQ(NodeOf(a), nullptr) << "a real source change must drop the object's node";
EXPECT_EQ(NodeOf(b), sharedNode) << "B must still hold the shared node";
// B is untouched: the compile it is waiting on still publishes, and its artifacts are
// the ones that source really produces.
ASSERT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
const GLuint programB = LinkWith(b);
ASSERT_EQ(QueryLinkStatus(programB), GL_TRUE);
EXPECT_GE(GetUniformLocation(programB, "uSeed200"), 0);
// A gets a genuinely fresh compile of the new text.
CompileShader(a);
EXPECT_NE(NodeOf(a), sharedNode);
ASSERT_EQ(QueryCompileStatus(a), GL_TRUE) << QueryShaderInfoLog(a);
const GLuint programA = LinkWith(a);
ASSERT_EQ(QueryLinkStatus(programA), GL_TRUE);
EXPECT_GE(GetUniformLocation(programA, "uSeed201"), 0);
EXPECT_EQ(GetUniformLocation(programA, "uSeed200"), -1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glDeleteShader on one sharer. The name goes immediately (no wait for a worker) and the
// object is destroyed, so this covers the DESTRUCTOR release as well as the orphan sweep's.
TEST_F(ShaderCompileAdoptionTest, DeletingOneSharerLeavesTheOtherIntact) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(210);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
const ShaderCompileTask* sharedNode = NodeOf(b);
ASSERT_NE(sharedNode, nullptr);
ASSERT_EQ(NodeOf(a), sharedNode);
DeleteShader(a);
EXPECT_EQ(IsShader(a), GL_FALSE) << "an unattached deleted shader's name goes immediately";
EXPECT_EQ(NodeOf(b), sharedNode);
ASSERT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
const GLuint program = LinkWith(b);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_GE(GetUniformLocation(program, "uSeed210"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The window DeletingOneSharerLeavesTheOtherIntact cannot reach: there, A's compile has
// always already finished (or not yet started) by the time B adopts, because the pool is
// merely BUSY with other backlog. Here A's OWN node is still Running - a worker is inside
// RunBody() for it - when the last holder releases it. ReleaseCompileNode fires Cancel(),
// but JobNode::Cancel on a Running node only sets the cancellation-REQUEST flag; the state
// stays Running until the worker's body returns and JobNode::Run forces the final transition
// to Cancelled (see JobNode::Run's tail: it takes Cancelled instead of Complete whenever
// m_cancelled is set, regardless of how the body finished). FindAdoptable must refuse a node
// in that in-between state - not just one already settled as Cancelled - or C inherits a
// doomed node and glGetShaderiv reports GL_FALSE with an empty info log for valid source.
TEST_F(ShaderCompileAdoptionTest, RunningCancelRequestedNodeIsNotAdopted) {
const AsyncModeScope async(true);
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(1);
const String source = MakeVeryHeavySource(310);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
// Spin on the GL thread until the single worker is actually inside A's body. The source
// is sized to make that window observable rather than instantaneous.
const ShaderCompileTask* node = NodeOf(a);
ASSERT_NE(node, nullptr);
bool sawRunning = false;
for (int i = 0; i < 200000 && !node->IsTerminal(); ++i) {
if (node->State() == MG_Util::Async::JobState::Running) {
sawRunning = true;
break;
}
std::this_thread::sleep_for(std::chrono::microseconds(20));
}
ASSERT_TRUE(sawRunning) << "could not observe A's compile Running; the synthetic source "
"needs to be heavier, or the pool did not have a free worker";
// A is the ONLY holder, so this release brings the adopter count to zero and (with no
// link pin) fires Cancel() on a node that is still Running.
DeleteShader(a);
ASSERT_EQ(node->State(), MG_Util::Async::JobState::Running)
<< "the node already settled; the race window closed before the assertions below "
"could observe it - widen MakeVeryHeavySource's loop count";
ASSERT_TRUE(node->IsCancellationRequested());
ASSERT_FALSE(node->IsCancelled()) << "the window this test targets does not exist here";
// A brand-new shader name, byte-identical source, nothing wrong with it.
const GLuint c = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(c, 1, &text, nullptr);
CompileShader(c);
EXPECT_NE(NodeOf(c), node) << "C adopted a cancellation-requested, still-Running node";
ASSERT_EQ(QueryCompileStatus(c), GL_TRUE)
<< "valid source reported GL_FALSE; info log: [" << QueryShaderInfoLog(c) << "]";
EXPECT_EQ(GetError(), GL_NO_ERROR);
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(
MG_Util::Async::ShaderCompilePool::Get().GetThreadCount());
}
// The deferred half of glDeleteShader: A is ATTACHED, so the delete only flags it and the
// name is freed by ReleaseShaderNameIfOrphaned when the detach removes the last GL-visible
// attachment. That sweep is the other caller of the release path, and it must not cancel the
// node B is sharing.
TEST_F(ShaderCompileAdoptionTest, OrphanSweepOnOneSharerLeavesTheOtherIntact) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(220);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
const ShaderCompileTask* sharedNode = NodeOf(b);
ASSERT_NE(sharedNode, nullptr);
ASSERT_EQ(NodeOf(a), sharedNode);
// Attach A, flag it for deletion (name survives), then detach: the sweep fires here, with
// NO link ever posted, so the stage-4 pin is NOT what is protecting the node - only the
// adopter count is.
const GLuint program = CreateProgram();
AttachShader(program, a);
DeleteShader(a);
EXPECT_EQ(IsShader(a), GL_TRUE) << "an attached deleted shader keeps its name";
DetachShader(program, a);
EXPECT_EQ(IsShader(a), GL_FALSE) << "the detach must free the flagged shader's name";
EXPECT_EQ(NodeOf(b), sharedNode);
ASSERT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
const GLuint programB = LinkWith(b);
ASSERT_EQ(QueryLinkStatus(programB), GL_TRUE);
EXPECT_GE(GetUniformLocation(programB, "uSeed220"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The same sweep, now with the stage-4 link pin also in play: A's program is LINKED (so the
// node is MarkLinkReferenced) and then A is detached and deleted, while B still shares the
// node. Both protections have to hold at once - the link must report GL_TRUE and B must
// still compile.
TEST_F(ShaderCompileAdoptionTest, OrphanSweepWithALinkPinnedSharedNodeHoldsBoth) {
const AsyncModeScope async(true);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(230);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
const ShaderCompileTask* sharedNode = NodeOf(b);
ASSERT_NE(sharedNode, nullptr);
ASSERT_EQ(NodeOf(a), sharedNode);
// The ordinary teardown order: link, then detach, then delete. No status read in between,
// so the link's own prologue is what joins the shared compile.
const GLuint vs = MakeAndCompile(GL_VERTEX_SHADER, kVs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, a);
LinkProgram(program);
DetachShader(program, a);
DeleteShader(a);
EXPECT_EQ(IsShader(a), GL_FALSE);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "the pinned shared compile must still publish";
EXPECT_GE(GetUniformLocation(program, "uSeed230"), 0);
EXPECT_EQ(NodeOf(b), sharedNode);
ASSERT_EQ(QueryCompileStatus(b), GL_TRUE) << QueryShaderInfoLog(b);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// Every sharer released, in turn, with nothing pinning the node: the LAST release is the one
// that may cancel, and afterwards the map must not hand the cancelled node to anybody. The
// property asserted is the one that matters and it is timing-free: whatever happened to the
// old node, a later object with the same source must end up with a CORRECT compile.
TEST_F(ShaderCompileAdoptionTest, AfterEverySharerIsGoneTheNextCompileIsStillCorrect) {
const AsyncModeScope async(true);
const CompilerThreadScope compilerThreads;
// One worker and a deep backlog: a node posted now is overwhelmingly likely to still be
// queued when its last holder drops it, which is the state in which the cancel bites.
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(1);
Vector<String> backlog;
SaturatePool(48, backlog);
const String source = MakeBulkySource(240);
const char* text = source.c_str();
const GLuint a = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(a, 1, &text, nullptr);
CompileShader(a);
const GLuint b = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(b, 1, &text, nullptr);
CompileShader(b);
ASSERT_EQ(NodeOf(a), NodeOf(b));
DeleteShader(a);
DeleteShader(b); // the last holder: this one is authorized to cancel
const GLuint c = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(c, 1, &text, nullptr);
CompileShader(c);
ASSERT_EQ(QueryCompileStatus(c), GL_TRUE)
<< "a cancelled node must never be adopted - it can only ever report GL_FALSE. "
<< QueryShaderInfoLog(c);
const GLuint program = LinkWith(c);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_GE(GetUniformLocation(program, "uSeed240"), 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// The bypasses: both must be byte-identical to the pre-stage-6 behaviour
// ---------------------------------------------------------------------------------------
// The kill switch. With the flag off, compilation is synchronous and NOTHING is adopted -
// the map is not even consulted, so the counter cannot move.
TEST_F(ShaderCompileAdoptionTest, FlagOffAdoptsNothing) {
const AsyncModeScope async(false);
ASSERT_FALSE(MG_Util::Async::AsyncShaderCompileEnabled());
const String source = MakeBulkySource(300);
const char* text = source.c_str();
const Uint64 before = AdoptionCount();
Vector<GLuint> shaders;
for (int i = 0; i < 6; ++i) {
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
EXPECT_EQ(AdoptionCount(), before) << "the flag-off path must not consult the adoption map";
for (SizeT i = 1; i < shaders.size(); ++i) {
EXPECT_NE(NodeOf(shaders[i]), NodeOf(shaders[0])) << "flag off means one node per object";
}
for (const GLuint fs : shaders) {
EXPECT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// glMaxShaderCompilerThreadsKHR(0) puts compilation back on the application's thread even
// though the extension stays advertised. Adoption keys off the same predicate, so a
// suspended context shares nothing either - which is what keeps a subsequent
// GL_COMPLETION_STATUS_KHR immediately GL_TRUE without any reasoning about shared nodes.
TEST_F(ShaderCompileAdoptionTest, SuspendedCompilationAdoptsNothing) {
const AsyncModeScope async(true);
const CompilerThreadScope compilerThreads;
MaxShaderCompilerThreadsKHR(0);
ASSERT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
const String source = MakeBulkySource(310);
const char* text = source.c_str();
const Uint64 before = AdoptionCount();
Vector<GLuint> shaders;
for (int i = 0; i < 4; ++i) {
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
GLint complete = GL_FALSE;
GetShaderiv(fs, GL_COMPLETION_STATUS_KHR, &complete);
EXPECT_EQ(complete, GL_TRUE) << "a zero compiler-thread count leaves nothing in flight";
}
EXPECT_EQ(AdoptionCount(), before);
for (SizeT i = 1; i < shaders.size(); ++i) {
EXPECT_NE(NodeOf(shaders[i]), NodeOf(shaders[0]));
}
for (const GLuint fs : shaders) {
EXPECT_EQ(QueryCompileStatus(fs), GL_TRUE) << QueryShaderInfoLog(fs);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// Stress
// ---------------------------------------------------------------------------------------
// The shaderpack shape: 48 objects over 6 distinct sources, all enqueued before anything is
// read, on a two-worker pool. 42 of the 48 compiles must simply vanish, and all 48 objects
// must still be individually correct - each with its own name, its own status, and its own
// link (which means 48 claims against 6 shared parses).
TEST_F(ShaderCompileAdoptionTest, StressFortyEightObjectsOverSixSources) {
const AsyncModeScope async(true);
const CompilerThreadScope compilerThreads;
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(2);
constexpr int kDistinct = 6;
constexpr int kDuplicates = 8;
Vector<String> sources;
sources.reserve(kDistinct);
for (int i = 0; i < kDistinct; ++i) {
sources.push_back(MakeBulkySource(400 + i));
}
const Uint64 before = AdoptionCount();
Vector<GLuint> shaders;
for (int duplicate = 0; duplicate < kDuplicates; ++duplicate) {
for (int i = 0; i < kDistinct; ++i) {
const char* text = sources[static_cast<SizeT>(i)].c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
}
const Uint64 adoptions = AdoptionCount() - before;
// The floor the stage contracts for, with room for any future scheduling slack...
ASSERT_GE(adoptions, 30u) << "48 objects over 6 sources adopted only " << adoptions << " times";
// ...and the number this design actually produces, because the decision is made on the GL
// thread before anything is posted and therefore does not depend on the workers at all.
EXPECT_EQ(adoptions, static_cast<Uint64>(kDistinct * (kDuplicates - 1)));
for (SizeT s = 0; s < shaders.size(); ++s) {
const GLuint fs = shaders[s];
ASSERT_EQ(QueryCompileStatus(fs), GL_TRUE) << "shader " << s << ": " << QueryShaderInfoLog(fs);
const GLuint program = LinkWith(fs);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "shader index " << s;
const String uniform = "uSeed" + std::to_string(400 + static_cast<int>(s % kDistinct));
EXPECT_GE(GetUniformLocation(program, uniform.c_str()), 0) << uniform;
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The adversarial interleaving, with duplicates everywhere: compile, query, re-source,
// re-compile, delete, all with the pool busy and most objects sharing nodes. Nothing here
// asserts timing - what it hunts for is a node cancelled out from under a sharer, which
// surfaces as a wrong status, a wrong uniform, or a crash.
TEST_F(ShaderCompileAdoptionTest, StressSharedNodesUnderResourceAndDelete) {
const AsyncModeScope async(true);
constexpr int kRounds = 6;
constexpr int kPerRound = 12;
for (int round = 0; round < kRounds; ++round) {
Vector<String> sources;
sources.reserve(4);
for (int i = 0; i < 4; ++i) {
sources.push_back(MakeBulkySource(round * 100 + i));
}
const String replacement = MakeBulkySource(round * 100 + 50);
const char* replacementText = replacement.c_str();
Vector<GLuint> shaders;
for (int i = 0; i < kPerRound; ++i) {
const char* text = sources[static_cast<SizeT>(i % 4)].c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
// Re-source a third of them onto ONE new shared source, so the survivors of each
// original node keep waiting on it while the movers pile onto a new one.
for (int i = 0; i < kPerRound; i += 3) {
ShaderSource(shaders[static_cast<SizeT>(i)], 1, &replacementText, nullptr);
CompileShader(shaders[static_cast<SizeT>(i)]);
}
// And delete another third outright, while their nodes are still shared.
for (int i = 1; i < kPerRound; i += 3) {
DeleteShader(shaders[static_cast<SizeT>(i)]);
}
for (int i = 0; i < kPerRound; ++i) {
if (i % 3 == 1) continue; // deleted
const GLuint shader = shaders[static_cast<SizeT>(i)];
ASSERT_EQ(QueryCompileStatus(shader), GL_TRUE)
<< "round " << round << " shader " << i << ": " << QueryShaderInfoLog(shader);
const GLuint program = LinkWith(shader);
ASSERT_EQ(QueryLinkStatus(program), GL_TRUE) << "round " << round << " shader " << i;
const String expected =
"uSeed" + std::to_string(i % 3 == 0 ? round * 100 + 50 : round * 100 + (i % 4));
EXPECT_GE(GetUniformLocation(program, expected.c_str()), 0)
<< "round " << round << " shader " << i << " expected " << expected;
DeleteProgram(program);
}
for (int i = 0; i < kPerRound; ++i) {
if (i % 3 != 1) DeleteShader(shaders[static_cast<SizeT>(i)]);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// ---------------------------------------------------------------------------------------
// The map itself, driven directly
// ---------------------------------------------------------------------------------------
// Two of the map's rules cannot be forced deterministically through the GL surface - a
// cancelled node depends on beating a worker to it, and a CompileEnv re-capture needs a
// backend swap. Both are unconditional properties of the class, so they are asserted here
// against the class.
namespace {
SharedPtr<ShaderCompileTask> MakeNode(const String& text, const ShaderStage stage,
const SharedPtr<const MG_Util::ShaderTranspiler::CompileEnv>& env) {
auto source = MakeShared<const String>(text);
const Uint64 hash = ShaderPreprocessCache::HashSource(*source);
return MakeShared<ShaderCompileTask>(stage, source, hash, env, nullptr, 0);
}
} // namespace
TEST(ShaderCompileAdoptionMapTest, RegisteredNodeIsAdoptedOnAnExactMatch) {
const auto& env = MG_Util::ShaderTranspiler::GetDefaultCompileEnv();
ShaderCompileAdoptionMap map;
const String text = "#version 460\nvoid main() {}\n";
const SharedPtr<ShaderCompileTask> node = MakeNode(text, ShaderStage::Fragment, env);
map.Register(node);
EXPECT_EQ(map.FindAdoptable(ShaderStage::Fragment, ShaderPreprocessCache::HashSource(text), text,
env->fingerprint),
node);
EXPECT_EQ(map.GetAdoptionCount(), 1u);
// Every discriminator in the key is load-bearing.
EXPECT_EQ(map.FindAdoptable(ShaderStage::Vertex, ShaderPreprocessCache::HashSource(text), text,
env->fingerprint),
nullptr);
const String other = text + "\n";
EXPECT_EQ(map.FindAdoptable(ShaderStage::Fragment, ShaderPreprocessCache::HashSource(other), other,
env->fingerprint),
nullptr);
EXPECT_EQ(map.GetAdoptionCount(), 1u) << "a miss must not count as an adoption";
}
// A memo must never be handed back under an environment other than the one it was computed
// against: the compute local-size verdict inside the pipeline reads CompileEnv's device
// limits, so a node captured under one backend's limits is not a valid answer under
// another's. The fingerprint is what enforces that, and it is part of the key.
TEST(ShaderCompileAdoptionMapTest, EnvFingerprintMismatchIsNotAdopted) {
const auto& env = MG_Util::ShaderTranspiler::GetDefaultCompileEnv();
ShaderCompileAdoptionMap map;
const String text = "#version 460\nvoid main() {}\n";
map.Register(MakeNode(text, ShaderStage::Fragment, env));
// A genuinely different environment: different device limits, hence a different
// fingerprint, hence a different key.
auto otherEnv = MakeShared<MG_Util::ShaderTranspiler::CompileEnv>(*env);
otherEnv->maxComputeWorkGroupInvocations = env->maxComputeWorkGroupInvocations + 1;
otherEnv->fingerprint = MG_Util::ShaderTranspiler::ComputeCompileEnvFingerprint(*otherEnv);
ASSERT_NE(otherEnv->fingerprint, env->fingerprint);
EXPECT_EQ(map.FindAdoptable(ShaderStage::Fragment, ShaderPreprocessCache::HashSource(text), text,
otherEnv->fingerprint),
nullptr);
EXPECT_EQ(map.GetAdoptionCount(), 0u);
}
// A node that settled as Cancelled published nothing, so adopting it would hand the new
// object a compile that can only ever report GL_FALSE. It must be a miss, and the dead entry
// must be pruned where it is found rather than waiting for the amortized sweep.
TEST(ShaderCompileAdoptionMapTest, CancelledNodeIsNotAdoptedAndIsPruned) {
const auto& env = MG_Util::ShaderTranspiler::GetDefaultCompileEnv();
ShaderCompileAdoptionMap map;
const String text = "#version 460\nvoid main() {}\n";
const SharedPtr<ShaderCompileTask> node = MakeNode(text, ShaderStage::Fragment, env);
map.Register(node);
// Never posted, so this settles the node as Cancelled right here.
node->Cancel();
ASSERT_TRUE(node->IsCancelled());
ASSERT_EQ(map.GetEntryCount(), 1u);
EXPECT_EQ(map.FindAdoptable(ShaderStage::Fragment, ShaderPreprocessCache::HashSource(text), text,
env->fingerprint),
nullptr);
EXPECT_EQ(map.GetEntryCount(), 0u) << "the dead entry must be pruned on the lookup that found it";
EXPECT_EQ(map.GetAdoptionCount(), 0u);
}
// The map is an index, never an owner: once the last real holder is gone the entry expires
// and is pruned, so a node's artifacts can never be kept alive by the map alone.
TEST(ShaderCompileAdoptionMapTest, ExpiredNodeIsNotAdoptedAndIsPruned) {
const auto& env = MG_Util::ShaderTranspiler::GetDefaultCompileEnv();
ShaderCompileAdoptionMap map;
const String text = "#version 460\nvoid main() {}\n";
{
map.Register(MakeNode(text, ShaderStage::Fragment, env));
}
ASSERT_EQ(map.GetEntryCount(), 1u);
EXPECT_EQ(map.FindAdoptable(ShaderStage::Fragment, ShaderPreprocessCache::HashSource(text), text,
env->fingerprint),
nullptr);
EXPECT_EQ(map.GetEntryCount(), 0u);
}
// The amortized sweep keeps the index O(live nodes) instead of O(compiles ever issued).
TEST(ShaderCompileAdoptionMapTest, SweepReclaimsDeadEntries) {
const auto& env = MG_Util::ShaderTranspiler::GetDefaultCompileEnv();
ShaderCompileAdoptionMap map;
// Every one of these dies immediately, so nothing but dead weight accumulates - and the
// map must not grow without bound because of it.
for (SizeT i = 0; i < ShaderCompileAdoptionMap::kMinSweepThreshold * 4; ++i) {
map.Register(MakeNode("#version 460\nvoid main() { float x" + std::to_string(i) + " = 0.0; }\n",
ShaderStage::Fragment, env));
}
EXPECT_LE(map.GetEntryCount(), ShaderCompileAdoptionMap::kMinSweepThreshold)
<< "expired entries must be reclaimed, not accumulated";
}
@@ -0,0 +1,984 @@
// MobileGL - MobileGL/MG_Test/Program/XfbFrontendOrderInvarianceTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// The frontend's answer for a transform-feedback program must not depend on what
// was linked before it. This binary asserts exactly that, headlessly: it links a
// clip_distance-shaped program A, then an XFB-shaped program B, and diffs B's
// whole frontend output (xfb varyings and their offsets, strides, buffer mode,
// scattered-capture and geometry-strip verdicts, uniform blocks, attribute and
// uniform counts, and every SPIR-V module byte-for-byte plus its Location /
// Component / Index / Offset / XfbBuffer / XfbStride / BuiltIn / Binding /
// DescriptorSet decorations) against the same B linked with no A ahead of it.
//
// It was written to arbitrate an order-triggered CTS failure - after
// KHR-GLxx.clip_distance.functional ran, every later transform_feedback capture
// case failed on DirectVulkan - and its verdict was NEGATIVE, which is what made
// it worth keeping: B's frontend output is bit-identical under every ordering,
// every flag state (MOBILEGL_ASYNC_SHADER_COMPILE on/off, THREADS unset/1/8) and
// every isolation level below. That ruled out the whole frontend - the P0b
// preprocess cache, the stage-6 adoption map, ProgramState, glslang's shared
// built-in symbol tables, the pool workers' thread_locals - and sent the hunt
// downstream, where the defect actually was: DirectVulkan's per-VAO vertex
// binding memo keyed on recycled heap addresses (see
// MG_IntegrationTest/Scenarios/XfbAfterClipDistanceScenario.cpp). Keep it as the
// standing guard on the negative half of that split: if the frontend ever DOES
// acquire cross-program order sensitivity, this is what says so.
//
// Isolation model - three levels, all in one binary:
// * FRESH CONTEXT MG_State::Init() reinstalls pGLContext, which is what
// drops the P0b cache, the adoption map and ProgramState.
// Process globals (glslang tables, prewarm latch, pool
// worker thread_locals) deliberately SURVIVE it, which is
// what makes the fresh-context control a bisection step
// rather than just a reset.
// * FRESH PROCESS ctest runs each gtest case in this binary in the same
// process, so the "control first, poisoned second" and
// "poisoned first, control second" orderings are split
// into two cases whose names sort in opposite orders and
// which each capture BOTH snapshots themselves. A truly
// fresh process is available by running one case with
// --gtest_filter (see the FreshProcess* cases).
// * CACHE-CLEARED context kept, but the source text of B is made unique
// per run so no P0b/adoption hit is possible at all.
#include <gtest/gtest.h>
#include <algorithm>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
// ---------------------------------------------------------------------------------
// Flag plumbing (same shape AsyncCompileTest uses)
// ---------------------------------------------------------------------------------
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async)
: m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
// ---------------------------------------------------------------------------------
// A: the clip_distance.functional shape
// glcClipDistance.cpp, FunctionalTest::m_vertex_shader_code with
// CLIP_DISTANCE_REDECLARATION = m_explicit_redeclaration and
// CLIP_DISTANCE_SETUP = m_dynamic_array_setter, clip function 0.
// ${VERSION} for a KHR-GL40 run is "#version 400".
// ---------------------------------------------------------------------------------
String ClipDistanceVs(const int clipCount, const char* version) {
const String n = std::to_string(clipCount);
return String(version) +
"\n"
"\n"
"out float gl_ClipDistance[" + n + "];\n"
"\n"
"float f(int i)\n"
"{\n"
" return 0.0;\n"
"}\n"
"\n"
"in vec4 position;\n"
"\n"
"void main()\n"
"{\n"
" for(int i = 0; i < " + n + "; i++)\n"
" {\n"
" gl_ClipDistance[i] = f(i);\n"
" }\n"
"\n"
" gl_Position = position;\n"
"}\n";
}
String ClipDistanceFs(const char* version) {
return String(version) +
"\n"
"\n"
"\n"
"out highp vec4 color;\n"
"\n"
"void main()\n"
"{\n"
" color = vec4(1.0, 0.0, 0.0, 1.0);\n"
"}\n";
}
// ---------------------------------------------------------------------------------
// B1: the transform_feedback3 skip_components shape
// gl3cTransformFeedback3Tests.cpp, TransformFeedbackBaseTestCase::m_shader_vert
// at "#version 150", captured with the gl_SkipComponents* varying list.
// This is the case whose failure text is the crispest:
// "compareArrays(GLfloat):index 1 value -2 != 1"
// ---------------------------------------------------------------------------------
String SkipComponentsVs(const char* version, const String& saltComment = String()) {
return String(version) + "\n" + saltComment +
" in vec4 vertex;\n"
" out vec4 value1;\n"
" out vec4 value2;\n"
" out vec4 value3;\n"
" out vec4 value4;\n"
"\n"
" void main (void)\n"
" {\n"
" vec4 temp = vertex;\n"
"\n"
" gl_Position = temp;\n"
"\n"
" value1 = abs(temp) * 1.0;\n"
" value2 = abs(temp) * 2.0;\n"
" value3 = abs(temp) * 3.0;\n"
" value4 = abs(temp) * 4.0;\n"
" }\n";
}
String SkipComponentsFs(const char* version) {
return String(version) +
"\n"
" out vec4 fragColor;\n"
" void main (void)\n"
" {\n"
" fragColor = vec4(0.0, 0.0, 0.0, 1.0);\n"
" }\n";
}
Vector<String> SkipComponentsVaryings() {
return {"gl_SkipComponents1", "value1", "gl_SkipComponents2", "gl_SkipComponents1", "value2",
"gl_SkipComponents3", "gl_SkipComponents2", "value3", "gl_SkipComponents4", "value4"};
}
// ---------------------------------------------------------------------------------
// B2: the capture_vertex_interleaved shape
// gl3cTransformFeedbackTests.cpp, CaptureVertexInterleaved::
// s_vertex_shader_source_code_template at "#version 130", with
// MAX_TRANSFORM_FEEDBACK_INTERLEAVED_COMPONENTS/4 - 1 user vec4 outputs
// plus gl_Position as the final captured varying.
// ---------------------------------------------------------------------------------
String CaptureInterleavedVs(const int userVaryings, const char* version) {
String declarations;
String setters;
for (int i = 0; i < userVaryings; ++i) {
const String name = "result_" + std::to_string(i);
declarations += "out vec4 " + name + ";\n";
setters += " " + name + " = vec4(" + std::to_string(i * 4) + ".0, " +
std::to_string(i * 4 + 1) + ".0, " + std::to_string(i * 4 + 2) + ".0, " +
std::to_string(i * 4 + 3) + ".0);\n";
}
return String(version) + "\n\n" + declarations + "\n" +
"void main()\n"
"{\n" +
setters +
"\n"
" vec4 position = vec4(0.0);\n"
"\n"
" switch(gl_VertexID)\n"
" {\n"
" case 0:\n"
" position = vec4(-1.0 + 0.0625, 1.0 - 0.0625, 0.0, 1.0);\n"
" break;\n"
" case 1:\n"
" position = vec4( 1.0 - 0.0625, 1.0 - 0.0625, 0.0, 1.0);\n"
" break;\n"
" case 2:\n"
" position = vec4(-1.0 + 0.0625, -1.0 + 0.0625, 0.0, 1.0);\n"
" break;\n"
" case 3:\n"
" position = vec4( 1.0 - 0.0625, -1.0 + 0.0625, 0.0, 1.0);\n"
" break;\n"
" }\n"
"\n"
" gl_Position = position;\n"
"}\n";
}
String CaptureInterleavedFs(const char* version) {
return String(version) +
"\n"
"\n"
"out vec4 color;\n"
"\n"
"void main()\n"
"{\n"
" color = vec4(0.5);\n"
"}\n";
}
Vector<String> CaptureInterleavedVaryings(const int userVaryings) {
Vector<String> names;
for (int i = 0; i < userVaryings; ++i) names.push_back("result_" + std::to_string(i));
names.push_back("gl_Position");
return names;
}
// ---------------------------------------------------------------------------------
// B3: the capture_geometry_interleaved shape. The only shape that reaches
// ResolveGsTriangleStripCapture, i.e. the gsStripTriangles / gsStripCaptureFixup
// artifacts - and triangle_strip is the sub-case that needs the fixup.
// ---------------------------------------------------------------------------------
const char* kGeometryBlankVs = "#version 130\n"
"\n"
"void main()\n"
"{\n"
"}\n";
String CaptureGeometryGs(const int userVaryings, const char* outPrimitive) {
String declarations;
String setters;
for (int i = 0; i < userVaryings; ++i) {
const String name = "result_" + std::to_string(i);
declarations += "out vec4 " + name + ";\n";
setters += " " + name + " = vec4(" + std::to_string(i * 4) + ".0, " +
std::to_string(i * 4 + 1) + ".0, " + std::to_string(i * 4 + 2) + ".0, " +
std::to_string(i * 4 + 3) + ".0);\n";
}
String source = "#version 150\n"
"\n"
"layout(points) in;\n"
"layout(" +
String(outPrimitive) +
", max_vertices = 4) out;\n"
"\n" +
declarations + "\n" +
"void main()\n"
"{\n";
const char* positions[] = {"vec4(-1.0 + 0.0625, 1.0 - 0.0625, 0.0, 1.0)",
"vec4( 1.0 - 0.0625, 1.0 - 0.0625, 0.0, 1.0)",
"vec4(-1.0 + 0.0625, -1.0 + 0.0625, 0.0, 1.0)",
"vec4( 1.0 - 0.0625, -1.0 + 0.0625, 0.0, 1.0)"};
for (const char* position : positions) {
source += String("\n gl_Position = ") + position + ";\n";
source += setters;
source += " EmitVertex();\n";
}
source += "}\n";
return source;
}
GLuint BuildProgramWithGeometry(const String& vertexSource, const String& geometrySource,
const String& fragmentSource, const Vector<String>& xfbVaryings);
// ---------------------------------------------------------------------------------
// SPIR-V digest: hash + every decoration that could express a slot shift, resolved
// through OpName so the text is stable across id renumbering.
// ---------------------------------------------------------------------------------
constexpr Uint32 kOpName = 5;
constexpr Uint32 kOpMemberName = 6;
constexpr Uint32 kOpEntryPoint = 15;
constexpr Uint32 kOpDecorate = 71;
constexpr Uint32 kOpMemberDecorate = 72;
const char* DecorationName(const Uint32 decoration) {
switch (decoration) {
case 11: return "BuiltIn";
case 30: return "Location";
case 31: return "Component";
case 32: return "Index";
case 33: return "Binding";
case 34: return "DescriptorSet";
case 35: return "Offset";
case 36: return "XfbBuffer";
case 37: return "XfbStride";
case 38: return "FuncParamAttr";
default: return nullptr;
}
}
const char* BuiltInName(const Uint32 builtIn) {
switch (builtIn) {
case 0: return "Position";
case 1: return "PointSize";
case 3: return "ClipDistance";
case 4: return "CullDistance";
case 5: return "VertexId";
case 42: return "VertexIndex";
default: return nullptr;
}
}
String ReadSpirvString(const Vector<unsigned>& words, const SizeT firstWord, const SizeT endWord,
SizeT& outNextWord) {
String text;
SizeT w = firstWord;
for (; w < endWord; ++w) {
const Uint32 word = words[w];
Bool done = false;
for (int b = 0; b < 4; ++b) {
const char c = static_cast<char>((word >> (8 * b)) & 0xFF);
if (c == '\0') {
done = true;
break;
}
text.push_back(c);
}
if (done) {
++w;
break;
}
}
outNextWord = w;
return text;
}
Uint64 Fnv1a(const Vector<unsigned>& words) {
Uint64 hash = 1469598103934665603ULL;
for (const unsigned word : words) {
for (int b = 0; b < 4; ++b) {
hash ^= static_cast<Uint64>((word >> (8 * b)) & 0xFF);
hash *= 1099511628211ULL;
}
}
return hash;
}
struct SpirvDigest {
Uint64 hash = 0;
SizeT wordCount = 0;
Vector<String> decorations;
Vector<String> interfaceNames;
};
SpirvDigest DigestSpirv(const Vector<unsigned>& words) {
SpirvDigest digest;
digest.hash = Fnv1a(words);
digest.wordCount = words.size();
if (words.size() < 5 || words[0] != 0x07230203u) {
digest.decorations.push_back("<not a SPIR-V module>");
return digest;
}
UnorderedMap<Uint32, String> names;
Vector<Uint32> interfaceIds;
struct PendingDecoration {
Uint32 target;
Int member; // -1 for OpDecorate
Uint32 decoration;
Vector<Uint32> operands;
};
Vector<PendingDecoration> pending;
SizeT w = 5;
while (w < words.size()) {
const Uint32 header = words[w];
const Uint32 wordCount = header >> 16;
const Uint32 opcode = header & 0xFFFFu;
if (wordCount == 0 || w + wordCount > words.size()) break;
if (opcode == kOpName && wordCount >= 3) {
SizeT next = 0;
names[words[w + 1]] = ReadSpirvString(words, w + 2, w + wordCount, next);
} else if (opcode == kOpMemberName && wordCount >= 4) {
SizeT next = 0;
const String member = ReadSpirvString(words, w + 3, w + wordCount, next);
names[words[w + 1]] = names.count(words[w + 1]) ? names[words[w + 1]] : String("<struct>");
(void)member;
} else if (opcode == kOpEntryPoint && wordCount >= 4) {
SizeT next = 0;
(void)ReadSpirvString(words, w + 3, w + wordCount, next);
for (SizeT i = next; i < w + wordCount; ++i) interfaceIds.push_back(words[i]);
} else if (opcode == kOpDecorate && wordCount >= 3) {
PendingDecoration entry{words[w + 1], -1, words[w + 2], {}};
for (SizeT i = w + 3; i < w + wordCount; ++i) entry.operands.push_back(words[i]);
pending.push_back(Move(entry));
} else if (opcode == kOpMemberDecorate && wordCount >= 4) {
PendingDecoration entry{words[w + 1], static_cast<Int>(words[w + 2]), words[w + 3], {}};
for (SizeT i = w + 4; i < w + wordCount; ++i) entry.operands.push_back(words[i]);
pending.push_back(Move(entry));
}
w += wordCount;
}
const auto label = [&](const Uint32 id) {
const auto it = names.find(id);
if (it != names.end() && !it->second.empty()) return it->second;
return String("%") + std::to_string(id);
};
for (const auto& entry : pending) {
const char* decorationName = DecorationName(entry.decoration);
if (decorationName == nullptr) continue; // relocation-irrelevant decorations
String line = label(entry.target);
if (entry.member >= 0) line += "[member " + std::to_string(entry.member) + "]";
line += " ";
line += decorationName;
line += " =";
for (const Uint32 operand : entry.operands) {
if (entry.decoration == 11) {
const char* builtIn = BuiltInName(operand);
line += String(" ") + (builtIn != nullptr ? builtIn : std::to_string(operand));
} else {
line += " " + std::to_string(operand);
}
}
digest.decorations.push_back(line);
}
std::sort(digest.decorations.begin(), digest.decorations.end());
for (const Uint32 id : interfaceIds) digest.interfaceNames.push_back(label(id));
std::sort(digest.interfaceNames.begin(), digest.interfaceNames.end());
return digest;
}
// ---------------------------------------------------------------------------------
// The snapshot under test
// ---------------------------------------------------------------------------------
struct XfbSnapshot {
GLint linkStatus = GL_FALSE;
String infoLog;
GLenum bufferMode = 0;
Uint32 packedStride = 0;
Bool needsScattered = false;
Int varyingNameMaxLength = 0;
Vector<Uint32> strides;
Vector<String> varyings;
GLenum gsInputPrimitive = 0;
Bool gsStripCaptureFixup = false;
Vector<Uint32> gsStripTriangles;
Int uniformBlockCount = 0;
Vector<Uint> uniformBlockBindings;
Uint maxUniformLocation = 0;
GLint activeAttributes = 0;
GLint activeUniforms = 0;
Vector<SpirvDigest> spirv;
};
String QueryProgramInfoLog(const GLuint program) {
GLint length = 0;
GetProgramiv(program, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return String();
std::vector<GLchar> buffer(static_cast<size_t>(length));
GLsizei written = 0;
GetProgramInfoLog(program, length, &written, buffer.data());
return String(buffer.data(), static_cast<size_t>(written));
}
XfbSnapshot Capture(const GLuint program) {
XfbSnapshot snapshot;
GetProgramiv(program, GL_LINK_STATUS, &snapshot.linkStatus);
snapshot.infoLog = QueryProgramInfoLog(program);
const auto& object = MG_State::pGLContext->GetProgramObject(program);
if (object == nullptr) {
snapshot.infoLog += "<no program object>";
return snapshot;
}
snapshot.bufferMode = object->GetTransformFeedbackBufferMode();
snapshot.packedStride = object->GetTransformFeedbackPackedStride();
snapshot.needsScattered = object->NeedsScatteredTransformFeedbackCapture();
snapshot.varyingNameMaxLength = object->GetTransformFeedbackVaryingMaxLength();
snapshot.gsInputPrimitive = object->GetGeometryInputType();
snapshot.gsStripCaptureFixup = object->HasGsTriangleStripCaptureFixup();
snapshot.gsStripTriangles = object->GetGsStripTriangles();
// The rest of ProgramFactory::ComputeHash's input set, so "the backend cache key is
// unchanged" is something this binary measures rather than assumes.
snapshot.uniformBlockCount = object->GetActiveUniformBlocksCount();
for (Int i = 0; i < snapshot.uniformBlockCount; ++i) {
snapshot.uniformBlockBindings.push_back(object->GetUniformBlockBinding(static_cast<Uint>(i)));
}
snapshot.maxUniformLocation = object->GetMaxUniformLocation();
GetProgramiv(program, GL_ACTIVE_ATTRIBUTES, &snapshot.activeAttributes);
GetProgramiv(program, GL_ACTIVE_UNIFORMS, &snapshot.activeUniforms);
for (SizeT i = 0; i < object->GetTransformFeedbackBufferCount(); ++i) {
snapshot.strides.push_back(object->GetTransformFeedbackStride(static_cast<Uint32>(i)));
}
for (const auto& varying : object->GetTransformFeedbackVaryings()) {
snapshot.varyings.push_back(varying.name + " type=0x" + [&] {
char buffer[16];
std::snprintf(buffer, sizeof(buffer), "%04X", static_cast<unsigned>(varying.type));
return String(buffer);
}() + " size=" + std::to_string(varying.size) + " buf=" + std::to_string(varying.bufferIndex) +
" off=" + std::to_string(varying.offsetBytes) +
" bytes=" + std::to_string(varying.byteSize) +
" packedOff=" + std::to_string(varying.packedOffsetBytes));
}
for (const auto& module : object->GetGeneratedSpirv()) {
snapshot.spirv.push_back(DigestSpirv(module));
}
return snapshot;
}
// One text blob per snapshot, so a mismatch shows up as a readable gtest diff.
String Render(const XfbSnapshot& snapshot, const Bool includeSpirvHash) {
String text;
text += "linkStatus = " + std::to_string(snapshot.linkStatus) + "\n";
if (!snapshot.infoLog.empty()) text += "infoLog = " + snapshot.infoLog + "\n";
text += "xfbBufferMode = " + std::to_string(snapshot.bufferMode) + "\n";
text += "xfbPackedStride = " + std::to_string(snapshot.packedStride) + "\n";
text += "xfbNeedsScatter = " + std::to_string(static_cast<int>(snapshot.needsScattered)) + "\n";
text += "xfbNameMaxLength = " + std::to_string(snapshot.varyingNameMaxLength) + "\n";
text += "xfbStrides =";
for (const Uint32 stride : snapshot.strides) text += " " + std::to_string(stride);
text += "\n";
text += "xfbVaryings (" + std::to_string(snapshot.varyings.size()) + "):\n";
for (const String& varying : snapshot.varyings) text += " " + varying + "\n";
text += "gsInputPrimitive = " + std::to_string(snapshot.gsInputPrimitive) + "\n";
text += "gsStripFixup = " + std::to_string(static_cast<int>(snapshot.gsStripCaptureFixup)) + "\n";
text += "gsStripTriangles =";
for (const Uint32 triangle : snapshot.gsStripTriangles) text += " " + std::to_string(triangle);
text += "\n";
text += "uniformBlocks = " + std::to_string(snapshot.uniformBlockCount) + " bindings:";
for (const Uint binding : snapshot.uniformBlockBindings) text += " " + std::to_string(binding);
text += "\n";
text += "maxUniformLoc = " + std::to_string(snapshot.maxUniformLocation) + "\n";
text += "activeAttribs = " + std::to_string(snapshot.activeAttributes) + "\n";
text += "activeUniforms = " + std::to_string(snapshot.activeUniforms) + "\n";
for (SizeT i = 0; i < snapshot.spirv.size(); ++i) {
const SpirvDigest& digest = snapshot.spirv[i];
text += "spirv[" + std::to_string(i) + "] words=" + std::to_string(digest.wordCount);
if (includeSpirvHash) {
char buffer[32];
std::snprintf(buffer, sizeof(buffer), " hash=%016llX",
static_cast<unsigned long long>(digest.hash));
text += buffer;
}
text += "\n";
text += " interface:";
for (const String& name : digest.interfaceNames) text += " " + name;
text += "\n";
for (const String& decoration : digest.decorations) text += " " + decoration + "\n";
}
return text;
}
// ---------------------------------------------------------------------------------
// Program construction through the real GL entry points
// ---------------------------------------------------------------------------------
GLuint BuildProgram(const String& vertexSource, const String& fragmentSource,
const Vector<String>& xfbVaryings, const GLenum bufferMode) {
const GLuint vertexShader = CreateShader(GL_VERTEX_SHADER);
const char* vertexText = vertexSource.c_str();
ShaderSource(vertexShader, 1, &vertexText, nullptr);
CompileShader(vertexShader);
const GLuint fragmentShader = CreateShader(GL_FRAGMENT_SHADER);
const char* fragmentText = fragmentSource.c_str();
ShaderSource(fragmentShader, 1, &fragmentText, nullptr);
CompileShader(fragmentShader);
const GLuint program = CreateProgram();
AttachShader(program, vertexShader);
AttachShader(program, fragmentShader);
if (!xfbVaryings.empty()) {
std::vector<const GLchar*> names;
names.reserve(xfbVaryings.size());
for (const String& name : xfbVaryings) names.push_back(name.c_str());
TransformFeedbackVaryings(program, static_cast<GLsizei>(names.size()), names.data(), bufferMode);
}
LinkProgram(program);
DeleteShader(vertexShader);
DeleteShader(fragmentShader);
return program;
}
// A, exactly as the CTS builds it for the failing sub-case (1 clip distance, dynamic
// setter, clip function 0). Returns the program so the caller can keep it alive, which
// is what the CTS does too (it holds m_program across the whole case).
GLuint LinkClipDistanceProgram(const int clipCount, const char* version) {
return BuildProgram(ClipDistanceVs(clipCount, version), ClipDistanceFs(version), {},
GL_INTERLEAVED_ATTRIBS);
}
GLuint LinkSkipComponentsProgram(const char* version, const String& salt = String()) {
return BuildProgram(SkipComponentsVs(version, salt), SkipComponentsFs(version),
SkipComponentsVaryings(), GL_INTERLEAVED_ATTRIBS);
}
GLuint LinkCaptureInterleavedProgram(const int userVaryings, const char* version) {
return BuildProgram(CaptureInterleavedVs(userVaryings, version), CaptureInterleavedFs(version),
CaptureInterleavedVaryings(userVaryings), GL_INTERLEAVED_ATTRIBS);
}
GLuint BuildProgramWithGeometry(const String& vertexSource, const String& geometrySource,
const String& fragmentSource, const Vector<String>& xfbVaryings) {
const auto makeShader = [](const GLenum type, const String& source) {
const GLuint shader = CreateShader(type);
const char* text = source.c_str();
ShaderSource(shader, 1, &text, nullptr);
CompileShader(shader);
return shader;
};
const GLuint vertexShader = makeShader(GL_VERTEX_SHADER, vertexSource);
const GLuint geometryShader = makeShader(GL_GEOMETRY_SHADER, geometrySource);
const GLuint fragmentShader = makeShader(GL_FRAGMENT_SHADER, fragmentSource);
const GLuint program = CreateProgram();
AttachShader(program, vertexShader);
AttachShader(program, geometryShader);
AttachShader(program, fragmentShader);
std::vector<const GLchar*> names;
names.reserve(xfbVaryings.size());
for (const String& name : xfbVaryings) names.push_back(name.c_str());
TransformFeedbackVaryings(program, static_cast<GLsizei>(names.size()), names.data(),
GL_INTERLEAVED_ATTRIBS);
LinkProgram(program);
DeleteShader(vertexShader);
DeleteShader(geometryShader);
DeleteShader(fragmentShader);
return program;
}
GLuint LinkCaptureGeometryProgram(const int userVaryings, const char* outPrimitive) {
return BuildProgramWithGeometry(kGeometryBlankVs, CaptureGeometryGs(userVaryings, outPrimitive),
CaptureInterleavedFs("#version 130"),
CaptureInterleavedVaryings(userVaryings));
}
// Reinstalls pGLContext: new ProgramState, new P0b preprocess cache, new stage-6
// adoption map. glslang's process globals are untouched on purpose.
void FreshContext() { MG_State::Init(); }
class XfbFrontendOrderInvarianceTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
void TearDown() override { FreshContext(); }
};
// The two B shapes, run through one lambda so every case tests both.
struct BCase {
const char* label;
GLuint (*link)();
};
GLuint LinkSkip150() { return LinkSkipComponentsProgram("#version 150"); }
GLuint LinkCapture130() { return LinkCaptureInterleavedProgram(15, "#version 130"); }
GLuint LinkSkip400() { return LinkSkipComponentsProgram("#version 400"); }
GLuint LinkCapture400() { return LinkCaptureInterleavedProgram(15, "#version 400"); }
GLuint LinkGeometryPoints() { return LinkCaptureGeometryProgram(15, "points"); }
GLuint LinkGeometryTriangleStrip() { return LinkCaptureGeometryProgram(15, "triangle_strip"); }
const BCase kBCases[] = {
{"skip_components@150", &LinkSkip150},
{"capture_interleaved@130", &LinkCapture130},
{"skip_components@400", &LinkSkip400},
{"capture_interleaved@400", &LinkCapture400},
{"capture_geometry@points", &LinkGeometryPoints},
{"capture_geometry@triangle_strip", &LinkGeometryTriangleStrip},
};
// ---------------------------------------------------------------------------------
// The core A/B comparison, parameterized on everything that could matter.
// ---------------------------------------------------------------------------------
struct AbResult {
String control;
String poisoned;
};
AbResult RunAb(const BCase& bCase, const int clipCount, const char* clipVersion,
const Bool freshContextForControl, const Bool includeSpirvHash) {
AbResult result;
// CONTROL: B alone, in a context that has never seen A.
if (freshContextForControl) FreshContext();
{
const GLuint program = bCase.link();
result.control = Render(Capture(program), includeSpirvHash);
DeleteProgram(program);
}
// POISONED: A first, then B, in ONE context - the glcts shape.
FreshContext();
{
const GLuint clipProgram = LinkClipDistanceProgram(clipCount, clipVersion);
GLint clipLinked = GL_FALSE;
GetProgramiv(clipProgram, GL_LINK_STATUS, &clipLinked);
// The A program is deliberately kept alive across B's link, exactly as the CTS
// holds its program object for the duration of the case.
const GLuint program = bCase.link();
result.poisoned = Render(Capture(program), includeSpirvHash);
if (clipLinked != GL_TRUE) {
result.poisoned += "\n<<< A DID NOT LINK: " + QueryProgramInfoLog(clipProgram) + " >>>\n";
}
DeleteProgram(program);
DeleteProgram(clipProgram);
}
return result;
}
} // namespace
// -------------------------------------------------------------------------------------
// 1. The headline question, both flag states, both B shapes, several clip counts.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, AsyncOn_ClipDistanceBeforeXfbChangesNothingInTheFrontend) {
const AsyncModeScope async(true);
ASSERT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
for (const BCase& bCase : kBCases) {
for (const int clipCount : {1, 4, 8}) {
for (const char* clipVersion : {"#version 400", "#version 150", "#version 130"}) {
const AbResult result = RunAb(bCase, clipCount, clipVersion, true, true);
EXPECT_EQ(result.control, result.poisoned)
<< "async=1 B=" << bCase.label << " clipCount=" << clipCount
<< " clipVersion=" << clipVersion;
}
}
}
}
TEST_F(XfbFrontendOrderInvarianceTest, AsyncOff_ClipDistanceBeforeXfbChangesNothingInTheFrontend) {
const AsyncModeScope async(false);
for (const BCase& bCase : kBCases) {
for (const int clipCount : {1, 4, 8}) {
for (const char* clipVersion : {"#version 400", "#version 150", "#version 130"}) {
const AbResult result = RunAb(bCase, clipCount, clipVersion, true, true);
EXPECT_EQ(result.control, result.poisoned)
<< "async=0 B=" << bCase.label << " clipCount=" << clipCount
<< " clipVersion=" << clipVersion;
}
}
}
}
// -------------------------------------------------------------------------------------
// 2. Repetition: the CTS incidence with async off is ~2.4%, i.e. roughly 1 in 40 runs, so
// a single comparison would miss it. 60 repetitions of the same A->B pair inside one
// process, each with its own fresh context, is the headless equivalent.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, RepeatedAbPairsAreBitStable) {
for (const Bool async : {true, false}) {
const AsyncModeScope scope(async);
String reference;
for (int repetition = 0; repetition < 60; ++repetition) {
const AbResult result = RunAb(kBCases[0], 1, "#version 400", repetition == 0, true);
if (repetition == 0) {
reference = result.control;
ASSERT_EQ(reference, result.poisoned) << "async=" << async << " first repetition";
}
EXPECT_EQ(reference, result.poisoned) << "async=" << async << " repetition " << repetition;
}
}
}
// -------------------------------------------------------------------------------------
// 3. Same context, no reset between A and B, and B's source made unique so neither the
// P0b preprocess cache nor the stage-6 adoption map can serve it. If the divergence
// survives this, no per-source memo is carrying it.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, NoMemoHitPossibleForB) {
for (const Bool async : {true, false}) {
const AsyncModeScope scope(async);
FreshContext();
const GLuint controlProgram = LinkSkipComponentsProgram("#version 150", "// salt control\n");
const String control = Render(Capture(controlProgram), false);
DeleteProgram(controlProgram);
FreshContext();
const GLuint clipProgram = LinkClipDistanceProgram(1, "#version 400");
const GLuint poisonedProgram = LinkSkipComponentsProgram("#version 150", "// salt poisoned\n");
const String poisoned = Render(Capture(poisonedProgram), false);
DeleteProgram(poisonedProgram);
DeleteProgram(clipProgram);
EXPECT_EQ(control, poisoned) << "async=" << async << " (SPIR-V hash excluded: the salt comment "
"is stripped by the preprocessor but ids can still renumber)";
}
}
// -------------------------------------------------------------------------------------
// 4. Bisection: A and B in one context WITHOUT the reset in between, so ProgramState, the
// P0b cache and the adoption map all carry over exactly as they do in glcts, compared
// against A and B separated by a fresh context. A difference here but not in case 1
// would put the poison in per-context state; no difference in either puts it outside
// the frontend entirely.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, PerContextStateBisection) {
for (const Bool async : {true, false}) {
const AsyncModeScope scope(async);
// (a) A, fresh context, then B: per-context state cleared, process globals kept.
FreshContext();
const GLuint clipA = LinkClipDistanceProgram(1, "#version 400");
DeleteProgram(clipA);
FreshContext();
const GLuint separated = LinkSkipComponentsProgram("#version 150");
const String separatedText = Render(Capture(separated), true);
DeleteProgram(separated);
// (b) A then B, same context, A kept alive.
FreshContext();
const GLuint clipB = LinkClipDistanceProgram(1, "#version 400");
const GLuint together = LinkSkipComponentsProgram("#version 150");
const String togetherText = Render(Capture(together), true);
DeleteProgram(together);
DeleteProgram(clipB);
EXPECT_EQ(separatedText, togetherText) << "async=" << async;
}
}
// -------------------------------------------------------------------------------------
// 5. The interleaving glcts actually produces: many cases in a row, A somewhere in the
// middle, every B compared against the very first B. This is the one that catches a
// poison that needs more than one link to develop.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, LongCaseSequenceLikeGlcts) {
for (const Bool async : {true, false}) {
const AsyncModeScope scope(async);
FreshContext();
String reference;
Vector<GLuint> keepAlive;
for (int step = 0; step < 12; ++step) {
if (step == 4) {
// The clip_distance case: every clip count, both setters' shapes.
for (const int clipCount : {1, 2, 4, 8}) {
keepAlive.push_back(LinkClipDistanceProgram(clipCount, "#version 400"));
}
}
const GLuint program = LinkSkipComponentsProgram("#version 150");
const String text = Render(Capture(program), true);
if (step == 0) {
reference = text;
} else {
EXPECT_EQ(reference, text) << "async=" << async << " step " << step;
}
keepAlive.push_back(program);
}
for (const GLuint program : keepAlive) DeleteProgram(program);
}
}
// -------------------------------------------------------------------------------------
// 6. Fresh-process controls. Run exactly one of these with --gtest_filter to get a
// process that has linked nothing else, then diff the two printed blobs by hand:
// ./XfbFrontendOrderInvarianceTest --gtest_filter='*FreshProcessControlB*'
// ./XfbFrontendOrderInvarianceTest --gtest_filter='*FreshProcessAThenB*'
// Both print their snapshot to stdout; they never fail on their own.
// -------------------------------------------------------------------------------------
// -------------------------------------------------------------------------------------
// 7. The one shape only async can produce: A's link is still IN FLIGHT when B's shaders
// are compiled and B is linked. Nothing joins A until after B has published. If the
// poison rode a worker thread_local (glslang's pool allocator, its TLS parse context)
// rather than any per-context container, this is where it would show.
// N copies of A are enqueued first so the pool really has a backlog.
// -------------------------------------------------------------------------------------
TEST_F(XfbFrontendOrderInvarianceTest, BLinksWhileAIsStillInFlight) {
const AsyncModeScope async(true);
ASSERT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
FreshContext();
const GLuint controlProgram = LinkSkipComponentsProgram("#version 150");
const String control = Render(Capture(controlProgram), true);
DeleteProgram(controlProgram);
for (int repetition = 0; repetition < 20; ++repetition) {
FreshContext();
Vector<GLuint> clipPrograms;
// Enqueued, never read: every one of these links is outstanding while B goes
// through compile + link on the same pool.
for (const int clipCount : {1, 2, 3, 4, 5, 6, 7, 8}) {
clipPrograms.push_back(LinkClipDistanceProgram(clipCount, "#version 400"));
}
const GLuint program = LinkSkipComponentsProgram("#version 150");
const String poisoned = Render(Capture(program), true);
EXPECT_EQ(control, poisoned) << "repetition " << repetition;
DeleteProgram(program);
for (const GLuint clipProgram : clipPrograms) DeleteProgram(clipProgram);
}
}
// Sanity: every shape this binary compares must actually LINK, otherwise "control ==
// poisoned" is the trivially true statement that two failures look alike.
TEST_F(XfbFrontendOrderInvarianceTest, EveryShapeActuallyLinks) {
const AsyncModeScope async(true);
for (const int clipCount : {1, 2, 4, 8}) {
for (const char* version : {"#version 400", "#version 150", "#version 130"}) {
FreshContext();
const GLuint program = LinkClipDistanceProgram(clipCount, version);
GLint linked = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &linked);
EXPECT_EQ(linked, GL_TRUE) << "A clipCount=" << clipCount << " " << version << ": "
<< QueryProgramInfoLog(program);
DeleteProgram(program);
}
}
for (const BCase& bCase : kBCases) {
FreshContext();
const GLuint program = bCase.link();
GLint linked = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &linked);
EXPECT_EQ(linked, GL_TRUE) << "B " << bCase.label << ": " << QueryProgramInfoLog(program);
std::printf("=== B shape %s ===\n%s\n", bCase.label, Render(Capture(program), true).c_str());
DeleteProgram(program);
}
}
// Writes B's raw SPIR-V modules next to the binary so they can be run through spirv-dis
// by hand. MOBILEGL_XFB_INVARIANCE_DUMP_DIR selects the directory; unset means no dump.
TEST_F(XfbFrontendOrderInvarianceTest, DumpBSpirvForDisassembly) {
const char* directory = std::getenv("MOBILEGL_XFB_INVARIANCE_DUMP_DIR");
if (directory == nullptr) {
GTEST_SKIP() << "set MOBILEGL_XFB_INVARIANCE_DUMP_DIR to dump";
}
const AsyncModeScope async(true);
struct Dump {
const char* tag;
Bool withClipDistanceFirst;
};
for (const Dump& dump : {Dump{"control", false}, Dump{"poisoned", true}}) {
for (const BCase& bCase : kBCases) {
FreshContext();
GLuint clipProgram = 0;
if (dump.withClipDistanceFirst) clipProgram = LinkClipDistanceProgram(1, "#version 400");
const GLuint program = bCase.link();
const auto& object = MG_State::pGLContext->GetProgramObject(program);
const auto& modules = object->GetGeneratedSpirv();
for (SizeT i = 0; i < modules.size(); ++i) {
String path = String(directory) + "/" + dump.tag + "-" + bCase.label + "-" +
std::to_string(i) + ".spv";
std::replace(path.begin() + std::strlen(directory) + 1, path.end(), '@', '_');
std::FILE* file = std::fopen(path.c_str(), "wb");
ASSERT_NE(file, nullptr) << path;
std::fwrite(modules[i].data(), sizeof(unsigned), modules[i].size(), file);
std::fclose(file);
}
DeleteProgram(program);
if (clipProgram != 0) DeleteProgram(clipProgram);
}
}
}
TEST_F(XfbFrontendOrderInvarianceTest, FreshProcessControlB) {
const AsyncModeScope async(true);
const GLuint program = LinkSkipComponentsProgram("#version 150");
std::printf("=== FreshProcessControlB ===\n%s\n", Render(Capture(program), true).c_str());
DeleteProgram(program);
}
TEST_F(XfbFrontendOrderInvarianceTest, FreshProcessAThenB) {
const AsyncModeScope async(true);
const GLuint clipProgram = LinkClipDistanceProgram(1, "#version 400");
const GLuint program = LinkSkipComponentsProgram("#version 150");
std::printf("=== FreshProcessAThenB ===\n%s\n", Render(Capture(program), true).c_str());
DeleteProgram(program);
DeleteProgram(clipProgram);
}
+27
View File
@@ -0,0 +1,27 @@
cmake_minimum_required(VERSION 3.14)
add_executable(
ObjectLifetimeIdTest
ObjectLifetimeIdTest.cpp
)
target_include_directories(ObjectLifetimeIdTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
${MGL_ROOT}/3rdparty/xxHash
${MGL_ROOT}/3rdparty/Vulkan-Headers/include
${MGL_ROOT}/3rdparty/SPIRV-Reflect
)
target_link_libraries(
ObjectLifetimeIdTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
if (MSVC)
target_compile_options(ObjectLifetimeIdTest PRIVATE /Zc:preprocessor)
endif()
include(GoogleTest)
gtest_discover_tests(ObjectLifetimeIdTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
@@ -0,0 +1,140 @@
// MobileGL - MobileGL/MG_Test/State/ObjectLifetimeIdTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// The invariant every backend memo keyed on a state object now rests on: a heap
// ADDRESS is not an identity, a lifetime id is.
//
// DirectVulkan memoises resolved vertex bindings per VertexArrayObject and folds
// the bound BufferObject's identity into the content hash that validates them.
// Both used to be heap addresses, and the allocator hands a freed address
// straight back: a VAO and a vertex buffer destroyed and immediately recreated
// under a byte-identical attribute layout reproduced BOTH the memo key and the
// validating hash, so the new draw fetched the destroyed buffer's GPU slice.
// GetLifetimeId() is what makes that impossible, so it is worth a test that
// needs no GPU, no context and no driver - only the allocator.
//
// The test does not simulate reuse; it waits for the real allocator to do it
// (which a LIFO free-list does on the very next allocation) and then asserts the
// id differs. If the allocator never repeats an address the run proves nothing,
// and the case says so with a skip rather than passing quietly.
#include <gtest/gtest.h>
#include <cstdint>
#include <memory>
#include <unordered_map>
#include "Includes.h"
#include <MG_State/GLState/BufferState/BufferObject.h>
#include <MG_State/GLState/VertexArrayState/VertexArrayObject.h>
using namespace MobileGL;
namespace {
// The allocation must actually happen: C++ permits eliding a new/delete pair,
// and an elided one would let two objects share an address for reasons that
// have nothing to do with the allocator - which is the only thing under test
// here. Publishing every pointer through a volatile sink keeps the pairs.
void* volatile g_addressSink = nullptr;
// Constructs and destroys `ObjectT` on the heap kAttempts times, watching for
// the allocator to hand back an address it already used. Every repeat must
// carry a lifetime id the dead occupant did not have. Returns how many repeats
// were seen, so the caller can tell "proven" from "never got the chance".
//
// Each object type has its own id counter, so a VertexArrayObject and a
// BufferObject may well both be id 1; ids are only ever compared within a
// type, which is exactly how the memos use them.
template <typename ObjectT>
int ProbeLifetimeIdAcrossAddressReuse(const char* typeName) {
constexpr int kAttempts = 64;
std::unordered_map<std::uintptr_t, Uint64> idAtAddress;
int reuseCount = 0;
Uint64 previousId = 0;
for (int attempt = 0; attempt < kAttempts; ++attempt) {
auto object = std::make_unique<ObjectT>(0u);
g_addressSink = object.get();
const auto address = reinterpret_cast<std::uintptr_t>(object.get());
const Uint64 lifetimeId = object->GetLifetimeId();
// 0 is the "this slot holds nothing" value in every memo that stores an
// id, so a live object must never be able to answer to a zeroed slot.
EXPECT_NE(lifetimeId, 0u) << typeName << " handed out lifetime id 0 (attempt " << attempt
<< "), which is the value a zero-initialised memo slot already carries";
EXPECT_GT(lifetimeId, previousId)
<< typeName << " lifetime ids must be strictly increasing, so an id is never handed out twice "
<< "(attempt " << attempt << ")";
previousId = lifetimeId;
const auto inserted = idAtAddress.emplace(address, lifetimeId);
if (!inserted.second) {
// The allocator reproduced an address: this is precisely the state in
// which a memo keyed on the address alone would hit a dead object's
// entry. The id is the thing that has to say no.
++reuseCount;
EXPECT_NE(lifetimeId, inserted.first->second)
<< typeName << " reconstructed at the address of a destroyed one reports the DEAD object's "
<< "lifetime id - a backend memo keyed on it would serve the dead object's resolved state "
<< "to this object's draws (attempt " << attempt << ")";
inserted.first->second = lifetimeId;
}
// Freed before the next construction on purpose: that ordering is what
// makes the allocator reuse the block, and it is the ordering the GL
// workload has (glDeleteVertexArrays, then the next glGenVertexArrays).
object.reset();
}
return reuseCount;
}
// Guards against a degenerate "id" that is really just the address in disguise:
// objects alive at the same time must differ too.
template <typename ObjectT>
void ExpectDistinctIdsWhileBothAlive(const char* typeName) {
auto first = std::make_unique<ObjectT>(0u);
auto second = std::make_unique<ObjectT>(0u);
g_addressSink = first.get();
g_addressSink = second.get();
EXPECT_NE(first->GetLifetimeId(), second->GetLifetimeId())
<< "two live " << typeName << "s share a lifetime id";
}
} // namespace
TEST(ObjectLifetimeIdTest, VertexArrayObjectAtARecycledAddressCarriesAFreshLifetimeId) {
using MG_State::GLState::VertexArrayObject;
const int reuseCount = ProbeLifetimeIdAcrossAddressReuse<VertexArrayObject>("VertexArrayObject");
if (reuseCount == 0) {
GTEST_SKIP() << "inconclusive, not proven: this allocator never handed the same address back across 64 "
"construct/destroy rounds, so the recycled-address case was never exercised";
}
RecordProperty("address_reuses_observed", reuseCount);
}
TEST(ObjectLifetimeIdTest, BufferObjectAtARecycledAddressCarriesAFreshLifetimeId) {
using MG_State::GLState::BufferObject;
const int reuseCount = ProbeLifetimeIdAcrossAddressReuse<BufferObject>("BufferObject");
if (reuseCount == 0) {
GTEST_SKIP() << "inconclusive, not proven: this allocator never handed the same address back across 64 "
"construct/destroy rounds, so the recycled-address case was never exercised";
}
RecordProperty("address_reuses_observed", reuseCount);
}
TEST(ObjectLifetimeIdTest, LiveVertexArrayObjectsHaveDistinctLifetimeIds) {
ExpectDistinctIdsWhileBothAlive<MG_State::GLState::VertexArrayObject>("VertexArrayObject");
}
TEST(ObjectLifetimeIdTest, LiveBufferObjectsHaveDistinctLifetimeIds) {
ExpectDistinctIdsWhileBothAlive<MG_State::GLState::BufferObject>("BufferObject");
}