[Feat, Test] (MG_Backend/DirectVulkan, MG_Test): Magma advertisement + the parallel-compile test net

Same gated push as the Espryt commit. The tests ride here because they
exercise both backends' advertisement paths and every piece of the
extension: ParallelShaderCompileTest (13 unit cases - the held-job proof
that GL_FALSE is observable and a second poll still shows outstanding work,
program equivalent, untouched objects read TRUE, always-TRUE with async
off, unknown pnames still INVALID_ENUM, zero-count join+inline for compiles
AND links, nonzero restores while Initialize() does not, clamping and
0xFFFFFFFF and KHR/ARB sharing one state, the getter vs the budget, the
string tracking configuration through glGetString AND glGetStringi) and
AsyncCompileScenario (5 real-GPU cases per the design: 64-compile polling,
forced-join correctness, string/thread-count checks against a live driver,
zero-count synchronous settlement, and async-vs-sync frames rendered
byte-identical with quadrant signatures so two identically-wrong images
cannot pass). Verified 5/5 on NVIDIA in the full 2x2 backend x flag matrix
with MOBILEGL_ITEST_REQUIRE_GPU=1.
This commit is contained in:
BZLZHH
2026-08-08 13:17:58 -04:00
parent f15cb8900f
commit d98f72447d
5 changed files with 1010 additions and 0 deletions
@@ -16,6 +16,7 @@
#include "MG_Util/Converters/MGToStr/TextureEnumConverter.h"
#include "MG_Util/Converters/MGToVk/TextureEnumConverter.h"
#include "MG_Util/Texture/TextureFormatProcessor.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#include <Config.h>
#include <cmath>
@@ -523,6 +524,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (shaderSubgroupSupported && !MG_Config::Features.DisableSubgroup) {
extensions.push_back(E_GL_KHR_shader_subgroup);
}
// GL_KHR_parallel_shader_compile is MobileGL's own capability, not the Vulkan
// device's: the compiler threads belong to MobileGL's shader pool and
// glCompileShader/glLinkProgram are serviced entirely inside the frontend, so there
// is no device feature to condition this on.
//
// Gated on the async flag deliberately, and this is the whole reason the gate
// exists. Advertising the string is the one part of asynchronous compilation that a
// recorded trace can never cover: Iris and Sodium change their SUBMISSION SCHEDULE
// the moment they see it - they enqueue whole pipeline batches and poll
// GL_COMPLETION_STATUS_KHR instead of compiling one program at a time - so
// MOBILEGL_ASYNC_SHADER_COMPILE=0 has to withdraw the application-visible behaviour
// change as well as the threading, or the kill switch would only be half a switch.
if (MG_Util::Async::AsyncShaderCompileEnabled()) {
extensions.push_back(E_GL_KHR_parallel_shader_compile);
}
// GL_ARB_timer_query gates MC's F3 GPU% (LWJGL checks the extension string);
// only advertised when the device actually supports timestamp queries and the
// MOBILEGL_DISABLE_TIMERQUERY escape hatch is off.
@@ -50,6 +50,7 @@ add_executable(MobileGLIntegrationTest
Scenarios/CrossFrameBufferScenario.cpp
Scenarios/ResidentIndexScenario.cpp
Scenarios/MultiDrawScenario.cpp
Scenarios/AsyncCompileScenario.cpp
)
target_include_directories(MobileGLIntegrationTest PRIVATE
@@ -0,0 +1,467 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/AsyncCompileScenario.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario E - asynchronous shader compilation and GL_KHR_parallel_shader_compile
// on a REAL driver.
//
// WHY THIS EXISTS ALONGSIDE THE UNIT SUITES. MG_Test/Program's async suites already
// drive the same GL entry points, but they stop at the frontend: nothing there ever
// reaches a driver, so nothing there can catch the failure this scenario is built for
// - artifacts produced on a worker thread that the BACKEND then rejects, mis-binds or
// renders differently from the ones the GL thread produced. The frontend cannot tell
// the two apart; a pixel can.
//
// The five things it pins, in order:
//
// (a) 64 heavy compiles are enqueued and polled through GL_COMPLETION_STATUS_KHR.
// At least one must be observed GL_FALSE - i.e. the query really answers while
// work is outstanding rather than silently joining. Skipped, never failed, when
// the machine drained the whole batch before the first poll: a fast box must not
// be able to turn this into a red.
// (b) Forcing the join afterwards produces the right answer for every one of them:
// GL_COMPILE_STATUS true, an empty info log, and a program that links.
// (c) The extension string matches the configuration. This is the half a recorded
// trace can never cover - Iris and Sodium change their submission schedule the
// moment they see the string - so it is asserted against a real backend's real
// GL_EXTENSIONS, through both glGetString and glGetStringi.
// (d) glMaxShaderCompilerThreadsKHR(0) leaves nothing in flight: every subsequent
// GL_COMPLETION_STATUS_KHR reads GL_TRUE immediately, and compilation after it
// is synchronous. That is what the extension requires of a zero count.
// (e) THE ONE THAT NEEDS A GPU: the same frame, drawn with programs compiled and
// linked asynchronously and then with programs compiled and linked inline, must
// come out byte-identical under glReadPixels. Anything the worker thread got
// wrong about the compile environment, the reflection or the SPIR-V shows up
// here as a pixel difference and nowhere else.
//
// Backend selection is the module's usual one process, one backend (MOBILEGL_BACKEND_TYPE),
// so this file runs twice per ctest invocation.
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#include "Config.h"
#include "MG_Util/Async/ShaderCompilePool.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
// GL_KHR_parallel_shader_compile. Spelled out rather than relying on the host's
// glext.h: this module is built against whatever GL headers the machine has, and an
// older one has neither token. Both are also GL_*_ARB with identical values.
#ifndef GL_MAX_SHADER_COMPILER_THREADS_KHR
#define GL_MAX_SHADER_COMPILER_THREADS_KHR 0x91B0
#endif
#ifndef GL_COMPLETION_STATUS_KHR
#define GL_COMPLETION_STATUS_KHR 0x91B1
#endif
// The entry point under test, resolved by the linker straight into MobileGL_s like
// every other gl* call in this module. Declared here for the same reason as the
// tokens above.
extern "C" void glMaxShaderCompilerThreadsKHR(GLuint count);
namespace MGITest {
namespace {
using MobileGL::MG_Config::QuirkOverride;
// Same shape as the other scenarios: a two-attribute pass-through, so the only
// thing that can differ between the two compilation modes is the compilation.
constexpr const char* kVertexSource = R"(#version 330 core
in vec2 aPos;
in vec3 aColor;
out vec3 vColor;
void main() {
vColor = aColor;
gl_Position = vec4(aPos, 0.0, 1.0);
}
)";
constexpr const char* kFragmentSource = R"(#version 330 core
in vec3 vColor;
out vec4 oColor;
void main() {
oColor = vec4(vColor, 1.0);
}
)";
// Asymmetric in both axes, so a mode difference that also happens to be a
// symmetry of the image cannot hide (the same reason OrientationScenario draws
// quadrants rather than stripes).
struct Vertex {
float x, y;
float r, g, b;
};
void AppendQuad(std::vector<Vertex>& out, float x0, float x1, float y0, float y1, float r, float g, float b) {
const Vertex bl{x0, y0, r, g, b};
const Vertex br{x1, y0, r, g, b};
const Vertex tr{x1, y1, r, g, b};
const Vertex tl{x0, y1, r, g, b};
out.insert(out.end(), {bl, br, tr, bl, tr, tl});
}
std::vector<Vertex> QuadrantGeometry() {
std::vector<Vertex> vertices;
vertices.reserve(24);
AppendQuad(vertices, -1.0f, 0.0f, -1.0f, 0.0f, 0.0f, 0.0f, 1.0f); // bottom-left: blue
AppendQuad(vertices, 0.0f, 1.0f, -1.0f, 0.0f, 0.0f, 1.0f, 0.0f); // bottom-right: green
AppendQuad(vertices, -1.0f, 0.0f, 0.0f, 1.0f, 1.0f, 0.0f, 0.0f); // top-left: red
AppendQuad(vertices, 0.0f, 1.0f, 0.0f, 1.0f, 1.0f, 1.0f, 1.0f); // top-right: white
return vertices;
}
// Expensive enough that a compile is not instantaneous, and distinct per index so
// the source-hash memo never turns one into a no-op: without both properties the
// pool has no backlog and (a) has nothing to observe.
std::string BulkyFragmentSource(int index) {
std::string source = "#version 330 core\n";
source += "in vec3 vColor;\nout vec4 oColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 320; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " oColor = vec4(vColor * acc, 1.0);\n}\n";
return source;
}
// MOBILEGL_ASYNC_SHADER_COMPILE decides the ambient mode; a scenario that wants
// the other one says so here and gets the ambient one back on scope exit. Forcing
// it in-process is what lets ONE ctest run compare the two modes against each
// other - the whole point of (e).
class AsyncModeScope {
public:
explicit AsyncModeScope(bool async) : m_saved(MobileGL::MG_Config::Features.AsyncShaderCompile) {
MobileGL::MG_Config::Features.AsyncShaderCompile =
async ? QuirkOverride::ForceOn : QuirkOverride::ForceOff;
}
~AsyncModeScope() { MobileGL::MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const QuirkOverride m_saved;
};
// glMaxShaderCompilerThreadsKHR writes process-wide state; a scenario that calls
// it has to put the pool back or it changes how every scenario after it compiles.
class CompilerThreadScope {
public:
CompilerThreadScope() = default;
~CompilerThreadScope() {
MobileGL::MG_Util::Async::SetAsyncShaderCompileSuspended(false);
auto& pool = MobileGL::MG_Util::Async::ShaderCompilePool::Get();
pool.SetMaxConcurrency(pool.GetThreadCount());
}
CompilerThreadScope(const CompilerThreadScope&) = delete;
CompilerThreadScope& operator=(const CompilerThreadScope&) = delete;
};
GLint ShaderCompletion(GLuint shader) {
GLint status = -1;
glGetShaderiv(shader, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint ShaderCompileStatus(GLuint shader) {
GLint status = GL_FALSE;
glGetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
std::string ShaderInfoLog(GLuint shader) {
GLint length = 0;
glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &length);
if (length <= 0) return std::string();
std::vector<char> buffer(static_cast<std::size_t>(length));
GLsizei written = 0;
glGetShaderInfoLog(shader, length, &written, buffer.data());
return std::string(buffer.data(), static_cast<std::size_t>(written));
}
class AsyncCompileScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
const std::vector<Vertex> vertices = QuadrantGeometry();
m_vertexCount = static_cast<int>(vertices.size());
glGenVertexArrays(1, &m_vao);
glBindVertexArray(m_vao);
glGenBuffers(1, &m_vbo);
glBindBuffer(GL_ARRAY_BUFFER, m_vbo);
glBufferData(GL_ARRAY_BUFFER, GLsizeiptr(vertices.size() * sizeof(Vertex)), vertices.data(),
GL_STATIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), reinterpret_cast<void*>(0));
glEnableVertexAttribArray(1);
glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), reinterpret_cast<void*>(8));
glBindVertexArray(0);
ASSERT_EQ(FirstGLError(), GLenum(GL_NO_ERROR)) << "setup left a GL error behind";
}
void TearDown() override {
if (!Ready()) return;
if (m_vbo != 0) glDeleteBuffers(1, &m_vbo);
if (m_vao != 0) glDeleteVertexArrays(1, &m_vao);
}
// A fresh program every time, compiled and linked in whatever mode is in
// force. Reusing one would defeat the comparison: the second mode would just
// read the first mode's artifacts back out of the memo.
GLuint BuildProgram() {
std::string error;
const GLuint program = CompileProgram(kVertexSource, kFragmentSource, &error);
EXPECT_NE(program, 0u) << error;
return program;
}
Image DrawFrameWith(GLuint program) {
BindDefaultFramebuffer();
ClearTo(0.0f, 0.0f, 0.0f, 1.0f);
glDisable(GL_DEPTH_TEST);
glDisable(GL_BLEND);
glUseProgram(program);
glBindVertexArray(m_vao);
glDrawArrays(GL_TRIANGLES, 0, m_vertexCount);
glBindVertexArray(0);
Image image = ReadPixels(Gl().Width(), Gl().Height());
Gl().EndFrame();
return image;
}
// Enqueues `count` distinct heavy compiles and returns their names WITHOUT
// reading anything back, so the pool is left with a real backlog.
std::vector<GLuint> EnqueueBacklog(int count, int seedBase) {
std::vector<GLuint> shaders;
shaders.reserve(static_cast<std::size_t>(count));
m_sources.reserve(m_sources.size() + static_cast<std::size_t>(count));
for (int i = 0; i < count; ++i) {
m_sources.push_back(BulkyFragmentSource(seedBase + i));
const char* text = m_sources.back().c_str();
const GLuint shader = glCreateShader(GL_FRAGMENT_SHADER);
glShaderSource(shader, 1, &text, nullptr);
glCompileShader(shader);
shaders.push_back(shader);
}
return shaders;
}
GLuint m_vao = 0;
GLuint m_vbo = 0;
int m_vertexCount = 0;
// Kept alive for the whole case: glShaderSource copies, but keeping the
// strings makes a failure message able to name the source it came from.
std::vector<std::string> m_sources;
};
// ---- (a) + (b) ------------------------------------------------------------
// A backlog is enqueued, polled without joining, then forced to settle and
// checked for correctness. Both halves in one case on purpose: (b) is only
// interesting for shaders that (a) proved were genuinely still outstanding.
TEST_F(AsyncCompileScenario, CompletionStatusPollingThenForcedJoin) {
if (!Ready()) return;
const AsyncModeScope async(true);
const CompilerThreadScope threads;
// One worker, so the queue behind it is what the poll observes.
glMaxShaderCompilerThreadsKHR(1);
const std::vector<GLuint> shaders = EnqueueBacklog(64, 6000);
int outstanding = 0;
for (const GLuint shader : shaders) {
const GLint completion = ShaderCompletion(shader);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE)
<< "GL_COMPLETION_STATUS_KHR returned " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "this machine drained 64 heavy compiles before the first poll; "
"nothing was outstanding to observe";
}
// (b) Forced join: every one of them is correct, and usable.
for (const GLuint shader : shaders) {
EXPECT_EQ(ShaderCompileStatus(shader), GL_TRUE) << ShaderInfoLog(shader);
EXPECT_TRUE(ShaderInfoLog(shader).empty());
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE) << "GL_COMPILE_STATUS must have joined";
}
// And a link over one of them really produces a usable program on this driver.
const GLuint vs = glCreateShader(GL_VERTEX_SHADER);
glShaderSource(vs, 1, &kVertexSource, nullptr);
glCompileShader(vs);
const GLuint program = glCreateProgram();
glAttachShader(program, vs);
glAttachShader(program, shaders.front());
glBindAttribLocation(program, 0, "aPos");
glBindAttribLocation(program, 1, "aColor");
glLinkProgram(program);
GLint linked = GL_FALSE;
glGetProgramiv(program, GL_LINK_STATUS, &linked);
EXPECT_EQ(linked, GL_TRUE);
EXPECT_GE(glGetUniformLocation(program, "uSeed6000"), 0);
glDeleteProgram(program);
glDeleteShader(vs);
for (const GLuint shader : shaders) glDeleteShader(shader);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (c) ------------------------------------------------------------------
// The extension string, read from a real backend that really brought a driver
// up. No mode forcing here: a backend builds its advertised list once, from the
// configuration in force at its first use, so the meaningful assertion is
// against the AMBIENT configuration - which is exactly what makes this case
// worth running in both of the suite's flag states.
TEST_F(AsyncCompileScenario, ExtensionStringMatchesTheConfiguration) {
if (!Ready()) return;
const bool expected = MobileGL::MG_Util::Async::AsyncShaderCompileEnabled();
const char* extensions = reinterpret_cast<const char*>(glGetString(GL_EXTENSIONS));
ASSERT_NE(extensions, nullptr);
const std::string extensionString(extensions);
const bool inString = extensionString.find("GL_KHR_parallel_shader_compile") != std::string::npos;
EXPECT_EQ(inString, expected)
<< "backend " << Gl().BackendName() << " GL_EXTENSIONS = " << extensionString;
// LWJGL builds GLCapabilities from the INDEXED form on a core profile, so the
// two spellings disagreeing would be invisible to the check above and fatal
// to a real application.
GLint count = 0;
glGetIntegerv(GL_NUM_EXTENSIONS, &count);
ASSERT_GT(count, 0);
bool inIndexed = false;
for (GLint i = 0; i < count; ++i) {
const char* name = reinterpret_cast<const char*>(glGetStringi(GL_EXTENSIONS, GLuint(i)));
if (name != nullptr && std::string(name) == "GL_KHR_parallel_shader_compile") inIndexed = true;
}
EXPECT_EQ(inIndexed, expected);
// The companion query, which an application reads right after the string.
GLint maxThreads = -1;
glGetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &maxThreads);
if (expected) {
EXPECT_GE(maxThreads, 1);
} else {
EXPECT_EQ(maxThreads, 0);
}
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (d) ------------------------------------------------------------------
// A zero count must leave nothing in flight and keep it that way.
TEST_F(AsyncCompileScenario, ZeroCompilerThreadsSettlesEverythingImmediately) {
if (!Ready()) return;
const AsyncModeScope async(true);
const CompilerThreadScope threads;
glMaxShaderCompilerThreadsKHR(1);
const std::vector<GLuint> backlog = EnqueueBacklog(48, 6200);
glMaxShaderCompilerThreadsKHR(0);
for (const GLuint shader : backlog) {
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE)
<< "glMaxShaderCompilerThreadsKHR(0) must join everything still in flight";
EXPECT_EQ(ShaderCompileStatus(shader), GL_TRUE) << ShaderInfoLog(shader);
}
// Compilation after the zero count is synchronous too.
const std::vector<GLuint> serial = EnqueueBacklog(6, 6300);
for (const GLuint shader : serial) {
EXPECT_EQ(ShaderCompletion(shader), GL_TRUE) << "a compile after a zero count must be synchronous";
}
for (const GLuint shader : backlog) glDeleteShader(shader);
for (const GLuint shader : serial) glDeleteShader(shader);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// ---- (e) ------------------------------------------------------------------
// The one that needs the GPU. Two programs, identical source, one built with
// compilation and linking on worker threads and one built inline; the frames
// they draw must be byte-identical.
//
// Compared through the DEFAULT framebuffer deliberately: that is where the
// backend's orientation and present path live, so the comparison covers the
// whole pipeline rather than the reflection tables alone.
TEST_F(AsyncCompileScenario, AsyncAndSyncProgramsRenderIdenticalFrames) {
if (!Ready()) return;
Image asyncImage;
{
const AsyncModeScope async(true);
const GLuint program = BuildProgram();
ASSERT_NE(program, 0u);
asyncImage = DrawFrameWith(program);
glDeleteProgram(program);
}
Image syncImage;
{
const AsyncModeScope async(false);
const GLuint program = BuildProgram();
ASSERT_NE(program, 0u);
syncImage = DrawFrameWith(program);
glDeleteProgram(program);
}
ASSERT_FALSE(asyncImage.Empty());
ASSERT_FALSE(syncImage.Empty());
// The frame is the expected one in the first place - two identically WRONG
// frames would otherwise pass.
EXPECT_EQ(asyncImage.QuadrantSignature(), "blue,green,red,white")
<< "the asynchronously compiled program did not draw the expected frame";
EXPECT_EQ(asyncImage, syncImage)
<< "asynchronous and synchronous compilation rendered different frames ("
<< asyncImage.ByteDiffCount(syncImage) << " bytes differ); backend " << Gl().BackendName();
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
// The same comparison over a batch, which is the shape a shaderpack load has:
// many programs enqueued before any of them is read back, then each one drawn.
// A per-worker state leak (glslang's thread-local pools are the obvious
// candidate) shows up here and not in the single-program case above.
TEST_F(AsyncCompileScenario, ABatchOfAsyncProgramsAllRenderCorrectly) {
if (!Ready()) return;
constexpr int kPrograms = 12;
std::vector<GLuint> programs;
{
const AsyncModeScope async(true);
const CompilerThreadScope threads;
glMaxShaderCompilerThreadsKHR(1);
// Everything enqueued before anything is read: the only shape in which
// more than one job is in flight at a time.
for (int i = 0; i < kPrograms; ++i) {
programs.push_back(BuildProgram());
}
}
for (int i = 0; i < kPrograms; ++i) {
ASSERT_NE(programs[static_cast<std::size_t>(i)], 0u) << "program " << i;
const Image image = DrawFrameWith(programs[static_cast<std::size_t>(i)]);
EXPECT_EQ(image.QuadrantSignature(), "blue,green,red,white") << "program " << i;
}
for (const GLuint program : programs) glDeleteProgram(program);
EXPECT_EQ(FirstGLError(), GLenum(GL_NO_ERROR));
}
} // namespace
} // namespace MGITest
+18
View File
@@ -60,6 +60,22 @@ target_link_libraries(
${LINK_LIBRARIES}
)
add_executable(
ParallelShaderCompileTest
ParallelShaderCompileTest.cpp
)
target_include_directories(ParallelShaderCompileTest PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
ParallelShaderCompileTest PRIVATE
GTest::gtest_main
${LINK_LIBRARIES}
)
# Its own binary on purpose: this one calls MobileGL::Destroy(), and ShaderCompilePool's
# stop is a one-way latch for the whole process - every case declared after it in the same
# binary would silently run its compiles and links inline.
@@ -97,4 +113,6 @@ gtest_discover_tests(ProgramTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
# compile pool so there is something in flight to race against.
gtest_discover_tests(AsyncCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncLinkTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
# Same reason: the GL_COMPLETION_STATUS_KHR cases saturate a one-worker pool on purpose.
gtest_discover_tests(ParallelShaderCompileTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
gtest_discover_tests(AsyncTeardownTest DISCOVERY_TIMEOUT 60 PROPERTIES LABELS unit TIMEOUT 300)
@@ -0,0 +1,508 @@
// MobileGL - MobileGL/MG_Test/Program/ParallelShaderCompileTest.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// P1 stage 5: the GL_KHR_parallel_shader_compile application surface.
//
// Four things are under test, and they are the four an application actually touches:
// * GL_COMPLETION_STATUS_KHR on shaders and programs, which MUST NOT JOIN - the whole
// point of the query is to answer while the work is still outstanding;
// * glMaxShaderCompilerThreadsKHR / ...ARB, including the count == 0 mode switch the
// extension mandates and what lifts it again;
// * GL_MAX_SHADER_COMPILER_THREADS_KHR;
// * the extension string itself, which must appear if and only if asynchronous
// compilation is enabled - the kill switch has to revert the application-visible
// behaviour change, not only the threading.
//
// Like the other async suites, every case drives the real GL entry points and flips
// MG_Config::Features.AsyncShaderCompile itself, so the file behaves identically whether or
// not the suite was launched with MOBILEGL_ASYNC_SHADER_COMPILE=1.
#include <gtest/gtest.h>
#include <algorithm>
#include <string>
#include <vector>
#include "Config.h"
#include "Includes.h"
#include "Init.h"
#include "MG_Backend/BackendObjects.h"
#include "MG_Backend/DirectGLES/BackendObject_DirectGLES.h"
#include "MG_Backend/DirectVulkan/BackendObject_DirectVulkan.h"
#include "MG_Impl/GLImpl/Getter/GL_Getter.h"
#include "MG_Impl/GLImpl/Program/GL_Program.h"
#include "MG_State/GLState/Core.h"
#include "MG_Util/Async/ShaderCompilePool.h"
using namespace MobileGL;
using namespace MobileGL::MG_Impl::GLImpl;
namespace {
class AsyncModeScope {
public:
explicit AsyncModeScope(const Bool async) : m_saved(MG_Config::Features.AsyncShaderCompile) {
MG_Config::Features.AsyncShaderCompile =
async ? MG_Config::QuirkOverride::ForceOn : MG_Config::QuirkOverride::ForceOff;
}
~AsyncModeScope() { MG_Config::Features.AsyncShaderCompile = m_saved; }
AsyncModeScope(const AsyncModeScope&) = delete;
AsyncModeScope& operator=(const AsyncModeScope&) = delete;
private:
const MG_Config::QuirkOverride m_saved;
};
// glMaxShaderCompilerThreadsKHR writes PROCESS-wide state (the pool's concurrency budget
// and the suspension latch), so a case that touches it has to put both back or it
// poisons every case declared after it in this binary.
class CompilerThreadScope {
public:
CompilerThreadScope() = default;
~CompilerThreadScope() {
MG_Util::Async::SetAsyncShaderCompileSuspended(false);
MG_Util::Async::ShaderCompilePool::Get().SetMaxConcurrency(
MG_Util::Async::ShaderCompilePool::Get().GetThreadCount());
}
CompilerThreadScope(const CompilerThreadScope&) = delete;
CompilerThreadScope& operator=(const CompilerThreadScope&) = delete;
};
const char* kVs = R"(#version 460
layout(location = 0) in vec3 aPos;
uniform vec4 uColor;
out vec4 vColor;
void main() {
vColor = uColor;
gl_Position = vec4(aPos, 1.0);
}
)";
// Deliberately expensive, and distinct per index so the source-hash memo never turns a
// second instance into a no-op: a saturated pool is the only way to observe an
// outstanding job without asserting on timing.
String MakeBulkySource(const int index) {
String source = "#version 460\nlayout(location = 0) out vec4 fragColor;\n";
source += "uniform float uSeed" + std::to_string(index) + ";\n";
source += "void main() {\n float acc = uSeed" + std::to_string(index) + ";\n";
for (int i = 0; i < 320; ++i) {
source += " acc = acc * 1.0001 + sin(acc + " + std::to_string(i) + ".0) * cos(acc);\n";
}
source += " fragColor = vec4(acc, acc, acc, 1.0);\n}\n";
return source;
}
GLuint MakeShader(const GLenum type, const char* source) {
const GLuint shader = CreateShader(type);
ShaderSource(shader, 1, &source, nullptr);
return shader;
}
GLint QueryShaderCompletion(const GLuint shader) {
GLint status = -1;
GetShaderiv(shader, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint QueryProgramCompletion(const GLuint program) {
GLint status = -1;
GetProgramiv(program, GL_COMPLETION_STATUS_KHR, &status);
return status;
}
GLint QueryCompileStatus(const GLuint shader) {
GLint status = GL_FALSE;
GetShaderiv(shader, GL_COMPILE_STATUS, &status);
return status;
}
GLint QueryLinkStatus(const GLuint program) {
GLint status = GL_FALSE;
GetProgramiv(program, GL_LINK_STATUS, &status);
return status;
}
// Enqueues `count` distinct heavy compiles and returns their names without reading
// anything back, leaving the pool with a real backlog.
Vector<GLuint> EnqueueBacklog(const int count, const int seedBase, Vector<String>& sourceStorage) {
Vector<GLuint> shaders;
shaders.reserve(static_cast<SizeT>(count));
for (int i = 0; i < count; ++i) {
sourceStorage.push_back(MakeBulkySource(seedBase + i));
const char* text = sourceStorage.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
shaders.push_back(fs);
}
return shaders;
}
Bool Advertises(const Vector<GLExtension>& extensions, const GLExtension wanted) {
return std::find(extensions.begin(), extensions.end(), wanted) != extensions.end();
}
class ParallelShaderCompileTest : public ::testing::Test {
protected:
void SetUp() override { MobileGL::Initialize(); }
};
} // namespace
// ---------------------------------------------------------------------------------------
// GL_COMPLETION_STATUS_KHR must not join
// ---------------------------------------------------------------------------------------
// The load-bearing case of the whole stage. A single-worker pool is saturated with heavy
// compiles, so jobs are demonstrably still queued; GL_COMPLETION_STATUS_KHR then has to
// report GL_FALSE for at least one of them *and leave it outstanding*. If the query joined -
// which is what happens if it is ever routed through the ordinary Compiled() gate - it could
// only ever return GL_TRUE, and the extension would be a lie that costs an application the
// exact stall it added the polling loop to avoid.
//
// Skipped rather than failed when the machine drained the backlog first, so it can never be
// a false red on a fast box.
TEST_F(ParallelShaderCompileTest, ShaderCompletionStatusReportsFalseWithoutJoining) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
// One worker: the queue behind it is the thing being observed.
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(64, 4000, sources);
int outstanding = 0;
for (const GLuint shader : shaders) {
const GLint completion = QueryShaderCompletion(shader);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE) << "completion = " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "the pool drained 64 heavy compiles before the first query; nothing outstanding to observe";
}
// Asking again must still not have settled anything: the query is a peek, so a second
// one cannot have made progress happen. (A joining implementation would report every
// shader complete by now.)
int stillOutstanding = 0;
for (const GLuint shader : shaders) {
if (QueryShaderCompletion(shader) == GL_FALSE) ++stillOutstanding;
}
EXPECT_GT(stillOutstanding, 0) << "GL_COMPLETION_STATUS_KHR joined - every shader settled just by being asked";
// And once the real (joining) query is used, everything is complete and correct.
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE) << "GL_COMPILE_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The program half: a link enqueued behind a saturated pool cannot be complete either, and
// asking must not drag it forward.
TEST_F(ParallelShaderCompileTest, ProgramCompletionStatusReportsFalseWithoutJoining) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
EnqueueBacklog(48, 4200, sources);
Vector<GLuint> programs;
for (int i = 0; i < 8; ++i) {
sources.push_back(MakeBulkySource(4400 + i));
const char* text = sources.back().c_str();
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &text, nullptr);
CompileShader(fs);
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
programs.push_back(program);
}
int outstanding = 0;
for (const GLuint program : programs) {
const GLint completion = QueryProgramCompletion(program);
ASSERT_TRUE(completion == GL_TRUE || completion == GL_FALSE) << "completion = " << completion;
if (completion == GL_FALSE) ++outstanding;
}
if (outstanding == 0) {
GTEST_SKIP() << "the pool drained the whole backlog before the first query; nothing outstanding to observe";
}
for (const GLuint program : programs) {
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE) << "GL_LINK_STATUS must have joined";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// "Nothing outstanding" is the answer for an object that was never compiled or linked at
// all: the query asks whether work is pending, not whether work ever happened.
TEST_F(ParallelShaderCompileTest, CompletionStatusIsTrueForUntouchedObjects) {
const AsyncModeScope async(true);
const GLuint shader = MakeShader(GL_FRAGMENT_SHADER, "#version 460\nvoid main() {}\n");
const GLuint program = CreateProgram();
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// With the flag off nothing is ever in flight, so the query is constant GL_TRUE - and, just
// as importantly, still a recognized pname rather than a GL_INVALID_ENUM.
TEST_F(ParallelShaderCompileTest, CompletionStatusIsAlwaysTrueWithAsyncOff) {
const AsyncModeScope async(false);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(8, 4600, sources);
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE);
}
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, shaders.front());
LinkProgram(program);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The pname is new; the rejection of everything else must be untouched.
TEST_F(ParallelShaderCompileTest, UnknownPnamesStillRaiseInvalidEnum) {
const GLuint shader = MakeShader(GL_FRAGMENT_SHADER, "#version 460\nvoid main() {}\n");
const GLuint program = CreateProgram();
GLint value = 0;
GetShaderiv(shader, GL_TEXTURE_2D, &value);
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
GetProgramiv(program, GL_TEXTURE_2D, &value);
EXPECT_EQ(GetError(), GL_INVALID_ENUM);
}
// ---------------------------------------------------------------------------------------
// glMaxShaderCompilerThreadsKHR / ...ARB
// ---------------------------------------------------------------------------------------
// count == 0 is the mode switch the extension defines: no compiler threads. Two obligations
// follow, and both are asserted here - everything already in flight is settled by the time
// the call returns (so every GL_COMPLETION_STATUS_KHR reads GL_TRUE straight away), and
// compilation that happens AFTERWARDS is synchronous too.
TEST_F(ParallelShaderCompileTest, ZeroCompilerThreadsJoinsEverythingAndCompilesInline) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> backlog = EnqueueBacklog(48, 4800, sources);
MaxShaderCompilerThreadsKHR(0);
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
EXPECT_FALSE(MG_Util::Async::AsyncShaderCompileActive());
// The configuration flag itself is untouched: the extension is still advertised, the
// application just asked for serial compilation.
EXPECT_TRUE(MG_Util::Async::AsyncShaderCompileEnabled());
for (const GLuint shader : backlog) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE)
<< "glMaxShaderCompilerThreadsKHR(0) must leave nothing in flight";
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
}
// Anything compiled from here on is finished before its glCompileShader returns.
Vector<String> serialSources;
const Vector<GLuint> serial = EnqueueBacklog(6, 4900, serialSources);
for (const GLuint shader : serial) {
EXPECT_EQ(QueryShaderCompletion(shader), GL_TRUE) << "a compile after a zero count must be synchronous";
}
// Links too, not just compiles.
const GLuint vs = MakeShader(GL_VERTEX_SHADER, kVs);
CompileShader(vs);
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, serial.front());
LinkProgram(program);
EXPECT_EQ(QueryProgramCompletion(program), GL_TRUE) << "a link after a zero count must be synchronous";
EXPECT_EQ(QueryLinkStatus(program), GL_TRUE);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ...and a later NONZERO count is what lifts it. Nothing else does: not a new context, not a
// join, not eglInitialize. That is the documented contract, so it gets an assertion.
TEST_F(ParallelShaderCompileTest, NonzeroCompilerThreadsRestoresAsynchronousCompilation) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(0);
ASSERT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
// Re-initializing must NOT quietly re-arm it - the application asked for serial
// compilation and has not taken that back.
MobileGL::Initialize();
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
MaxShaderCompilerThreadsKHR(4);
EXPECT_FALSE(MG_Util::Async::IsAsyncShaderCompileSuspended());
EXPECT_TRUE(MG_Util::Async::AsyncShaderCompileActive());
// And work really is being enqueued again: with the budget back at one worker a heavy
// backlog leaves something outstanding (skip-not-fail if the box drained it first).
MaxShaderCompilerThreadsKHR(1);
Vector<String> sources;
const Vector<GLuint> shaders = EnqueueBacklog(64, 5000, sources);
const Bool anyOutstanding = std::any_of(shaders.begin(), shaders.end(), [](const GLuint shader) {
return QueryShaderCompletion(shader) == GL_FALSE;
});
if (!anyOutstanding) {
GTEST_SKIP() << "the pool drained the backlog before the first query; asynchrony not observable here";
}
for (const GLuint shader : shaders) {
EXPECT_EQ(QueryCompileStatus(shader), GL_TRUE);
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// The three count cases map onto the pool's concurrency budget: a request above the thread
// count cannot conjure threads, 0xFFFFFFFF means "implementation maximum", and an ordinary
// value is taken as given (clamped to at least one).
TEST_F(ParallelShaderCompileTest, CompilerThreadCountIsClampedToTheThreadCount) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
auto& pool = MG_Util::Async::ShaderCompilePool::Get();
const Uint threadCount = pool.GetThreadCount();
ASSERT_GE(threadCount, 1u);
MaxShaderCompilerThreadsKHR(1);
EXPECT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsKHR(threadCount + 1000);
EXPECT_EQ(pool.GetMaxConcurrency(), threadCount) << "asking for more threads than exist cannot create any";
MaxShaderCompilerThreadsKHR(1);
ASSERT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsKHR(0xFFFFFFFFu);
EXPECT_EQ(pool.GetMaxConcurrency(), threadCount) << "0xFFFFFFFF is the implementation maximum";
// The ARB spelling is the same entry point, not a second piece of state.
MaxShaderCompilerThreadsARB(1);
EXPECT_EQ(pool.GetMaxConcurrency(), 1u);
MaxShaderCompilerThreadsARB(0);
EXPECT_TRUE(MG_Util::Async::IsAsyncShaderCompileSuspended());
MaxShaderCompilerThreadsKHR(threadCount);
EXPECT_FALSE(MG_Util::Async::IsAsyncShaderCompileSuspended())
<< "the KHR and ARB names must share one piece of state";
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// A zero count with the feature switched off is legal and does nothing observable: there is
// nothing to suspend, and the call must not fail just because MobileGL never had threads.
TEST_F(ParallelShaderCompileTest, CompilerThreadCallsAreHarmlessWithAsyncOff) {
const AsyncModeScope async(false);
const CompilerThreadScope threads;
MaxShaderCompilerThreadsKHR(0);
MaxShaderCompilerThreadsKHR(8);
MaxShaderCompilerThreadsARB(0xFFFFFFFFu);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// GL_MAX_SHADER_COMPILER_THREADS_KHR
// ---------------------------------------------------------------------------------------
TEST_F(ParallelShaderCompileTest, MaxShaderCompilerThreadsGetter) {
{
const AsyncModeScope async(true);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, static_cast<GLint>(MG_Util::Async::ShaderCompilePool::Get().GetThreadCount()));
EXPECT_GE(value, 1);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
{
// No compiler threads exist in this configuration, and the extension is not
// advertised either, so zero is the honest answer.
const AsyncModeScope async(false);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, 0);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
}
// The reported maximum is the pool's THREAD count, not its current concurrency budget: an
// application that lowered the budget still wants to know what the implementation can do.
TEST_F(ParallelShaderCompileTest, MaxShaderCompilerThreadsIgnoresTheCurrentBudget) {
const AsyncModeScope async(true);
const CompilerThreadScope threads;
const Uint threadCount = MG_Util::Async::ShaderCompilePool::Get().GetThreadCount();
MaxShaderCompilerThreadsKHR(1);
GLint value = -1;
GetIntegerv(GL_MAX_SHADER_COMPILER_THREADS_KHR, &value);
EXPECT_EQ(value, static_cast<GLint>(threadCount));
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
// ---------------------------------------------------------------------------------------
// The extension string
// ---------------------------------------------------------------------------------------
// The advertisement is the riskiest half of P1 - a recorded trace cannot cover it, because
// Iris and Sodium change their submission schedule the moment they see the string - so
// MOBILEGL_ASYNC_SHADER_COMPILE=0 has to withdraw it. Asserted on both backends' own
// BuildAdvertisedExtensions, which is the single source of truth each of them (and the
// driver POST) builds the list from.
TEST_F(ParallelShaderCompileTest, BothBackendsAdvertiseTheExtensionIffAsyncIsEnabled) {
{
const AsyncModeScope async(true);
EXPECT_TRUE(Advertises(MG_Backend::DirectGLES::BuildAdvertisedExtensions(false, false),
E_GL_KHR_parallel_shader_compile));
EXPECT_TRUE(Advertises(MG_Backend::DirectVulkan::BuildAdvertisedExtensions(false, false, false),
E_GL_KHR_parallel_shader_compile));
}
{
const AsyncModeScope async(false);
EXPECT_FALSE(Advertises(MG_Backend::DirectGLES::BuildAdvertisedExtensions(false, false),
E_GL_KHR_parallel_shader_compile))
<< "MOBILEGL_ASYNC_SHADER_COMPILE=0 must withdraw the extension, not only the threading";
EXPECT_FALSE(Advertises(MG_Backend::DirectVulkan::BuildAdvertisedExtensions(false, false, false),
E_GL_KHR_parallel_shader_compile))
<< "MOBILEGL_ASYNC_SHADER_COMPILE=0 must withdraw the extension, not only the threading";
}
}
// The same fact through the GL surface an application actually reads. No flag flipping here:
// a backend's advertised list is built once, at its first use, from the configuration that
// was in force then - so this case asserts against the AMBIENT configuration, which is
// exactly what makes it meaningful in both of the suite's two runs (with and without
// MOBILEGL_ASYNC_SHADER_COMPILE=1 exported).
TEST_F(ParallelShaderCompileTest, GLExtensionStringTracksTheAmbientConfiguration) {
UniquePtr<MG_Backend::BackendObject> previousBackend = Move(MG_Backend::pActiveBackendObject);
MG_Backend::pActiveBackendObject = MakeUnique<MG_Backend::DirectGLES::BackendObject_DirectGLES>();
const char* extensions = reinterpret_cast<const char*>(GetString(GL_EXTENSIONS));
ASSERT_NE(extensions, nullptr);
const String extensionString(extensions);
const Bool advertised = extensionString.find("GL_KHR_parallel_shader_compile") != String::npos;
EXPECT_EQ(advertised, MG_Util::Async::AsyncShaderCompileEnabled()) << "GL_EXTENSIONS = " << extensionString;
// glGetStringi must agree with the monolithic string - LWJGL builds GLCapabilities from
// the indexed form on a core profile.
GLint count = 0;
GetIntegerv(GL_NUM_EXTENSIONS, &count);
ASSERT_GT(count, 0);
Bool foundIndexed = false;
for (GLint i = 0; i < count; ++i) {
const char* name = reinterpret_cast<const char*>(GetStringi(GL_EXTENSIONS, static_cast<GLuint>(i)));
if (name != nullptr && std::string(name) == "GL_KHR_parallel_shader_compile") foundIndexed = true;
}
EXPECT_EQ(foundIndexed, advertised);
MG_Backend::pActiveBackendObject = Move(previousBackend);
EXPECT_EQ(GetError(), GL_NO_ERROR);
}