Compare commits

..
Author SHA1 Message Date
BZLZHH 3025284a6e [Perf, Test] (MG_Util): libfork execution engine for the shader compile pool, runtime-selectable
Adds libfork v3.8.0 (3rdparty submodule, header-only, wired like the asio
precedent) as a second execution engine behind ShaderCompilePool, selected
per process by MOBILEGL_ASYNC_POOL=asio|libfork (default asio; unknown
values warn and fall back). The engine boundary is deliberately tiny: the
queue, the concurrency budget and its clamping, the suspension latch,
cancel request-vs-outcome, the stopped-is-synchronous fallback and the
drain all stay in the shared Impl - an engine only answers how a
budget-cleared job reaches a worker.

The libfork engine runs detached root tasks as CHAINS: a finished body
takes the next queued job in the same coroutine on the same worker, so
the refills a worker posts are absorbed without a scheduler round trip
(the naive dispatch-thread shape measured 4x worse than asio on short
jobs). Absorption is bounded at one job per live chain - unbounded
absorption serialized bursts posted from inside the pool, which is the
shipped shape (one compile settling fans out link jobs via SubmitAfter
and the adoption map), caught by the review and pinned by a permanent
peak-concurrency regression test (pre-fix: libfork peak 1 vs asio peak 4
on a 16-job worker-posted burst). External submissions go through a
round-robin adaptor instead of lf::lazy_pool::schedule, which both
avoids a data race on lazy_pool's unsynchronized xoshiro under
concurrent submits and beats birthday-collision placement by ~1.3x at
budget == thread count.

The measured answer to "does asio scale poorly": no - the executor was
never the bottleneck. On real pack corpora extracted from the trace
fixtures (BSL 61 shaders, Complementary 277), interleaved best-of-5 per
cell, the engines are within noise of each other at every thread count
(complementary: 4965/2506/1376/824 ms at 1/2/4/8 threads for asio;
libfork within 1%), both ~6x at 8 threads. perf counters show the
flattening past 4 threads is machine-level (instructions flat at 22.1e9
from 1 to 16 threads - no added work, no lock spinning - while cycles
and LLC misses double: memory-stall bound), and the separating control
- N fully independent single-threaded processes with no shared
scheduler at all - scales WORSE than the pool (5.27x vs 5.94x at 8).
The pool microbenchmark does favor libfork on pure dispatch (518 vs
530 ns/job at 1 worker, growing with worker count), but a real compile
body is 1-100 ms, so dispatch is under 0.1% either way. asio therefore
stays the default; this branch exists to make the comparison
reproducible (MG_Test/Util/AsyncPoolBench drives either engine over a
corpus directory) and to keep the alternative viable.

613/613 unit tests in all four combos ({asio, libfork} x {async default
on, kill switch}), integration scenarios byte-identical between engines
on both backends.
2026-08-09 17:24:01 -04:00
BZLZHH d8d7530011 [Fix, Test] (MG_Backend/DirectGLES, MG_Util, MG_Impl): widen three-channel render targets wherever the driver refuses them
Complementary Reimagined would not load through Espryt on Mali: Iris got
GL_FRAMEBUFFER_UNSUPPORTED building its composite framebuffer, because
colortex1 is RGB8_SNORM and colortex2 is RGB16F - three-channel formats
that no real ES driver can render to (EXT_render_snorm covers R/RG/RGBA
only, and the float extensions exclude the RGB forms). The frontend's
probe cache diagnosed this correctly and then had nothing to offer: the
NoThreeChannelRenderTarget widening machinery existed but was gated to
multisample targets alone. llvmpipe turns out to refuse most of the same
attachments - CI retrace stayed green only because a replay never
branches on glCheckFramebufferStatus - so this was never a desktop-vs-
device split, just an unlit path.

The widening now applies to every color-attachable image, renderbuffers
included, riding the driver-probe branch so the native format is still
tried first and substituted only on refusal. One ThreeChannelWidening
table owns the widened (internalformat, format, type) triple per source
format - the previous per-case branches disagreed with each other and
could emit an unuploadable (RGBA16F, GL_RGB, GL_BYTE) combination or
widen into another three-channel format the driver refuses just the
same. Uploads repack three-component client data to four with the
format's own one in the alpha channel (127 is not 1 for RGB8I - the
integer arms carry integer ones); readback drops the synthetic alpha,
derived from the actual image being read, not the bound framebuffer,
so glGetTexImage through a scratch FBO cannot be confused by an
unrelated widened attachment.

Stored alpha on a widened attachment is now an invariant 1.0 rather
than an accident: the color-mask sync clears the alpha bit per draw
buffer (glColorMaski for MRT mixes), and clears route through
glClearBufferfv with alpha substituted on widened slots only -
scissored clears inherit the discipline for free, integer color
buffers keep their explicit integer-clear path, and glGet still
answers the application's own mask. GL_DST_ALPHA blending, blits and
readback therefore all see 1.0 without further interception.

DriverPost grows the rows this bug earned: EXT_color_buffer_float
detection (previously unreferenced anywhere) with a FAIL row when
absent, the missing EXT_render_snorm row, and a three-channel-
attachment row that reports one representative per widening class -
graded so a half-float-only driver warns about the 32-bit float gap
instead of being declared unsupported.

Gates: 606/606 unit at default and with the async kill switch; full
retrace, both backends - the complementary fixtures now run with the
widening ACTIVE on llvmpipe and pass with a slightly better SSIM than
before; ext caselist DirectGLES holds 3914/4867 with zero set drift
while 54 cases move from NotSupported to genuinely passing; on the
Mali-G77 device, Complementary Reimagined builds its pipeline and
renders in-world through Espryt (md5-verified build), BSL still green.
A new ThreeChannelAttachmentScenario pins the frontend answer -
COMPLETE where it used to say UNSUPPORTED - on the real driver.
2026-08-09 15:28:34 -04:00
BZLZHH 0f394fa46f [Test] (tools/trace_replay): pin the Iris glyph-death bug with a BSL pause-menu fixture
minecraft-1.21.4-fabric-iris-bsl-esc-menu-854: Minecraft 1.21.4 Fabric +
Sodium + Iris + BSL 10.1.3 through Espryt on Mali-G77, paused at the ESC
menu over the BSL-blurred world, captured at 854x480 through the FCL
trace-capture skill. The trimmed trace keeps the whole session preamble
deliberately: the 107669b3 bug class is triggered while Iris BUILDS the
pipeline (a by-name texture call swapping the active unit's binding under
the sync memo), and the pause menu afterwards is where its damage is
legible - every glyph alpha-discards against the zeroed lightmap while
the button frames survive.

Ablation-proven as a net before landing: against a build with both layers
of 107669b3 reverted the case fails at ssim 0.949284 and the diff image
is text and only text - menu title, all button labels, the tutorial
toast; against the fixed library it matches the golden exactly, and
Magma passes the same golden at 0.998402 with no alternate needed.
Replays in about a minute per backend on llvmpipe/lavapipe (the 1.3M-call
preamble; well under the timeout and smaller than several existing
fixtures). Existing fixtures spot-checked green.
2026-08-09 12:49:10 -04:00
BZLZHH 107669b3db [Fix, Test] (MG_Impl, MG_Backend/DirectGLES): DSA by-name texture calls corrupted borrowed-slot memo pairings - process-wide glyph death under Iris
Field report: on Espryt with a BSL Iris pipeline built, every glyph in the
game died - HUD, menu labels, even the vanilla title screen after leaving
the world - while sprites kept rendering. Captured on-device (FCL apitrace
rig), reproduced headlessly on llvmpipe, and pinned with a three-way replay:
the same trace renders full text on raw Mesa desktop GL and on Magma, so
the stream was intact and the execution was Espryt's.

MECHANISM. WithTemporarilyBoundNamedTexture implements the by-name (DSA)
texture entry points by binding the named texture onto the active unit's
real slot, running the bound-texture code, and restoring - without moving
the texture bind generation on either edge. DirectGLES's per-draw texture
sync memo keys on that generation and BORROWS the slot pointer, so a memo
built for texture A kept passing every key while a by-name call had
texture B sitting in the slot: A's backend twin was driven with B's
frontend object, and SyncMipmapsToBackend re-specified A's storage with
B's shape. In the trace, a by-name upload to a BSL 2048x2048 map while
the 16x16 lightmap was bound re-specified the lightmap's GL texture
2048x2048-NULL and back 16x16-NULL. The lightmap exists only as render
output - no glTexSubImage2D ever touches it - so it stayed zero forever,
and rendertype_text (vertexColor = Color * texelFetch(lightmap, ...)),
alpha-discards every glyph. Background quads never sample the lightmap,
which is why only text died.

FIX, class-level, two layers:
- Frontend (shared, closes the same hole for DirectVulkan's generation-
  keyed memos): the temporary bind and the restore each bump the texture
  bind generation (only when the slot actually changed), and the restore
  is an RAII scope guard so a throwing body can no longer leak the
  temporary binding - a second latent bug of the same class. Deliberately
  a generation bump and not a touched-unit note: the high-water mark must
  not chase by-name calls, and a completed bind/restore pair leaves the
  content epoch unchanged, so the cost is an owner-compare re-walk, not a
  memo rebuild.
- DirectGLES defense in depth: both borrowed-pair memos
  (g_unitTextureSyncList, g_fboTextureSyncList) record which frontend
  texture each backend twin was paired with and re-check it before any
  replay (last in the key conjunction, behind the context-id compare). A
  stale pairing now costs a list rebuild instead of silent cross-texture
  storage corruption.

Tests, both red with their own layer reverted:
TextureTest.NamedTextureCallKeepsUnitBindingAccountingCoherent (the
accounting contract) and DirectGLESTextureSync.UnitMemoRefusesToDriveA-
TwinFromAnotherTexture (the corrupting sequence shape against a mock GLES
table, asserting the resident texture's storage is never re-specified).
595/595 unit at default and with the async kill switch. Replay evidence:
the captured BSL ESC-menu trace renders all text through Espryt post-fix,
byte-comparable to the Mesa-direct and Magma replays; the no-shaderpack
control is unchanged. A trace fixture wiring this scene into CI follows
in a separate commit.
2026-08-09 11:28:24 -04:00
32 changed files with 3650 additions and 216 deletions
+3
View File
@@ -34,3 +34,6 @@
[submodule "3rdparty/asio"] [submodule "3rdparty/asio"]
path = 3rdparty/asio path = 3rdparty/asio
url = https://github.com/chriskohlhoff/asio.git url = https://github.com/chriskohlhoff/asio.git
[submodule "3rdparty/libfork"]
path = 3rdparty/libfork
url = https://github.com/ConorWilliams/libfork.git
Vendored Submodule
+1
Submodule 3rdparty/libfork added at 9b2b844a5f
+7
View File
@@ -373,6 +373,13 @@ set(MOBILEGL_INCLUDE_DIR
# MG_Util/Async/ShaderCompilePool.cpp includes it, and it stays behind that file's # MG_Util/Async/ShaderCompilePool.cpp includes it, and it stays behind that file's
# pimpl so no consumer target needs this path. # pimpl so no consumer target needs this path.
${CMAKE_SOURCE_DIR}/3rdparty/asio/asio/include ${CMAKE_SOURCE_DIR}/3rdparty/asio/asio/include
# The second shader-compile execution engine (MOBILEGL_ASYNC_POOL=libfork), on the
# same terms as Asio above: header-only, no add_subdirectory (its CMakeLists only
# declares an INTERFACE target plus install/test scaffolding we do not want), no link
# target, and reachable from exactly one translation unit. libfork's own
# target_compile_features asks for cxx_std_23, which this project already sets
# globally, so its C++20 coroutines need no per-source standard override.
${CMAKE_SOURCE_DIR}/3rdparty/libfork/include
) )
add_library(${CMAKE_PROJECT_NAME} SHARED add_library(${CMAKE_PROJECT_NAME} SHARED
+6
View File
@@ -66,6 +66,12 @@ namespace MobileGL::MG_Config {
// - DISPLAY: X11 session variable, not MobileGL configuration. // - DISPLAY: X11 session variable, not MobileGL configuration.
// - MOBILEGL_LOG_FILE_PATH: log-file init runs before MG_ConfigLoader::Init // - MOBILEGL_LOG_FILE_PATH: log-file init runs before MG_ConfigLoader::Init
// (see MG_Util/Debug/Log.cpp). // (see MG_Util/Debug/Log.cpp).
// - MOBILEGL_ASYNC_POOL: a ShaderCompilePool is constructed by binaries that never call
// MobileGL::Initialize() and so never run MG_ConfigLoader::Init - MG_Test's
// JobNodeTest builds pools directly, and it is the suite that runs the whole async
// matrix against both execution engines. Mirroring it here would resolve to the
// default in exactly the tests that exist to tell the engines apart (see
// MG_Util/Async/ShaderCompilePool.cpp, DetectAsyncPoolEngine).
struct FeaturesTable { struct FeaturesTable {
// MOBILEGL_DISABLE_TIMERQUERY: do not advertise or use GPU timer queries. // MOBILEGL_DISABLE_TIMERQUERY: do not advertise or use GPU timer queries.
Bool DisableTimerQuery = false; Bool DisableTimerQuery = false;
@@ -210,7 +210,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
reasons.push_back("GL_DEPTH_COMPONENT32 native probe failed on OpenGL ES"); reasons.push_back("GL_DEPTH_COMPONENT32 native probe failed on OpenGL ES");
} }
if (options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget) { if (options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget) {
reasons.push_back("no three-channel multisample storage format on OpenGL ES"); reasons.push_back("no colour-renderable three-channel format on OpenGL ES");
} }
if (options & PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget) { if (options & PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget) {
reasons.push_back("EXT_render_snorm not supported"); reasons.push_back("EXT_render_snorm not supported");
@@ -553,26 +553,60 @@ namespace MobileGL::MG_Backend::DirectGLES {
for (SizeT targetIndex = 0; targetIndex < kFormatCapabilityTextureTargetCount; ++targetIndex) { for (SizeT targetIndex = 0; targetIndex < kFormatCapabilityTextureTargetCount; ++targetIndex) {
const auto target = static_cast<TextureTarget>(targetIndex); const auto target = static_cast<TextureTarget>(targetIndex);
// A multisample texture can only ever be rendered into, so its storage format // Colour-attachable targets need a colour-renderable fallback; the ordinary
// has to stay colour-renderable; the ordinary fallback for a three-channel // fallback for a three-channel format is another three-channel one, which ES
// format is a three-channel one, which ES accepts as a texture but rejects as // accepts as a texture but never as an attachment. Recompute the fallback per
// multisample storage. Recompute the fallback per target so those formats get // target so those formats get widened where the target demands it.
// widened here and nowhere else. const Flags<PixelFormatNormalizeOptionBit> renderTargetOptions =
Flags<PixelFormatNormalizeOptionBit> targetOptions; TextureImpl::GetRenderTargetNormalizeOptions(capabilities, targetIndex);
if (IsGLESProbeMultisampleTarget(target)) { // Multisample storage has no three-channel form on ES at all, so its widening
targetOptions |= PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget; // is unconditional and skips the native probe (which cannot succeed). Every
if (!capabilities.SupportsRenderSnorm || !capabilities.SupportsNorm16Texture) { // other target keeps the widening on the DRIVER branch, behind the native
targetOptions |= PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget; // probe: `shouldProbeFallback = !nativeCreated || !nativeRenderable` below is
} // what makes the substitution conditional on the driver actually refusing, so
} // a driver that does render to a three-channel image keeps allocating it byte
// for byte. That is a per-format runtime answer, NOT a desktop-vs-device
// split: llvmpipe renders to GL_RGB16F but refuses GL_RGB8_SNORM, GL_SRGB8,
// GL_RGB32F and the RGB integer formats, so the CI driver widens those eight
// too. Re-run the retrace fixtures and the glcts suites on any change here.
const Bool widenUnconditionally = IsGLESProbeMultisampleTarget(target);
GLESProbeFormatInfo fallbackInfo = outerFallbackInfo; GLESProbeFormatInfo fallbackInfo = outerFallbackInfo;
Bool hasForcedFallback = outerHasForcedFallback; Bool hasForcedFallback = outerHasForcedFallback;
if (targetOptions) { if (renderTargetOptions) {
hasForcedFallback = BuildFallbackProbeFormatInfo( // Folded into the forced options only when a forced fallback already
requestedInternalFormat, forcedOptions | targetOptions, true, fallbackInfo); // applies, so the render-target bits never *create* one: ANGLE's forced
if (!hasForcedFallback) { // GL_RGB8_SNORM -> GL_RGB16F is still three-channel and still needs
BuildFallbackProbeFormatInfo(requestedInternalFormat, driverOptions | targetOptions, false, // widening, but a non-ANGLE driver must not lose its native probe.
const Flags<PixelFormatNormalizeOptionBit> forcedProbeOptions =
(outerHasForcedFallback || widenUnconditionally) ? forcedOptions | renderTargetOptions
: forcedOptions;
hasForcedFallback =
BuildFallbackProbeFormatInfo(requestedInternalFormat, forcedProbeOptions, true,
fallbackInfo); fallbackInfo);
if (!hasForcedFallback) {
BuildFallbackProbeFormatInfo(requestedInternalFormat,
driverOptions | renderTargetOptions, false, fallbackInfo);
}
// HONEST STATUS OF THE FORCED PATH. A forced fallback is only ever built
// for ANGLE (GetForcedPixelFormatNormalizeOptions returns nothing for any
// other renderer), and it SKIPS the native probe entirely - the widened
// format is asserted rather than measured on this device. That assertion
// is validated on exactly one configuration, the android-angle retrace
// golden; it is NOT covered by the headless llvmpipe suites, which take
// the driver branch below and prove nothing about ANGLE's answers. So log
// the choice at INFO rather than the usual MGLOG_D caveat: on any other
// ANGLE device the device report is the only evidence there is of which
// storage format the image really got. Once per format on the ordinary 2D
// target - repeating it for all ten targets would bury the report.
if (hasForcedFallback && target == TextureTarget::Texture2D &&
(MG_Util::TextureFormatProcessor::GetApplicablePixelFormatNormalizeOptions(
requestedInternalFormat, renderTargetOptions) &
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget)) {
MGLOG_I("Three-channel widening (FORCED path, no native probe): %s stored as %s. "
"Reason: %s. Device-validated on the android-angle golden only.",
MG_Util::ConvertTextureInternalFormatToString(logicalFormat).c_str(),
ConvertFallbackInternalFormatToString(fallbackInfo.InternalFormat).c_str(),
fallbackInfo.Reason.c_str());
} }
} }
@@ -618,8 +652,26 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
const SizeT renderbufferTargetIndex = GetRenderbufferFormatCapabilityTargetIndex(); const SizeT renderbufferTargetIndex = GetRenderbufferFormatCapabilityTargetIndex();
Bool shouldProbeFallbackRenderbuffer = outerHasForcedFallback; // A renderbuffer exists only to be attached, so it needs the same three-channel
if (!outerHasForcedFallback) { // widening the colour-attachable texture targets get - and on the same terms: the
// native storage is probed first, so a driver that renders to it keeps it.
const Flags<PixelFormatNormalizeOptionBit> renderbufferOptions =
TextureImpl::GetRenderTargetNormalizeOptions(capabilities, renderbufferTargetIndex);
GLESProbeFormatInfo renderbufferFallbackInfo = outerFallbackInfo;
Bool renderbufferHasForcedFallback = outerHasForcedFallback;
if (renderbufferOptions) {
const Flags<PixelFormatNormalizeOptionBit> forcedProbeOptions =
outerHasForcedFallback ? forcedOptions | renderbufferOptions : forcedOptions;
renderbufferHasForcedFallback = BuildFallbackProbeFormatInfo(
requestedInternalFormat, forcedProbeOptions, true, renderbufferFallbackInfo);
if (!renderbufferHasForcedFallback) {
BuildFallbackProbeFormatInfo(requestedInternalFormat, driverOptions | renderbufferOptions,
false, renderbufferFallbackInfo);
}
}
Bool shouldProbeFallbackRenderbuffer = renderbufferHasForcedFallback;
if (!renderbufferHasForcedFallback) {
const Bool nativeRenderbufferComplete = const Bool nativeRenderbufferComplete =
ProbeRenderbuffer(gl, nativeInfo.InternalFormat, logicalFormat, false, 1); ProbeRenderbuffer(gl, nativeInfo.InternalFormat, logicalFormat, false, 1);
if (nativeRenderbufferComplete) { if (nativeRenderbufferComplete) {
@@ -633,16 +685,16 @@ namespace MobileGL::MG_Backend::DirectGLES {
shouldProbeFallbackRenderbuffer = true; shouldProbeFallbackRenderbuffer = true;
} }
} }
if (shouldProbeFallbackRenderbuffer && outerFallbackInfo.InternalFormat != GL_UNKNOWN_MGL && if (shouldProbeFallbackRenderbuffer && renderbufferFallbackInfo.InternalFormat != GL_UNKNOWN_MGL &&
ProbeRenderbuffer(gl, outerFallbackInfo.InternalFormat, logicalFormat, false, 1)) { ProbeRenderbuffer(gl, renderbufferFallbackInfo.InternalFormat, logicalFormat, false, 1)) {
if (AddCaveatFormatCaps(cache, renderbufferTargetIndex, formatIndex, if (AddCaveatFormatCaps(cache, renderbufferTargetIndex, formatIndex,
GetRenderbufferFeatureCaps(logicalFormat))) { GetRenderbufferFeatureCaps(logicalFormat))) {
LogGLESFormatCaveat(logicalFormat, renderbufferTargetIndex, outerFallbackInfo); LogGLESFormatCaveat(logicalFormat, renderbufferTargetIndex, renderbufferFallbackInfo);
} }
const Int maxSamples = const Int maxSamples =
GetGLESFormatMaxSamples(capabilities, logicalFormat, outerFallbackInfo.ImageFormat); GetGLESFormatMaxSamples(capabilities, logicalFormat, renderbufferFallbackInfo.ImageFormat);
cache.SampleCounts[renderbufferTargetIndex][formatIndex] = cache.SampleCounts[renderbufferTargetIndex][formatIndex] = ProbeRenderbufferSampleCounts(
ProbeRenderbufferSampleCounts(gl, outerFallbackInfo.InternalFormat, logicalFormat, maxSamples); gl, renderbufferFallbackInfo.InternalFormat, logicalFormat, maxSamples);
} }
} }
} }
+249 -46
View File
@@ -1127,10 +1127,29 @@ namespace MobileGL::MG_Backend::DirectGLES {
// slot the key covers still holds a reference to it. Holding either side by shared_ptr // slot the key covers still holds a reference to it. Holding either side by shared_ptr
// instead would keep dead frontend textures alive and defeat the registry's // instead would keep dead frontend textures alive and defeat the registry's
// weak-reference GC. // weak-reference GC.
//
// `texture` records WHICH frontend object `backend` was paired with when the entry was
// built, and PairingsIntact re-checks it before any replay. The keys above are the
// primary guard, but they are all derived state: a slot swap that never reaches the
// bind generation (the DSA by-name emulation used to swap a slot silently) would leave
// every key matching while the borrowed slot pointed at a different texture, and the
// replay would then drive texture A's backend twin from texture B's frontend state -
// re-specifying A's backend storage with B's shape and destroying A's contents. A raw
// pointer compare per entry is far cheaper than the walk it guards, and a stale pairing
// costs only a list rebuild, so this stays as the structural net under the keys.
struct UnitTextureSyncEntry { struct UnitTextureSyncEntry {
const SharedPtr<MG_State::GLState::ITextureObject>* slot = nullptr; const SharedPtr<MG_State::GLState::ITextureObject>* slot = nullptr;
MG_State::GLState::ITextureObject* texture = nullptr;
BackendTextureObject* backend = nullptr; BackendTextureObject* backend = nullptr;
}; };
// True while every entry's borrowed slot still holds the texture the entry was paired
// with. Callers put it LAST in the key conjunction so it only runs on a key hit.
static Bool PairingsIntact(const Vector<UnitTextureSyncEntry>& list) {
for (const auto& entry : list) {
if (entry.slot->get() != entry.texture) return false;
}
return true;
}
static Vector<UnitTextureSyncEntry> g_unitTextureSyncList; static Vector<UnitTextureSyncEntry> g_unitTextureSyncList;
static Bool g_unitTextureSyncListValid = false; static Bool g_unitTextureSyncListValid = false;
static Uint64 g_unitTextureSyncListContextId = 0; static Uint64 g_unitTextureSyncListContextId = 0;
@@ -1197,7 +1216,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_unitTextureSyncListMaxUnit == maxTouchedUnit && g_unitTextureSyncListMaxUnit == maxTouchedUnit &&
g_unitTextureSyncListContextGeneration == g_textureContextGeneration && g_unitTextureSyncListContextGeneration == g_textureContextGeneration &&
g_unitTextureSyncListEpoch == unitBindingsEpoch && g_unitTextureSyncListEpoch == unitBindingsEpoch &&
g_unitTextureSyncListSamplingGeneration == samplingGeneration) { g_unitTextureSyncListSamplingGeneration == samplingGeneration &&
PairingsIntact(g_unitTextureSyncList)) {
for (const auto& entry : g_unitTextureSyncList) { for (const auto& entry : g_unitTextureSyncList) {
// Aggregate gate == the conjunction of the three callees' own // Aggregate gate == the conjunction of the three callees' own
// early-outs (see IsDrawSyncClean); skipping on true is // early-outs (see IsDrawSyncClean); skipping on true is
@@ -1219,8 +1239,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
// An image-less default texture (name 0) is the slot's initial / "unbound" // An image-less default texture (name 0) is the slot's initial / "unbound"
// state; it has nothing to sync, so skip it as cheaply as the old null slot. // state; it has nothing to sync, so skip it as cheaply as the old null slot.
if (textureObject && !MG_State::GLState::IsUndefinedDefaultTexture(textureObject.get())) { if (textureObject && !MG_State::GLState::IsUndefinedDefaultTexture(textureObject.get())) {
g_unitTextureSyncList.push_back( g_unitTextureSyncList.push_back({&textureObject, textureObject.get(),
{&textureObject, SyncTextureObjectToBackend(textureObject).get()}); SyncTextureObjectToBackend(textureObject).get()});
} }
} }
} }
@@ -1254,7 +1274,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_fboTextureSyncListSlotVersion == fboSlotVersion && g_fboTextureSyncListSlotVersion == fboSlotVersion &&
g_fboTextureSyncListObjectVersion == fboObjectVersion && g_fboTextureSyncListObjectVersion == fboObjectVersion &&
g_fboTextureSyncListContextId == keys.contextId && g_fboTextureSyncListContextId == keys.contextId &&
g_fboTextureSyncListContextGeneration == g_textureContextGeneration; g_fboTextureSyncListContextGeneration == g_textureContextGeneration &&
PairingsIntact(g_fboTextureSyncList);
if (fboListValid) { if (fboListValid) {
for (const auto& entry : g_fboTextureSyncList) { for (const auto& entry : g_fboTextureSyncList) {
// Same aggregate gate as the unit list above. // Same aggregate gate as the unit list above.
@@ -1273,8 +1294,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (!attachment.IsTexture()) continue; if (!attachment.IsTexture()) continue;
auto& textureObject = attachment.GetTexture(); auto& textureObject = attachment.GetTexture();
if (textureObject) { if (textureObject) {
g_fboTextureSyncList.push_back( g_fboTextureSyncList.push_back({&textureObject, textureObject.get(),
{&textureObject, SyncTextureObjectToBackend(textureObject).get()}); SyncTextureObjectToBackend(textureObject).get()});
} }
} }
g_fboTextureSyncListFbo = currentFBO.get(); g_fboTextureSyncListFbo = currentFBO.get();
@@ -1380,7 +1401,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
if (currentFBO == MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo->defaultFBO) { if (currentFBO == MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo->defaultFBO) {
// Default FBO, nothing to sync // Default FBO, nothing to sync - except the widened-attachment mask, which is
// only ever WRITTEN by SyncToBackend and would otherwise still describe the
// user FBO that was draw-bound before. The window surface is a real RGBA
// buffer, so nothing here is ever widened.
if (target == FramebufferTarget::Draw) {
g_alphaWidenedDrawBufferMask = 0;
g_integerColorDrawBufferMask = 0;
}
StampSyncedFBO(target, slotVersion, objectVersion, currentPtr); StampSyncedFBO(target, slotVersion, objectVersion, currentPtr);
continue; continue;
} }
@@ -1438,20 +1466,35 @@ namespace MobileGL::MG_Backend::DirectGLES {
// direct_state_access.renderbuffers_storage. One unconditional push settles the whole // direct_state_access.renderbuffers_storage. One unconditional push settles the whole
// block rather than the one cap that happened to be noticed. // block rather than the one cap that happened to be noticed.
static Bool g_forceFullRenderStateResync = true; static Bool g_forceFullRenderStateResync = true;
// Which draw buffers' alpha channel the colour mask last pushed to the driver had forced
// OFF - i.e. the value of `appliedWidenMask` in the last SyncRenderState that reached the
// colour-mask block. NOT derivable from the frontend parameter block: it depends on the
// bound DRAW framebuffer's attachment formats and on whether the caller is a draw or a
// clear, neither of which bumps the frontend render-state version. Without it a
// clear-then-draw pair on an unchanged parameter block early-outs and the draw inherits
// the clear's undoctored mask.
static Uint32 g_syncedColorMaskAlphaWidenMask = 0;
void InvalidateSyncedRenderState() { void InvalidateSyncedRenderState() {
g_forceFullRenderStateResync = true; g_forceFullRenderStateResync = true;
g_hasSyncedRenderState = false; g_hasSyncedRenderState = false;
g_syncedBackendViewport = IntVec4(-1, -1, -1, -1); g_syncedBackendViewport = IntVec4(-1, -1, -1, -1);
g_syncedBackendScissorBox = IntVec4(-1, -1, -1, -1); g_syncedBackendScissorBox = IntVec4(-1, -1, -1, -1);
} }
void SyncRenderState() { void SyncRenderState(Bool forColorClear) {
#ifdef TRACY_ENABLE #ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND); ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif #endif
Uint16 currentRenderStateVersion = MG_State::pGLContext->GetRenderStateParametersVersion(); Uint16 currentRenderStateVersion = MG_State::pGLContext->GetRenderStateParametersVersion();
const Bool forceFullPush = g_forceFullRenderStateResync; const Bool forceFullPush = g_forceFullRenderStateResync;
g_forceFullRenderStateResync = false; g_forceFullRenderStateResync = false;
if (!forceFullPush && g_hasSyncedRenderState && currentRenderStateVersion == g_syncedRenderStateVersion) { // The alpha discipline for widened colour attachments (see the header comment on
// SyncRenderState): a DRAW must not be able to move the stored alpha off 1.0, a CLEAR
// is what puts it there. So the draw path masks alpha off on every widened draw
// buffer and the clear path masks nothing.
const Uint32 appliedWidenMask = forColorClear ? 0u : FramebufferImpl::g_alphaWidenedDrawBufferMask;
const Bool colorMaskWidenDirty = appliedWidenMask != g_syncedColorMaskAlphaWidenMask;
if (!forceFullPush && !colorMaskWidenDirty && g_hasSyncedRenderState &&
currentRenderStateVersion == g_syncedRenderStateVersion) {
return; return;
} }
@@ -1757,23 +1800,42 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
} }
if (tailSpanDirty) { // Color mask. Uniform masks use the non-indexed glColorMask (works everywhere); divergent if (tailSpanDirty || colorMaskWidenDirty) { // Color mask. Uniform masks use the non-indexed glColorMask
// per-draw-buffer masks use the indexed glColorMaski when draw_buffers_indexed is // (works everywhere); divergent per-draw-buffer masks use the indexed glColorMaski when
// available, otherwise fall back to broadcasting draw buffer 0. Mirrors the blend block. // draw_buffers_indexed is available, otherwise fall back to broadcasting draw buffer 0.
// Mirrors the blend block.
using FBO = MG_State::GLState::FramebufferObject; using FBO = MG_State::GLState::FramebufferObject;
const auto& targetMasks = parameters.ColorMasks; const auto& targetMasks = parameters.ColorMasks;
const auto& syncedMasks = g_syncedRenderStateParameters.ColorMasks; const auto& syncedMasks = g_syncedRenderStateParameters.ColorMasks;
Bool anyDirty = forceFullPush; // What the DRIVER is told for draw buffer i. Identical to the application's mask
// except on a widened attachment during a draw, where alpha is forced off; the
// frontend's own array is never written, so glGet(GL_COLOR_WRITEMASK) keeps
// answering with the application's value.
const auto driverMask = [&](Uint i) -> BoolVec4 {
BoolVec4 m = targetMasks[i];
if (i < 32 && (appliedWidenMask & (1u << i)) != 0) {
m.w() = false;
}
return m;
};
Bool anyDirty = forceFullPush || colorMaskWidenDirty;
Bool allSame = true; Bool allSame = true;
const BoolVec4 driverMask0 = driverMask(0);
for (Uint i = 0; i < FBO::MAX_DRAW_BUFFERS; ++i) { for (Uint i = 0; i < FBO::MAX_DRAW_BUFFERS; ++i) {
if (targetMasks[i] != syncedMasks[i]) anyDirty = true; if (targetMasks[i] != syncedMasks[i]) anyDirty = true;
if (i > 0 && targetMasks[i] != targetMasks[0]) allSame = false; if (i > 0 && driverMask(i) != driverMask0) allSame = false;
} }
if (anyDirty) { if (anyDirty) {
if (allSame || !g_GLESCapabilities.SupportsIndexedColorMask) { if (allSame || !g_GLESCapabilities.SupportsIndexedColorMask) {
const BoolVec4& m = targetMasks[0]; // Without draw_buffers_indexed there is only one mask for the whole
// framebuffer, so a widened draw buffer 0 costs every other buffer its
// alpha writes. ES 3.2 makes glColorMaski core and ES 3.1 has it as
// EXT/OES; the only devices that reach this line are ES 3.0-class, where
// MRT with a mixed widened/native colour attachment set is already rare.
const BoolVec4& m = driverMask0;
g_GLESFuncs.glColorMask(ToGLBoolean(m.x()), ToGLBoolean(m.y()), ToGLBoolean(m.z()), g_GLESFuncs.glColorMask(ToGLBoolean(m.x()), ToGLBoolean(m.y()), ToGLBoolean(m.z()),
ToGLBoolean(m.w())); ToGLBoolean(m.w()));
} else { } else {
@@ -1781,14 +1843,18 @@ namespace MobileGL::MG_Backend::DirectGLES {
: g_GLESFuncs.glColorMaskiEXT ? g_GLESFuncs.glColorMaskiEXT : g_GLESFuncs.glColorMaskiEXT ? g_GLESFuncs.glColorMaskiEXT
: g_GLESFuncs.glColorMaskiOES; : g_GLESFuncs.glColorMaskiOES;
for (Uint i = 0; i < FBO::MAX_DRAW_BUFFERS; ++i) { for (Uint i = 0; i < FBO::MAX_DRAW_BUFFERS; ++i) {
if (forceFullPush || targetMasks[i] != syncedMasks[i]) { // colorMaskWidenDirty forces every slot: the previous push may have
const BoolVec4& m = targetMasks[i]; // been the non-indexed glColorMask above (which set all of them), and
// the per-slot diff below only knows about the application's array.
if (forceFullPush || colorMaskWidenDirty || targetMasks[i] != syncedMasks[i]) {
const BoolVec4 m = driverMask(i);
colorMaskiFn(i, ToGLBoolean(m.x()), ToGLBoolean(m.y()), ToGLBoolean(m.z()), colorMaskiFn(i, ToGLBoolean(m.x()), ToGLBoolean(m.y()), ToGLBoolean(m.z()),
ToGLBoolean(m.w())); ToGLBoolean(m.w()));
} }
} }
} }
} }
g_syncedColorMaskAlphaWidenMask = appliedWidenMask;
} }
if (tailSpanDirty) { // Polygon mode. GLES core has no glPolygonMode; use NV/ANGLE_polygon_mode when present. if (tailSpanDirty) { // Polygon mode. GLES core has no glPolygonMode; use NV/ANGLE_polygon_mode when present.
@@ -2068,6 +2134,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
ZoneScopedC(TRACY_ZONECOLOR_BACKEND); ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif #endif
if (!framebuffer || framebuffer == MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo->defaultFBO) { if (!framebuffer || framebuffer == MG_Impl::GLImpl::FramebufferImpl::pDefaultFramebufferInfo->defaultFBO) {
// Same reset as SyncCurrentFBO's default-framebuffer branch: SyncToBackend is the
// only writer of the widened-attachment mask, so a path that skips it has to say so
// explicitly. It matters here because the DSA clears and glBlitFramebuffer briefly
// sync a DIFFERENT framebuffer as DRAW and then restore the application's through
// ForceBindCurrentFBO - which lands right here when that one is the default.
if (target == FramebufferTarget::Draw) {
FramebufferImpl::g_alphaWidenedDrawBufferMask = 0;
FramebufferImpl::g_integerColorDrawBufferMask = 0;
}
FramebufferImpl::BindFramebufferId( FramebufferImpl::BindFramebufferId(
target == FramebufferTarget::Draw ? GL_DRAW_FRAMEBUFFER : GL_READ_FRAMEBUFFER, 0); target == FramebufferTarget::Draw ? GL_DRAW_FRAMEBUFFER : GL_READ_FRAMEBUFFER, 0);
return; return;
@@ -2988,7 +3063,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
#endif #endif
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
FramebufferImpl::SyncCurrentFBO(); FramebufferImpl::SyncCurrentFBO();
RenderStateImpl::SyncRenderState(); // A colour clear is exactly the operation that is allowed to write a widened
// attachment's alpha - it is what puts the 1.0 there that every later draw is masked
// away from. SyncCurrentFBO ran first, so g_alphaWidenedDrawBufferMask already describes
// the framebuffer this clear will land on.
RenderStateImpl::SyncRenderState(/*forColorClear=*/(mask & GL_COLOR_BUFFER_BIT) != 0);
BindCurrentFBO(FramebufferTarget::Draw); BindCurrentFBO(FramebufferTarget::Draw);
@@ -3026,10 +3105,26 @@ namespace MobileGL::MG_Backend::DirectGLES {
const FloatVec4& cc = MG_State::pGLContext->GetRenderStateParameters().ClearColor; const FloatVec4& cc = MG_State::pGLContext->GetRenderStateParameters().ClearColor;
const Bool outOfRange = cc.x() < 0.f || cc.x() > 1.f || cc.y() < 0.f || cc.y() > 1.f || cc.z() < 0.f || const Bool outOfRange = cc.x() < 0.f || cc.x() > 1.f || cc.y() < 0.f || cc.y() > 1.f || cc.z() < 0.f ||
cc.z() > 1.f || cc.w() < 0.f || cc.w() > 1.f; cc.z() > 1.f || cc.w() < 0.f || cc.w() > 1.f;
// A widened attachment's stored alpha has to end up 1.0, and glClear applies ONE
// clear colour to every draw buffer - so a framebuffer that mixes a widened
// attachment with a native one cannot be served by doctoring glClearColor. Take the
// same per-draw-buffer glClearBufferfv route the out-of-range case already uses and
// substitute the alpha only where it belongs. Scissor and the colour write mask apply
// to glClearBufferfv exactly as they do to glClear, so a scissored clear stays
// scissored and an application that masked alpha off still gets its way (the storage
// then keeps the 1.0 an earlier clear left, which is the same answer).
//
// glClearBufferfv on an INTEGER colour buffer is GL_INVALID_OPERATION, so a
// framebuffer with one of those as a draw buffer keeps plain glClear - which ES
// leaves undefined for integer colour buffers anyway, and which an application that
// wants a defined answer must replace with glClearBufferuiv/iv (those DO substitute
// the widened alpha). The out-of-range trigger is left exactly as it was.
const Uint32 widenedDrawBuffers = FramebufferImpl::g_alphaWidenedDrawBufferMask;
const Bool widenedColorClear =
widenedDrawBuffers != 0 && FramebufferImpl::g_integerColorDrawBufferMask == 0;
GLint clearDrawFbo = 0; GLint clearDrawFbo = 0;
g_GLESFuncs.glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &clearDrawFbo); g_GLESFuncs.glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &clearDrawFbo);
if (outOfRange && clearDrawFbo != 0) { if ((outOfRange || widenedColorClear) && clearDrawFbo != 0) {
const GLfloat value[4] = {cc.x(), cc.y(), cc.z(), cc.w()};
GLint maxDrawBuffers = 0; GLint maxDrawBuffers = 0;
GLint clearedCount = 0; GLint clearedCount = 0;
GLint firstDb = -1; GLint firstDb = -1;
@@ -3039,6 +3134,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_GLESFuncs.glGetIntegerv(GL_DRAW_BUFFER0 + static_cast<GLenum>(i), &db); g_GLESFuncs.glGetIntegerv(GL_DRAW_BUFFER0 + static_cast<GLenum>(i), &db);
if (i == 0) firstDb = db; if (i == 0) firstDb = db;
if (db != GL_NONE) { if (db != GL_NONE) {
const Bool widened = i < 32 && (widenedDrawBuffers & (1u << i)) != 0;
const GLfloat value[4] = {cc.x(), cc.y(), cc.z(), widened ? 1.0f : cc.w()};
g_GLESFuncs.glClearBufferfv(GL_COLOR, i, value); g_GLESFuncs.glClearBufferfv(GL_COLOR, i, value);
++clearedCount; ++clearedCount;
} }
@@ -5295,37 +5392,82 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_GLESFuncs.glClearBufferfi(buffer, drawbuffer, depth, stencil); g_GLESFuncs.glClearBufferfi(buffer, drawbuffer, depth, stencil);
} }
namespace {
using FramebufferImpl::SubstituteWidenedClearAlpha;
// A colour attachment the backend widened from three channels to four has to end up
// holding alpha 1.0 - the value GL reports for a channel the application's format does
// not have - so an explicit per-buffer clear of it writes 1.0 rather than whatever the
// application passed (SubstituteWidenedClearAlpha, in Managers.h). Draws can never move
// it again: their alpha write mask is forced off, see SyncRenderState. That pairing is
// what makes GL_DST_ALPHA blending, glReadPixels and glBlitFramebuffer all see the right
// value without any of them being intercepted.
// Whether draw buffer `drawbuffer` of the framebuffer currently bound as DRAW is such an
// attachment. Answered from the mask SyncCurrentFBO just recomputed, so it costs nothing.
Bool IsWidenedBoundDrawBuffer(GLenum buffer, GLint drawbuffer) {
return buffer == GL_COLOR && drawbuffer >= 0 && drawbuffer < 32 &&
(FramebufferImpl::g_alphaWidenedDrawBufferMask & (1u << drawbuffer)) != 0;
}
// The same question for an explicitly named framebuffer (the DSA clears), which is NOT
// the one g_alphaWidenedDrawBufferMask describes at the point these run.
Bool IsWidenedNamedDrawBuffer(const SharedPtr<MG_State::GLState::FramebufferObject>& framebuffer,
GLenum buffer, GLint drawbuffer) {
using FBO = MG_State::GLState::FramebufferObject;
if (buffer != GL_COLOR || !framebuffer || drawbuffer < 0 ||
drawbuffer >= static_cast<GLint>(FBO::MAX_DRAW_BUFFERS)) {
return false;
}
const auto frontendBuf = framebuffer->GetDrawBuffers()[static_cast<SizeT>(drawbuffer)];
if (frontendBuf < FramebufferAttachmentType::Color0 ||
frontendBuf > FramebufferAttachmentType::Color31) {
return false;
}
return FramebufferImpl::IsAlphaWidenedColorAttachment(framebuffer->GetAttachment(frontendBuf));
}
} // namespace
void ClearBufferfv(GLenum buffer, GLint drawbuffer, const GLfloat* value) { void ClearBufferfv(GLenum buffer, GLint drawbuffer, const GLfloat* value) {
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
FramebufferImpl::SyncCurrentFBO(); FramebufferImpl::SyncCurrentFBO();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
BindCurrentFBO(FramebufferTarget::Draw); BindCurrentFBO(FramebufferTarget::Draw);
g_GLESFuncs.glClearBufferfv(buffer, drawbuffer, value); GLfloat widenedValue[4] = {};
g_GLESFuncs.glClearBufferfv(
buffer, drawbuffer,
SubstituteWidenedClearAlpha(value, IsWidenedBoundDrawBuffer(buffer, drawbuffer), 1.0f, widenedValue));
} }
void ClearBufferiv(GLenum buffer, GLint drawbuffer, const GLint* value) { void ClearBufferiv(GLenum buffer, GLint drawbuffer, const GLint* value) {
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
FramebufferImpl::SyncCurrentFBO(); FramebufferImpl::SyncCurrentFBO();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
// SyncCurrentFBO early-outs for the default framebuffer, so without this // SyncCurrentFBO early-outs for the default framebuffer, so without this
// bind a user-FBO -> default-FBO switch would leave the clear landing on // bind a user-FBO -> default-FBO switch would leave the clear landing on
// the stale driver DRAW binding (the fi/fv/uiv siblings all bind too). // the stale driver DRAW binding (the fi/fv/uiv siblings all bind too).
BindCurrentFBO(FramebufferTarget::Draw); BindCurrentFBO(FramebufferTarget::Draw);
g_GLESFuncs.glClearBufferiv(buffer, drawbuffer, value); GLint widenedValue[4] = {};
g_GLESFuncs.glClearBufferiv(
buffer, drawbuffer,
SubstituteWidenedClearAlpha(value, IsWidenedBoundDrawBuffer(buffer, drawbuffer), GLint(1), widenedValue));
} }
void ClearBufferuiv(GLenum buffer, GLint drawbuffer, const GLuint* value) { void ClearBufferuiv(GLenum buffer, GLint drawbuffer, const GLuint* value) {
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
FramebufferImpl::SyncCurrentFBO(); FramebufferImpl::SyncCurrentFBO();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
BindCurrentFBO(FramebufferTarget::Draw); BindCurrentFBO(FramebufferTarget::Draw);
g_GLESFuncs.glClearBufferuiv(buffer, drawbuffer, value); GLuint widenedValue[4] = {};
g_GLESFuncs.glClearBufferuiv(
buffer, drawbuffer,
SubstituteWidenedClearAlpha(value, IsWidenedBoundDrawBuffer(buffer, drawbuffer), GLuint(1), widenedValue));
} }
void ClearNamedFramebufferfv(const SharedPtr<MG_State::GLState::FramebufferObject>& framebuffer, void ClearNamedFramebufferfv(const SharedPtr<MG_State::GLState::FramebufferObject>& framebuffer,
@@ -5334,9 +5476,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
DebugImpl::OpenGLScopeMarker marker(__func__); DebugImpl::OpenGLScopeMarker marker(__func__);
#endif #endif
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true); SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true);
GLfloat widenedValue[4] = {};
value = SubstituteWidenedClearAlpha(value, IsWidenedNamedDrawBuffer(framebuffer, buffer, drawbuffer), 1.0f,
widenedValue);
g_GLESFuncs.glClearBufferfv(buffer, drawbuffer, value); g_GLESFuncs.glClearBufferfv(buffer, drawbuffer, value);
DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) { DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) {
MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str()); MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str());
@@ -5368,9 +5513,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
DebugImpl::OpenGLScopeMarker marker(__func__); DebugImpl::OpenGLScopeMarker marker(__func__);
#endif #endif
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true); SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true);
GLint widenedValue[4] = {};
value = SubstituteWidenedClearAlpha(value, IsWidenedNamedDrawBuffer(framebuffer, buffer, drawbuffer),
GLint(1), widenedValue);
g_GLESFuncs.glClearBufferiv(buffer, drawbuffer, value); g_GLESFuncs.glClearBufferiv(buffer, drawbuffer, value);
DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) { DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) {
MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str()); MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str());
@@ -5385,9 +5533,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
DebugImpl::OpenGLScopeMarker marker(__func__); DebugImpl::OpenGLScopeMarker marker(__func__);
#endif #endif
TextureImpl::SyncNeccessaryTextures(); TextureImpl::SyncNeccessaryTextures();
RenderStateImpl::SyncRenderState(); RenderStateImpl::SyncRenderState(/*forColorClear=*/buffer == GL_COLOR);
SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true); SyncAndBindFramebufferObject(framebuffer, FramebufferTarget::Draw, true);
GLuint widenedValue[4] = {};
value = SubstituteWidenedClearAlpha(value, IsWidenedNamedDrawBuffer(framebuffer, buffer, drawbuffer),
GLuint(1), widenedValue);
g_GLESFuncs.glClearBufferuiv(buffer, drawbuffer, value); g_GLESFuncs.glClearBufferuiv(buffer, drawbuffer, value);
DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) { DebugImpl::ErrorLopper::Loop([file = __FILE__, line = __LINE__](auto err) {
MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str()); MGLOG_D("ES error (%s:%d): %s", file, line, MG_Util::ConvertGLEnumToString(err).c_str());
@@ -5660,16 +5811,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
format == GL_RGBA_INTEGER; format == GL_RGBA_INTEGER;
} }
// Expands a tightly-packed narrow read (1-3 channels per texel) into the 4-channel wide RGBA // The bit pattern of 1.0 in a wide-read component type: what GL reports for a channel the
// layout ConvertWideReadbackRow expects. Missing G/B read zero; missing A reads one, encoded in // attachment's format does not have.
// the source component type. static void FillWideReadOneBits(GLenum componentType, Uint8* oneBits) {
static void ExpandNarrowWideRead(Vector<Uint8>& data, SizeT pixelCount, Int srcChannels, GLenum componentType) {
const SizeT componentSize = GetReadbackComponentSize(componentType);
if (componentSize == 0 || srcChannels <= 0 || srcChannels >= 4) {
return;
}
Uint8 zeroBits[4] = {0, 0, 0, 0};
Uint8 oneBits[4] = {0, 0, 0, 0};
switch (componentType) { switch (componentType) {
case GL_UNSIGNED_BYTE: case GL_UNSIGNED_BYTE:
oneBits[0] = 0xFF; oneBits[0] = 0xFF;
@@ -5706,6 +5850,35 @@ namespace MobileGL::MG_Backend::DirectGLES {
default: default:
break; break;
} }
}
// Overwrites the alpha of a 4-channel wide read with the format's implied 1.0. Used for an
// attachment the backend widened from three channels to keep it colour-renderable: the storage
// has a real alpha channel holding whatever the draw wrote, but the format the application
// asked for has none, and GL reads a missing channel back as one.
static void ForceWideReadAlphaToOne(Vector<Uint8>& data, SizeT pixelCount, GLenum componentType) {
const SizeT componentSize = GetReadbackComponentSize(componentType);
if (componentSize == 0 || data.size() < pixelCount * 4 * componentSize) {
return;
}
Uint8 oneBits[4] = {0, 0, 0, 0};
FillWideReadOneBits(componentType, oneBits);
for (SizeT i = 0; i < pixelCount; ++i) {
Memcpy(data.data() + (i * 4 + 3) * componentSize, oneBits, componentSize);
}
}
// Expands a tightly-packed narrow read (1-3 channels per texel) into the 4-channel wide RGBA
// layout ConvertWideReadbackRow expects. Missing G/B read zero; missing A reads one, encoded in
// the source component type.
static void ExpandNarrowWideRead(Vector<Uint8>& data, SizeT pixelCount, Int srcChannels, GLenum componentType) {
const SizeT componentSize = GetReadbackComponentSize(componentType);
if (componentSize == 0 || srcChannels <= 0 || srcChannels >= 4) {
return;
}
Uint8 zeroBits[4] = {0, 0, 0, 0};
Uint8 oneBits[4] = {0, 0, 0, 0};
FillWideReadOneBits(componentType, oneBits);
Vector<Uint8> expanded(pixelCount * 4 * componentSize); Vector<Uint8> expanded(pixelCount * 4 * componentSize);
for (SizeT i = 0; i < pixelCount; ++i) { for (SizeT i = 0; i < pixelCount; ++i) {
@@ -5750,9 +5923,16 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Reads the current READ framebuffer as wide RGBA(_INTEGER) and repacks the pixels into the client's // Reads the current READ framebuffer as wide RGBA(_INTEGER) and repacks the pixels into the client's
// (format, type) layout. Returns false when the combination is not convertible (the caller keeps its // (format, type) layout. Returns false when the combination is not convertible (the caller keeps its
// "not implemented" skip); returns true when the request was handled, even if it degraded to a logged no-op. // "not implemented" skip); returns true when the request was handled, even if it degraded to a logged no-op.
// `forceOpaqueAlpha`: the source image is a three-channel format the backend widened to four to
// keep it colour-renderable, so its alpha channel holds whatever the draw wrote and has to be
// answered with the 1.0 the application's format implies. Passed in rather than derived here:
// glReadPixels reads the bound READ framebuffer, but glGetTexImage reads a texture through a
// scratch framebuffer, so the frontend's READ binding describes a different image entirely -
// consulting it there would both miss real widenings and corrupt readbacks of ordinary
// textures taken while some unrelated widened attachment happened to be bound.
static Bool ReadPixelsViaFormatConversion(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, static Bool ReadPixelsViaFormatConversion(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format,
GLenum type, void* pixels, Bool honorPackImageParams = false, GLenum type, void* pixels, Bool honorPackImageParams,
Bool applyFixedPointReadClamp = true) { Bool applyFixedPointReadClamp, Bool forceOpaqueAlpha) {
ReadbackChannelMapping mapping{}; ReadbackChannelMapping mapping{};
if (!GetReadbackChannelMapping(format, mapping)) { if (!GetReadbackChannelMapping(format, mapping)) {
return false; return false;
@@ -5879,6 +6059,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
ExpandNarrowWideRead(wide, static_cast<SizeT>(width) * static_cast<SizeT>(height), readChannels, wideType); ExpandNarrowWideRead(wide, static_cast<SizeT>(width) * static_cast<SizeT>(height), readChannels, wideType);
} }
// Undo the three-channel widening (see the parameter's comment). Deliberately not gated on
// applyFixedPointReadClamp: that flag implements GL_CLAMP_READ_COLOR, which glGetTexImage
// is exempt from, whereas "a format without alpha reads as 1.0" is the format's own
// semantics and applies to every read.
if (forceOpaqueAlpha) {
ForceWideReadAlphaToOne(wide, static_cast<SizeT>(width) * static_cast<SizeT>(height), wideType);
}
// GL clamps a read from a fixed-point colour buffer to [0,1] (GL_CLAMP_READ_COLOR // GL clamps a read from a fixed-point colour buffer to [0,1] (GL_CLAMP_READ_COLOR
// defaults to GL_FIXED_ONLY). Formats the backend substitutes with a floating-point // defaults to GL_FIXED_ONLY). Formats the backend substitutes with a floating-point
// one keep the out-of-range value the app stored, so apply the clamp here - a // one keep the out-of-range value the app stored, so apply the clamp here - a
@@ -6037,11 +6225,19 @@ namespace MobileGL::MG_Backend::DirectGLES {
// the driver accepts for the current attachment. GL_PACK_SWAP_BYTES has no ES equivalent, so // the driver accepts for the current attachment. GL_PACK_SWAP_BYTES has no ES equivalent, so
// it always takes the conversion path (which swaps on the CPU). // it always takes the conversion path (which swaps on the CPU).
const Bool packSwapBytes = MG_State::pGLContext->GetPixelStoreParameters(false).SwapBytes; const Bool packSwapBytes = MG_State::pGLContext->GetPixelStoreParameters(false).SwapBytes;
const Bool nativeFastPair = !packSwapBytes && // The read buffer is what glReadPixels reads, so the frontend's READ binding is exactly
// the right thing to ask here.
const Bool forceOpaqueAlpha = FramebufferImpl::IsAlphaWidenedFallbackReadAttachment();
// An attachment widened from three channels to stay colour-renderable also has to leave
// the fast pair: only the conversion path knows to answer its alpha with the 1.0 the
// application's format implies instead of whatever the draw wrote into the added channel.
const Bool nativeFastPair = !packSwapBytes && !forceOpaqueAlpha &&
((format == GL_RGBA && type == GL_UNSIGNED_BYTE) || ((format == GL_RGBA && type == GL_UNSIGNED_BYTE) ||
(format == GL_RGBA_INTEGER && (type == GL_UNSIGNED_INT || type == GL_INT))); (format == GL_RGBA_INTEGER && (type == GL_UNSIGNED_INT || type == GL_INT)));
if (convertible && !nativeFastPair) { if (convertible && !nativeFastPair) {
if (ReadPixelsViaFormatConversion(x, y, width, height, format, type, pixels)) { if (ReadPixelsViaFormatConversion(x, y, width, height, format, type, pixels,
/*honorPackImageParams=*/false, /*applyFixedPointReadClamp=*/true,
forceOpaqueAlpha)) {
MGLOG_D("ReadPixels: finished via client-format conversion"); MGLOG_D("ReadPixels: finished via client-format conversion");
return; return;
} }
@@ -6100,7 +6296,9 @@ namespace MobileGL::MG_Backend::DirectGLES {
MGLOG_D("ReadPixels: native read of %s/%s failed (%s), retrying via client-format conversion", MGLOG_D("ReadPixels: native read of %s/%s failed (%s), retrying via client-format conversion",
MG_Util::ConvertGLEnumToString(format).c_str(), MG_Util::ConvertGLEnumToString(type).c_str(), MG_Util::ConvertGLEnumToString(format).c_str(), MG_Util::ConvertGLEnumToString(type).c_str(),
MG_Util::ConvertGLEnumToString(nativeReadError).c_str()); MG_Util::ConvertGLEnumToString(nativeReadError).c_str());
if (ReadPixelsViaFormatConversion(x, y, width, height, format, type, pixels)) { if (ReadPixelsViaFormatConversion(x, y, width, height, format, type, pixels,
/*honorPackImageParams=*/false, /*applyFixedPointReadClamp=*/true,
forceOpaqueAlpha)) {
MGLOG_D("ReadPixels: finished via client-format conversion after native failure"); MGLOG_D("ReadPixels: finished via client-format conversion after native failure");
return; return;
} }
@@ -6281,6 +6479,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
// for normalized attachments), while the conversion path reads a wide format that is always // for normalized attachments), while the conversion path reads a wide format that is always
// accepted and repacks on the CPU. // accepted and repacks on the CPU.
if (convertible) { if (convertible) {
// The image being read is this texture, not whatever the application left bound to
// GL_READ_FRAMEBUFFER, so the widening question has to be asked of the texture.
const Bool forceOpaqueAlpha =
TextureImpl::BackendTextureFormatAddsAlpha(textureObject->GetFormat(), textureObject->GetTarget());
// GL_PACK_IMAGE_HEIGHT/GL_PACK_SKIP_IMAGES only apply to 3D/array image // GL_PACK_IMAGE_HEIGHT/GL_PACK_SKIP_IMAGES only apply to 3D/array image
// readbacks (cube-map arrays address as arrays); 2D targets must ignore // readbacks (cube-map arrays address as arrays); 2D targets must ignore
// them (GL 3.3 section 6.1.4). // them (GL 3.3 section 6.1.4).
@@ -6326,7 +6528,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
void* sliceDst = static_cast<Uint8*>(pixels) + sliceOffset; void* sliceDst = static_cast<Uint8*>(pixels) + sliceOffset;
if (!ReadPixelsViaFormatConversion(0, 0, size.x(), size.y(), format, type, sliceDst, if (!ReadPixelsViaFormatConversion(0, 0, size.x(), size.y(), format, type, sliceDst,
/*honorPackImageParams=*/false, /*honorPackImageParams=*/false,
/*applyFixedPointReadClamp=*/false)) { /*applyFixedPointReadClamp=*/false, forceOpaqueAlpha)) {
allSlicesRead = false; allSlicesRead = false;
break; break;
} }
@@ -6348,7 +6550,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
if (tempFBOComplete && ReadPixelsViaFormatConversion(0, 0, size.x(), size.y(), format, type, pixels, if (tempFBOComplete && ReadPixelsViaFormatConversion(0, 0, size.x(), size.y(), format, type, pixels,
applyPackImageParams, applyPackImageParams,
/*applyFixedPointReadClamp=*/false)) { /*applyFixedPointReadClamp=*/false,
forceOpaqueAlpha)) {
MGLOG_D("GetTexImage: finished via client-format conversion"); MGLOG_D("GetTexImage: finished via client-format conversion");
return; return;
} }
@@ -188,6 +188,26 @@ namespace MobileGL::MG_Backend::DirectGLES {
void OnBackendContextDestroyed(); void OnBackendContextDestroyed();
} // namespace XfbImpl } // namespace XfbImpl
namespace RenderStateImpl {
// Pushes the frontend's render-state block to the ES driver, diffed against what was
// last pushed.
//
// `forColorClear` names the CALLER, and the only thing it changes is the colour write
// mask handed to the driver. A draw into a colour attachment the backend widened from
// three channels to four gets that buffer's alpha channel masked OFF, so nothing can
// move the stored alpha away from the 1.0 the application's three-channel format
// implies (see FramebufferImpl::g_alphaWidenedDrawBufferMask). A CLEAR is how that 1.0
// gets there in the first place, so it must be allowed to write alpha - hence the flag
// rather than an unconditional doctoring. It is part of the sync memo, so a clear
// followed by a draw re-pushes the mask instead of early-outing on an unchanged
// frontend version.
//
// The application's own colour mask is never modified: glGet(GL_COLOR_WRITEMASK)
// answers from the frontend state, which this function only reads.
void SyncRenderState(Bool forColorClear = false);
void InvalidateSyncedRenderState();
} // namespace RenderStateImpl
extern MG_External::EGLFunctionsTable g_EGLFuncs; extern MG_External::EGLFunctionsTable g_EGLFuncs;
extern MG_External::GLESFunctionsTable g_GLESFuncs; extern MG_External::GLESFunctionsTable g_GLESFuncs;
extern MG_External::GLESCapabilities g_GLESCapabilities; extern MG_External::GLESCapabilities g_GLESCapabilities;
+319 -13
View File
@@ -1869,6 +1869,154 @@ namespace MobileGL::MG_Backend::DirectGLES {
} }
} }
// Components per texel the frontend format's client data carries. Only the three-channel
// formats that can be widened to a four-channel render target need an answer (see
// PrepareChannelWidenedUpload); everything else keeps its own layout and reports 0.
Uint GetWidenableClientComponentCount(TextureInternalFormat format) {
switch (format) {
case TextureInternalFormat::RGB8Snorm:
case TextureInternalFormat::RGB16Snorm:
case TextureInternalFormat::RGB16:
case TextureInternalFormat::RGB10: // stored as RGB16 (UNorm16 shadow)
case TextureInternalFormat::RGB12: // stored as RGB16 (UNorm16 shadow)
case TextureInternalFormat::RGB16F:
case TextureInternalFormat::RGB32F:
case TextureInternalFormat::SRGB8:
case TextureInternalFormat::RGB8I:
case TextureInternalFormat::RGB8UI:
case TextureInternalFormat::RGB16I:
case TextureInternalFormat::RGB16UI:
case TextureInternalFormat::RGB32I:
case TextureInternalFormat::RGB32UI:
return 3;
default:
return 0;
}
}
// True when the widened format's client data is integer rather than normalized. The two
// classes share every narrow component type - GL_RGB8I and GL_RGB8_SNORM are both uploaded
// as GL_BYTE - but their "1.0" differs: an integer channel's one is the integer 1, a
// normalized channel's is the saturated field. The type alone cannot tell them apart, so
// the source format has to.
Bool IsIntegerWidenableFormat(TextureInternalFormat format) {
switch (format) {
case TextureInternalFormat::RGB8I:
case TextureInternalFormat::RGB8UI:
case TextureInternalFormat::RGB16I:
case TextureInternalFormat::RGB16UI:
case TextureInternalFormat::RGB32I:
case TextureInternalFormat::RGB32UI:
return true;
default:
return false;
}
}
// The bit pattern of 1.0 in an upload component type: what a format without alpha reads
// back as, and therefore what the synthetic fourth channel of a widened render target has
// to hold. Integer components carry the integer one, not a saturated field - and since
// GL_BYTE/GL_SHORT/GL_UNSIGNED_BYTE/GL_UNSIGNED_SHORT serve both classes, `integerData`
// is what decides, not the type.
static Bool GetUploadComponentOneBits(GLenum uploadType, Bool integerData, Uint8* outOneBits,
SizeT* outComponentSize) {
switch (uploadType) {
case GL_BYTE: {
const Int8 one = integerData ? Int8(1) : Int8(0x7F);
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_UNSIGNED_BYTE: {
const Uint8 one = integerData ? Uint8(1) : Uint8(0xFF);
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_SHORT: {
const Int16 one = integerData ? Int16(1) : Int16(0x7FFF);
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_UNSIGNED_SHORT: {
const Uint16 one = integerData ? Uint16(1) : Uint16(0xFFFF);
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_HALF_FLOAT: {
const Uint16 one = 0x3C00; // half 1.0
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_FLOAT: {
const Float one = 1.0f;
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_INT: {
const Int32 one = 1;
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
case GL_UNSIGNED_INT: {
const Uint32 one = 1;
Memcpy(outOneBits, &one, sizeof(one));
*outComponentSize = sizeof(one);
return true;
}
default:
return false;
}
}
// A three-channel format widened to four to keep a colour attachment renderable (see
// NormalizePixelFormat) is described to the driver as a four-component transfer, so the
// three-component client data has to be repacked with an alpha of 1.0 - otherwise the
// driver walks three texels' worth of data per four-texel row and the image shears.
// `componentCount` is the SOURCE component count and `byteSize` the source's size, so this
// runs after any type conversion (which keeps the component count) has already happened.
const void* PrepareChannelWidenedUpload(Uint componentCount, const IntVec3& texelSize,
const void* data, SizeT byteSize, GLenum uploadType,
Vector<Uint8>& widenedData, Bool integerData) {
Uint8 oneBits[8] = {};
SizeT componentSize = 0;
if (componentCount != 3 || data == nullptr || byteSize == 0 ||
!GetUploadComponentOneBits(uploadType, integerData, oneBits, &componentSize)) {
return data;
}
const SizeT srcTexelBytes = componentSize * componentCount;
// Sized from the level, never from the source: the driver reads a full
// width*height*depth*4 components for the transfer it was handed, so a source that
// somehow holds fewer texels must still leave a full destination behind (its tail
// reads as transparent black with the format's implied opaque alpha) rather than a
// short buffer the driver would run off the end of.
const SizeT texelCount = static_cast<SizeT>(std::max(texelSize.x(), 0)) *
static_cast<SizeT>(std::max(texelSize.y(), 0)) *
static_cast<SizeT>(std::max(texelSize.z(), 1));
if (texelCount == 0) {
return data;
}
const SizeT copyTexelCount = std::min(texelCount, byteSize / srcTexelBytes);
widenedData.assign(texelCount * componentSize * 4, 0);
const auto* src = static_cast<const Uint8*>(data);
Uint8* dst = widenedData.data();
for (SizeT i = 0; i < texelCount; ++i, dst += componentSize * 4) {
if (i < copyTexelCount) {
Memcpy(dst, src, srcTexelBytes);
src += srcTexelBytes;
}
Memcpy(dst + srcTexelBytes, oneBits, componentSize);
}
return widenedData.data();
}
static const void* PrepareNormFloatFallbackUpload(TextureInternalFormat format, static const void* PrepareNormFloatFallbackUpload(TextureInternalFormat format,
const IntVec3& texelSize, const IntVec3& texelSize,
const void* data, const void* data,
@@ -1914,6 +2062,39 @@ namespace MobileGL::MG_Backend::DirectGLES {
return convertedData.data(); return convertedData.data();
} }
// The two shadow -> upload conversions a fallback storage format can need, in order:
// the component type first (SNORM/UNORM shadows into the float the fallback stores), then
// the component count (three-channel client data into a four-channel widened render
// target). They compose: GL_RGB8_SNORM on a driver with no renderable three-channel
// format becomes GL_RGBA16F, so its Int8x3 shadow is converted to Float x3 and then
// repacked as Float x4 with alpha 1.0.
//
// Both scratch buffers belong to the caller so they outlive the returned pointer; the
// return value is `data` itself whenever neither conversion applies, which is what the
// sub-rect upload fast path tests for.
static const void* PrepareFallbackUpload(TextureInternalFormat format, TextureTarget target,
const IntVec3& texelSize, const void* data, SizeT byteSize,
GLenum uploadType, Vector<Float>& convertedData,
Vector<Uint8>& widenedData) {
const void* uploadData =
PrepareNormFloatFallbackUpload(format, texelSize, data, byteSize, uploadType, convertedData);
// The component-count switch first: it rules out every format that cannot be widened
// (which is nearly all of them, including GL_RGBA8) without touching the capability
// cache, so an ordinary atlas upload does not pay for a per-level cache lookup.
const Uint componentCount = GetWidenableClientComponentCount(format);
if (componentCount == 0 || !TextureImpl::BackendTextureFormatAddsAlpha(format, target)) {
return uploadData;
}
// The type conversion above rewrites the level into `convertedData` at four bytes per
// component while keeping the component count, so the widening's source size is that
// buffer's, not the shadow's.
const SizeT uploadByteSize = (!convertedData.empty() && uploadData == convertedData.data())
? convertedData.size() * sizeof(Float)
: byteSize;
return PrepareChannelWidenedUpload(componentCount, texelSize, uploadData, uploadByteSize, uploadType,
widenedData, IsIntegerWidenableFormat(format));
}
// RGB565/RGB5_A1 shadow data is stored as 8-bit unorm; uploading it as GL_UNSIGNED_BYTE // RGB565/RGB5_A1 shadow data is stored as 8-bit unorm; uploading it as GL_UNSIGNED_BYTE
// leaves the 8-bit -> 5/6-bit requantization to the driver, whose rounding direction is // leaves the 8-bit -> 5/6-bit requantization to the driver, whose rounding direction is
// implementation-defined: Adreno rounds to nearest (lossless round trip) but Mali floors, // implementation-defined: Adreno rounds to nearest (lossless round trip) but Mali floors,
@@ -2121,9 +2302,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
? textureMipmapObject->MapMipmapData(uploadTarget, level) ? textureMipmapObject->MapMipmapData(uploadTarget, level)
: nullptr; : nullptr;
Vector<Float> convertedUploadData; Vector<Float> convertedUploadData;
const void* uploadData = PrepareNormFloatFallbackUpload( Vector<Uint8> widenedUploadData;
textureMipmapObject->GetFormat(), levelTexelSize, pData, levelByteSize, glType, const void* uploadData = PrepareFallbackUpload(
convertedUploadData); textureMipmapObject->GetFormat(), targetInternal, levelTexelSize, pData,
levelByteSize, glType, convertedUploadData, widenedUploadData);
Vector<Uint8> packedUploadData; Vector<Uint8> packedUploadData;
uploadData = PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize, uploadData = PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize,
uploadData, levelByteSize, &glType, packedUploadData); uploadData, levelByteSize, &glType, packedUploadData);
@@ -2253,9 +2435,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
auto glUploadTarget = ConvertTextureUploadTargetToBackendGLEnum(uploadTarget); auto glUploadTarget = ConvertTextureUploadTargetToBackendGLEnum(uploadTarget);
auto* pData = textureMipmapObject->MapMipmapData(uploadTarget, level); auto* pData = textureMipmapObject->MapMipmapData(uploadTarget, level);
Vector<Float> convertedUploadData; Vector<Float> convertedUploadData;
const void* uploadData = PrepareNormFloatFallbackUpload( Vector<Uint8> widenedUploadData;
textureMipmapObject->GetFormat(), levelTexelSize, pData, levelByteSize, glType, const void* uploadData = PrepareFallbackUpload(
convertedUploadData); textureMipmapObject->GetFormat(), targetInternal, levelTexelSize, pData,
levelByteSize, glType, convertedUploadData, widenedUploadData);
Vector<Uint8> packedUploadData; Vector<Uint8> packedUploadData;
uploadData = uploadData =
PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize, PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize,
@@ -2312,9 +2495,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
? textureMipmapObject->MapMipmapData(uploadTarget, level) ? textureMipmapObject->MapMipmapData(uploadTarget, level)
: nullptr; : nullptr;
Vector<Float> convertedUploadData; Vector<Float> convertedUploadData;
const void* uploadData = PrepareNormFloatFallbackUpload( Vector<Uint8> widenedUploadData;
textureMipmapObject->GetFormat(), levelTexelSize, pData, levelByteSize, glType, const void* uploadData = PrepareFallbackUpload(
convertedUploadData); textureMipmapObject->GetFormat(), targetInternal, levelTexelSize, pData,
levelByteSize, glType, convertedUploadData, widenedUploadData);
Vector<Uint8> packedUploadData; Vector<Uint8> packedUploadData;
uploadData = uploadData =
PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize, PreparePackedNormUpload(textureMipmapObject->GetFormat(), levelTexelSize,
@@ -2422,9 +2606,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
auto texelSize = textureMipmapObject->GetMipmapTexelSize(uploadTarget, level); auto texelSize = textureMipmapObject->GetMipmapTexelSize(uploadTarget, level);
const void* mipData = textureMipmapObject->MapMipmapData(uploadTarget, level); const void* mipData = textureMipmapObject->MapMipmapData(uploadTarget, level);
Vector<Float> convertedUploadData; Vector<Float> convertedUploadData;
const void* uploadData = PrepareNormFloatFallbackUpload( Vector<Uint8> widenedUploadData;
textureMipmapObject->GetFormat(), texelSize, mipData, byteSize, glType, const void* uploadData = PrepareFallbackUpload(
convertedUploadData); textureMipmapObject->GetFormat(), targetInternal, texelSize, mipData, byteSize,
glType, convertedUploadData, widenedUploadData);
Vector<Uint8> packedUploadData; Vector<Uint8> packedUploadData;
uploadData = PreparePackedNormUpload(textureMipmapObject->GetFormat(), texelSize, uploadData = PreparePackedNormUpload(textureMipmapObject->GetFormat(), texelSize,
uploadData, byteSize, &glType, packedUploadData); uploadData, byteSize, &glType, packedUploadData);
@@ -2827,7 +3012,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
MGLOG_D("%s(%s:%d) ES error %s", func, file, line, MG_Util::ConvertGLEnumToString(err).c_str()); MGLOG_D("%s(%s:%d) ES error %s", func, file, line, MG_Util::ConvertGLEnumToString(err).c_str());
}); });
// A three-channel format widened to four for a multisample target (see // A three-channel format widened to four to keep the image colour-renderable (see
// NormalizePixelFormat) gains an alpha channel the frontend format does not have, and // NormalizePixelFormat) gains an alpha channel the frontend format does not have, and
// whatever the draw that filled it wrote there is not what GL would report: a format // whatever the draw that filled it wrote there is not what GL would report: a format
// without alpha reads back as 1.0. Answer the ALPHA swizzle source with ONE so the // without alpha reads back as 1.0. Answer the ALPHA swizzle source with ONE so the
@@ -3139,6 +3324,113 @@ namespace MobileGL::MG_Backend::DirectGLES {
return false; return false;
} }
// The colour attachment glReadPixels/glGetTexImage would read from, or nullptr when the
// read buffer names no colour attachment at all.
static const MG_State::GLState::FramebufferAttachmentObject* GetReadColorAttachment() {
const auto& readFBO =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Read).GetBoundObject();
if (!readFBO) {
return nullptr;
}
const auto readBuffer = readFBO->GetReadBuffer();
if (readBuffer < FramebufferAttachmentType::Color0 || readBuffer > FramebufferAttachmentType::Color31) {
return nullptr;
}
return &readFBO->GetAttachment(readBuffer);
}
Bool IsAlphaWidenedColorAttachment(
const MG_State::GLState::FramebufferAttachmentObject& attachmentObject) {
if (attachmentObject.IsTexture()) {
const auto& textureObject = attachmentObject.GetTexture();
return textureObject && TextureImpl::BackendTextureFormatAddsAlpha(textureObject->GetFormat(),
textureObject->GetTarget());
}
if (attachmentObject.IsRenderbuffer()) {
const auto& renderbufferObject = attachmentObject.GetRenderbuffer();
return renderbufferObject &&
TextureImpl::BackendRenderbufferFormatAddsAlpha(renderbufferObject->GetInternalFormat());
}
return false;
}
Uint32 g_alphaWidenedDrawBufferMask = 0;
Uint32 g_integerColorDrawBufferMask = 0;
static Bool IsIntegerColorFormat(TextureInternalFormat format) {
switch (format) {
case TextureInternalFormat::R8I:
case TextureInternalFormat::R8UI:
case TextureInternalFormat::R16I:
case TextureInternalFormat::R16UI:
case TextureInternalFormat::R32I:
case TextureInternalFormat::R32UI:
case TextureInternalFormat::RG8I:
case TextureInternalFormat::RG8UI:
case TextureInternalFormat::RG16I:
case TextureInternalFormat::RG16UI:
case TextureInternalFormat::RG32I:
case TextureInternalFormat::RG32UI:
case TextureInternalFormat::RGB8I:
case TextureInternalFormat::RGB8UI:
case TextureInternalFormat::RGB16I:
case TextureInternalFormat::RGB16UI:
case TextureInternalFormat::RGB32I:
case TextureInternalFormat::RGB32UI:
case TextureInternalFormat::RGBA8I:
case TextureInternalFormat::RGBA8UI:
case TextureInternalFormat::RGBA16I:
case TextureInternalFormat::RGBA16UI:
case TextureInternalFormat::RGBA32I:
case TextureInternalFormat::RGBA32UI:
case TextureInternalFormat::RGB10A2UI:
return true;
default:
return false;
}
}
static Bool IsIntegerColorAttachment(
const MG_State::GLState::FramebufferAttachmentObject& attachmentObject) {
if (attachmentObject.IsTexture()) {
const auto& textureObject = attachmentObject.GetTexture();
return textureObject && IsIntegerColorFormat(textureObject->GetFormat());
}
if (attachmentObject.IsRenderbuffer()) {
const auto& renderbufferObject = attachmentObject.GetRenderbuffer();
return renderbufferObject && IsIntegerColorFormat(renderbufferObject->GetInternalFormat());
}
return false;
}
Uint32 ComputeAlphaWidenedDrawBufferMask(const MG_State::GLState::FramebufferObject& fbo) {
using FBO = MG_State::GLState::FramebufferObject;
const auto& drawBuffers = fbo.GetDrawBuffers();
Uint32 mask = 0;
for (Uint i = 0; i < FBO::MAX_DRAW_BUFFERS && i < 32; ++i) {
const auto frontendBuf = drawBuffers[i];
if (frontendBuf < FramebufferAttachmentType::Color0 ||
frontendBuf > FramebufferAttachmentType::Color31) {
continue;
}
if (IsAlphaWidenedColorAttachment(fbo.GetAttachment(frontendBuf))) {
mask |= (1u << i);
}
}
return mask;
}
// The read attachment's storage carries an alpha channel its frontend format does not
// (the three-channel colour-renderable widening). GL answers such a read with 1.0, but
// the storage holds whatever the draw wrote there, so the readback has to overwrite it.
Bool IsAlphaWidenedFallbackReadAttachment() {
const auto* attachmentObject = GetReadColorAttachment();
if (attachmentObject == nullptr) {
return false;
}
return IsAlphaWidenedColorAttachment(*attachmentObject);
}
Bool IsFixedPointFallbackReadAttachment() { Bool IsFixedPointFallbackReadAttachment() {
const auto& readFBO = const auto& readFBO =
MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Read).GetBoundObject(); MG_State::pGLContext->GetFramebufferBindingSlot(FramebufferTarget::Read).GetBoundObject();
@@ -3344,6 +3636,8 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (asTarget == FramebufferTarget::Draw) { if (asTarget == FramebufferTarget::Draw) {
Uint32 snormClampOutputMask = 0; Uint32 snormClampOutputMask = 0;
Uint32 unormClampOutputMask = 0; Uint32 unormClampOutputMask = 0;
Uint32 alphaWidenedMask = 0;
Uint32 integerColorMask = 0;
for (Uint i = 0; i < FramebufferObject::MAX_DRAW_BUFFERS && i < 32; ++i) { for (Uint i = 0; i < FramebufferObject::MAX_DRAW_BUFFERS && i < 32; ++i) {
const auto frontendBuf = stateDrawBuffers[i]; const auto frontendBuf = stateDrawBuffers[i];
if (frontendBuf < FramebufferAttachmentType::Color0 || if (frontendBuf < FramebufferAttachmentType::Color0 ||
@@ -3356,9 +3650,21 @@ namespace MobileGL::MG_Backend::DirectGLES {
} else if (IsUnormFallbackAttachment(attachmentObject)) { } else if (IsUnormFallbackAttachment(attachmentObject)) {
unormClampOutputMask |= (1u << i); unormClampOutputMask |= (1u << i);
} }
// Independent of the two above: a widened attachment can be SNORM
// (GL_RGB8_SNORM -> GL_RGBA16F, which also clamps) or not (GL_SRGB8 ->
// GL_SRGB8_ALPHA8, which does not), so it gets its own bit rather than an
// `else if` branch of theirs.
if (IsAlphaWidenedColorAttachment(attachmentObject)) {
alphaWidenedMask |= (1u << i);
}
if (IsIntegerColorAttachment(attachmentObject)) {
integerColorMask |= (1u << i);
}
} }
PrgramImpl::g_snormFallbackClampOutputMask = snormClampOutputMask; PrgramImpl::g_snormFallbackClampOutputMask = snormClampOutputMask;
PrgramImpl::g_unormFallbackClampOutputMask = unormClampOutputMask; PrgramImpl::g_unormFallbackClampOutputMask = unormClampOutputMask;
g_alphaWidenedDrawBufferMask = alphaWidenedMask;
g_integerColorDrawBufferMask = integerColorMask;
} }
// 2. Remap read buffer. glReadBuffer writes the READ-bound FBO's state, so // 2. Remap read buffer. glReadBuffer writes the READ-bound FBO's state, so
+84
View File
@@ -515,6 +515,25 @@ namespace MobileGL::MG_Backend::DirectGLES {
return target == TextureTarget::Texture3D || target == TextureTarget::TextureCubeMap; return target == TextureTarget::Texture3D || target == TextureTarget::TextureCubeMap;
} }
// Components per texel the frontend format's client data carries, for the three-channel
// formats that can be widened to a four-channel colour-renderable target; 0 for everything
// else. See PrepareChannelWidenedUpload.
Uint GetWidenableClientComponentCount(TextureInternalFormat format);
// True when a widenable format's components are integer rather than normalized, which is
// what decides the synthetic alpha's value: GL_RGB8I and GL_RGB8_SNORM are both uploaded
// as GL_BYTE, but their 1.0 is 1 and 0x7F respectively.
Bool IsIntegerWidenableFormat(TextureInternalFormat format);
// Repacks three-component client data as four components with an alpha of 1.0 in
// `uploadType`, for a format the backend widened to keep a colour attachment renderable.
// Returns `data` untouched when no widening applies. Pure CPU and context-free so a unit
// test can exercise the exact packing the driver is handed; `widenedData` is the caller's
// scratch buffer and has to outlive the returned pointer.
const void* PrepareChannelWidenedUpload(Uint componentCount, const IntVec3& texelSize, const void* data,
SizeT byteSize, GLenum uploadType, Vector<Uint8>& widenedData,
Bool integerData = false);
struct StateTextureBasicInfo { // Used for tracking texture state changes struct StateTextureBasicInfo { // Used for tracking texture state changes
TextureInternalFormat internalFormat = TextureInternalFormat::Unknown; TextureInternalFormat internalFormat = TextureInternalFormat::Unknown;
SizeT width = 0; SizeT width = 0;
@@ -630,6 +649,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
SharedPtr<BackendTextureObject>& SyncTextureObjectToBackend( SharedPtr<BackendTextureObject>& SyncTextureObjectToBackend(
const SharedPtr<MG_State::GLState::ITextureObject>& textureObject, const SharedPtr<MG_State::GLState::ITextureObject>& textureObject,
Bool imageBindableStorageRequired = false); Bool imageBindableStorageRequired = false);
// Brings every texture the next draw reads - the touched units' bindings and the draw
// FBO's texture attachments - onto the backend, through the two borrowed-pair memos
// documented at their definitions. Declared here so tests can drive those memos directly.
void SyncNeccessaryTextures();
extern Array<Array<BackendTextureObject*, (SizeT)TextureTarget::TextureTargetCount>, extern Array<Array<BackendTextureObject*, (SizeT)TextureTarget::TextureTargetCount>,
MG_State::GLState::TextureState::MAX_TEXTURE_IMAGE_UNITS> MG_State::GLState::TextureState::MAX_TEXTURE_IMAGE_UNITS>
g_boundTexturesCache; g_boundTexturesCache;
@@ -710,6 +733,67 @@ namespace MobileGL::MG_Backend::DirectGLES {
// has to apply the clamp itself. // has to apply the clamp itself.
Bool IsFixedPointFallbackReadAttachment(); Bool IsFixedPointFallbackReadAttachment();
// True when the read buffer names a three-channel attachment the backend actually stores
// in a four-channel format (the colour-renderable widening). A format without alpha reads
// back as 1.0, so the readback path has to overwrite the alpha the draw left behind -
// unconditionally, since this is the format's own semantics rather than the
// GL_CLAMP_READ_COLOR rule the clamp above implements.
Bool IsAlphaWidenedFallbackReadAttachment();
// True when this attachment's storage carries an alpha channel its frontend format does
// not (the three-channel colour-renderable widening).
Bool IsAlphaWidenedColorAttachment(const MG_State::GLState::FramebufferAttachmentObject& attachmentObject);
// Bit i set = DRAW BUFFER i of `fbo` resolves to a colour attachment the backend widened
// from three channels to four. Indexed by draw-buffer slot, not by attachment point,
// because that is what glColorMaski / glClearBufferfv address.
Uint32 ComputeAlphaWidenedDrawBufferMask(const MG_State::GLState::FramebufferObject& fbo);
// The same mask for whatever is currently bound to GL_DRAW_FRAMEBUFFER, recomputed by
// SyncCurrentFBO (BackendFramebufferObject::SyncToBackend for the DRAW target, and reset
// to 0 on the default framebuffer). Read by the draw/clear state sync, so it is only
// trustworthy after SyncCurrentFBO has run in the same entry point.
//
// WHY IT EXISTS (the dst-alpha discipline). A widened attachment has a real alpha channel
// the application's format does not, and GL says a missing channel reads as 1.0. Readback
// can paper over that (ForceWideReadAlphaToOne), but GL_DST_ALPHA /
// GL_ONE_MINUS_DST_ALPHA blending and glBlitFramebuffer read the STORED alpha inside the
// driver where no interception is possible. So the stored alpha is kept at 1.0 instead:
// a clear touching a widened buffer writes alpha 1.0, and every draw into it has its
// alpha write mask forced off, so nothing can ever move it again. The application's own
// colour mask is untouched - glGet(GL_COLOR_WRITEMASK) still reports what it set.
extern Uint32 g_alphaWidenedDrawBufferMask;
// Bit i set = DRAW BUFFER i of the framebuffer bound as DRAW resolves to a colour
// attachment with an INTEGER format. Recomputed beside the mask above and for its sake:
// glClearBufferfv on an integer colour buffer is GL_INVALID_OPERATION, so the
// per-draw-buffer clear route the widening needs has to stand down when one is present.
// (glClear on an integer colour buffer is left undefined by ES in the first place, and
// an application that wants a defined answer has to call glClearBufferuiv/iv - which does
// carry the widened alpha substitution.)
extern Uint32 g_integerColorDrawBufferMask;
// The colour a clear has to hand the driver for one draw buffer: the application's value,
// except that a widened attachment's alpha is replaced by the 1.0 its three-channel
// format implies. `one` is 1.0 encoded in the clear call's own component type - the
// integer clears carry the integer 1, the float clear carries 1.0f.
//
// Returns `value` itself when nothing is substituted, so the ordinary path allocates and
// copies nothing; `scratch` is the caller's buffer and has to outlive the returned
// pointer. Free of GL state on purpose, so the substitution can be unit-tested exactly as
// the driver sees it.
template <typename T>
const T* SubstituteWidenedClearAlpha(const T* value, Bool widened, T one, T (&scratch)[4]) {
if (!widened || value == nullptr) {
return value;
}
scratch[0] = value[0];
scratch[1] = value[1];
scratch[2] = value[2];
scratch[3] = one;
return scratch;
}
// What SyncCurrentFBO last pushed for each target, as a (binding, object, revision) // What SyncCurrentFBO last pushed for each target, as a (binding, object, revision)
// triple; it re-syncs unless all three still match. Stamped by SyncCurrentFBO and // triple; it re-syncs unless all three still match. Stamped by SyncCurrentFBO and
// ForceBindCurrentFBO, cleared by InvalidateFramebufferBindingCache. The three are // ForceBindCurrentFBO, cleared by InvalidateFramebufferBindingCache. The three are
+69 -36
View File
@@ -61,29 +61,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
requestedInternalFormat, GetDriverPixelFormatNormalizeOptions() | extraOptions); requestedInternalFormat, GetDriverPixelFormatNormalizeOptions() | extraOptions);
} }
// Multisample textures can only ever be rendered into, never uploaded to, so a fallback
// format for them has to stay colour-renderable - a three-channel float fallback is a legal
// ES texture format but not a legal multisample storage format. Widening to four channels
// is safe here precisely because there is no transfer path that would have to expand
// three-channel client data, and the alpha the draw writes for a three-channel source is
// already the 1.0 the frontend format implies.
Bool TargetRequiresRenderableFormat(SizeT targetIndex) {
return targetIndex == static_cast<SizeT>(TextureTarget::Texture2DMultisample) ||
targetIndex == static_cast<SizeT>(TextureTarget::Texture2DMultisampleArray);
}
Flags<PixelFormatNormalizeOptionBit> GetRenderTargetNormalizeOptions(SizeT targetIndex) {
Flags<PixelFormatNormalizeOptionBit> options;
if (!TargetRequiresRenderableFormat(targetIndex)) {
return options;
}
options |= PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget;
if (!g_GLESCapabilities.SupportsRenderSnorm || !g_GLESCapabilities.SupportsNorm16Texture) {
options |= PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget;
}
return options;
}
Bool HasCachedFormatCapability(TextureInternalFormat internalFormat, Bool HasCachedFormatCapability(TextureInternalFormat internalFormat,
SizeT targetIndex, SizeT targetIndex,
Bool caveat, Bool caveat,
@@ -141,14 +118,61 @@ namespace MobileGL::MG_Backend::DirectGLES {
const GLenum requestedInternalFormat = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat); const GLenum requestedInternalFormat = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat);
Flags<PixelFormatNormalizeOptionBit> options; Flags<PixelFormatNormalizeOptionBit> options;
if (!pActiveBackendObject || ShouldUseCaveatFormat(internalFormat, targetIndex)) { if (!pActiveBackendObject || ShouldUseCaveatFormat(internalFormat, targetIndex)) {
options = GetRuntimeFallbackNormalizeOptions(requestedInternalFormat, options = GetRuntimeFallbackNormalizeOptions(
GetRenderTargetNormalizeOptions(targetIndex)); requestedInternalFormat,
TextureImpl::GetRenderTargetNormalizeOptions(g_GLESCapabilities, targetIndex));
} }
NormalizePixelFormat(requestedInternalFormat, options, outInternalFormat, outFormat, outType); NormalizePixelFormat(requestedInternalFormat, options, outInternalFormat, outFormat, outType);
} }
} // namespace } // namespace
namespace TextureImpl { namespace TextureImpl {
// Every image that can back a colour attachment needs a colour-renderable storage format,
// and ES has no renderable three-channel format at all: a three-channel float fallback is
// a legal ES texture but neither legal multisample storage nor a legal attachment, so
// GL_RGB8_SNORM / GL_RGB16F / ... have to be widened to four channels for any of them.
// This used to cover the multisample pair alone, on the grounds that only those can never
// be uploaded to; the transfer paths now expand three-channel client data themselves
// (Managers.cpp PrepareFallbackUpload) and hide the added alpha again on sample and
// readback, so the same substitution is available everywhere.
//
// The widening only ever *happens* where the driver refuses the native form (see
// PopulateFormatCapabilitiesImpl: outside multisample storage it rides the driver branch,
// behind the native probe), so a driver that does render to a three-channel image keeps
// allocating it byte for byte.
//
// Do NOT read that as "nothing changes off-device". Measured on Mesa 26.1.6 llvmpipe
// (the headless CI driver), an ES 3.2 GL_TEXTURE_2D colour attachment is COMPLETE for
// GL_RGB8 and GL_RGB16F but INCOMPLETE_ATTACHMENT for GL_RGB8_SNORM, GL_SRGB8 and every
// RGB integer format, and UNSUPPORTED for GL_RGB32F. Those eight formats therefore DO
// take the widened path on llvmpipe, which is where the retrace fixtures and the glcts
// green suites run - the substitution is driver-conditional, not desktop-exempt.
//
// A buffer texture is the one image that can never be an attachment; its storage is the
// buffer object's, and widening it would misdescribe the application's data.
Bool TargetRequiresRenderableFormat(SizeT targetIndex) {
if (targetIndex >= kFormatCapabilityTargetCount) {
return false;
}
if (targetIndex == kFormatCapabilityRenderbufferTargetIndex) {
return true;
}
return static_cast<TextureTarget>(targetIndex) != TextureTarget::TextureBuffer;
}
Flags<PixelFormatNormalizeOptionBit> GetRenderTargetNormalizeOptions(
const MG_External::GLESCapabilities& capabilities, SizeT targetIndex) {
Flags<PixelFormatNormalizeOptionBit> options;
if (!TargetRequiresRenderableFormat(targetIndex)) {
return options;
}
options |= PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget;
if (!capabilities.SupportsRenderSnorm || !capabilities.SupportsNorm16Texture) {
options |= PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget;
}
return options;
}
void GenerateTextureFormatInfo(TextureInternalFormat internalFormat, GLenum* outInternalFormat, void GenerateTextureFormatInfo(TextureInternalFormat internalFormat, GLenum* outInternalFormat,
GLenum* outFormat, GLenum* outType, TextureTarget target) { GLenum* outFormat, GLenum* outType, TextureTarget target) {
#ifdef TRACY_ENABLE #ifdef TRACY_ENABLE
@@ -178,20 +202,29 @@ namespace MobileGL::MG_Backend::DirectGLES {
return ShouldUseCaveatFormat(internalFormat, GetRenderbufferFormatCapabilityTargetIndex()); return ShouldUseCaveatFormat(internalFormat, GetRenderbufferFormatCapabilityTargetIndex());
} }
namespace {
Bool BackendFormatAddsAlpha(TextureInternalFormat internalFormat, SizeT targetIndex) {
if (!TargetRequiresRenderableFormat(targetIndex)) {
return false;
}
if (pActiveBackendObject && !ShouldUseCaveatFormat(internalFormat, targetIndex)) {
return false;
}
const GLenum requestedInternalFormat = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat);
const Flags<PixelFormatNormalizeOptionBit> options = GetRuntimeFallbackNormalizeOptions(
requestedInternalFormat, GetRenderTargetNormalizeOptions(g_GLESCapabilities, targetIndex));
return static_cast<Bool>(options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget);
}
} // namespace
Bool BackendTextureFormatAddsAlpha(TextureInternalFormat internalFormat, TextureTarget target) { Bool BackendTextureFormatAddsAlpha(TextureInternalFormat internalFormat, TextureTarget target) {
const SizeT targetIndex = const SizeT targetIndex =
target == TextureTarget::Unknown ? kFormatCapabilityTargetCount : GetFormatCapabilityTargetIndex(target); target == TextureTarget::Unknown ? kFormatCapabilityTargetCount : GetFormatCapabilityTargetIndex(target);
if (!TargetRequiresRenderableFormat(targetIndex)) { return BackendFormatAddsAlpha(internalFormat, targetIndex);
return false; }
}
if (pActiveBackendObject && !ShouldUseCaveatFormat(internalFormat, targetIndex)) { Bool BackendRenderbufferFormatAddsAlpha(TextureInternalFormat internalFormat) {
return false; return BackendFormatAddsAlpha(internalFormat, GetRenderbufferFormatCapabilityTargetIndex());
}
const GLenum requestedInternalFormat = MG_Util::ConvertTextureInternalFormatToGLEnum(internalFormat);
const Flags<PixelFormatNormalizeOptionBit> options =
GetRuntimeFallbackNormalizeOptions(requestedInternalFormat,
GetRenderTargetNormalizeOptions(targetIndex));
return static_cast<Bool>(options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget);
} }
} // namespace TextureImpl } // namespace TextureImpl
namespace PrgramImpl { namespace PrgramImpl {
+17 -3
View File
@@ -9,6 +9,8 @@
#pragma once #pragma once
#include <Includes.h> #include <Includes.h>
#include <MG_State/GLState/Core.h> #include <MG_State/GLState/Core.h>
#include <MG_Util/BackendLoaders/OpenGL/Loader.h>
#include <MG_Util/Texture/TextureFormatProcessor.h>
namespace MobileGL::MG_Backend::DirectGLES { namespace MobileGL::MG_Backend::DirectGLES {
namespace DebugImpl { namespace DebugImpl {
@@ -34,6 +36,16 @@ namespace MobileGL::MG_Backend::DirectGLES {
} // namespace VertexArrayImpl } // namespace VertexArrayImpl
namespace TextureImpl { namespace TextureImpl {
// Whether images on this format-capability target can back a colour attachment, and so
// need a colour-renderable storage format even when the frontend asked for a
// three-channel one ES never renders to. Shared by the capability probe (which passes the
// capabilities it has just queried, before the globals are published) and by the
// allocation path (which reads the active backend's), so the format the cache was probed
// with is always the format the image is created with.
Bool TargetRequiresRenderableFormat(SizeT targetIndex);
Flags<PixelFormatNormalizeOptionBit> GetRenderTargetNormalizeOptions(
const MG_External::GLESCapabilities& capabilities, SizeT targetIndex);
void GenerateTextureFormatInfo(TextureInternalFormat internalFormat, GLenum* outInternalFormat, void GenerateTextureFormatInfo(TextureInternalFormat internalFormat, GLenum* outInternalFormat,
GLenum* outFormat, GLenum* outType, GLenum* outFormat, GLenum* outType,
TextureTarget target = TextureTarget::Unknown); TextureTarget target = TextureTarget::Unknown);
@@ -41,10 +53,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
GLenum* outFormat, GLenum* outType); GLenum* outFormat, GLenum* outType);
Bool ShouldUseCaveatTextureFormat(TextureInternalFormat internalFormat, TextureTarget target); Bool ShouldUseCaveatTextureFormat(TextureInternalFormat internalFormat, TextureTarget target);
// True when the format the texture is actually created with has an alpha channel the // True when the format the image is actually created with has an alpha channel the
// frontend format does not (the three-channel multisample widening). GL reads such a // frontend format does not (the three-channel colour-renderable widening). GL reads such
// channel back as 1.0, so any swizzle source of ALPHA has to be answered with ONE. // a channel back as 1.0, so any swizzle source of ALPHA has to be answered with ONE and
// any readback of the image has to overwrite the alpha the draw happened to leave there.
Bool BackendTextureFormatAddsAlpha(TextureInternalFormat internalFormat, TextureTarget target); Bool BackendTextureFormatAddsAlpha(TextureInternalFormat internalFormat, TextureTarget target);
Bool BackendRenderbufferFormatAddsAlpha(TextureInternalFormat internalFormat);
Bool ShouldUseCaveatRenderbufferFormat(TextureInternalFormat internalFormat); Bool ShouldUseCaveatRenderbufferFormat(TextureInternalFormat internalFormat);
} // namespace TextureImpl } // namespace TextureImpl
@@ -106,9 +106,11 @@ namespace MobileGL::MG_Impl::GLImpl {
// `capabilityTargetIndex` is the row of the cache the attachment actually lives in; // `capabilityTargetIndex` is the row of the cache the attachment actually lives in;
// kFormatCapabilityTargetCount asks about the format in general. Asking per target matters // kFormatCapabilityTargetCount asks about the format in general. Asking per target matters
// because a capability recorded for one of them says nothing about the others: DirectGLES // because a capability recorded for one of them says nothing about the others: DirectGLES
// widens three-channel formats to four channels to keep them renderable as *multisample* // decides each target's substitution against that target's own probe, and a buffer texture
// storage, and a format that survives only through that substitution is still texture-only // never gets one at all. This is also where the three-channel widening becomes visible to
// on every ordinary target. // the application - a GL_RGB8_SNORM colour attachment on a driver with no renderable
// three-channel format answers COMPLETE because the backend stores it as GL_RGBA16F and
// recorded FramebufferRenderable in CaveatCaps.
Bool IsColorInternalFormatRenderable(TextureInternalFormat format, SizeT capabilityTargetIndex) { Bool IsColorInternalFormatRenderable(TextureInternalFormat format, SizeT capabilityTargetIndex) {
const SizeT formatIndex = static_cast<SizeT>(format); const SizeT formatIndex = static_cast<SizeT>(format);
if (MG_Backend::pActiveBackendObject && formatIndex < MG_Backend::kFormatCapabilityFormatCount) { if (MG_Backend::pActiveBackendObject && formatIndex < MG_Backend::kFormatCapabilityFormatCount) {
+51 -3
View File
@@ -86,17 +86,65 @@ namespace MobileGL::MG_Impl::GLImpl {
return false; return false;
} }
// DSA emulation: the by-name entry points are implemented by putting the named texture
// on the active unit's slot for their target, running the classic bound-texture code,
// then putting the previous binding back.
//
// Both of those binds are REAL changes to "which texture is bound at this unit" for as
// long as `fn` runs, so both have to move the texture bind generation. Backends memoise
// per-unit work keyed on that generation and BORROW the binding slot (they hold a
// pointer to the slot's shared_ptr, not a copy); a slot swap the generation never saw
// let such a memo replay texture A's backend twin against texture B now sitting in the
// slot - which re-specified A's backend storage with B's shape and silently destroyed
// A's GPU-rendered contents (Minecraft's lightmap, blanked by a by-name upload to an
// Iris shadow map, which then discarded every glyph).
//
// The generation is bumped directly rather than through NoteTextureUnitTouched because
// the touched-unit HIGH-WATER MARK must NOT move: glActiveTexture does not advance it,
// so a DSA-only app would otherwise have every later draw walk up to the highest unit it
// ever aimed a by-name call at. Not advancing it is also sufficient - a unit above the
// mark is outside every memo's coverage and outside the epoch walk, so nothing can
// observe the transient swap there; at or below it, the bump is exactly what makes the
// epoch re-derive. Bumping only on a real change keeps the very common redundant case (a
// by-name call on the texture already bound to the active unit) free.
//
// The restore is a scope guard because `fn` can throw (the unsupported-state paths use
// THROW_EXCEPTION): leaking the temporary binding would leave the wrong texture bound to
// a live unit for the rest of the context's life.
template <typename Fn> template <typename Fn>
void WithTemporarilyBoundNamedTexture(const SharedPtr<MG_State::GLState::ITextureObject>& textureObject, void WithTemporarilyBoundNamedTexture(const SharedPtr<MG_State::GLState::ITextureObject>& textureObject,
Fn&& fn) { Fn&& fn) {
if (!textureObject) return; if (!textureObject) return;
auto& activeUnit = MG_State::pGLContext->GetTextureUnitObject(MG_State::pGLContext->GetActiveTextureUnit()); const Int activeUnitIndex = MG_State::pGLContext->GetActiveTextureUnit();
auto& activeUnit = MG_State::pGLContext->GetTextureUnitObject(activeUnitIndex);
auto& bindingSlot = activeUnit.GetBindingSlot(textureObject->GetTarget()); auto& bindingSlot = activeUnit.GetBindingSlot(textureObject->GetTarget());
const auto previousBinding = bindingSlot.GetBoundObject(); const auto previousBinding = bindingSlot.GetBoundObject();
bindingSlot.Bind(textureObject);
using SlotType = std::remove_reference_t<decltype(bindingSlot)>;
class ScopedSlotRestore {
public:
ScopedSlotRestore(SlotType& slot, SharedPtr<MG_State::GLState::ITextureObject> previous)
: m_slot(slot), m_previous(Move(previous)) {}
~ScopedSlotRestore() {
if (m_slot.Bind(m_previous)) {
MG_State::pGLContext->BumpTextureBindGeneration();
}
}
ScopedSlotRestore(const ScopedSlotRestore&) = delete;
ScopedSlotRestore& operator=(const ScopedSlotRestore&) = delete;
private:
SlotType& m_slot;
SharedPtr<MG_State::GLState::ITextureObject> m_previous;
};
if (bindingSlot.Bind(textureObject)) {
MG_State::pGLContext->BumpTextureBindGeneration();
}
ScopedSlotRestore restore(bindingSlot, previousBinding);
fn(MG_Util::ConvertTextureTargetToGLEnum(textureObject->GetTarget())); fn(MG_Util::ConvertTextureTargetToGLEnum(textureObject->GetTarget()));
bindingSlot.Bind(previousBinding);
} }
SizeT ComputeTextureStorageByteSize(TextureInternalFormat textureInternalFormat, GLsizei width, GLsizei height, SizeT ComputeTextureStorageByteSize(TextureInternalFormat textureInternalFormat, GLsizei width, GLsizei height,
@@ -52,6 +52,7 @@ add_executable(MobileGLIntegrationTest
Scenarios/MultiDrawScenario.cpp Scenarios/MultiDrawScenario.cpp
Scenarios/AsyncCompileScenario.cpp Scenarios/AsyncCompileScenario.cpp
Scenarios/XfbAfterClipDistanceScenario.cpp Scenarios/XfbAfterClipDistanceScenario.cpp
Scenarios/ThreeChannelAttachmentScenario.cpp
) )
target_include_directories(MobileGLIntegrationTest PRIVATE target_include_directories(MobileGLIntegrationTest PRIVATE
@@ -0,0 +1,300 @@
// MobileGL - MobileGL/MG_IntegrationTest/Scenarios/ThreeChannelAttachmentScenario.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
//
// Scenario - THREE-CHANNEL COLOUR ATTACHMENTS, on a live driver.
//
// The bug: no OpenGL ES driver renders to a three-channel image. EXT_render_snorm covers
// R/RG/RGBA only, EXT_color_buffer_float excludes RGB16F, and RGB integer formats are not
// colour-renderable anywhere. Complementary Reimagined declares colortex1 = RGB8_SNORM and
// colortex2 = RGB16F, so every framebuffer Iris built from them answered
// GL_FRAMEBUFFER_UNSUPPORTED and Iris refused to load the shaderpack. DirectGLES now stores such
// an attachment in its four-channel sibling (GL_RGB8_SNORM -> GL_RGBA16F) and reports the
// substitution as a caveat capability, which is what makes glCheckFramebufferStatus say COMPLETE.
//
// WHY THIS SCENARIO EXISTS RATHER THAN A UNIT TEST. The unit tests in
// MG_Test/Framebuffer/FramebufferTest.cpp drive a HAND-BUILT capability cache: they prove the
// frontend accepts a caveat capability, and prove the colour-mask/clear discipline that keeps a
// widened attachment's stored alpha at 1.0, but they cannot prove that a real driver's probe
// actually PRODUCES that caveat. Only a live glCheckFramebufferStatus can, and the answer is
// per-driver, not per-platform:
//
// Mesa llvmpipe (the headless CI driver), ES 3.2, GL_TEXTURE_2D colour attachment:
// COMPLETE GL_RGB8, GL_RGB16F, GL_R11F_G11F_B10F, every RGBA*
// INCOMPLETE_ATTACHMENT GL_RGB8_SNORM, GL_SRGB8, every RGB integer format
// UNSUPPORTED GL_RGB32F
//
// So the widening is LIVE on llvmpipe - "the desktop build is unaffected" was simply wrong, and
// the CI retraces were green before the fix only because retrace ignores what
// glCheckFramebufferStatus returns. This scenario is the gate that actually looks.
//
// DirectGLES only. DirectVulkan's format story is its own (Vulkan exposes R8G8B8_SNORM on almost
// nothing, and Magma substitutes on different terms); asserting Espryt's answers there would
// only pin a coincidence.
#include <cmath>
#include <string>
#include <vector>
#include "../Harness/HeadlessGL.h"
#include "../Harness/ScenarioFixture.h"
#ifdef GLAPI
#undef GLAPI
#endif
#define GL_GLEXT_PROTOTYPES
#include <GL/gl.h>
#include <GL/glcorearb.h>
#undef GL_GLEXT_PROTOTYPES
namespace MGITest {
namespace {
constexpr const char* kVS = R"(#version 330 core
in vec2 aPos;
void main() {
gl_Position = vec4(aPos, 0.0, 1.0);
}
)";
// Two outputs so the mixed case is covered: draw buffer 0 is a natively renderable
// four-channel format whose alpha the application owns, draw buffer 1 is the widened
// three-channel one whose alpha the format says is 1.0. Both alphas are deliberately
// NOT 1.0 in the shader, so an implementation that simply passed the value through would
// fail the second assertion.
constexpr const char* kFS = R"(#version 330 core
layout(location = 0) out vec4 oNative;
layout(location = 1) out vec4 oWidened;
void main() {
oNative = vec4(1.0, 0.0, 0.0, 0.25);
oWidened = vec4(0.0, 1.0, 0.0, 0.75);
}
)";
constexpr int kSize = 16;
class ThreeChannelAttachmentScenario : public ScenarioTest {
protected:
void SetUp() override {
ScenarioTest::SetUp();
if (!Ready()) return;
if (Gl().BackendName() != "DirectGLES") {
GTEST_SKIP() << "three-channel widening is a DirectGLES substitution; backend is "
<< Gl().BackendName();
}
}
// A single-level 2D texture in `internalFormat`, or 0 when the driver rejects the
// storage outright (which is a different failure from rejecting the ATTACHMENT).
static GLuint MakeTexture(GLenum internalFormat) {
GLuint texture = 0;
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexStorage2D(GL_TEXTURE_2D, 1, internalFormat, kSize, kSize);
if (glGetError() != GL_NO_ERROR) {
glDeleteTextures(1, &texture);
return 0;
}
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glBindTexture(GL_TEXTURE_2D, 0);
return texture;
}
static GLenum SingleAttachmentStatus(GLenum internalFormat) {
const GLuint texture = MakeTexture(internalFormat);
if (texture == 0) return GL_NONE;
GLuint fbo = 0;
glGenFramebuffers(1, &fbo);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, fbo);
glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0);
const GLenum status = glCheckFramebufferStatus(GL_DRAW_FRAMEBUFFER);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &fbo);
glDeleteTextures(1, &texture);
return status;
}
};
// THE regression gate for the frontend's answer: this is the exact call Iris makes, and
// GL_FRAMEBUFFER_UNSUPPORTED here is the whole shaderpack load failure.
TEST_F(ThreeChannelAttachmentScenario, ThreeChannelColorAttachmentsReportComplete) {
if (!Ready() || IsSkipped()) return;
// GL_RGB8 is the control: colour-renderable in ES core, so it must pass with or
// without any substitution. If it ever fails, nothing below means anything.
EXPECT_EQ(SingleAttachmentStatus(GL_RGB8), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE))
<< "GL_RGB8 is ES-core colour-renderable";
// Complementary Reimagined's colortex1 and colortex2.
EXPECT_EQ(SingleAttachmentStatus(GL_RGB8_SNORM), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE))
<< "colortex1 (RGB8_SNORM) must be renderable through the four-channel widening";
EXPECT_EQ(SingleAttachmentStatus(GL_RGB16F), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE))
<< "colortex2 (RGB16F) must be renderable, natively or through the widening";
// The other formats the widening covers. GL_RGB32F only reaches a renderable
// four-channel form when EXT_color_buffer_float is present, so a half-float-only
// driver legitimately answers UNSUPPORTED for it - see the POST's per-format row.
EXPECT_EQ(SingleAttachmentStatus(GL_SRGB8), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
EXPECT_EQ(SingleAttachmentStatus(GL_RGB8UI), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
EXPECT_EQ(FirstGLError(), 0u) << GLErrorName(FirstGLError());
}
// The other half: the substitution has to be INVISIBLE. A three-channel format has no
// alpha, so GL answers 1.0 for it - and that answer has to hold after a draw that wrote
// something else into the widened storage's real alpha channel, which is what the
// colour-mask discipline in SyncRenderState is for. GL_DST_ALPHA blending and
// glBlitFramebuffer read that stored alpha inside the driver, where no readback fixup can
// reach it, so "the storage really holds 1.0" is the only workable invariant.
TEST_F(ThreeChannelAttachmentScenario, WidenedAttachmentReadsBackOpaqueWhileItsNeighbourKeepsItsAlpha) {
if (!Ready() || IsSkipped()) return;
std::string error;
const GLuint program = CompileProgram(kVS, kFS, &error);
ASSERT_NE(program, 0u) << error;
const GLuint nativeTexture = MakeTexture(GL_RGBA16F);
const GLuint widenedTexture = MakeTexture(GL_RGB8_SNORM);
ASSERT_NE(nativeTexture, 0u);
ASSERT_NE(widenedTexture, 0u);
GLuint fbo = 0;
glGenFramebuffers(1, &fbo);
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, nativeTexture, 0);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, widenedTexture, 0);
const GLenum drawBuffers[2] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1};
glDrawBuffers(2, drawBuffers);
ASSERT_EQ(glCheckFramebufferStatus(GL_FRAMEBUFFER), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
glViewport(0, 0, kSize, kSize);
// Alpha 0.0 on purpose: the widened attachment must come back 1.0 anyway, and the
// native one must come back 0.0 where the draw does not cover it.
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT);
const float quad[] = {-1.0f, -1.0f, 1.0f, -1.0f, -1.0f, 1.0f, 1.0f, 1.0f};
GLuint vao = 0;
GLuint vbo = 0;
glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, sizeof(quad), quad, GL_STATIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 2 * sizeof(float), nullptr);
glUseProgram(program);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
std::vector<float> pixels(static_cast<std::size_t>(kSize) * kSize * 4, -1.0f);
glReadBuffer(GL_COLOR_ATTACHMENT1);
glReadPixels(0, 0, kSize, kSize, GL_RGBA, GL_FLOAT, pixels.data());
EXPECT_NEAR(pixels[0], 0.0f, 0.02f) << "widened attachment red";
EXPECT_NEAR(pixels[1], 1.0f, 0.02f) << "widened attachment green";
EXPECT_NEAR(pixels[2], 0.0f, 0.02f) << "widened attachment blue";
EXPECT_NEAR(pixels[3], 1.0f, 0.001f)
<< "a three-channel format has no alpha channel, so GL must report 1.0 for it";
glReadBuffer(GL_COLOR_ATTACHMENT0);
glReadPixels(0, 0, kSize, kSize, GL_RGBA, GL_FLOAT, pixels.data());
EXPECT_NEAR(pixels[0], 1.0f, 0.02f) << "native attachment red";
EXPECT_NEAR(pixels[3], 0.25f, 0.02f)
<< "the alpha discipline must not leak onto a natively renderable attachment";
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &fbo);
glDeleteBuffers(1, &vbo);
glDeleteVertexArrays(1, &vao);
glDeleteTextures(1, &nativeTexture);
glDeleteTextures(1, &widenedTexture);
glDeleteProgram(program);
EXPECT_EQ(FirstGLError(), 0u) << GLErrorName(FirstGLError());
}
// The case above can be satisfied by the readback fixup alone (ForceWideReadAlphaToOne
// rewrites glReadPixels' alpha), so it does NOT prove the STORED alpha is 1.0. This one
// does, by asking the driver to read that alpha itself: GL_DST_ALPHA blending multiplies
// by the destination alpha inside the raster pipeline, where nothing MobileGL does can
// intervene. Same reason GL_ONE_MINUS_DST_ALPHA and glBlitFramebuffer are covered for
// free once this holds - and the reason the discipline is a write mask rather than a
// readback patch.
//
// Ablation-checked on llvmpipe, each half separately: disable the alpha doctoring in
// SyncRenderState and the opaque draw leaves 0.25 in the stored alpha; disable the clear
// substitution in Clear() and it stays at the application's 0.0. Either way this case
// reads back the wrong number, which is what makes it a gate rather than a description.
TEST_F(ThreeChannelAttachmentScenario, DstAlphaBlendingSeesOneInAWidenedAttachment) {
if (!Ready() || IsSkipped()) return;
static constexpr const char* kSingleOutFS = R"(#version 330 core
out vec4 oColor;
uniform vec4 uColor;
void main() { oColor = uColor; }
)";
std::string error;
const GLuint program = CompileProgram(kVS, kSingleOutFS, &error);
ASSERT_NE(program, 0u) << error;
const GLint colorLocation = glGetUniformLocation(program, "uColor");
ASSERT_GE(colorLocation, 0);
const GLuint widenedTexture = MakeTexture(GL_RGB8_SNORM);
ASSERT_NE(widenedTexture, 0u);
GLuint fbo = 0;
glGenFramebuffers(1, &fbo);
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, widenedTexture, 0);
ASSERT_EQ(glCheckFramebufferStatus(GL_FRAMEBUFFER), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
const float quad[] = {-1.0f, -1.0f, 1.0f, -1.0f, -1.0f, 1.0f, 1.0f, 1.0f};
GLuint vao = 0;
GLuint vbo = 0;
glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, sizeof(quad), quad, GL_STATIC_DRAW);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 2 * sizeof(float), nullptr);
glUseProgram(program);
glViewport(0, 0, kSize, kSize);
// The clear's alpha is 0.0 and the draw's is 0.25 - neither is the 1.0 the format
// implies, so both halves of the discipline have to fire for the blend below to see
// 1.0: the clear substitutes it, and the draw is masked away from it.
glDisable(GL_BLEND);
glClearColor(0.0f, 0.0f, 0.0f, 0.0f);
glClear(GL_COLOR_BUFFER_BIT);
glUniform4f(colorLocation, 0.0f, 1.0f, 0.0f, 0.25f);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
// dst = stored alpha; src factor GL_DST_ALPHA, dst factor GL_ZERO, source white
// => the destination colour becomes (storedAlpha, storedAlpha, storedAlpha).
glEnable(GL_BLEND);
glBlendFunc(GL_DST_ALPHA, GL_ZERO);
glUniform4f(colorLocation, 1.0f, 1.0f, 1.0f, 1.0f);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glDisable(GL_BLEND);
std::vector<float> pixels(static_cast<std::size_t>(kSize) * kSize * 4, -1.0f);
glReadBuffer(GL_COLOR_ATTACHMENT0);
glReadPixels(0, 0, kSize, kSize, GL_RGBA, GL_FLOAT, pixels.data());
EXPECT_NEAR(pixels[0], 1.0f, 0.02f)
<< "GL_DST_ALPHA read the stored alpha of a three-channel attachment; it must be 1.0";
glBindFramebuffer(GL_FRAMEBUFFER, 0);
glDeleteFramebuffers(1, &fbo);
glDeleteBuffers(1, &vbo);
glDeleteVertexArrays(1, &vao);
glDeleteTextures(1, &widenedTexture);
glDeleteProgram(program);
EXPECT_EQ(FirstGLError(), 0u) << GLErrorName(FirstGLError());
}
} // namespace
} // namespace MGITest
@@ -13,10 +13,13 @@
#include "Includes.h" #include "Includes.h"
#include "Init.h" #include "Init.h"
#include <MG_Backend/BackendObjects.h> #include <MG_Backend/BackendObjects.h>
#include <MG_Backend/DirectGLES/DirectGLES.h>
#include <MG_Backend/DirectGLES/Managers.h>
#include <MG_Backend/DirectGLES/Utils.h> #include <MG_Backend/DirectGLES/Utils.h>
#include <MG_Impl/GLImpl/Buffer/GL_Buffer.h> #include <MG_Impl/GLImpl/Buffer/GL_Buffer.h>
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h> #include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Getter/GL_Getter.h> #include <MG_Impl/GLImpl/Getter/GL_Getter.h>
#include <MG_Impl/GLImpl/RenderState/GL_RenderState.h>
#include <MG_Impl/GLImpl/Texture/GL_Texture.h> #include <MG_Impl/GLImpl/Texture/GL_Texture.h>
#include <MG_State/GLState/Core.h> #include <MG_State/GLState/Core.h>
@@ -858,3 +861,394 @@ TEST_F(FramebufferTest, NonRenderableColorFormatsReportUnsupportedFramebuffer) {
MG_Impl::GLImpl::ReadPixels(0, 0, 4, 4, GL_RGBA, GL_UNSIGNED_BYTE, pixelStorage); MG_Impl::GLImpl::ReadPixels(0, 0, 4, 4, GL_RGBA, GL_UNSIGNED_BYTE, pixelStorage);
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_INVALID_FRAMEBUFFER_OPERATION); EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_INVALID_FRAMEBUFFER_OPERATION);
} }
// ---- Three-channel colour attachments: the Complementary Reimagined / Iris load failure --------
//
// Complementary declares colortex1 = RGB8_SNORM and colortex2 = RGB16F. No real OpenGL ES driver
// renders to a three-channel image (EXT_render_snorm covers R/RG/RGBA only; EXT_color_buffer_float
// excludes RGB16F), so the DirectGLES probe records those formats as creatable-but-not-renderable
// and the frontend answered every framebuffer built from them GL_FRAMEBUFFER_UNSUPPORTED - which
// Iris turns into a hard "Draw buffers [0, 1] Status: 36061" load failure. The backend now records
// the four-channel substitution it will actually allocate as a caveat capability, and the frontend
// has to accept that as renderable.
namespace {
class ThreeChannelAttachmentBackend final : public MG_Backend::BackendObject {
public:
// `substituted` stands in for a driver where the four-channel widening probe succeeded, i.e.
// for what PopulateFormatCapabilitiesImpl records on Mali. false is the pre-fix state: the
// native form is creatable, nothing is renderable, and no fallback was ever built.
explicit ThreeChannelAttachmentBackend(Bool substituted) {
auto& cache = MutableFormatCapabilities();
const auto texture2DIndex = MG_Backend::GetFormatCapabilityTargetIndex(TextureTarget::Texture2D);
// IsColorInternalFormatRenderable only trusts the cache once it looks populated, which
// it decides from RGBA8 being creatable somewhere. Without this the static deny-list
// answers instead and the caveat below would never be consulted.
const auto rgba8Index = static_cast<SizeT>(TextureInternalFormat::RGBA8);
cache.FullCaps[texture2DIndex][rgba8Index] |= MG_Backend::FormatCapability::Creatable;
cache.FullCaps[texture2DIndex][rgba8Index] |= MG_Backend::FormatCapability::FramebufferRenderable;
cache.FullCaps[texture2DIndex][rgba8Index] |= MG_Backend::FormatCapability::ColorAttachment;
for (const TextureInternalFormat format :
{TextureInternalFormat::RGB8Snorm, TextureInternalFormat::RGB16F}) {
const auto formatIndex = static_cast<SizeT>(format);
// Creatable and samplable as an ordinary texture, but the driver's
// glCheckFramebufferStatus said no - exactly Mali r32p1's answer.
cache.FullCaps[texture2DIndex][formatIndex] |= MG_Backend::FormatCapability::Creatable;
cache.FullCaps[texture2DIndex][formatIndex] |= MG_Backend::FormatCapability::Sampled;
if (substituted) {
cache.CaveatCaps[texture2DIndex][formatIndex] |=
MG_Backend::FormatCapability::FramebufferRenderable;
cache.CaveatCaps[texture2DIndex][formatIndex] |= MG_Backend::FormatCapability::ColorAttachment;
}
}
}
void Initialize() override {}
Bool InitCapabilities() override { return true; }
Bool InitWindowSurface() override { return true; }
const RendererInfo& GetRendererInfo() const override {
static RendererInfo info = {};
return info;
}
String GetBackendAPIVersionString() const override { return {}; }
const MG_Backend::GlobalBackendFunctionsTable& GetBackendFunctions() const override {
static MG_Backend::GlobalBackendFunctionsTable table = {};
return table;
}
const MG_Backend::DynamicBackendParameters& GetDynamicParameters() const override {
static MG_Backend::DynamicBackendParameters params = {};
return params;
}
BackendType GetBackendType() const override { return BackendType::Unknown; }
};
class ScopedBackendOverride {
public:
explicit ScopedBackendOverride(UniquePtr<MG_Backend::BackendObject> backend):
m_previous(Move(MG_Backend::pActiveBackendObject)) {
MG_Backend::pActiveBackendObject = Move(backend);
}
~ScopedBackendOverride() { MG_Backend::pActiveBackendObject = Move(m_previous); }
private:
UniquePtr<MG_Backend::BackendObject> m_previous;
};
GLenum CheckSingleColorAttachmentStatus(GLenum internalFormat) {
GLuint framebuffer = 0;
GLuint texture = 0;
MG_Impl::GLImpl::CreateFramebuffers(1, &framebuffer);
MG_Impl::GLImpl::CreateTextures(GL_TEXTURE_2D, 1, &texture);
MG_Impl::GLImpl::TextureStorage2D(texture, 1, internalFormat, 4, 4);
MG_Impl::GLImpl::NamedFramebufferTexture(framebuffer, GL_COLOR_ATTACHMENT0, texture, 0);
return MG_Impl::GLImpl::CheckNamedFramebufferStatus(framebuffer, GL_DRAW_FRAMEBUFFER);
}
} // namespace
TEST_F(FramebufferTest, ThreeChannelColorAttachmentsAreUnsupportedWithoutTheWidenedSubstitution) {
// The pre-fix behaviour, pinned so a regression is a red test rather than a shaderpack that
// silently stops loading: no caveat capability, so nothing makes these renderable.
ScopedBackendOverride backend(MakeUnique<ThreeChannelAttachmentBackend>(/*substituted=*/false));
EXPECT_EQ(CheckSingleColorAttachmentStatus(GL_RGB8_SNORM), static_cast<GLenum>(GL_FRAMEBUFFER_UNSUPPORTED));
EXPECT_EQ(CheckSingleColorAttachmentStatus(GL_RGB16F), static_cast<GLenum>(GL_FRAMEBUFFER_UNSUPPORTED));
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
TEST_F(FramebufferTest, ThreeChannelColorAttachmentsAreCompleteThroughTheWidenedSubstitution) {
ScopedBackendOverride backend(MakeUnique<ThreeChannelAttachmentBackend>(/*substituted=*/true));
// Complementary's colortex1 (RGB8_SNORM) and colortex2 (RGB16F): both must come out COMPLETE,
// because the backend stores them as GL_RGBA16F. Shipping only the first would move the
// failure one composite pass down instead of fixing it.
EXPECT_EQ(CheckSingleColorAttachmentStatus(GL_RGB8_SNORM), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
EXPECT_EQ(CheckSingleColorAttachmentStatus(GL_RGB16F), static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
TEST_F(FramebufferTest, TwoAttachmentCompositeFramebufferMatchesIrisComplementaryPass) {
// The exact framebuffer Iris failed on: Complementary's `composite` pass draws to colortex7
// (RGBA16F, natively renderable) and colortex1 (RGB8_SNORM, only renderable widened). Iris
// logs it as "Draw buffers [0, 1]" - a two-attachment FBO, not colortex 0 and 1.
ScopedBackendOverride backend(MakeUnique<ThreeChannelAttachmentBackend>(/*substituted=*/true));
GLuint framebuffer = 0;
GLuint colortex7 = 0;
GLuint colortex1 = 0;
MG_Impl::GLImpl::CreateFramebuffers(1, &framebuffer);
MG_Impl::GLImpl::CreateTextures(GL_TEXTURE_2D, 1, &colortex7);
MG_Impl::GLImpl::CreateTextures(GL_TEXTURE_2D, 1, &colortex1);
MG_Impl::GLImpl::TextureStorage2D(colortex7, 1, GL_RGBA8, 4, 4);
MG_Impl::GLImpl::TextureStorage2D(colortex1, 1, GL_RGB8_SNORM, 4, 4);
MG_Impl::GLImpl::NamedFramebufferTexture(framebuffer, GL_COLOR_ATTACHMENT0, colortex7, 0);
MG_Impl::GLImpl::NamedFramebufferTexture(framebuffer, GL_COLOR_ATTACHMENT1, colortex1, 0);
EXPECT_EQ(MG_Impl::GLImpl::CheckNamedFramebufferStatus(framebuffer, GL_DRAW_FRAMEBUFFER),
static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
// Both entry points answer from the same helpers, and CheckFramebufferStatus is what Iris
// actually calls; they are near-verbatim duplicates, so assert they agree.
MG_Impl::GLImpl::BindFramebuffer(GL_DRAW_FRAMEBUFFER, framebuffer);
EXPECT_EQ(MG_Impl::GLImpl::CheckFramebufferStatus(GL_DRAW_FRAMEBUFFER),
static_cast<GLenum>(GL_FRAMEBUFFER_COMPLETE));
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
// ---- Widened attachments: the stored-alpha discipline -----------------------------------------
//
// A widened attachment has a real alpha channel the application's three-channel format does not,
// and GL says a channel a format lacks reads back as 1.0. glReadPixels and glGetTexImage can be
// made to say that (ForceWideReadAlphaToOne), but GL_DST_ALPHA / GL_ONE_MINUS_DST_ALPHA blending
// and glBlitFramebuffer read the STORED alpha inside the driver, where nothing can intercept it.
// So the stored alpha is held at 1.0 instead: a clear writes 1.0 into it, and every draw has that
// buffer's alpha write mask forced off so nothing can move it again.
//
// These cases pin the two halves of that pairing at the seam where they are visible - what the ES
// driver is actually handed - and pin the invariant that the application's own colour mask is
// never touched.
namespace {
struct RecordedColorMask {
Bool seen = false;
GLboolean r = GL_FALSE, g = GL_FALSE, b = GL_FALSE, a = GL_FALSE;
};
constexpr Uint kRecordedDrawBuffers = 8;
RecordedColorMask g_driverIndexedColorMasks[kRecordedDrawBuffers];
RecordedColorMask g_driverUniformColorMask;
void ResetRecordedColorMasks() {
for (auto& recorded : g_driverIndexedColorMasks) recorded = {};
g_driverUniformColorMask = {};
}
void StubColorMask(GLboolean r, GLboolean g, GLboolean b, GLboolean a) {
g_driverUniformColorMask = {true, r, g, b, a};
// The non-indexed call sets every draw buffer, so record it as such: a later assertion
// about draw buffer 1 must not read a stale indexed record the uniform push overwrote.
for (auto& recorded : g_driverIndexedColorMasks) recorded = {true, r, g, b, a};
}
void StubColorMaski(GLuint index, GLboolean r, GLboolean g, GLboolean b, GLboolean a) {
if (index < kRecordedDrawBuffers) g_driverIndexedColorMasks[index] = {true, r, g, b, a};
}
void StubViewport(GLint, GLint, GLsizei, GLsizei) {}
void StubScissor(GLint, GLint, GLsizei, GLsizei) {}
void StubEnable(GLenum) {}
void StubDisable(GLenum) {}
void StubEnablei(GLenum, GLuint) {}
void StubDisablei(GLenum, GLuint) {}
void StubBlendFuncSeparate(GLenum, GLenum, GLenum, GLenum) {}
void StubBlendFuncSeparatei(GLuint, GLenum, GLenum, GLenum, GLenum) {}
void StubBlendEquationSeparate(GLenum, GLenum) {}
void StubBlendEquationSeparatei(GLuint, GLenum, GLenum) {}
void StubBlendColor(GLfloat, GLfloat, GLfloat, GLfloat) {}
void StubDepthFunc(GLenum) {}
void StubDepthMask(GLboolean) {}
void StubDepthRangef(GLfloat, GLfloat) {}
void StubStencilFuncSeparate(GLenum, GLenum, GLint, GLuint) {}
void StubStencilMaskSeparate(GLenum, GLuint) {}
void StubStencilOpSeparate(GLenum, GLenum, GLenum, GLenum) {}
void StubClearColor(GLfloat, GLfloat, GLfloat, GLfloat) {}
void StubClearDepthf(GLfloat) {}
void StubClearStencil(GLint) {}
void StubCullFace(GLenum) {}
void StubFrontFace(GLenum) {}
void StubPolygonOffset(GLfloat, GLfloat) {}
void StubLineWidth(GLfloat) {}
void StubSampleCoverage(GLfloat, GLboolean) {}
// Replaces the ES function table with no-ops that record only what these cases assert on.
// The table is ZEROED first on purpose: SyncRenderState is long, and a call it makes that
// this fixture did not anticipate must crash here rather than silently reach a stale pointer
// into a driver that this process never made current.
class ScopedRenderStateDriverStubs {
public:
ScopedRenderStateDriverStubs():
m_funcs(MG_Backend::DirectGLES::g_GLESFuncs), m_caps(MG_Backend::DirectGLES::g_GLESCapabilities) {
auto& gl = MG_Backend::DirectGLES::g_GLESFuncs;
gl = MG_External::GLESFunctionsTable{};
gl.glViewport = StubViewport;
gl.glScissor = StubScissor;
gl.glEnable = StubEnable;
gl.glDisable = StubDisable;
gl.glEnablei = StubEnablei;
gl.glDisablei = StubDisablei;
gl.glBlendFuncSeparate = StubBlendFuncSeparate;
gl.glBlendFuncSeparatei = StubBlendFuncSeparatei;
gl.glBlendEquationSeparate = StubBlendEquationSeparate;
gl.glBlendEquationSeparatei = StubBlendEquationSeparatei;
gl.glBlendColor = StubBlendColor;
gl.glDepthFunc = StubDepthFunc;
gl.glDepthMask = StubDepthMask;
gl.glDepthRangef = StubDepthRangef;
gl.glStencilFuncSeparate = StubStencilFuncSeparate;
gl.glStencilMaskSeparate = StubStencilMaskSeparate;
gl.glStencilOpSeparate = StubStencilOpSeparate;
gl.glClearColor = StubClearColor;
gl.glClearDepthf = StubClearDepthf;
gl.glClearStencil = StubClearStencil;
gl.glCullFace = StubCullFace;
gl.glFrontFace = StubFrontFace;
gl.glPolygonOffset = StubPolygonOffset;
gl.glLineWidth = StubLineWidth;
gl.glSampleCoverage = StubSampleCoverage;
gl.glColorMask = StubColorMask;
gl.glColorMaski = StubColorMaski;
auto& caps = MG_Backend::DirectGLES::g_GLESCapabilities;
caps.SupportsIndexedColorMask = true;
caps.SupportsSrgbWriteControl = false;
caps.SupportsPolygonMode = false;
caps.SupportsDualSourceBlend = true;
ResetRecordedColorMasks();
// The viewport and scissor blocks fall back to querying the surface size when the
// frontend's rectangle is degenerate, and there is no surface in this process.
MG_Impl::GLImpl::Viewport(0, 0, 4, 4);
MG_Impl::GLImpl::Scissor(0, 0, 4, 4);
MG_Backend::DirectGLES::RenderStateImpl::InvalidateSyncedRenderState();
}
~ScopedRenderStateDriverStubs() {
MG_Backend::DirectGLES::FramebufferImpl::g_alphaWidenedDrawBufferMask = 0;
MG_Backend::DirectGLES::g_GLESFuncs = m_funcs;
MG_Backend::DirectGLES::g_GLESCapabilities = m_caps;
// The shadow now describes pushes that went to the stubs, not to any driver.
MG_Backend::DirectGLES::RenderStateImpl::InvalidateSyncedRenderState();
MG_Impl::GLImpl::ColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
}
private:
MG_External::GLESFunctionsTable m_funcs;
MG_External::GLESCapabilities m_caps;
};
} // namespace
TEST_F(FramebufferTest, WidenedDrawBufferIsIdentifiedPerDrawBufferSlotNotPerAttachmentPoint) {
ScopedBackendOverride backend(MakeUnique<ThreeChannelAttachmentBackend>(/*substituted=*/true));
// Complementary's `composite` framebuffer again: draw buffer 0 is a natively renderable
// RGBA8, draw buffer 1 is the widened RGB8_SNORM. Only the second may be doctored.
GLuint framebuffer = 0;
GLuint colortex7 = 0;
GLuint colortex1 = 0;
MG_Impl::GLImpl::CreateFramebuffers(1, &framebuffer);
MG_Impl::GLImpl::CreateTextures(GL_TEXTURE_2D, 1, &colortex7);
MG_Impl::GLImpl::CreateTextures(GL_TEXTURE_2D, 1, &colortex1);
MG_Impl::GLImpl::TextureStorage2D(colortex7, 1, GL_RGBA8, 4, 4);
MG_Impl::GLImpl::TextureStorage2D(colortex1, 1, GL_RGB8_SNORM, 4, 4);
MG_Impl::GLImpl::NamedFramebufferTexture(framebuffer, GL_COLOR_ATTACHMENT0, colortex7, 0);
MG_Impl::GLImpl::NamedFramebufferTexture(framebuffer, GL_COLOR_ATTACHMENT1, colortex1, 0);
auto& framebufferObject = MG_State::pGLContext->GetFramebufferObject(framebuffer);
ASSERT_NE(framebufferObject, nullptr);
framebufferObject->SetDrawBuffer(0, FramebufferAttachmentType::Color0);
framebufferObject->SetDrawBuffer(1, FramebufferAttachmentType::Color1);
EXPECT_EQ(MG_Backend::DirectGLES::FramebufferImpl::ComputeAlphaWidenedDrawBufferMask(*framebufferObject),
1u << 1);
// Swapping the draw-buffer array moves the bit with the SLOT, not with the attachment point:
// glColorMaski and glClearBufferfv both address slots.
framebufferObject->SetDrawBuffer(0, FramebufferAttachmentType::Color1);
framebufferObject->SetDrawBuffer(1, FramebufferAttachmentType::Color0);
EXPECT_EQ(MG_Backend::DirectGLES::FramebufferImpl::ComputeAlphaWidenedDrawBufferMask(*framebufferObject),
1u << 0);
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
TEST_F(FramebufferTest, DrawIntoAWidenedDrawBufferReachesTheDriverWithAlphaWritesMaskedOff) {
ScopedRenderStateDriverStubs driver;
MG_Backend::DirectGLES::FramebufferImpl::g_alphaWidenedDrawBufferMask = 1u << 1;
// What the application asked for: write every channel of every draw buffer.
MG_Impl::GLImpl::ColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
MG_Backend::DirectGLES::RenderStateImpl::SyncRenderState(/*forColorClear=*/false);
// What the driver was told. Draw buffer 0 is untouched; draw buffer 1 loses alpha.
ASSERT_TRUE(g_driverIndexedColorMasks[0].seen);
EXPECT_EQ(g_driverIndexedColorMasks[0].r, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[0].g, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[0].b, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[0].a, GL_TRUE);
ASSERT_TRUE(g_driverIndexedColorMasks[1].seen);
EXPECT_EQ(g_driverIndexedColorMasks[1].r, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[1].g, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[1].b, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[1].a, GL_FALSE) << "a widened draw buffer must not take alpha writes";
// And what the application sees back. The doctoring lives entirely on the push; the frontend
// state it is derived from is never written, so glGet still answers with the app's value.
GLboolean appMask[4] = {GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE};
MG_Impl::GLImpl::GetBooleanv(GL_COLOR_WRITEMASK, appMask);
EXPECT_EQ(appMask[0], GL_TRUE);
EXPECT_EQ(appMask[1], GL_TRUE);
EXPECT_EQ(appMask[2], GL_TRUE);
EXPECT_EQ(appMask[3], GL_TRUE) << "glGet(GL_COLOR_WRITEMASK) must report the application's mask";
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
TEST_F(FramebufferTest, ClearIntoAWidenedDrawBufferKeepsAlphaWritableAndSubstitutesOne) {
ScopedRenderStateDriverStubs driver;
MG_Backend::DirectGLES::FramebufferImpl::g_alphaWidenedDrawBufferMask = 1u << 1;
MG_Impl::GLImpl::ColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
// A draw first, so the mask really is doctored when the clear arrives...
MG_Backend::DirectGLES::RenderStateImpl::SyncRenderState(/*forColorClear=*/false);
ASSERT_EQ(g_driverIndexedColorMasks[1].a, GL_FALSE);
// ...and now the clear, with NOTHING changed in the frontend parameter block. The frontend's
// render-state version has not moved, so only the purpose-aware memo can force this push -
// without it the clear would inherit the draw's alpha-off mask and never write the 1.0.
ResetRecordedColorMasks();
MG_Backend::DirectGLES::RenderStateImpl::SyncRenderState(/*forColorClear=*/true);
ASSERT_TRUE(g_driverIndexedColorMasks[1].seen) << "the clear must re-push the colour mask";
EXPECT_EQ(g_driverIndexedColorMasks[1].a, GL_TRUE) << "a clear is what puts the 1.0 in the stored alpha";
// The value that clear writes: the application's RGB, alpha replaced by the 1.0 the
// three-channel format implies, and only on the widened buffer.
const GLfloat appColor[4] = {0.25f, 0.5f, 0.75f, 0.0f};
GLfloat scratch[4] = {};
const GLfloat* widened =
MG_Backend::DirectGLES::FramebufferImpl::SubstituteWidenedClearAlpha(appColor, true, 1.0f, scratch);
EXPECT_EQ(widened[0], 0.25f);
EXPECT_EQ(widened[1], 0.5f);
EXPECT_EQ(widened[2], 0.75f);
EXPECT_EQ(widened[3], 1.0f);
const GLfloat* untouched =
MG_Backend::DirectGLES::FramebufferImpl::SubstituteWidenedClearAlpha(appColor, false, 1.0f, scratch);
EXPECT_EQ(untouched, appColor) << "a native attachment's clear must not even be copied";
// An integer widened format (GL_RGB8UI -> GL_RGBA8UI) carries the INTEGER one, not a
// saturated field: glClearBufferuiv takes the value verbatim.
const GLuint appIntegerColor[4] = {7u, 8u, 9u, 0u};
GLuint integerScratch[4] = {};
const GLuint* widenedInteger = MG_Backend::DirectGLES::FramebufferImpl::SubstituteWidenedClearAlpha(
appIntegerColor, true, GLuint(1), integerScratch);
EXPECT_EQ(widenedInteger[3], 1u);
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
TEST_F(FramebufferTest, ApplicationAlphaMaskOffIsStillHonouredOnANativeDrawBuffer) {
// The doctoring only ever REMOVES alpha writes on a widened buffer; it must never add them
// back on a buffer the application masked itself, and must never touch a native one.
ScopedRenderStateDriverStubs driver;
MG_Backend::DirectGLES::FramebufferImpl::g_alphaWidenedDrawBufferMask = 1u << 1;
MG_Impl::GLImpl::ColorMaski(0, GL_TRUE, GL_TRUE, GL_TRUE, GL_FALSE);
MG_Impl::GLImpl::ColorMaski(1, GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
MG_Impl::GLImpl::ColorMaski(2, GL_FALSE, GL_TRUE, GL_FALSE, GL_TRUE);
MG_Backend::DirectGLES::RenderStateImpl::SyncRenderState(/*forColorClear=*/false);
EXPECT_EQ(g_driverIndexedColorMasks[0].a, GL_FALSE) << "the application's own alpha mask survives";
EXPECT_EQ(g_driverIndexedColorMasks[1].a, GL_FALSE) << "the widened buffer loses alpha";
EXPECT_EQ(g_driverIndexedColorMasks[2].r, GL_FALSE);
EXPECT_EQ(g_driverIndexedColorMasks[2].g, GL_TRUE);
EXPECT_EQ(g_driverIndexedColorMasks[2].b, GL_FALSE);
EXPECT_EQ(g_driverIndexedColorMasks[2].a, GL_TRUE) << "a native buffer keeps its alpha writes";
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
}
+129
View File
@@ -1898,3 +1898,132 @@ TEST(FastSTLSanity, ErasingTheOnlyElementReturnsEnd) {
EXPECT_EQ(next, map.end()); EXPECT_EQ(next, map.end());
EXPECT_TRUE(map.empty()); EXPECT_TRUE(map.empty());
} }
namespace {
// Records what the per-unit texture sync actually pushed at the driver: which backend
// texture id was current when each glTexImage2D landed, and the shape it was given.
struct TexSpecCall {
GLuint texture;
GLsizei width;
GLsizei height;
};
MobileGL::Vector<TexSpecCall>* g_texSpecCalls = nullptr;
GLuint g_texSpecBoundTexture = 0;
void TS_BindTexture(GLenum, GLuint texture) { g_texSpecBoundTexture = texture; }
void TS_ActiveTexture(GLenum) {}
void TS_TexParameteri(GLenum, GLenum, GLint) {}
void TS_TexParameterf(GLenum, GLenum, GLfloat) {}
void TS_TexParameterfv(GLenum, GLenum, const GLfloat*) {}
void TS_PixelStorei(GLenum, GLint) {}
void TS_BindBuffer(GLenum, GLuint) {}
void TS_TexImage2D(GLenum, GLint level, GLint, GLsizei width, GLsizei height, GLint, GLenum, GLenum,
const void*) {
if (g_texSpecCalls && level == 0) {
g_texSpecCalls->push_back({g_texSpecBoundTexture, width, height});
}
}
// Clears the recording hook even when a gtest assertion unwinds the test body.
struct ScopedTexSpecRecording {
explicit ScopedTexSpecRecording(MobileGL::Vector<TexSpecCall>& sink) {
g_texSpecCalls = &sink;
g_texSpecBoundTexture = 0;
}
~ScopedTexSpecRecording() { g_texSpecCalls = nullptr; }
ScopedTexSpecRecording(const ScopedTexSpecRecording&) = delete;
ScopedTexSpecRecording& operator=(const ScopedTexSpecRecording&) = delete;
};
// Gives `name` a complete single-level 2D image of the requested size without going through
// the frontend upload path (the mock table below wires only the state-pushing entry points).
MobileGL::SharedPtr<MobileGL::MG_State::GLState::ITextureObject> MakeComplete2DTexture(GLuint name,
MobileGL::Int size) {
using namespace MobileGL;
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, name);
auto object = MG_State::pGLContext->GetTextureUnitObject(0)
.GetBindingSlot(TextureTarget::Texture2D)
.GetBoundObject();
object->SetInternalFormat(TextureInternalFormat::RGBA8);
MG_State::GLState::AsMipmapTexture(object.get())
->AllocateStorage(TextureUploadTarget::Texture2D, 0, {{size, size, 1}, 4});
return object;
}
} // namespace
// The per-unit texture sync memo BORROWS the binding slot: an entry holds a pointer to the
// slot's shared_ptr plus the backend twin of whatever was in it when the entry was built. Its
// keys (context id, bind-generation epoch, high-water mark, sampling generation) are the primary
// guard, but they are all derived state - so the memo also has to survive a slot swap that never
// reached them.
//
// It did not. The DSA by-name emulation swapped a slot silently, every key still matched, and
// the replay drove texture A's backend twin from texture B's frontend object: A's backend
// storage was re-specified with B's shape, destroying anything A only ever had on the GPU. On
// Espryt + Iris/BSL that blanked Minecraft's 16x16 lightmap the moment a 2048x2048 shadow map
// was uploaded through a by-name call, and since the text shader multiplies by the lightmap,
// `if (color.a < 0.1) discard` then threw away every glyph in the process - HUD, menus and the
// vanilla title screen alike.
TEST(DirectGLESTextureSync, UnitMemoRefusesToDriveATwinFromAnotherTexture) {
using namespace MobileGL;
ScopedDirectGLESTextureBindings scoped; // fresh GLContext + registry + binding caches
Vector<TexSpecCall> specs;
ScopedTexSpecRecording recording(specs);
auto functions = MG_Backend::DirectGLES::g_GLESFuncs;
functions.glBindTexture = TS_BindTexture;
functions.glActiveTexture = TS_ActiveTexture;
functions.glTexImage2D = TS_TexImage2D;
functions.glTexParameteri = TS_TexParameteri;
functions.glTexParameterf = TS_TexParameterf;
functions.glTexParameterfv = TS_TexParameterfv;
functions.glPixelStorei = TS_PixelStorei;
functions.glBindBuffer = TS_BindBuffer;
MG_Backend::DirectGLES::SetGLESFuncsTable(functions);
GLuint names[2] = {};
MG_Impl::GLImpl::GenTextures(2, names);
// `foreign` stands in for the shadow map, `resident` for the lightmap. Both are fully
// specified BEFORE the first sync so that nothing between the two syncs can move the
// sampling-resolution generation and invalidate the memo for an unrelated reason.
const auto foreign = MakeComplete2DTexture(names[1], 32);
const auto resident = MakeComplete2DTexture(names[0], 16);
ASSERT_NE(foreign, nullptr);
ASSERT_NE(resident, nullptr);
// First sync: builds the memo with unit 0 -> `resident`, and gives `resident`'s twin its
// 16x16 backend storage.
MG_Backend::DirectGLES::TextureImpl::SyncNeccessaryTextures();
auto* residentSlot = MG_Backend::DirectGLES::TextureImpl::g_backendTextureObjects.Find(resident.get());
ASSERT_NE(residentSlot, nullptr);
ASSERT_NE(*residentSlot, nullptr);
const GLuint residentBackendId = (*residentSlot)->GetBackendTextureId();
ASSERT_NE(residentBackendId, 0u);
ASSERT_FALSE(specs.empty());
EXPECT_EQ(specs.back().texture, residentBackendId);
EXPECT_EQ(specs.back().width, 16);
// The hazard, reproduced at the state level: put `foreign` on the slot the memo borrows
// WITHOUT telling the binding accounting, exactly as the by-name emulation used to.
MG_State::pGLContext->GetTextureUnitObject(0).GetBindingSlot(TextureTarget::Texture2D).Bind(foreign);
const SizeT specsBeforeReplay = specs.size();
MG_Backend::DirectGLES::TextureImpl::SyncNeccessaryTextures();
// `foreign` must have been synced through its OWN twin...
auto* foreignSlot = MG_Backend::DirectGLES::TextureImpl::g_backendTextureObjects.Find(foreign.get());
ASSERT_NE(foreignSlot, nullptr);
ASSERT_NE(*foreignSlot, nullptr);
const GLuint foreignBackendId = (*foreignSlot)->GetBackendTextureId();
EXPECT_NE(foreignBackendId, residentBackendId);
// ...and above all, nothing may have re-specified the RESIDENT texture's backend storage.
// That single call is what destroyed the lightmap.
for (SizeT i = specsBeforeReplay; i < specs.size(); ++i) {
EXPECT_NE(specs[i].texture, residentBackendId)
<< "the stale memo entry re-specified the resident texture's backend storage with "
<< specs[i].width << "x" << specs[i].height;
}
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, 0);
}
+341
View File
@@ -15,6 +15,7 @@
#include <Config.h> #include <Config.h>
#include <MG_Backend/BackendObjects.h> #include <MG_Backend/BackendObjects.h>
#include <MG_Backend/DirectGLES/Managers.h> #include <MG_Backend/DirectGLES/Managers.h>
#include <MG_Backend/DirectGLES/Utils.h>
#include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h> #include <MG_Impl/GLImpl/Framebuffer/GL_Framebuffer.h>
#include <MG_Impl/GLImpl/Getter/GL_Getter.h> #include <MG_Impl/GLImpl/Getter/GL_Getter.h>
#include <MG_Impl/GLImpl/RenderState/GL_RenderState.h> #include <MG_Impl/GLImpl/RenderState/GL_RenderState.h>
@@ -27,6 +28,7 @@
#include <MG_Util/Converters/GLToMG/TextureEnumConverter.h> #include <MG_Util/Converters/GLToMG/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToGL/TextureEnumConverter.h> #include <MG_Util/Converters/MGToGL/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToMG/TextureEnumConverter.h> #include <MG_Util/Converters/MGToMG/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToStr/TextureEnumConverter.h>
#include <MG_Util/Math/SmallFloat.h> #include <MG_Util/Math/SmallFloat.h>
#include <MG_Util/Texture/PixelStoreProcessor.h> #include <MG_Util/Texture/PixelStoreProcessor.h>
#include <MG_Util/Texture/TextureFormatProcessor.h> #include <MG_Util/Texture/TextureFormatProcessor.h>
@@ -2835,3 +2837,342 @@ TEST_F(TextureTest, BindSamplerRejectsUnitsBeyondMaxCombinedTextureImageUnits) {
MG_Impl::GLImpl::DeleteSamplers(1, &sampler); MG_Impl::GLImpl::DeleteSamplers(1, &sampler);
DrainPendingGlErrors(); DrainPendingGlErrors();
} }
// The DSA by-name entry points are emulated by temporarily binding the named texture onto the
// active unit's slot for its target, running the classic bound-texture code, then putting the
// previous binding back. For as long as the emulated call runs, that swap is a REAL change to
// which texture is bound at that unit, so both transitions have to move the texture bind
// generation.
//
// They used to move nothing. Backends memoise per-unit work keyed on the bind generation and
// BORROW the binding slot (they hold a pointer to the slot's shared_ptr, not a copy), so a memo
// built while texture A sat in the slot stayed "valid" while B was temporarily in it - and the
// backend then drove A's backend twin from B's frontend state, re-specifying A's backend storage
// with B's shape. Any content A only ever had on the GPU was gone. That is what blanked
// Minecraft's lightmap when Iris uploaded to a BSL shadow map: the text shader multiplies by the
// lightmap, so `if (color.a < 0.1) discard` then threw away every glyph in the process.
TEST_F(TextureTest, NamedTextureCallKeepsUnitBindingAccountingCoherent) {
GLuint names[2] = {};
MG_Impl::GLImpl::GenTextures(2, names);
const GLuint boundName = names[0];
const GLuint namedName = names[1];
MG_Impl::GLImpl::ActiveTexture(GL_TEXTURE0);
// Instantiate both as 2D objects, then leave `boundName` on the unit.
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, namedName);
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, boundName);
ASSERT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
auto& slot = MG_State::pGLContext->GetTextureUnitObject(0).GetBindingSlot(TextureTarget::Texture2D);
const auto boundObject = slot.GetBoundObject();
ASSERT_NE(boundObject, nullptr);
ASSERT_EQ(boundObject->GetExternalIndex(), boundName);
// TextureParameteriv is one of the by-name calls that is emulated by binding: it reaches
// WithTemporarilyBoundNamedTexture, unlike the scalar TextureParameteri, which edits the
// object directly and never touches a unit.
const Uint64 base = MG_State::pGLContext->GetTextureBindGeneration();
const GLint maxLevel = 0;
MG_Impl::GLImpl::TextureParameteriv(namedName, GL_TEXTURE_MAX_LEVEL, &maxLevel);
EXPECT_EQ(MG_Impl::GLImpl::GetError(), GL_NO_ERROR);
// The emulation put `namedName` on the unit and took it off again. A generation-keyed memo
// must be able to see that the slot it borrows was not stable across the call.
EXPECT_GT(MG_State::pGLContext->GetTextureBindGeneration(), base)
<< "a by-name texture call swapped a live unit binding without moving the bind generation";
// ...and the application-visible binding is exactly what it was before the call.
EXPECT_EQ(slot.GetBoundObject(), boundObject);
MG_Impl::GLImpl::BindTexture(GL_TEXTURE_2D, 0);
MG_Impl::GLImpl::DeleteTextures(2, names);
DrainPendingGlErrors();
}
// ---- Three-channel colour-renderable widening (Complementary Reimagined / Iris) ----------------
//
// No real OpenGL ES driver renders to a three-channel image, so a colour attachment the
// application asked for as GL_RGB8_SNORM or GL_RGB16F has to be stored in the four-channel
// sibling. The bit that says so used to be reachable for multisample storage only, which is why
// an ordinary GL_TEXTURE_2D attachment in one of those formats had no fallback at all and the
// frontend could only answer GL_FRAMEBUFFER_UNSUPPORTED.
TEST_F(TextureTest, ColorAttachableTargetsRequestTheThreeChannelWidening) {
using MobileGL::MG_Backend::DirectGLES::TextureImpl::GetRenderTargetNormalizeOptions;
using MobileGL::MG_Backend::DirectGLES::TextureImpl::TargetRequiresRenderableFormat;
MG_External::GLESCapabilities capabilities{};
capabilities.SupportsRenderSnorm = true;
capabilities.SupportsNorm16Texture = true;
// Every image that can be a colour attachment, not just the multisample pair: an ordinary 2D
// texture is what Iris attaches, and it used to be excluded.
for (const TextureTarget target : {TextureTarget::Texture2D, TextureTarget::Texture3D,
TextureTarget::TextureCubeMap, TextureTarget::Texture2DArray,
TextureTarget::TextureCubeMapArray, TextureTarget::Texture2DMultisample,
TextureTarget::Texture2DMultisampleArray, TextureTarget::Texture1D,
TextureTarget::Texture1DArray, TextureTarget::TextureRectangle}) {
const SizeT targetIndex = MobileGL::MG_Backend::GetFormatCapabilityTargetIndex(target);
EXPECT_TRUE(TargetRequiresRenderableFormat(targetIndex))
<< "target " << MG_Util::ConvertTextureTargetToString(target);
EXPECT_TRUE(GetRenderTargetNormalizeOptions(capabilities, targetIndex) &
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget)
<< "target " << MG_Util::ConvertTextureTargetToString(target);
}
// A renderbuffer exists only to be attached.
EXPECT_TRUE(TargetRequiresRenderableFormat(MobileGL::MG_Backend::GetRenderbufferFormatCapabilityTargetIndex()));
// A buffer texture is the one image that can never be an attachment; its storage belongs to
// the buffer object, so widening it would misdescribe the application's data.
const SizeT bufferIndex = MobileGL::MG_Backend::GetFormatCapabilityTargetIndex(TextureTarget::TextureBuffer);
EXPECT_FALSE(TargetRequiresRenderableFormat(bufferIndex));
EXPECT_FALSE(GetRenderTargetNormalizeOptions(capabilities, bufferIndex));
// Without EXT_render_snorm a 16-bit SNORM render target cannot keep its encoding either.
MG_External::GLESCapabilities noSnormCapabilities{};
const SizeT texture2DIndex = MobileGL::MG_Backend::GetFormatCapabilityTargetIndex(TextureTarget::Texture2D);
EXPECT_TRUE(GetRenderTargetNormalizeOptions(noSnormCapabilities, texture2DIndex) &
PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget);
EXPECT_FALSE(GetRenderTargetNormalizeOptions(capabilities, texture2DIndex) &
PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget);
}
TEST_F(TextureTest, ThreeChannelRenderTargetOptionAppliesToEveryDeniedThreeChannelFormat) {
using MG_Util::TextureFormatProcessor::GetApplicablePixelFormatNormalizeOptions;
const Flags<PixelFormatNormalizeOptionBit> requested =
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget;
// GL_RGB16F in particular matched no case at all, so no option could ever apply to it and it
// fell through NormalizePixelFormat's default passthrough unchanged.
for (const GLenum internalFormat : {GL_RGB8_SNORM, GL_RGB16_SNORM, GL_RGB16, GL_RGB10, GL_RGB12, GL_RGB16F,
GL_RGB32F, GL_SRGB8, GL_RGB8I, GL_RGB8UI, GL_RGB16I, GL_RGB16UI, GL_RGB32I,
GL_RGB32UI}) {
EXPECT_TRUE(GetApplicablePixelFormatNormalizeOptions(internalFormat, requested) &
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget)
<< "internalformat 0x" << std::hex << internalFormat;
}
// Four-channel and shared-exponent formats are not widened: RGBA8_SNORM has its own always-on
// fallback, and GL_RGB9_E5 has no four-channel sibling that would not need the shared exponent
// unpacked on every transfer (nothing renders to it on desktop GL either).
for (const GLenum internalFormat : {GL_RGBA8_SNORM, GL_RGBA16F, GL_RGBA8, GL_RGB8, GL_RGB9_E5}) {
EXPECT_FALSE(GetApplicablePixelFormatNormalizeOptions(internalFormat, requested) &
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget)
<< "internalformat 0x" << std::hex << internalFormat;
}
}
TEST_F(TextureTest, ThreeChannelWideningRetargetsInternalFormatAndTransferPairTogether) {
using MG_Util::TextureFormatProcessor::NormalizePixelFormat;
struct Case {
GLenum requested;
Flags<PixelFormatNormalizeOptionBit> options;
GLenum internalFormat;
GLenum format;
GLenum type;
};
const Flags<PixelFormatNormalizeOptionBit> widen = PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget;
const Flags<PixelFormatNormalizeOptionBit> widenNoSnorm16 =
PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget |
PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget;
const Case cases[] = {
// Complementary's colortex1 and colortex2. The transfer pair used to stay three-channel
// and keep the *source* component type, emitting (GL_RGBA16F, GL_RGB, GL_BYTE) - which ES
// rejects for glTexImage2D outright, and which only went unnoticed because the bit was
// reachable for multisample storage alone (glTexStorage*Multisample takes no pair).
{GL_RGB8_SNORM, widen, GL_RGBA16F, GL_RGBA, GL_FLOAT},
{GL_RGB16F, widen, GL_RGBA16F, GL_RGBA, GL_HALF_FLOAT},
{GL_RGB32F, widen, GL_RGBA32F, GL_RGBA, GL_FLOAT},
// 16-bit SNORM keeps its encoding where EXT_render_snorm can render to it; a half float's
// 11-bit mantissa cannot represent a 16-bit SNORM channel exactly.
{GL_RGB16_SNORM, widen, GL_RGBA16_SNORM, GL_RGBA, GL_SHORT},
{GL_RGB16_SNORM, widenNoSnorm16, GL_RGBA16F, GL_RGBA, GL_FLOAT},
// 16-bit UNORM and the legacy 10/12-bit formats stored as RGB16.
{GL_RGB16, widen, GL_RGBA32F, GL_RGBA, GL_FLOAT},
{GL_RGB10, widen, GL_RGBA32F, GL_RGBA, GL_FLOAT},
{GL_RGB12, widen, GL_RGBA32F, GL_RGBA, GL_FLOAT},
// sRGB and the integer formats: the base format has to move to the four-channel one of the
// right class, GL_RGBA_INTEGER included.
{GL_SRGB8, widen, GL_SRGB8_ALPHA8, GL_RGBA, GL_UNSIGNED_BYTE},
{GL_RGB8I, widen, GL_RGBA8I, GL_RGBA_INTEGER, GL_BYTE},
{GL_RGB8UI, widen, GL_RGBA8UI, GL_RGBA_INTEGER, GL_UNSIGNED_BYTE},
{GL_RGB16I, widen, GL_RGBA16I, GL_RGBA_INTEGER, GL_SHORT},
{GL_RGB16UI, widen, GL_RGBA16UI, GL_RGBA_INTEGER, GL_UNSIGNED_SHORT},
{GL_RGB32I, widen, GL_RGBA32I, GL_RGBA_INTEGER, GL_INT},
{GL_RGB32UI, widen, GL_RGBA32UI, GL_RGBA_INTEGER, GL_UNSIGNED_INT},
// The widening outranks the other fallbacks, which all pick a three-channel storage the
// driver still refuses to render to (GL_RGB8_SNORM -> GL_RGB16F, GL_RGB16 -> GL_RGB32F).
{GL_RGB8_SNORM, widen | PixelFormatNormalizeOptionBit::NoSnorm8, GL_RGBA16F, GL_RGBA, GL_FLOAT},
{GL_RGB16, widen | PixelFormatNormalizeOptionBit::NoNorm16, GL_RGBA32F, GL_RGBA, GL_FLOAT},
// Control: without the bit nothing moves. The bit is only ever set for a target whose
// native probe failed, so this is the shape every driver that does render to the
// three-channel form keeps - per format, not per platform (llvmpipe renders to GL_RGB16F
// but not to GL_RGB8_SNORM, GL_SRGB8, GL_RGB32F or the RGB integer formats).
{GL_RGB8_SNORM, PixelFormatNormalizeOptionBit::None, GL_RGB8_SNORM, GL_RGB, GL_BYTE},
{GL_RGB16F, PixelFormatNormalizeOptionBit::None, GL_RGB16F, GL_RGB, GL_HALF_FLOAT},
{GL_RGB32F, PixelFormatNormalizeOptionBit::None, GL_RGB32F, GL_RGB, GL_FLOAT},
{GL_SRGB8, PixelFormatNormalizeOptionBit::None, GL_SRGB8, GL_RGB, GL_UNSIGNED_BYTE},
// Not widened even under the bit: no four-channel shared-exponent sibling exists.
{GL_RGB9_E5, widen, GL_RGB9_E5, GL_RGB, GL_UNSIGNED_INT_5_9_9_9_REV},
// Four-channel formats are unaffected by the bit; RGBA8_SNORM keeps its own fallback.
{GL_RGBA8_SNORM, widen, GL_RGBA8_SNORM, GL_RGBA, GL_BYTE},
{GL_RGBA8_SNORM, widen | PixelFormatNormalizeOptionBit::NoRGBA8Snorm, GL_RGBA16F, GL_RGBA, GL_FLOAT},
};
for (const auto& testCase : cases) {
GLenum internalFormat = 0;
GLenum format = 0;
GLenum type = 0;
NormalizePixelFormat(testCase.requested, testCase.options, &internalFormat, &format, &type);
EXPECT_EQ(internalFormat, testCase.internalFormat) << "requested 0x" << std::hex << testCase.requested;
EXPECT_EQ(format, testCase.format) << "requested 0x" << std::hex << testCase.requested;
EXPECT_EQ(type, testCase.type) << "requested 0x" << std::hex << testCase.requested;
}
}
TEST_F(TextureTest, WidenedRenderTargetUploadExpandsThreeChannelDataWithOpaqueAlpha) {
using MobileGL::MG_Backend::DirectGLES::TextureImpl::GetWidenableClientComponentCount;
using MobileGL::MG_Backend::DirectGLES::TextureImpl::PrepareChannelWidenedUpload;
// Only the three-channel formats that can be widened report a source component count; the
// repack is what keeps the driver from walking three texels' worth of data per four-texel row.
for (const TextureInternalFormat format :
{TextureInternalFormat::RGB8Snorm, TextureInternalFormat::RGB16F, TextureInternalFormat::RGB32F,
TextureInternalFormat::RGB16Snorm, TextureInternalFormat::RGB16, TextureInternalFormat::SRGB8,
TextureInternalFormat::RGB8UI, TextureInternalFormat::RGB32I}) {
EXPECT_EQ(GetWidenableClientComponentCount(format), 3u)
<< MG_Util::ConvertTextureInternalFormatToString(format);
}
EXPECT_EQ(GetWidenableClientComponentCount(TextureInternalFormat::RGBA8), 0u);
EXPECT_EQ(GetWidenableClientComponentCount(TextureInternalFormat::RGBA8Snorm), 0u);
EXPECT_EQ(GetWidenableClientComponentCount(TextureInternalFormat::RGB9E5), 0u);
const IntVec3 texelSize(2, 1, 1);
// GL_RGB8_SNORM -> GL_RGBA16F: PrepareNormFloatFallbackUpload has already turned the Int8
// shadow into floats, so what arrives here is three floats per texel.
{
const Float source[] = {0.25f, -0.5f, 0.75f, -1.0f, 0.0f, 1.0f};
Vector<Uint8> widened;
const auto* result = static_cast<const Float*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_FLOAT, widened));
ASSERT_NE(result, static_cast<const void*>(source));
ASSERT_EQ(widened.size(), 8 * sizeof(Float));
const Float expected[] = {0.25f, -0.5f, 0.75f, 1.0f, -1.0f, 0.0f, 1.0f, 1.0f};
for (SizeT i = 0; i < 8; ++i) {
EXPECT_FLOAT_EQ(result[i], expected[i]) << "component " << i;
}
}
// GL_RGB16F -> GL_RGBA16F uploads halves untouched, so the synthetic alpha is the half
// encoding of 1.0 rather than a saturated field.
{
const Uint16 source[] = {0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006};
Vector<Uint8> widened;
const auto* result = static_cast<const Uint16*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_HALF_FLOAT, widened));
ASSERT_NE(result, static_cast<const void*>(source));
const Uint16 expected[] = {0x0001, 0x0002, 0x0003, 0x3C00, 0x0004, 0x0005, 0x0006, 0x3C00};
for (SizeT i = 0; i < 8; ++i) {
EXPECT_EQ(result[i], expected[i]) << "component " << i;
}
}
// GL_SRGB8 -> GL_SRGB8_ALPHA8: fixed-point one is the saturated field.
{
const Uint8 source[] = {1, 2, 3, 4, 5, 6};
Vector<Uint8> widened;
const auto* result = static_cast<const Uint8*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_UNSIGNED_BYTE, widened));
const Uint8 expected[] = {1, 2, 3, 0xFF, 4, 5, 6, 0xFF};
ASSERT_NE(result, static_cast<const void*>(source));
EXPECT_EQ(std::memcmp(result, expected, sizeof(expected)), 0);
}
// GL_RGB16_SNORM -> GL_RGBA16_SNORM keeps GL_SHORT, whose 1.0 is the positive maximum.
{
const Int16 source[] = {-1, 2, -3, 4, -5, 6};
Vector<Uint8> widened;
const auto* result = static_cast<const Int16*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_SHORT, widened));
const Int16 expected[] = {-1, 2, -3, 0x7FFF, 4, -5, 6, 0x7FFF};
ASSERT_NE(result, static_cast<const void*>(source));
EXPECT_EQ(std::memcmp(result, expected, sizeof(expected)), 0);
}
// An integer format's added channel carries the integer one, not a saturated field.
{
const Uint32 source[] = {10, 20, 30, 40, 50, 60};
Vector<Uint8> widened;
const auto* result = static_cast<const Uint32*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_UNSIGNED_INT, widened, /*integerData=*/true));
const Uint32 expected[] = {10, 20, 30, 1, 40, 50, 60, 1};
ASSERT_NE(result, static_cast<const void*>(source));
EXPECT_EQ(std::memcmp(result, expected, sizeof(expected)), 0);
}
// GL_RGB8I -> GL_RGBA8I uploads as GL_BYTE, the very type GL_RGB8_SNORM uses, so the type
// alone cannot decide the added channel's value: the integer format's one is 1, the
// signed-normalized format's is 0x7F. Getting this wrong is invisible through sampling and
// glGetTexImage (both answer the alpha with the format's implied one) but escapes through a
// blit or glCopyTexSubImage out of the widened attachment.
{
const Int8 source[] = {-1, 2, -3, 4, -5, 6};
Vector<Uint8> widened;
const auto* asInteger = static_cast<const Int8*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_BYTE, widened, /*integerData=*/true));
const Int8 expectedInteger[] = {-1, 2, -3, 1, 4, -5, 6, 1};
ASSERT_NE(asInteger, static_cast<const void*>(source));
EXPECT_EQ(std::memcmp(asInteger, expectedInteger, sizeof(expectedInteger)), 0);
Vector<Uint8> widenedNorm;
const auto* asNormalized = static_cast<const Int8*>(PrepareChannelWidenedUpload(
3, texelSize, source, sizeof(source), GL_BYTE, widenedNorm, /*integerData=*/false));
const Int8 expectedNormalized[] = {-1, 2, -3, 0x7F, 4, -5, 6, 0x7F};
EXPECT_EQ(std::memcmp(asNormalized, expectedNormalized, sizeof(expectedNormalized)), 0);
}
// Which class a widenable format belongs to.
for (const TextureInternalFormat format :
{TextureInternalFormat::RGB8I, TextureInternalFormat::RGB8UI, TextureInternalFormat::RGB16I,
TextureInternalFormat::RGB16UI, TextureInternalFormat::RGB32I, TextureInternalFormat::RGB32UI}) {
EXPECT_TRUE(MobileGL::MG_Backend::DirectGLES::TextureImpl::IsIntegerWidenableFormat(format))
<< MG_Util::ConvertTextureInternalFormatToString(format);
}
for (const TextureInternalFormat format :
{TextureInternalFormat::RGB8Snorm, TextureInternalFormat::RGB16Snorm, TextureInternalFormat::RGB16,
TextureInternalFormat::RGB16F, TextureInternalFormat::RGB32F, TextureInternalFormat::SRGB8}) {
EXPECT_FALSE(MobileGL::MG_Backend::DirectGLES::TextureImpl::IsIntegerWidenableFormat(format))
<< MG_Util::ConvertTextureInternalFormatToString(format);
}
// The destination is sized from the level, never from the source. The driver reads a full
// width*height*4 components for the transfer it was handed, so a short source must still
// leave a full buffer behind - sizing it from the source would hand the driver a buffer it
// runs off the end of.
{
const Float shortSource[] = {0.5f, 0.25f, 0.125f};
Vector<Uint8> widened;
const auto* result = static_cast<const Float*>(PrepareChannelWidenedUpload(
3, IntVec3(2, 2, 1), shortSource, sizeof(shortSource), GL_FLOAT, widened));
ASSERT_NE(result, static_cast<const void*>(shortSource));
ASSERT_EQ(widened.size(), 4 * 4 * sizeof(Float));
const Float expected[] = {0.5f, 0.25f, 0.125f, 1.0f, 0.0f, 0.0f, 0.0f, 1.0f,
0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 1.0f};
for (SizeT i = 0; i < 16; ++i) {
EXPECT_FLOAT_EQ(result[i], expected[i]) << "component " << i;
}
}
// No widening in effect (or nothing to convert): the caller's pointer comes straight back, so
// the sub-rect upload fast path still recognises an unconverted level.
{
const Float source[] = {1.0f, 2.0f, 3.0f, 4.0f};
Vector<Uint8> widened;
EXPECT_EQ(PrepareChannelWidenedUpload(4, texelSize, source, sizeof(source), GL_FLOAT, widened),
static_cast<const void*>(source));
EXPECT_EQ(PrepareChannelWidenedUpload(0, texelSize, source, sizeof(source), GL_FLOAT, widened),
static_cast<const void*>(source));
EXPECT_EQ(PrepareChannelWidenedUpload(3, texelSize, nullptr, 0, GL_FLOAT, widened), nullptr);
}
}
+368
View File
@@ -0,0 +1,368 @@
// MobileGL - MobileGL/MG_Test/Util/AsyncPoolBench.cpp
// Copyright (c) 2025-2026 MobileGL-Dev
// Licensed under the GNU Lesser General Public License v3.0:
// https://www.gnu.org/licenses/gpl-3.0.txt
// https://www.gnu.org/licenses/lgpl-3.0.txt
// SPDX-License-Identifier: LGPL-3.0-only
// End of Source File Header
// A head-to-head harness for the two ShaderCompilePool execution engines
// (MOBILEGL_ASYNC_POOL=asio|libfork). Not a gtest: it measures one wall-clock interval per
// process, because most of what it drives is memoized per process (the shader preprocess
// cache and the compile-adoption map both live for the life of the GL context), so a second
// timed repetition inside one process would measure the cache, not the compiler. The driver
// script re-executes the binary for every repetition instead.
//
// Two modes:
//
// corpus - the REAL frontend path. glCreateShader/glShaderSource are done untimed, then
// the clock starts and glCompileShader/glLinkProgram submit every job, and stops
// once glGetProgramiv(GL_LINK_STATUS) has joined all of them. That is exactly the
// first-submit-to-all-joined interval a shaderpack load pays.
//
// micro - N trivial JobNodes straight through ShaderCompilePool::Post, isolating the
// executor's own dispatch overhead from any workload contention.
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <filesystem>
#include <fstream>
#include <sstream>
#include <string>
#include <vector>
#include "Includes.h"
#include "Init.h"
#include <Config.h>
#include <MG_Impl/GLImpl/Program/GL_Program.h>
#include <MG_Util/Async/JobNode.h>
#include <MG_Util/Async/ShaderCompilePool.h>
using namespace MobileGL;
using namespace MobileGL::MG_Util::Async;
namespace GLImpl = MobileGL::MG_Impl::GLImpl;
namespace fs = std::filesystem;
namespace {
using Clock = std::chrono::steady_clock;
double MillisSince(const Clock::time_point start) {
return std::chrono::duration<double, std::milli>(Clock::now() - start).count();
}
GLenum StageFromExtension(const std::string& ext) {
if (ext == ".vert") return GL_VERTEX_SHADER;
if (ext == ".frag") return GL_FRAGMENT_SHADER;
if (ext == ".geom") return GL_GEOMETRY_SHADER;
if (ext == ".comp") return GL_COMPUTE_SHADER;
if (ext == ".tesc") return GL_TESS_CONTROL_SHADER;
if (ext == ".tese") return GL_TESS_EVALUATION_SHADER;
return 0;
}
std::string ReadFile(const fs::path& path) {
std::ifstream in(path, std::ios::binary);
std::ostringstream buf;
buf << in.rdbuf();
return buf.str();
}
struct CorpusShader {
std::string name;
std::string source;
GLenum stage = 0;
};
// One program's worth of the corpus: the trace's link group. Shaders are indices into
// the flat shader list, because a source shared by several programs must stay ONE entry
// - that sharing is what the compile-adoption map sees in the real path too.
struct CorpusProgram {
std::vector<SizeT> shaders;
};
struct Corpus {
std::vector<CorpusShader> shaders;
std::vector<CorpusProgram> programs;
SizeT totalBytes = 0;
};
// Reads a corpus directory written by extract_corpus.py: one file per compiled shader,
// stage in the extension, plus manifest.txt naming the trace's link groups.
Corpus LoadCorpus(const fs::path& dir) {
Corpus corpus;
std::unordered_map<std::string, SizeT> byName;
const auto intern = [&](const std::string& name) -> SizeT {
if (const auto it = byName.find(name); it != byName.end()) return it->second;
const fs::path path = dir / name;
if (!fs::exists(path)) return static_cast<SizeT>(-1);
CorpusShader shader;
shader.name = name;
shader.source = ReadFile(path);
shader.stage = StageFromExtension(path.extension().string());
if (shader.stage == 0) return static_cast<SizeT>(-1);
corpus.totalBytes += shader.source.size();
corpus.shaders.push_back(Move(shader));
const SizeT index = corpus.shaders.size() - 1;
byName.emplace(name, index);
return index;
};
const fs::path manifest = dir / "manifest.txt";
if (fs::exists(manifest)) {
std::ifstream in(manifest);
std::string line;
while (std::getline(in, line)) {
if (line.empty() || line[0] == '#') continue;
CorpusProgram program;
std::istringstream fields(line);
std::string name;
while (fields >> name) {
const SizeT index = intern(name);
if (index != static_cast<SizeT>(-1)) program.shaders.push_back(index);
}
if (!program.shaders.empty()) corpus.programs.push_back(Move(program));
}
}
// Anything in the directory the manifest never linked still gets compiled, as a
// program-less group, so the corpus on disk and the corpus measured are the same set.
std::vector<fs::path> leftovers;
for (const auto& entry : fs::directory_iterator(dir)) {
if (!entry.is_regular_file()) continue;
const std::string name = entry.path().filename().string();
if (name == "manifest.txt") continue;
if (StageFromExtension(entry.path().extension().string()) == 0) continue;
if (byName.count(name) != 0) continue;
leftovers.push_back(entry.path());
}
std::sort(leftovers.begin(), leftovers.end());
for (const auto& path : leftovers) intern(path.filename().string());
return corpus;
}
struct CorpusResult {
double submitMs = 0; // first glCompileShader -> last glLinkProgram returned
double joinMs = 0; // last submit -> every program joined
double totalMs = 0; // the number that matters: first submit -> all joined
SizeT linkFailures = 0;
SizeT compileFailures = 0;
};
CorpusResult RunCorpus(const Corpus& corpus) {
// ---- Untimed: create every GL object and stage every source ----------------------
// glShaderSource is a memcpy into the shader object and glAttachShader is a pointer
// append; neither touches the pool. Keeping them outside the clock makes the measured
// interval exactly the compile+link critical path, which is what an application's
// loading screen waits on.
std::vector<GLuint> shaderNames(corpus.shaders.size(), 0);
for (SizeT i = 0; i < corpus.shaders.size(); ++i) {
const CorpusShader& shader = corpus.shaders[i];
const GLuint name = GLImpl::CreateShader(shader.stage);
const GLchar* text = shader.source.c_str();
const GLint length = static_cast<GLint>(shader.source.size());
GLImpl::ShaderSource(name, 1, &text, &length);
shaderNames[i] = name;
}
std::vector<GLuint> programNames(corpus.programs.size(), 0);
for (SizeT p = 0; p < corpus.programs.size(); ++p) {
const GLuint program = GLImpl::CreateProgram();
for (const SizeT shaderIndex : corpus.programs[p].shaders) {
GLImpl::AttachShader(program, shaderNames[shaderIndex]);
}
programNames[p] = program;
}
// ---- Timed ------------------------------------------------------------------------
const Clock::time_point start = Clock::now();
// Submission order follows the trace: a program's shaders, then its link. That order
// is what exercises ProgramLinkTask::SubmitAfter's dependency chaining rather than a
// flat burst of independent compiles.
std::vector<Bool> submitted(corpus.shaders.size(), false);
for (SizeT p = 0; p < corpus.programs.size(); ++p) {
for (const SizeT shaderIndex : corpus.programs[p].shaders) {
if (submitted[shaderIndex]) continue;
submitted[shaderIndex] = true;
GLImpl::CompileShader(shaderNames[shaderIndex]);
}
GLImpl::LinkProgram(programNames[p]);
}
for (SizeT i = 0; i < corpus.shaders.size(); ++i) {
if (submitted[i]) continue;
submitted[i] = true;
GLImpl::CompileShader(shaderNames[i]);
}
const Clock::time_point submitted_at = Clock::now();
CorpusResult result;
// GL_LINK_STATUS is a joining query (GL_COMPLETION_STATUS_KHR is the one that must
// not join), so this loop is the all-joined barrier.
for (const GLuint program : programNames) {
GLint status = 0;
GLImpl::GetProgramiv(program, GL_LINK_STATUS, &status);
if (status == GL_FALSE) ++result.linkFailures;
}
for (const GLuint shader : shaderNames) {
GLint status = 0;
GLImpl::GetShaderiv(shader, GL_COMPILE_STATUS, &status);
if (status == GL_FALSE) ++result.compileFailures;
}
result.totalMs = MillisSince(start);
result.submitMs = std::chrono::duration<double, std::milli>(submitted_at - start).count();
result.joinMs = result.totalMs - result.submitMs;
for (const GLuint program : programNames) GLImpl::DeleteProgram(program);
for (const GLuint shader : shaderNames) GLImpl::DeleteShader(shader);
return result;
}
// ---- Executor microbenchmark ----------------------------------------------------------
// The body is deliberately near-empty: what is being measured is Post -> engine ->
// RunOnWorker -> next dispatch, i.e. the executor's own cost per job, with no compiler
// work to hide it.
//
// The barrier is an all-jobs-ran latch, and it has to be. This bench used to stop the
// clock at StopAndDrain(), which is not a "wait for everything" - it is the teardown path,
// and its contract is to ABANDON whatever the budget has not dispatched yet (see
// ShaderCompilePool::StopAndDrain, and the JobNodeTest case that pins exactly that). With
// 100k jobs behind a budget of N, most of them were therefore cancelled rather than run,
// and the fraction that survived was decided by how fast the engine drained the queue
// relative to the posting loop - i.e. by the very quantity under test. Measured on this
// machine at 8 workers: Asio ran 75,906 of 100,000 and libfork 99,998, and both were
// scored as if they had run 100,000. The reported "libfork is 1.36x faster" was libfork
// being charged for 32% more work than Asio.
class TrivialJob final : public JobNode {
public:
TrivialJob(std::atomic<Uint64>* sink, const Uint64 total, std::mutex* mutex,
std::condition_variable* cv)
: m_sink(sink), m_total(total), m_mutex(mutex), m_cv(cv) {}
private:
void RunBody() override {
if (m_sink->fetch_add(1, std::memory_order_acq_rel) + 1 == m_total) {
// The last job wakes the timer. Under the lock, so the waiter cannot miss it
// between its predicate check and its wait.
const std::lock_guard<std::mutex> lock(*m_mutex);
m_cv->notify_all();
}
}
std::atomic<Uint64>* m_sink;
Uint64 m_total;
std::mutex* m_mutex;
std::condition_variable* m_cv;
};
struct MicroResult {
double ms = 0;
Uint64 ran = 0;
};
MicroResult RunMicrobench(const Uint threads, const SizeT jobs) {
ShaderCompilePool pool(threads);
std::atomic<Uint64> counter{0};
std::mutex mutex;
std::condition_variable cv;
const auto total = static_cast<Uint64>(jobs);
// Nodes are allocated up front: MakeShared is not what is under test, and leaving it
// inside the loop would put an allocator on the critical path in front of the
// dispatch path this is meant to isolate.
std::vector<SharedPtr<JobNode>> nodes;
nodes.reserve(jobs);
for (SizeT i = 0; i < jobs; ++i) {
nodes.push_back(MakeShared<TrivialJob>(&counter, total, &mutex, &cv));
}
const Clock::time_point start = Clock::now();
for (auto& node : nodes) pool.Post(Move(node));
{
std::unique_lock<std::mutex> lock(mutex);
cv.wait(lock, [&] { return counter.load(std::memory_order_acquire) >= total; });
}
const double ms = MillisSince(start);
MicroResult result;
result.ms = ms;
result.ran = counter.load(std::memory_order_acquire);
return result;
}
[[noreturn]] void Usage() {
std::fprintf(stderr,
"usage: AsyncPoolBench --corpus DIR\n"
" AsyncPoolBench --micro JOBS --threads N\n"
"env: MOBILEGL_ASYNC_POOL=asio|libfork, "
"MOBILEGL_ASYNC_SHADER_COMPILE_THREADS=N\n");
std::exit(2);
}
} // namespace
int main(int argc, char** argv) {
std::string corpusDir;
SizeT microJobs = 0;
Uint microThreads = 0;
for (int i = 1; i < argc; ++i) {
const std::string arg = argv[i];
const auto next = [&]() -> std::string {
if (i + 1 >= argc) Usage();
return argv[++i];
};
if (arg == "--corpus") corpusDir = next();
else if (arg == "--micro") microJobs = static_cast<SizeT>(std::stoull(next()));
else if (arg == "--threads") microThreads = static_cast<Uint>(std::stoul(next()));
else Usage();
}
if (corpusDir.empty() && microJobs == 0) Usage();
Initialize();
const AsyncPoolEngine engine = DetectAsyncPoolEngine();
const char* engineName = AsyncPoolEngineName(engine);
if (microJobs != 0) {
const Uint threads = microThreads != 0 ? microThreads : DetectShaderCompileThreadCount();
const MicroResult result = RunMicrobench(threads, microJobs);
// `ran` is printed, not just checked, so that a run in which the arms did different
// amounts of work is visible in the results file rather than on a stderr the driver
// script redirects to /dev/null. ns_per_job divides by what actually ran.
std::printf("RESULT mode=micro engine=%s threads=%u jobs=%zu ran=%llu total_ms=%.3f "
"ns_per_job=%.1f\n",
engineName, threads, microJobs, static_cast<unsigned long long>(result.ran),
result.ms, result.ms * 1e6 / static_cast<double>(result.ran));
return result.ran == microJobs ? 0 : 1;
}
const Corpus corpus = LoadCorpus(corpusDir);
if (corpus.shaders.empty()) {
std::fprintf(stderr, "AsyncPoolBench: no shaders found in %s\n", corpusDir.c_str());
return 1;
}
if (!AsyncShaderCompileActive()) {
std::fprintf(stderr, "AsyncPoolBench: asynchronous compilation is OFF; measuring the "
"inline path\n");
}
const CorpusResult result = RunCorpus(corpus);
const Uint threads = ShaderCompilePool::Get().GetThreadCount();
std::printf("RESULT mode=corpus engine=%s threads=%u corpus=%s shaders=%zu programs=%zu "
"bytes=%zu total_ms=%.3f submit_ms=%.3f join_ms=%.3f link_fail=%zu "
"compile_fail=%zu\n",
engineName, threads, corpusDir.c_str(), corpus.shaders.size(),
corpus.programs.size(), corpus.totalBytes, result.totalMs, result.submitMs,
result.joinMs, result.linkFailures, result.compileFailures);
return 0;
}
+25 -1
View File
@@ -10,11 +10,35 @@ target_include_directories(JobNodeTest PRIVATE
${MGL_ROOT}/MobileGL ${MGL_ROOT}/MobileGL
) )
# GTest::gtest, not GTest::gtest_main: JobNodeTest supplies its own main so that
# MOBILEGL_LOG_FILE_PATH is set before the first log write in the process. The engine
# -selection cases read the log back to assert that an unrecognized MOBILEGL_ASYNC_POOL value
# warns, and the desktop log sink is the file (MOBILEGL_LOG_ENABLE_CONSOLE is 0).
target_link_libraries( target_link_libraries(
JobNodeTest PRIVATE JobNodeTest PRIVATE
GTest::gtest_main GTest::gtest
${LINK_LIBRARIES} ${LINK_LIBRARIES}
) )
include(GoogleTest) include(GoogleTest)
gtest_discover_tests(JobNodeTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit) gtest_discover_tests(JobNodeTest DISCOVERY_TIMEOUT 30 PROPERTIES LABELS unit)
# The engine comparison harness. Deliberately NOT registered with add_test: it measures wall
# time, so it has no pass/fail verdict to give CI, and it is driven by a script that varies
# MOBILEGL_ASYNC_POOL and MOBILEGL_ASYNC_SHADER_COMPILE_THREADS across a matrix. It lives
# beside JobNodeTest because it drives the same pool through the same two engines; it links
# MobileGL_s for the real glCompileShader/glLinkProgram frontend path.
add_executable(
AsyncPoolBench
AsyncPoolBench.cpp
)
target_include_directories(AsyncPoolBench PRIVATE
${MGL_ROOT}/include
${MGL_ROOT}/MobileGL
)
target_link_libraries(
AsyncPoolBench PRIVATE
${LINK_LIBRARIES}
)
+344 -19
View File
@@ -9,8 +9,19 @@
#include <gtest/gtest.h> #include <gtest/gtest.h>
#include <chrono> #include <chrono>
#include <cstdlib>
#include <filesystem>
#include <fstream>
#include <stdexcept> #include <stdexcept>
#ifdef _WIN32
#include <process.h>
#define MGL_TEST_GETPID _getpid
#else
#include <unistd.h>
#define MGL_TEST_GETPID getpid
#endif
#include "Includes.h" #include "Includes.h"
#include <Config.h> #include <Config.h>
@@ -21,6 +32,29 @@ using namespace MobileGL;
using namespace MobileGL::MG_Util::Async; using namespace MobileGL::MG_Util::Async;
namespace { namespace {
// Where this binary's MobileGL log lands, set by main() below. The engine-selection cases
// read it back: MobileGL's desktop log sink is the FILE, not the console
// (MOBILEGL_LOG_ENABLE_CONSOLE is 0 in Defines.h), so gtest's stdout capture would see
// nothing, and "unrecognized value warns" is a contract worth pinning rather than
// assuming - a silent fallback makes a misspelt engine name look exactly like an unset
// variable.
String g_logFilePath;
// Log.cpp flushes the file after every line, so everything written before this call is
// already visible.
String ReadLogFrom(const std::streamoff offset) {
std::ifstream file(g_logFilePath, std::ios::binary);
if (!file) return {};
file.seekg(offset);
return String((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
}
std::streamoff LogSize() {
std::error_code error;
const auto size = std::filesystem::file_size(g_logFilePath, error);
return error ? 0 : static_cast<std::streamoff>(size);
}
// Every test drives its own pool instance rather than ShaderCompilePool::Get(): the // Every test drives its own pool instance rather than ShaderCompilePool::Get(): the
// process-wide pool is stopped permanently by StopAndDrain (that is the teardown // process-wide pool is stopped permanently by StopAndDrain (that is the teardown
// contract), so a test that drained the singleton would poison every test after it. // contract), so a test that drained the singleton would poison every test after it.
@@ -73,6 +107,21 @@ namespace {
Bool m_open = false; Bool m_open = false;
}; };
// Live thread count of this process. Linux only - /proc/self/task has one entry per
// thread - and 0 where that is not available, which is how the one case that uses it
// decides to skip rather than to assert something it cannot see.
SizeT LiveThreadCount() {
#ifdef __linux__
std::error_code error;
const auto count = static_cast<SizeT>(
std::distance(std::filesystem::directory_iterator("/proc/self/task", error),
std::filesystem::directory_iterator()));
return error ? 0 : count;
#else
return 0;
#endif
}
Bool WaitUntil(const std::function<Bool()>& predicate, Bool WaitUntil(const std::function<Bool()>& predicate,
const std::chrono::milliseconds timeout = std::chrono::seconds(10)) { const std::chrono::milliseconds timeout = std::chrono::seconds(10)) {
const auto deadline = std::chrono::steady_clock::now() + timeout; const auto deadline = std::chrono::steady_clock::now() + timeout;
@@ -89,11 +138,33 @@ namespace {
// --------------------------------------------------------------------------------------- // ---------------------------------------------------------------------------------------
TEST(ShaderCompilePoolLifecycle, ConstructingAPoolStartsNoThreadUntilSomethingIsPosted) { TEST(ShaderCompilePoolLifecycle, ConstructingAPoolStartsNoThreadUntilSomethingIsPosted) {
const SizeT before = LiveThreadCount();
ShaderCompilePool pool(kTestThreads); ShaderCompilePool pool(kTestThreads);
EXPECT_EQ(pool.GetThreadCount(), kTestThreads); EXPECT_EQ(pool.GetThreadCount(), kTestThreads);
EXPECT_EQ(pool.GetMaxConcurrency(), kTestThreads); EXPECT_EQ(pool.GetMaxConcurrency(), kTestThreads);
// Nothing observable to assert about thread creation from here; what this pins is that
// construction is side-effect free and the pool destructs cleanly without ever running. if (before == 0) {
// No thread census on this platform. The rest still holds: construction is
// side-effect free and the pool destructs cleanly without ever having run.
SUCCEED();
return;
}
// "A build that never posts pays nothing" is a real requirement, not a stylistic one -
// asynchronous compilation can be switched off entirely, and a switched-off pool that
// still spawned its workers would cost every such process its threads and their stacks.
// Worth asserting rather than asserting-by-comment now that an engine's thread shape is
// selectable: the libfork engine starts its workers AND a dispatch thread of its own, so
// a regression here would cost more than it used to.
EXPECT_EQ(LiveThreadCount(), before) << "constructing a pool started " << (LiveThreadCount() - before)
<< " thread(s) before anything was posted";
auto job = MakeShared<TestJob>();
pool.Post(job);
job->Wait();
EXPECT_GT(LiveThreadCount(), before) << "the first Post started no thread at all, so the engine did not "
"really run the job off the calling thread";
} }
TEST(ShaderCompilePoolLifecycle, StopAndDrainIsIdempotentAndSafeOnAnUnusedPool) { TEST(ShaderCompilePoolLifecycle, StopAndDrainIsIdempotentAndSafeOnAnUnusedPool) {
@@ -217,6 +288,94 @@ TEST(JobNodeSubmit, ManyJobsAllComplete) {
} }
} }
TEST(JobNodeSubmit, AJobBodyMayPostAnotherJobToTheSamePool) {
// The ProgramLinkTask::SubmitAfter shape, reduced to its scheduling core: the dependent is
// posted by whichever thread drove the dependency terminal, which for a job that finished
// on a worker is that WORKER. Every engine therefore has to accept a submission from
// inside its own pool.
//
// Not a hypothetical: libfork refuses this outright at its normal entry point
// (lf::schedule throws lf::schedule_in_worker, because a libfork worker may never block),
// which is why the libfork engine owns a dispatch thread of its own. Without this case a
// naive port passes every other test in the file and turns every dependency-released link
// job into a cancelled one on the real GL path.
ShaderCompilePool pool(kTestThreads);
std::atomic<Bool> innerSawPoolThread{false};
auto inner = MakeShared<TestJob>(
[&](TestJob&) { innerSawPoolThread.store(ShaderCompilePool::IsPoolThread(), std::memory_order_release); });
std::atomic<Bool> postedFromPoolThread{false};
auto outer = MakeShared<TestJob>([&](TestJob&) {
postedFromPoolThread.store(ShaderCompilePool::IsPoolThread(), std::memory_order_release);
pool.Post(inner);
});
pool.Post(outer);
outer->Wait();
inner->Wait();
EXPECT_TRUE(postedFromPoolThread.load()) << "the outer body did not run on a pool thread, so this case "
"did not exercise posting from inside the pool";
EXPECT_TRUE(outer->IsComplete());
// The load-bearing one: the inner job RAN. A dispatch the engine refused would have
// settled it Cancelled instead, and its body would never have executed.
EXPECT_TRUE(inner->IsComplete()) << "a job posted from a pool thread was not dispatched";
EXPECT_FALSE(inner->IsCancelled());
EXPECT_EQ(inner->ran.load(), 1u);
EXPECT_TRUE(innerSawPoolThread.load());
}
TEST(JobNodeSubmit, ABurstPostedFromInsideThePoolStillRunsInParallel) {
// The tail of a pack load: one compile job goes terminal and its continuations release
// several programs at once (ShaderCompileAdoptionMap lets one compile settle many), so a
// WORKER posts a burst into a pool that is otherwise idle. Every one of those posts clears
// the budget immediately, so the engine is handed `kBurst` runnable jobs from inside
// itself - and it has to spread them, not run them one behind another on the thread that
// submitted them.
//
// Asserting on peak concurrency rather than on wall time: the budget is the contract, and
// an engine that dispatches within the budget but executes serially has silently turned
// the budget into an upper bound nothing reaches.
constexpr Uint kBurst = 4; // == kTestThreads, so the budget can hold all of them at once
ShaderCompilePool pool(kTestThreads);
std::atomic<Uint> live{0};
std::atomic<Uint> peak{0};
std::atomic<Uint> finished{0};
Vector<SharedPtr<TestJob>> burst;
burst.reserve(kBurst);
for (Uint i = 0; i < kBurst; ++i) {
burst.push_back(MakeShared<TestJob>([&](TestJob&) {
const Uint now = live.fetch_add(1, std::memory_order_acq_rel) + 1;
Uint seen = peak.load(std::memory_order_acquire);
while (now > seen && !peak.compare_exchange_weak(seen, now, std::memory_order_acq_rel)) {
}
// Long enough that a serial engine cannot fake overlap, short enough to keep the
// case cheap: with any real spread every body is inside this window together.
std::this_thread::sleep_for(std::chrono::milliseconds(120));
live.fetch_sub(1, std::memory_order_acq_rel);
finished.fetch_add(1, std::memory_order_acq_rel);
}));
}
std::atomic<Bool> postedFromPoolThread{false};
auto seeder = MakeShared<TestJob>([&](TestJob&) {
postedFromPoolThread.store(ShaderCompilePool::IsPoolThread(), std::memory_order_release);
for (const auto& job : burst) pool.Post(job);
});
pool.Post(seeder);
seeder->Wait();
for (const auto& job : burst) job->Wait();
ASSERT_TRUE(postedFromPoolThread.load()) << "the burst was not posted from a pool thread";
EXPECT_EQ(finished.load(), kBurst);
EXPECT_GT(peak.load(), 1u) << "a burst posted from inside the pool ran strictly one at a time; the "
"engine serialized work the budget had already cleared";
}
TEST(JobNodeSubmit, ConcurrencyBudgetIsNeverExceeded) { TEST(JobNodeSubmit, ConcurrencyBudgetIsNeverExceeded) {
constexpr Uint kBudget = 2; constexpr Uint kBudget = 2;
constexpr Uint kJobs = 64; constexpr Uint kJobs = 64;
@@ -472,30 +631,58 @@ TEST(JobNodeException, AThrowingJobDoesNotPoisonTheWorkerForLaterJobs) {
// --------------------------------------------------------------------------------------- // ---------------------------------------------------------------------------------------
TEST(ShaderCompilePoolDrain, StopAndDrainWithAThousandQueuedJobsLeavesNoneRunningOrPending) { TEST(ShaderCompilePoolDrain, StopAndDrainWithAThousandQueuedJobsLeavesNoneRunningOrPending) {
constexpr Uint kJobs = 1000; constexpr Uint kQueued = 1000;
ShaderCompilePool pool(kTestThreads); ShaderCompilePool pool(kTestThreads);
pool.SetMaxConcurrency(1); // keep the vast majority queued behind the budget pool.SetMaxConcurrency(1); // one slot, so everything behind the first job stays queued
Vector<SharedPtr<TestJob>> jobs; // Pin that slot with a job that will not return until this test says so. Everything
jobs.reserve(kJobs); // posted behind it is then PROVABLY still in the queue, which is what makes the counts
for (Uint i = 0; i < kJobs; ++i) { // below exact.
jobs.push_back(MakeShared<TestJob>()); //
pool.Post(jobs.back()); // This case used to post a thousand trivial jobs and drain immediately, hoping the drain
// would beat the workers to some of them - and then assert only that "some" were
// cancelled. That hope does not survive an engine whose workers take their next job
// without a scheduler round trip: the libfork engine drained all thousand before the
// posting loop had finished, so the assertion failed about one run in fifty. The property
// being tested (a drain ABANDONS queued work rather than running it) is real and
// engine-independent; only the way it was provoked was a race.
Gate gate;
std::atomic<Bool> entered{false};
auto blocker = MakeShared<TestJob>([&](TestJob&) {
entered.store(true, std::memory_order_release);
gate.Wait();
});
pool.Post(blocker);
ASSERT_TRUE(WaitUntil([&] { return entered.load(); }));
Vector<SharedPtr<TestJob>> queued;
queued.reserve(kQueued);
for (Uint i = 0; i < kQueued; ++i) {
queued.push_back(MakeShared<TestJob>());
pool.Post(queued.back());
} }
for (const auto& job : queued) ASSERT_FALSE(job->IsTerminal());
pool.StopAndDrain(); std::thread drain([&] { pool.StopAndDrain(); });
// StopAndDrain settles the entire queue before it waits for the running body, so the
// first cancelled node proves it is past that point - and the gate can then be released
// without racing it.
ASSERT_TRUE(WaitUntil([&] { return queued.front()->IsTerminal(); }));
gate.Open();
drain.join();
// Every node is terminal, so nothing can be waiting on a worker that will never come. // The job that was already running still finished: an in-flight body is waited for, not
Uint complete = 0; // interrupted.
Uint cancelled = 0; EXPECT_TRUE(blocker->IsComplete());
for (const auto& job : jobs) { EXPECT_EQ(blocker->ran.load(), 1u);
// And every queued node is terminal, so nothing is left waiting on a worker that will
// never come - settled as cancelled, with its body never entered.
for (const auto& job : queued) {
ASSERT_TRUE(job->IsTerminal()); ASSERT_TRUE(job->IsTerminal());
if (job->IsComplete()) ++complete; EXPECT_TRUE(job->IsCancelled());
if (job->IsCancelled()) ++cancelled; EXPECT_EQ(job->ran.load(), 0u);
EXPECT_LE(job->ran.load(), 1u);
} }
EXPECT_EQ(complete + cancelled, kJobs);
EXPECT_GT(cancelled, 0u); // the drain really did abandon queued work rather than run it
} }
TEST(ShaderCompilePoolDrain, StopAndDrainWaitsForARunningBodyToReturn) { TEST(ShaderCompilePoolDrain, StopAndDrainWaitsForARunningBodyToReturn) {
@@ -535,3 +722,141 @@ TEST(ShaderCompilePoolDrain, JobsPostedAfterADrainStillRun) {
EXPECT_TRUE(job->IsComplete()); EXPECT_TRUE(job->IsComplete());
EXPECT_EQ(job->ran.load(), 1u); EXPECT_EQ(job->ran.load(), 1u);
} }
// ---------------------------------------------------------------------------------------
// Execution engine selection (MOBILEGL_ASYNC_POOL)
// ---------------------------------------------------------------------------------------
//
// The engine decides only HOW a job that the concurrency budget has already cleared reaches a
// worker thread. Everything else in this file - the budget, cancel request-vs-outcome, the
// continuation machinery, the inline fallback after a stop, the drain - is engine-independent
// by construction, which is why the whole suite is expected to pass unchanged with
// MOBILEGL_ASYNC_POOL unset and with it set to libfork. These cases pin the selection itself,
// so that a run of the matrix cannot silently test asio twice.
TEST(AsyncPoolEngineSelection, EveryAcceptedSpellingParsesToItsEngine) {
EXPECT_EQ(ParseAsyncPoolEngine("asio"), AsyncPoolEngine::Asio);
EXPECT_EQ(ParseAsyncPoolEngine("libfork"), AsyncPoolEngine::Libfork);
// Case-insensitive, like the other named-value variables (MOBILEGL_*_MULTIDRAW_MODE).
EXPECT_EQ(ParseAsyncPoolEngine("Libfork"), AsyncPoolEngine::Libfork);
EXPECT_EQ(ParseAsyncPoolEngine("LIBFORK"), AsyncPoolEngine::Libfork);
EXPECT_EQ(ParseAsyncPoolEngine("ASIO"), AsyncPoolEngine::Asio);
EXPECT_STREQ(AsyncPoolEngineName(AsyncPoolEngine::Asio), "asio");
EXPECT_STREQ(AsyncPoolEngineName(AsyncPoolEngine::Libfork), "libfork");
// Round trip: whatever the name prints is a spelling the variable accepts back.
EXPECT_EQ(ParseAsyncPoolEngine(AsyncPoolEngineName(AsyncPoolEngine::Asio)), AsyncPoolEngine::Asio);
EXPECT_EQ(ParseAsyncPoolEngine(AsyncPoolEngineName(AsyncPoolEngine::Libfork)), AsyncPoolEngine::Libfork);
}
TEST(AsyncPoolEngineSelection, EmptyAndAutoAreTheDefaultEngineAndSaySoSilently) {
// Unset resolves through the empty string, and "auto" is the spelling the other named
// -value variables accept for "no preference". Neither is a mistake, so neither warns.
const std::streamoff before = LogSize();
EXPECT_EQ(ParseAsyncPoolEngine(""), AsyncPoolEngine::Asio);
EXPECT_EQ(ParseAsyncPoolEngine("auto"), AsyncPoolEngine::Asio);
EXPECT_EQ(ReadLogFrom(before).find("MOBILEGL_ASYNC_POOL"), String::npos)
<< "a legitimate value warned; only an unrecognized one may";
}
TEST(AsyncPoolEngineSelection, AnUnrecognizedEngineNameFallsBackToAsioAndWarns) {
const std::streamoff before = LogSize();
EXPECT_EQ(ParseAsyncPoolEngine("libfrok"), AsyncPoolEngine::Asio);
// The warning is the other half of the contract: a misspelt engine name that fell back
// silently would be indistinguishable from an unset variable, and a scaling measurement
// taken against the wrong engine is worse than no measurement.
//
// Guarded because MGLOG_W is a compile-time no-op unless the build's log level admits it -
// and the shipped level does not (Log.h orders the levels DEBUG=0, WARN=1, ERROR=2, INFO=3,
// FATAL=4 and gates on `ACTIVE <= LEVEL`, so the default INFO build enables only INFO and
// FATAL). Nothing is skipped: the fallback above is pinned in every build, and this half is
// checked by a build configured with
// -DMOBILEGL_LOG_ACTIVE_LEVEL=MOBILEGL_LOG_LEVEL_WARN. The same guard is what makes the
// preceding "says so silently" case honest rather than vacuously true.
#if MOBILEGL_LOG_ACTIVE_LEVEL <= MOBILEGL_LOG_LEVEL_WARN
const String logged = ReadLogFrom(before);
EXPECT_NE(logged.find("MOBILEGL_ASYNC_POOL"), String::npos) << "no warning names the variable; log tail: " << logged;
EXPECT_NE(logged.find("libfrok"), String::npos)
<< "the warning does not quote the rejected value; log tail: " << logged;
EXPECT_NE(logged.find("asio"), String::npos)
<< "the warning does not say what it fell back to; log tail: " << logged;
#else
(void)before;
#endif
}
TEST(AsyncPoolEngineSelection, TheDetectedEngineIsTheOneTheEnvironmentAskedFor) {
// Read the variable directly rather than through the pool, so this really compares the
// process's answer against the environment the runner exported. This is the case that
// makes "the suite passed with MOBILEGL_ASYNC_POOL=libfork" mean something.
const char* const raw = std::getenv("MOBILEGL_ASYNC_POOL");
const AsyncPoolEngine expected = ParseAsyncPoolEngine(raw != nullptr ? String(raw) : String());
EXPECT_EQ(DetectAsyncPoolEngine(), expected);
// Stable: resolved once per process, so it cannot drift between calls.
EXPECT_EQ(DetectAsyncPoolEngine(), DetectAsyncPoolEngine());
if (DetectAsyncPoolEngine() != AsyncPoolEngine::Asio) {
// Selecting a non-default engine announces itself at INFO, which the shipped log level
// does admit - so on the libfork half of the matrix this doubles as the positive
// control for the log plumbing the preceding two cases read: it proves
// MOBILEGL_LOG_FILE_PATH took effect and that ReadLogFrom really sees MobileGL's
// output, rather than passing because the file is always empty.
const String logged = ReadLogFrom(0);
EXPECT_NE(logged.find("MOBILEGL_ASYNC_POOL"), String::npos)
<< "the selected engine was never announced, so this binary's log capture proves nothing";
EXPECT_NE(logged.find(AsyncPoolEngineName(DetectAsyncPoolEngine())), String::npos);
}
}
TEST(AsyncPoolEngineSelection, EveryPoolReportsTheProcessEngineAndRunsWorkOnIt) {
ShaderCompilePool first(kTestThreads);
ShaderCompilePool second(kTestThreads);
EXPECT_EQ(first.GetEngine(), DetectAsyncPoolEngine());
EXPECT_EQ(second.GetEngine(), first.GetEngine())
<< "two pools in one process disagree about the engine; a process must never run both";
// And the engine it reports is the one that actually executed the work: the body ran off
// the calling thread, on a thread the pool owns.
const auto callingThread = std::this_thread::get_id();
std::atomic<Bool> sawPoolThread{false};
std::thread::id bodyThread{};
auto job = MakeShared<TestJob>([&](TestJob&) {
sawPoolThread.store(ShaderCompilePool::IsPoolThread(), std::memory_order_release);
bodyThread = std::this_thread::get_id();
});
first.Post(job);
job->Wait();
ASSERT_TRUE(job->IsComplete());
EXPECT_TRUE(sawPoolThread.load());
EXPECT_NE(bodyThread, callingThread);
}
// gtest_main is replaced here for one reason: the engine-selection cases above assert that an
// unrecognized MOBILEGL_ASYNC_POOL value WARNS, and MobileGL's desktop log sink is the log
// file - MOBILEGL_LOG_ENABLE_CONSOLE is 0 in Defines.h, so there is nothing on stdout to
// capture. MOBILEGL_LOG_FILE_PATH is read by Log.cpp's InitFile() at the first log write in
// the process, so it has to be set before any test body runs.
int main(int argc, char** argv) {
const std::filesystem::path logPath =
std::filesystem::temp_directory_path() /
("mobilegl-jobnodetest-" + std::to_string(static_cast<long long>(MGL_TEST_GETPID())) + ".log");
g_logFilePath = logPath.string();
std::filesystem::remove(logPath);
#ifdef _WIN32
::_putenv_s("MOBILEGL_LOG_FILE_PATH", g_logFilePath.c_str());
#else
::setenv("MOBILEGL_LOG_FILE_PATH", g_logFilePath.c_str(), 1);
#endif
::testing::InitGoogleTest(&argc, argv);
const int result = RUN_ALL_TESTS();
// Best-effort: leaving a log file per test process in the temp directory would be litter,
// and a failed run has already printed the tail it needed into the gtest output.
std::error_code ignored;
std::filesystem::remove(logPath, ignored);
return result;
}
+503 -35
View File
@@ -12,8 +12,14 @@
#include <asio/post.hpp> #include <asio/post.hpp>
#include <asio/thread_pool.hpp> #include <asio/thread_pool.hpp>
#include <libfork/core.hpp>
#include <libfork/schedule/lazy_pool.hpp>
#include <cstdio> #include <cstdio>
#include <cstdlib>
#include <deque> #include <deque>
#include <functional>
#include <span>
namespace MobileGL::MG_Util::Async { namespace MobileGL::MG_Util::Async {
namespace { namespace {
@@ -135,46 +141,506 @@ namespace MobileGL::MG_Util::Async {
return std::clamp(DetectBigCoreCount(), 1u, kMaxAutoShaderCompileThreads); return std::clamp(DetectBigCoreCount(), 1u, kMaxAutoShaderCompileThreads);
} }
// ---- Engine selection -----------------------------------------------------------------
const char* AsyncPoolEngineName(const AsyncPoolEngine engine) {
switch (engine) {
case AsyncPoolEngine::Libfork: return "libfork";
case AsyncPoolEngine::Asio: break;
}
return "asio";
}
AsyncPoolEngine ParseAsyncPoolEngine(const String& value) {
String lowered = value;
std::transform(lowered.begin(), lowered.end(), lowered.begin(),
[](const unsigned char c) { return static_cast<char>(std::tolower(c)); });
if (lowered == "libfork") return AsyncPoolEngine::Libfork;
if (lowered == "asio" || lowered == "auto" || lowered.empty()) return AsyncPoolEngine::Asio;
// Not silent: a misspelt engine name resolving to the default would be
// indistinguishable from not having set the variable at all, and the only reason to
// set it is to know which engine ran.
MGLOG_W("Config: Ignoring invalid env variable MOBILEGL_ASYNC_POOL='%s'; expected asio|libfork, "
"using asio",
value.c_str());
return AsyncPoolEngine::Asio;
}
AsyncPoolEngine DetectAsyncPoolEngine() {
// A live std::getenv rather than an MG_Config::Features mirror, and deliberately so:
// a ShaderCompilePool is constructed by binaries that never call MobileGL::Initialize()
// and therefore never run MG_ConfigLoader::Init() - MG_Test/Util/JobNodeTest builds
// pools directly, and it is the suite that exercises the engines against each other.
// Reading Features there would silently resolve to the default and the libfork half of
// the test matrix would prove nothing. See the exemption list in Config.h.
//
// Resolved once per process (a function-local static): every pool in a process gets
// the same engine, so a process can never end up running two.
static const AsyncPoolEngine engine = [] {
const char* value = std::getenv("MOBILEGL_ASYNC_POOL");
const AsyncPoolEngine resolved = ParseAsyncPoolEngine(value != nullptr ? String(value) : String());
if (resolved != AsyncPoolEngine::Asio) {
MGLOG_I("ShaderCompilePool: MOBILEGL_ASYNC_POOL selected the %s execution engine",
AsyncPoolEngineName(resolved));
}
return resolved;
}();
return engine;
}
namespace {
// ---- The engine boundary ----------------------------------------------------------
// Submit() has exactly asio::post's contract, and ShaderCompilePool::Impl leans on all
// four halves of it:
// * it NEVER runs `fn` on the calling thread. DispatchLocked calls it while holding
// the pool's plain, non-recursive mutex, and a job body (or a terminal
// continuation it releases) is free to call Post() again - an inline run would
// deadlock on the lock this frame already owns.
// * it is callable from ANY thread, a worker of this very pool included:
// ProgramLinkTask::OnDepSettled posts the link job from whichever thread drove the
// last compile terminal, which is a worker.
// * it may throw, and when it does it must not have consumed the caller's job node,
// so Post/DispatchLocked can settle the node instead of stranding it Pending with
// a joiner blocked forever.
// * once it has accepted `fn`, `fn` WILL run. A dropped callable is a node nothing
// ever settles, so the engines run it themselves rather than discard it.
class JobExecutor {
public:
virtual ~JobExecutor() = default;
JobExecutor() = default;
JobExecutor(const JobExecutor&) = delete;
JobExecutor& operator=(const JobExecutor&) = delete;
virtual void Submit(std::function<void()> fn) = 0;
// Returns once every callable ever handed to Submit has finished running. The
// guarantee StopAndDrain sells to library teardown: after it returns, no worker is
// still inside a job body that could touch glslang's process globals.
virtual void JoinAll() = 0;
};
// ---- Engine 1: Asio (the shipped default) -----------------------------------------
class AsioJobExecutor final : public JobExecutor {
public:
explicit AsioJobExecutor(const Uint threads) : m_pool(threads) {}
// asio::post only enqueues; it never runs the handler on the calling thread, which
// is what makes calling it under the pool mutex safe.
void Submit(std::function<void()> fn) override { asio::post(m_pool, Move(fn)); }
void JoinAll() override { m_pool.join(); }
private:
asio::thread_pool m_pool;
};
// ---- Engine 2: libfork ------------------------------------------------------------
//
// libfork is a continuation-stealing fork-join runtime, and the shape that fits here is
// NOT fork-join: a job body is one coarse, blocking, non-forking unit (a glslang
// compile), and the concurrency budget that bounds peak RSS is Impl's, not the
// scheduler's. So libfork is used as a job executor - each dispatched job is a detached
// root task - and what it is being asked to beat is Asio's single scheduler queue with
// its per-worker work-stealing deques and sleeping workers.
//
// The one thing libfork forbids is the thing this pool does constantly: lf::schedule
// (which lf::detach is built on) THROWS lf::schedule_in_worker when the calling thread
// is a libfork worker, because workers may never block. Yet a worker submits on every
// job completion - RunOnWorker's tail refills the budget - and again whenever a
// terminal continuation posts (ProgramLinkTask::OnDepSettled). Routing those through a
// separate dispatch thread works but costs two thread wakeups per job, which measured
// 4x worse than Asio on short jobs. So instead a dispatched root is a CHAIN: when its
// body returns it takes the next queued job itself and runs it in the same coroutine
// on the same worker. The refill a worker submits is therefore absorbed by the very
// chain that submitted it - no scheduler round trip, no wakeup - and libfork is only
// entered for work that arrives from outside the pool.
//
// Absorption is bounded at one job per running chain, though, because a chain is one
// worker: past that bound the queue would be jobs the budget has already cleared,
// waiting behind each other on a single thread. See Submit.
//
// Why none of this can strand a job: the queue below is only ever added to from inside
// a running chain (tl_chainOwner == this), and a chain exits only when it finds the
// queue empty - unconditionally, whatever the bound says. Every other submitter goes
// to the dispatch thread or straight to lf::detach.
class LibforkJobExecutor;
// Which executor's chain, if any, is running on this thread. Deliberately narrower
// than ShaderCompilePool::IsPoolThread(): that flag is process-wide and latched
// forever, so a worker of a DIFFERENT pool would read as "mine" and queue a job into a
// chain that will never drain it. This says exactly "a chain of *this* executor is
// executing on this thread, and it will look at the queue again before it exits".
thread_local LibforkJobExecutor* tl_chainOwner = nullptr;
// One dispatched job, heap-owned. It reaches its coroutine as a POINTER passed BY
// VALUE: libfork forwards a root task's arguments into the coroutine frame, so a
// by-value pointer is copied into the frame, whereas anything passed by reference
// would dangle the moment lf::detach returns - and detach, unlike sync_wait, does not
// outlive the task.
struct LibforkJob {
std::function<void()> body;
LibforkJobExecutor* owner;
};
// A scheduler adaptor for lf::detach: it places external submissions round-robin over
// lf::lazy_pool's worker contexts instead of letting the pool pick one at random.
// Both reasons are load-bearing, and the second was worth 1.3x at a budget equal to
// the worker count - the configuration MobileGL actually ships, since maxConcurrency
// is clamped to the thread count:
// * lf::lazy_pool::schedule chooses its victim with a
// std::uniform_int_distribution over a lazy_pool-member xoshiro generator -
// unsynchronized mutable state, so two concurrent submissions are a data race
// inside libfork itself. An atomic cursor is not.
// * A worker's SUBMISSION list is drained only by that worker
// (worker_context::try_pop_all is documented "for use only by the owning worker
// thread"); a thief takes from the task deque, which is a different queue. So a
// job placed on a worker that is inside a long blocking body waits for that body
// rather than being stolen - and random placement of `budget` submissions over
// `budget` workers collides by the birthday rule. Round-robin lands the GL
// thread's burst one per worker, which is exactly the intended shape.
struct RoundRobinSubmitter {
std::span<lf::worker_context*> contexts;
std::atomic<Uint64>* cursor;
void schedule(const lf::submit_handle job) const {
const Uint64 index = cursor->fetch_add(1, std::memory_order_relaxed);
contexts[static_cast<SizeT>(index % contexts.size())]->schedule(job);
}
};
void RunLibforkChain(LibforkJob* raw) noexcept;
// The root task every dispatched chain runs as. libfork async function objects are
// copyable, captureless callables returning lf::task<>, whose first parameter is the
// combinator's synthesized first argument (unused here: this task neither forks nor
// joins). The coroutine exists purely as libfork's entry protocol; the loop is in
// RunLibforkChain.
inline constexpr auto kLibforkChainTask = [](auto /*self*/, LibforkJob* job) -> lf::task<void> {
RunLibforkChain(job);
co_return;
};
class LibforkJobExecutor final : public JobExecutor {
public:
explicit LibforkJobExecutor(const Uint threads)
: m_pool(static_cast<std::size_t>(std::max(1u, threads))), m_contexts(m_pool.contexts()),
m_fallback([this] { FallbackLoop(); }) {}
~LibforkJobExecutor() override {
JoinAll();
{
const std::lock_guard<std::mutex> lock(m_mutex);
m_fallbackStop = true;
}
m_fallbackCv.notify_all();
if (m_fallback.joinable()) m_fallback.join();
// m_pool is destroyed last, and only here: lf::lazy_pool may not be destructed
// while any submitted task can still run or submit more. JoinAll() has
// established the first and the joined fallback thread the second. Its
// destructor then joins the worker threads, so a worker still unwinding a
// finished coroutine frame is waited for rather than pulled out from under.
}
void Submit(std::function<void()> fn) override {
if (tl_chainOwner == this) {
const std::lock_guard<std::mutex> lock(m_mutex);
// The hot path: ONE job per running chain. A chain picks up exactly one
// queued job each time its body returns, so a queue no longer than the
// number of live chains is a queue every entry of which has a distinct
// worker waiting to take it - which is precisely the steady state this
// absorption exists for (every worker finishes a job and refills its own
// slot, all at once, with no scheduler round trip between them).
//
// Past that it is oversubscription, and absorbing it would be a
// correctness-preserving way to destroy the pool's parallelism: the
// budget would still say `maxConcurrency` jobs are in flight while one
// worker ran them one behind another. That is not hypothetical - it is
// the tail of a pack load, where one compile going terminal releases
// several programs at once (ShaderCompileAdoptionMap lets a single
// compile settle many) and the worker that drove it posts the whole
// burst into an otherwise idle pool. Measured before this branch existed:
// four such jobs took 4x one job's wall time on libfork and 1x on Asio.
//
// The overflow cannot go to lf::detach from here - a libfork worker may
// not schedule - so it goes to the dispatch thread, which detaches it to
// a worker of its own. That costs one thread wakeup; serializing costs a
// whole compile.
//
// The count is taken AFTER the push, not before: deque::push_back is
// strongly exception-safe, so an allocation failure here leaves `fn`
// intact for DispatchLocked to settle - but a count incremented in front
// of it would be a count nothing ever gives back, and JoinAll would wait
// on it forever.
const Bool takeable = m_chainQueue.size() < m_liveChains;
if (takeable) {
m_chainQueue.push_back(Move(fn));
++m_outstanding;
} else {
m_fallbackQueue.push_back(Move(fn));
++m_outstanding;
m_fallbackCv.notify_one();
}
return;
}
{
// Counted before anything can run it, so JoinAll cannot observe a zero
// that this job would have broken.
const std::lock_guard<std::mutex> lock(m_mutex);
++m_outstanding;
}
try {
DetachChain(Move(fn));
} catch (const lf::schedule_in_worker&) {
// Submitted from a libfork worker that is not running one of my chains -
// a worker of another ShaderCompilePool. libfork will not take a
// submission from there at all, and the queue above is not safe for it
// (no chain of mine is running on that thread to drain it), so it goes to
// the fallback thread, which is neither. DetachChain restored `fn` before
// it threw.
const std::lock_guard<std::mutex> lock(m_mutex);
m_fallbackQueue.push_back(Move(fn));
m_fallbackCv.notify_one();
} catch (...) {
// Out of memory. Give the count back and let the caller settle its node:
// that is Submit's contract and what DispatchLocked is written against.
Retire();
throw;
}
}
void JoinAll() override {
std::unique_lock<std::mutex> lock(m_mutex);
m_idleCv.wait(lock, [this] { return m_outstanding == 0; });
}
// A chain announces itself before it runs its first body, so that Submit's
// absorption rule can count the workers that are going to come back and ask for
// more. Under-counting is the only direction this can be wrong in (a detached
// chain is not counted until it starts), and under-counting only sends work to
// the dispatch thread that a chain could have taken - never the reverse.
void EnterChain() noexcept {
const std::lock_guard<std::mutex> lock(m_mutex);
++m_liveChains;
}
// The end of one job in a chain. Returns true having loaded `body` with the next
// job to run on this same worker, false when there is nothing left - after which
// the caller must touch neither `this` nor anything owned by it, because the
// count this drops to zero may be the one JoinAll is waiting for.
//
// `body` must arrive empty: the finished job's captures (a strong reference to its
// JobNode) are released by the chain, outside this lock, so that no JobNode
// destructor ever runs inside the executor's critical section.
Bool RetireAndTakeNext(std::function<void()>& body) noexcept {
const std::lock_guard<std::mutex> lock(m_mutex);
--m_outstanding;
if (!m_chainQueue.empty()) {
// Unconditional, and it has to stay that way: a chain that exited while
// the queue was non-empty could be the last one, and the entry would then
// be waiting on a worker that never comes. That is what makes the
// absorption bound in Submit a scheduling policy rather than a liveness
// requirement.
//
// swap, not move-assign: std::function's move assignment is not noexcept,
// and this function is.
body.swap(m_chainQueue.front());
m_chainQueue.pop_front();
return true; // the taken job's own count stays held
}
--m_liveChains;
// Notified while STILL HOLDING the lock, which is the whole reason this is not
// the usual notify-after-unlock. The wakeup this sends can be the one that
// lets JoinAll return and ~LibforkJobExecutor destroy m_idleCv - and a
// std::condition_variable may not be destroyed while another thread is inside
// notify_all() on it. Holding the lock across the notify means the waiter
// cannot re-acquire the mutex, and therefore cannot leave wait(), until this
// thread is out of both the notify and the unlock. ThreadSanitizer catches the
// other order immediately (pthread_cond_destroy vs pthread_cond_broadcast).
if (m_outstanding == 0) m_idleCv.notify_all();
return false;
}
private:
// Builds the root task and hands it to libfork. On any failure `fn` is restored,
// so the caller can still decide what to do with the job.
void DetachChain(std::function<void()>&& fn) {
// `new T{...}` allocates before it constructs, so a throwing operator new
// leaves `fn` untouched; the member move is std::function's noexcept one.
LibforkJob* job = new LibforkJob{Move(fn), this};
try {
lf::detach(RoundRobinSubmitter{m_contexts, &m_cursor}, kLibforkChainTask, job);
} catch (...) {
// lf::schedule upholds the strong exception guarantee, so nothing was
// scheduled and the payload is still ours.
const UniquePtr<LibforkJob> owned(job);
fn = Move(owned->body);
throw;
}
}
void Retire() noexcept {
// Under the lock, for the reason RetireAndTakeNext spells out.
const std::lock_guard<std::mutex> lock(m_mutex);
if (--m_outstanding == 0) m_idleCv.notify_all();
}
// The dispatch thread. It exists because lf::detach is illegal on a libfork worker
// and legal here, and it serves the two cases Submit cannot take itself: a
// submission from another pool's worker, and a chain's overflow past the
// one-job-per-chain bound. It sleeps otherwise, and it dispatches rather than
// executes - a body only ever runs here if libfork refuses the job outright.
void FallbackLoop() {
for (;;) {
std::function<void()> fn;
{
std::unique_lock<std::mutex> lock(m_mutex);
m_fallbackCv.wait(lock, [this] { return !m_fallbackQueue.empty() || m_fallbackStop; });
// Emptiness is checked before the stop flag so that a stop can never
// strand accepted work: an accepted job always runs, because the node
// behind it has a joiner that would otherwise block forever.
if (m_fallbackQueue.empty()) return;
fn.swap(m_fallbackQueue.front());
m_fallbackQueue.pop_front();
}
try {
DetachChain(Move(fn));
} catch (...) {
MGLOG_E("ShaderCompilePool: libfork refused a fallback dispatch; running the job on "
"the dispatch thread instead of dropping it");
RunHere(Move(fn));
}
}
}
// Last resort. Running the body here costs this engine its parallelism for one
// job; dropping it would cost a joiner its wakeup forever.
void RunHere(std::function<void()>&& fn) noexcept {
try {
if (fn) fn();
} catch (...) {
MGLOG_E("ShaderCompilePool: a job body escaped its own containment on the dispatch "
"thread; it has been swallowed to keep the thread alive");
}
fn = nullptr;
Retire();
}
lf::lazy_pool m_pool;
// Fixed for the pool's lifetime, so it is read once rather than per submission.
std::span<lf::worker_context*> m_contexts;
std::atomic<Uint64> m_cursor{0};
std::mutex m_mutex;
std::condition_variable m_fallbackCv;
std::condition_variable m_idleCv;
// Refills and continuations submitted from inside a chain: drained by the chains.
std::deque<std::function<void()>> m_chainQueue;
// Chains currently executing, i.e. workers that will look at m_chainQueue again
// before they exit. The bound on how much Submit may absorb into a chain.
Uint m_liveChains = 0;
// Submissions from another pool's libfork worker, and the overflow of the rule
// above: drained by m_fallback, which detaches each one to a worker.
std::deque<std::function<void()>> m_fallbackQueue;
// Everything submitted and not yet finished, whichever queue it is in and whether
// or not it has reached a worker, so JoinAll needs a single predicate.
Uint m_outstanding = 0;
Bool m_fallbackStop = false;
std::thread m_fallback;
};
void RunLibforkChain(LibforkJob* const raw) noexcept {
UniquePtr<LibforkJob> job(raw);
LibforkJobExecutor* const owner = job->owner;
std::function<void()> body;
body.swap(job->body);
job.reset();
LibforkJobExecutor* const savedOwner = tl_chainOwner;
tl_chainOwner = owner;
owner->EnterChain();
for (;;) {
try {
if (body) body();
} catch (...) {
// JobNode::Run contains every body exception already; this is the backstop
// for the wrapper itself. An exception escaping here would be stashed in
// the root task's shared state, which lf::detach discards - i.e. silently
// lost - and would abandon the rest of the chain.
MGLOG_E("ShaderCompilePool: a job body escaped its own containment on a libfork worker; "
"it has been swallowed to keep the chain alive");
}
// Release the finished job's captures (its strong JobNode reference) HERE,
// outside the executor's lock: a JobNode destructor is arbitrary code.
body = nullptr;
if (!owner->RetireAndTakeNext(body)) break;
}
// `owner` may already be destroyed - RetireAndTakeNext returning false can be the
// call that releases a JoinAll. Nothing below touches it.
tl_chainOwner = savedOwner;
}
UniquePtr<JobExecutor> MakeJobExecutor(const AsyncPoolEngine engine, const Uint threads) {
switch (engine) {
case AsyncPoolEngine::Libfork: return MakeUnique<LibforkJobExecutor>(threads);
case AsyncPoolEngine::Asio: break;
}
return MakeUnique<AsioJobExecutor>(threads);
}
} // namespace
struct ShaderCompilePool::Impl { struct ShaderCompilePool::Impl {
explicit Impl(const Uint threads) : threadCount(std::max(1u, threads)), maxConcurrency(threadCount) {} explicit Impl(const Uint threads)
: threadCount(std::max(1u, threads)), engine(DetectAsyncPoolEngine()), maxConcurrency(threadCount) {}
const Uint threadCount; const Uint threadCount;
// Latched at construction, not re-read: a pool may not change engines under its own
// workers, and GetEngine() is what the tests compare against the environment.
const AsyncPoolEngine engine;
std::mutex mutex; std::mutex mutex;
// Created on the first dispatched Post, never in the constructor: asio::thread_pool // Created on the first dispatched Post, never in the constructor: both engines spawn
// spawns its threads eagerly, and a build with async off must not pay for threads it // their threads eagerly (asio::thread_pool its workers, lf::lazy_pool its workers plus
// this file's dispatch thread), and a build with async off must not pay for threads it
// will never use. // will never use.
UniquePtr<asio::thread_pool> pool; UniquePtr<JobExecutor> executor;
std::deque<SharedPtr<JobNode>> queue; std::deque<SharedPtr<JobNode>> queue;
Uint inFlight = 0; Uint inFlight = 0;
Uint maxConcurrency; Uint maxConcurrency;
std::atomic<Bool> stopped{false}; std::atomic<Bool> stopped{false};
// Callers hold `mutex`. Hands as many queued nodes to Asio as the concurrency budget // Callers hold `mutex`. Hands as many queued nodes to the engine as the concurrency
// allows. Posting under the lock is safe and is what keeps `pool` from being moved // budget allows. Submitting under the lock is safe and is what keeps `executor` from
// out by a concurrent StopAndDrain between the decision and the dispatch: asio::post // being moved out by a concurrent StopAndDrain between the decision and the dispatch:
// only enqueues, it never runs the handler on the calling thread, so it cannot // Submit only enqueues, it never runs the callable on the calling thread, so it cannot
// re-enter this mutex. // re-enter this mutex.
// //
// A node asio::post fails to hand off is appended to `toCancel` instead of being // A node the engine fails to accept is appended to `toCancel` instead of being
// Cancel()'d here: Cancel() runs the node's OnTerminal continuations inline (stage 4 // Cancel()'d here: Cancel() runs the node's OnTerminal continuations inline (stage 4
// added ProgramLinkTask::OnDepSettled as a real one), and a continuation is free to // added ProgramLinkTask::OnDepSettled as a real one), and a continuation is free to
// call ShaderCompilePool::Post() again. Every caller of DispatchLocked holds `mutex` // call ShaderCompilePool::Post() again. Every caller of DispatchLocked holds `mutex`
// (a plain, non-recursive std::mutex) - Cancel()'ing in here would let that // (a plain, non-recursive std::mutex) - Cancel()'ing in here would let that
// re-entrant Post() deadlock on the very lock this frame already owns. The caller // re-entrant Post() deadlock on the very lock this frame already owns. The caller
// drains `toCancel` after releasing the lock. // drains `toCancel` after releasing the lock.
//
// The `stopped` check is also what keeps this loop from dereferencing a null
// `executor`: StopAndDrain sets the flag and moves the executor out in the same
// critical section, so a stopped pool never reaches the Submit below.
void DispatchLocked(Vector<SharedPtr<JobNode>>& toCancel) { void DispatchLocked(Vector<SharedPtr<JobNode>>& toCancel) {
while (!queue.empty() && inFlight < maxConcurrency && !stopped.load(std::memory_order_acquire)) { while (!queue.empty() && inFlight < maxConcurrency && !stopped.load(std::memory_order_acquire)) {
// Copy rather than move into the handler: if asio::post throws (it allocates) // Copy rather than move into the callable: if Submit throws (both engines
// the local SharedPtr is still valid, so the node can be settled instead of // allocate) the local SharedPtr is still valid, so the node can be settled
// being stranded Pending in a queue nothing will dispatch from again - a // instead of being stranded Pending in a queue nothing will dispatch from
// joiner would block on it forever. Reclaiming the slot matters just as much: // again - a joiner would block on it forever. Reclaiming the slot matters just
// a leaked `inFlight` shrinks the pool's concurrency budget permanently. // as much: a leaked `inFlight` shrinks the pool's concurrency budget
// permanently.
SharedPtr<JobNode> node = queue.front(); SharedPtr<JobNode> node = queue.front();
queue.pop_front(); queue.pop_front();
++inFlight; ++inFlight;
try { try {
asio::post(*pool, [this, node]() mutable { RunOnWorker(Move(node)); }); executor->Submit([this, node]() mutable { RunOnWorker(Move(node)); });
} catch (...) { } catch (...) {
--inFlight; --inFlight;
toCancel.push_back(Move(node)); toCancel.push_back(Move(node));
@@ -184,7 +650,7 @@ namespace MobileGL::MG_Util::Async {
void RunOnWorker(SharedPtr<JobNode> node) { void RunOnWorker(SharedPtr<JobNode> node) {
tl_isPoolThread = true; tl_isPoolThread = true;
// A node that was already handed to Asio when StopAndDrain ran still arrives // A node that was already handed to the engine when StopAndDrain ran still arrives
// here; cancelling it first turns the dispatch into a state transition instead of // here; cancelling it first turns the dispatch into a state transition instead of
// a full compile, so the drain's join() returns promptly. This Cancel() runs // a full compile, so the drain's join() returns promptly. This Cancel() runs
// before `mutex` is ever taken in this frame, so it is not subject to the // before `mutex` is ever taken in this frame, so it is not subject to the
@@ -234,13 +700,15 @@ namespace MobileGL::MG_Util::Async {
return m_impl->maxConcurrency; return m_impl->maxConcurrency;
} }
AsyncPoolEngine ShaderCompilePool::GetEngine() const { return m_impl->engine; }
void ShaderCompilePool::SetMaxConcurrency(const Uint n) { void ShaderCompilePool::SetMaxConcurrency(const Uint n) {
Vector<SharedPtr<JobNode>> toCancel; Vector<SharedPtr<JobNode>> toCancel;
{ {
const std::lock_guard<std::mutex> lock(m_impl->mutex); const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount); m_impl->maxConcurrency = std::clamp(n, 1u, m_impl->threadCount);
// Raising the budget releases whatever the old one was holding back. // Raising the budget releases whatever the old one was holding back.
if (m_impl->pool) m_impl->DispatchLocked(toCancel); if (m_impl->executor) m_impl->DispatchLocked(toCancel);
} }
// Outside the lock: see DispatchLocked's comment. // Outside the lock: see DispatchLocked's comment.
for (const auto& n2 : toCancel) { for (const auto& n2 : toCancel) {
@@ -252,23 +720,23 @@ namespace MobileGL::MG_Util::Async {
if (!node) return; if (!node) return;
EnsureProcessTeardownSentinel(); EnsureProcessTeardownSentinel();
// Enqueueing can throw: the thread_pool construction and asio::post both allocate, // Enqueueing can throw: building the engine and submitting to it both allocate (and
// and under memory pressure a throw here would escape glCompileShader leaving the // both spawn threads), and under memory pressure a throw here would escape
// node Pending with nothing left to dispatch it - the first observable read would // glCompileShader leaving the node Pending with nothing left to dispatch it - the
// then block the GL thread forever. Settle the node instead: a cancelled node is a // first observable read would then block the GL thread forever. Settle the node
// state every joiner already handles. // instead: a cancelled node is a state every joiner already handles.
// //
// `node` is still valid in the catch for every throw this try can produce. The // `node` is still valid in the catch for every throw this try can produce. The engine
// thread_pool construction runs before the move; deque::push_back is strongly // construction runs before the move; deque::push_back is strongly exception-safe and
// exception-safe and SharedPtr's move constructor is noexcept, so a throwing // SharedPtr's move constructor is noexcept, so a throwing push_back never consumed it;
// push_back never consumed it; and DispatchLocked contains its own asio::post // and DispatchLocked contains its own Submit failures rather than propagating them
// failures rather than propagating them (see above). Keep it that way. // (see above). Keep it that way.
Bool enqueued = false; Bool enqueued = false;
Vector<SharedPtr<JobNode>> toCancel; Vector<SharedPtr<JobNode>> toCancel;
try { try {
const std::lock_guard<std::mutex> lock(m_impl->mutex); const std::lock_guard<std::mutex> lock(m_impl->mutex);
if (!m_impl->stopped.load(std::memory_order_acquire) && !InProcessTeardown()) { if (!m_impl->stopped.load(std::memory_order_acquire) && !InProcessTeardown()) {
if (!m_impl->pool) m_impl->pool = MakeUnique<asio::thread_pool>(m_impl->threadCount); if (!m_impl->executor) m_impl->executor = MakeJobExecutor(m_impl->engine, m_impl->threadCount);
m_impl->queue.push_back(Move(node)); m_impl->queue.push_back(Move(node));
m_impl->DispatchLocked(toCancel); m_impl->DispatchLocked(toCancel);
enqueued = true; enqueued = true;
@@ -304,17 +772,17 @@ namespace MobileGL::MG_Util::Async {
} }
void ShaderCompilePool::StopAndDrain() { void ShaderCompilePool::StopAndDrain() {
// asio::thread_pool::join() from a pool thread would deadlock on itself, and the // Waiting for the workers from a worker would deadlock on itself (asio's join() says
// whole point of this call is that the GL thread waits for the workers. // so outright), and the whole point of this call is that the GL thread waits.
MOBILEGL_ASSERT(!IsPoolThread(), "ShaderCompilePool::StopAndDrain() called from a pool thread"); MOBILEGL_ASSERT(!IsPoolThread(), "ShaderCompilePool::StopAndDrain() called from a pool thread");
std::deque<SharedPtr<JobNode>> abandoned; std::deque<SharedPtr<JobNode>> abandoned;
UniquePtr<asio::thread_pool> pool; UniquePtr<JobExecutor> executor;
{ {
const std::lock_guard<std::mutex> lock(m_impl->mutex); const std::lock_guard<std::mutex> lock(m_impl->mutex);
m_impl->stopped.store(true, std::memory_order_release); m_impl->stopped.store(true, std::memory_order_release);
abandoned.swap(m_impl->queue); abandoned.swap(m_impl->queue);
pool = Move(m_impl->pool); executor = Move(m_impl->executor);
} }
// Queued but never dispatched: settle them so anything chained behind them is // Queued but never dispatched: settle them so anything chained behind them is
@@ -323,9 +791,9 @@ namespace MobileGL::MG_Util::Async {
if (node) node->Cancel(); if (node) node->Cancel();
} }
if (pool) { if (executor) {
pool->join(); // returns once every handler already handed to Asio has finished executor->JoinAll(); // returns once every job already handed to the engine is done
pool.reset(); executor.reset(); // and this stops the engine's threads
} }
const std::lock_guard<std::mutex> lock(m_impl->mutex); const std::lock_guard<std::mutex> lock(m_impl->mutex);
+36 -5
View File
@@ -11,11 +11,12 @@
#include <MG_Util/Types.h> #include <MG_Util/Types.h>
#include <MG_Util/Async/JobNode.h> #include <MG_Util/Async/JobNode.h>
// This header deliberately includes NO Asio header: asio::thread_pool lives behind the pimpl // This header deliberately includes NO Asio and NO libfork header: both execution engines
// in ShaderCompilePool.cpp. Asio stays a private implementation detail of one translation // live behind the pimpl in ShaderCompilePool.cpp. They stay private implementation details of
// unit, so no consumer target (MG_Test, MG_IntegrationTest, MG_Benchmark - each with its own // one translation unit, so no consumer target (MG_Test, MG_IntegrationTest, MG_Benchmark -
// target_include_directories) needs the Asio include path, and no consumer pays its compile // each with its own target_include_directories) needs either include path, and no consumer
// time. Do not add one here. // pays their compile time. libfork in particular is a C++20-coroutine header set whose
// instantiation cost nothing outside the pool has any reason to carry. Do not add one here.
namespace MobileGL::MG_Util::Async { namespace MobileGL::MG_Util::Async {
// Stage 7: on by default. The gate behind the flip (2026-08-09, headless Mesa, both // Stage 7: on by default. The gate behind the flip (2026-08-09, headless Mesa, both
@@ -66,6 +67,31 @@ namespace MobileGL::MG_Util::Async {
// MOBILEGL_ASYNC_SHADER_COMPILE_THREADS overrides it outright. // MOBILEGL_ASYNC_SHADER_COMPILE_THREADS overrides it outright.
Uint DetectShaderCompileThreadCount(); Uint DetectShaderCompileThreadCount();
// ---- MOBILEGL_ASYNC_POOL: which engine drives the worker threads ----------------------
// The engine is ONLY the execution engine. The job queue, the concurrency budget and its
// clamping, the suspension latch, cancel request-vs-outcome, the stopped-is-synchronous
// fallback and the drain are all engine-independent - they live in ShaderCompilePool::Impl
// and are shared verbatim by both engines, which is what lets the whole async suite run
// unchanged against either one. An engine answers exactly one question: how does a job
// that the budget has already cleared reach a worker thread?
enum class AsyncPoolEngine : Uint8 {
Asio, // asio::thread_pool: one shared queue behind Asio's scheduler lock
Libfork, // lf::lazy_pool: per-worker work-stealing deques, workers sleep when idle
};
// "asio" / "libfork" - the spelling the environment variable accepts and the log prints.
const char* AsyncPoolEngineName(AsyncPoolEngine engine);
// Parses one MOBILEGL_ASYNC_POOL value. Case-insensitive; empty, "auto" and anything
// unrecognized resolve to Asio, and an unrecognized value warns (a misspelt engine name
// would otherwise be indistinguishable from the default, and the whole point of the
// variable is to know which engine ran).
AsyncPoolEngine ParseAsyncPoolEngine(const String& value);
// The process's engine, resolved from MOBILEGL_ASYNC_POOL on first call and cached. Every
// pool constructed afterwards reports the same answer, so a process never mixes engines.
AsyncPoolEngine DetectAsyncPoolEngine();
class ShaderCompilePool { class ShaderCompilePool {
public: public:
explicit ShaderCompilePool(Uint threadCount); explicit ShaderCompilePool(Uint threadCount);
@@ -97,6 +123,11 @@ namespace MobileGL::MG_Util::Async {
Uint GetThreadCount() const; Uint GetThreadCount() const;
Uint GetMaxConcurrency() const; Uint GetMaxConcurrency() const;
// The engine this pool was built with, latched at construction from
// DetectAsyncPoolEngine(). Reported rather than re-resolved so that a pool cannot
// change engines under its own workers.
AsyncPoolEngine GetEngine() const;
// Bounded concurrency doubles as the memory bound, and is how // Bounded concurrency doubles as the memory bound, and is how
// glMaxShaderCompilerThreadsKHR(n) is honoured: a 300-program pack load cannot put // glMaxShaderCompilerThreadsKHR(n) is honoured: a 300-program pack load cannot put
// 300 glslang arenas in flight at once. Clamped to [1, thread count]. // 300 glslang arenas in flight at once. Clamped to [1, thread count].
@@ -838,6 +838,12 @@ namespace MobileGL::MG_Util::BackendLoader {
if (std::strcmp(extension, "GL_EXT_render_snorm") == 0) { if (std::strcmp(extension, "GL_EXT_render_snorm") == 0) {
caps.SupportsRenderSnorm = true; caps.SupportsRenderSnorm = true;
} }
if (std::strcmp(extension, "GL_EXT_color_buffer_float") == 0) {
caps.SupportsColorBufferFloat = true;
}
if (std::strcmp(extension, "GL_EXT_color_buffer_half_float") == 0) {
caps.SupportsColorBufferHalfFloat = true;
}
if (std::strcmp(extension, "GL_EXT_sRGB_write_control") == 0) { if (std::strcmp(extension, "GL_EXT_sRGB_write_control") == 0) {
caps.SupportsSrgbWriteControl = true; caps.SupportsSrgbWriteControl = true;
} }
@@ -1034,6 +1034,16 @@ namespace MobileGL {
// GL_EXT_render_snorm is present, so the signed-normalized formats are colour-renderable // GL_EXT_render_snorm is present, so the signed-normalized formats are colour-renderable
// (and usable as multisample texture storage) rather than texture-only. // (and usable as multisample texture storage) rather than texture-only.
Bool SupportsRenderSnorm = false; Bool SupportsRenderSnorm = false;
// GL_EXT_color_buffer_float is present, so GL_RGBA16F / GL_RGBA32F / GL_R11F_G11F_B10F
// (and the R/RG float formats) are colour-renderable. ES 3.x core makes them
// texture-only, and every Iris shaderpack renders into at least R11F_G11F_B10F, so
// without this no shaderpack can work at all.
Bool SupportsColorBufferFloat = false;
// GL_EXT_color_buffer_half_float is present: the half-float subset of the above, for
// drivers that ship only the smaller extension. Note it does NOT rescue GL_RGB16F -
// the extension nominally lists it but disclaims it under ES 3.x, and real drivers
// reject it, which is why three-channel float attachments are widened instead.
Bool SupportsColorBufferHalfFloat = false;
// GL_EXT_sRGB_write_control is present, so GL_FRAMEBUFFER_SRGB can be turned off. // GL_EXT_sRGB_write_control is present, so GL_FRAMEBUFFER_SRGB can be turned off.
// GLES has no such switch in core: writes into an sRGB attachment are ALWAYS encoded, // GLES has no such switch in core: writes into an sRGB attachment are ALWAYS encoded,
// while desktop GL leaves GL_FRAMEBUFFER_SRGB disabled by default and writes raw. // while desktop GL leaves GL_FRAMEBUFFER_SRGB disabled by default and writes raw.
+158 -4
View File
@@ -16,7 +16,12 @@
// Only for the compile-time MAX_VERTEX_ATTRIBS constant asserted below. The POST still executes no // Only for the compile-time MAX_VERTEX_ATTRIBS constant asserted below. The POST still executes no
// MG_State code: it runs standalone, before MG_State::Init(). // MG_State code: it runs standalone, before MG_State::Init().
#include <MG_State/GLState/VertexArrayState/VertexArrayObject.h> #include <MG_State/GLState/VertexArrayState/VertexArrayObject.h>
#include <MG_Backend/DirectGLES/Utils.h>
#include <MG_Util/Converters/GLToStr/GLEnumConverter.h>
#include <MG_Util/Converters/MGToGL/TextureEnumConverter.h>
#include <MG_Util/Converters/MGToStr/GLExtensionConverter.h> #include <MG_Util/Converters/MGToStr/GLExtensionConverter.h>
#include <MG_Util/Converters/MGToStr/TextureEnumConverter.h>
#include <MG_Util/Texture/TextureFormatProcessor.h>
#include <MG_Util/Async/ShaderCompilePool.h> #include <MG_Util/Async/ShaderCompilePool.h>
#include <chrono> #include <chrono>
#include <thread> #include <thread>
@@ -146,12 +151,20 @@ namespace MobileGL::MG_Util::SelfTest {
return; return;
} }
const Uint threads = MG_Util::Async::DetectShaderCompileThreadCount(); const Uint threads = MG_Util::Async::DetectShaderCompileThreadCount();
// The execution engine is named here too. It changes no observable GL behaviour -
// both engines run the same job queue under the same budget - but when a scaling
// or stall report comes back from a device, "which engine was this?" is the first
// question, and a POST page is the one artefact that always accompanies it.
const char* const engineName =
MG_Util::Async::AsyncPoolEngineName(MG_Util::Async::DetectAsyncPoolEngine());
builder.Pass(rowName, builder.Pass(rowName,
format("on with {} compiler thread{}; GL_KHR_parallel_shader_compile is advertised " format("on with {} compiler thread{} on the {} execution engine; "
"GL_KHR_parallel_shader_compile is advertised "
"and GL_MAX_SHADER_COMPILER_THREADS_KHR = {} (set environment variable " "and GL_MAX_SHADER_COMPILER_THREADS_KHR = {} (set environment variable "
"MOBILEGL_ASYNC_SHADER_COMPILE=0 to disable it, or " "MOBILEGL_ASYNC_SHADER_COMPILE=0 to disable it, "
"MOBILEGL_ASYNC_SHADER_COMPILE_THREADS=n to change the count)", "MOBILEGL_ASYNC_SHADER_COMPILE_THREADS=n to change the count, or "
threads, threads == 1 ? "" : "s", threads)); "MOBILEGL_ASYNC_POOL=asio|libfork to change the engine)",
threads, threads == 1 ? "" : "s", engineName, threads));
} }
// Appends the four "MobileGL reported ..." rows for one backend section. // Appends the four "MobileGL reported ..." rows for one backend section.
@@ -450,6 +463,35 @@ namespace MobileGL::MG_Util::SelfTest {
builder.Warn("GL_EXT_texture_norm16", builder.Warn("GL_EXT_texture_norm16",
"not supported; 16-bit normalized texture formats need emulation"); "not supported; 16-bit normalized texture formats need emulation");
} }
if (caps.SupportsRenderSnorm) {
builder.Pass("GL_EXT_render_snorm",
"supported (signed-normalized formats are colour-renderable, so an "
"SNORM render target keeps its own encoding instead of a float substitute)");
} else {
builder.Warn("GL_EXT_render_snorm",
"not supported; signed-normalized formats are texture-only, so every SNORM "
"render target is stored as a float (GL_RGBA8_SNORM/GL_RGB8_SNORM -> "
"GL_RGBA16F) and its fragment outputs are clamped to [-1,1] in software");
}
// FAIL, not WARN: ES 3.x core makes every float format texture-only, and every Iris
// shaderpack renders into at least GL_R11F_G11F_B10F (Complementary's colortex0, BSL's
// colortex0). Without this extension there is no substitute format left - a half float
// is not renderable either - so shaderpacks cannot work at all on such a driver.
if (caps.SupportsColorBufferFloat) {
builder.Pass("GL_EXT_color_buffer_float",
"supported (GL_R11F_G11F_B10F / GL_RGBA16F / GL_RGBA32F are "
"colour-renderable, which is what every shaderpack renders into)");
} else if (caps.SupportsColorBufferHalfFloat) {
builder.Warn("GL_EXT_color_buffer_float",
"not supported, but GL_EXT_color_buffer_half_float is; 16-bit float render "
"targets work, 32-bit float ones (GL_RGBA32F, and the GL_RGBA16 fallback "
"that lands on it) do not");
} else {
builder.Fail("GL_EXT_color_buffer_float",
"not supported, and neither is GL_EXT_color_buffer_half_float; no floating-point "
"format is colour-renderable on this driver, so no shaderpack can create its "
"render targets (Iris reports GL_FRAMEBUFFER_UNSUPPORTED and refuses to load)");
}
// INFO, never WARN: this is the HOST driver's ability to compile its own ESSL on // INFO, never WARN: this is the HOST driver's ability to compile its own ESSL on
// its own threads, and MobileGL's asynchronous compilation does not depend on it // its own threads, and MobileGL's asynchronous compilation does not depend on it
@@ -783,6 +825,117 @@ namespace MobileGL::MG_Util::SelfTest {
} }
} }
// No real ES driver renders to a three-channel image, but desktop GL applications ask for
// one constantly - Complementary Reimagined's colortex1 is GL_RGB8_SNORM and its colortex2
// is GL_RGB16F, and Iris refuses to load when a framebuffer built from them is not
// COMPLETE. DirectGLES substitutes the four-channel sibling, and this row names the
// outcome per format so the failure mode is a five-second read instead of an
// investigation. Answered from the capability cache that was just probed on this very
// driver, so it costs no extra GL work.
void ReportThreeChannelColorAttachments(ReportBuilder& builder, const MG_External::GLESCapabilities& caps,
const MG_Backend::FormatCapabilityCache& cache) {
// GL_RGB8 is the control: it is ES-core renderable, and it is exactly why BSL loads on
// the same driver where Complementary does not. The rest are one representative of
// each widening class - signed-normalized, half float, 32-bit float, sRGB, integer -
// so the row says which CLASS of shaderpack target a device cannot serve rather than
// just "three-channel formats".
constexpr TextureInternalFormat kProbedFormats[] = {
TextureInternalFormat::RGB8, TextureInternalFormat::RGB8Snorm, TextureInternalFormat::RGB16F,
TextureInternalFormat::RGB32F, TextureInternalFormat::SRGB8, TextureInternalFormat::RGB8UI};
const SizeT targetIndex = MG_Backend::GetFormatCapabilityTargetIndex(TextureTarget::Texture2D);
const Flags<PixelFormatNormalizeOptionBit> renderTargetOptions =
MG_Backend::DirectGLES::TextureImpl::GetRenderTargetNormalizeOptions(caps, targetIndex);
String nativeList;
String widenedList;
String unusableList;
// GL_RGB8 is colour-renderable in ES 3.0 CORE. A driver that answers no to it is
// broken (or the probe itself is), and that is the ONLY three-channel verdict that
// deserves a FAIL on its own - see the verdict block below.
Bool controlFormatBroken = false;
const auto append = [](String& list, const String& entry) {
if (!list.empty()) list += ", ";
list += entry;
};
for (const TextureInternalFormat probedFormat : kProbedFormats) {
const SizeT formatIndex = static_cast<SizeT>(probedFormat);
const String name = MG_Util::ConvertTextureInternalFormatToString(probedFormat);
if (MG_Backend::HasFormatCapability(cache.FullCaps[targetIndex][formatIndex],
MG_Backend::FormatCapability::FramebufferRenderable)) {
append(nativeList, name);
continue;
}
if (probedFormat == TextureInternalFormat::RGB8) {
controlFormatBroken = true;
}
if (MG_Backend::HasFormatCapability(cache.CaveatCaps[targetIndex][formatIndex],
MG_Backend::FormatCapability::FramebufferRenderable)) {
GLenum widenedInternalFormat = GL_UNKNOWN_MGL;
MG_Util::TextureFormatProcessor::NormalizePixelFormat(
MG_Util::ConvertTextureInternalFormatToGLEnum(probedFormat), renderTargetOptions,
&widenedInternalFormat, nullptr, nullptr);
append(widenedList, name + " -> " + MG_Util::ConvertGLEnumToString(widenedInternalFormat));
continue;
}
append(unusableList, name);
}
String detail;
if (!nativeList.empty()) detail += "renderable natively: " + nativeList;
if (!widenedList.empty()) {
if (!detail.empty()) detail += "; ";
detail += "widened to stay renderable: " + widenedList;
}
if (!unusableList.empty()) {
if (!detail.empty()) detail += "; ";
detail += "NOT renderable and not substitutable: " + unusableList;
}
// The verdict deliberately does NOT track "every probed format came out usable".
//
// GL_RGB32F widens to GL_RGBA32F, and GL_RGBA32F is colour-renderable only under
// GL_EXT_color_buffer_float. A perfectly healthy half-float-only driver (the common
// mobile shape: EXT_color_buffer_half_float and nothing more) therefore reports
// GL_RGB32F as unusable while every format a shaderpack actually renders into works.
// FAILing that device would make the POST's hardest verdict fire on a configuration
// MobileGL runs fine on, which is exactly how a report stops being read.
//
// So FAIL is reserved for the two answers that really are broken:
// * the ES-core control (GL_RGB8) is not renderable - the probe or the driver is
// wrong about something much more basic than three-channel widening; and
// * a widenable format has no usable fallback ON A DRIVER THAT ADVERTISES
// GL_EXT_color_buffer_float - the extension promises the widened float targets
// are renderable, so a gap here is a real, unexplained refusal.
// Everything else is a WARN carrying the exact per-format status, which is what the
// row is for. The "no float render targets at all" case is already a FAIL of its own
// on the GL_EXT_color_buffer_float row above; repeating it here would only double-count.
if (controlFormatBroken) {
builder.Fail("Three-channel colour attachments",
detail + " - GL_RGB8 is colour-renderable in OpenGL ES 3.0 core, so a driver "
"that refuses it cannot render to ANY three-channel attachment and the "
"capability probe itself is suspect");
} else if (!unusableList.empty() && caps.SupportsColorBufferFloat) {
builder.Fail("Three-channel colour attachments",
detail + " - GL_EXT_color_buffer_float is supported, so the widened "
"four-channel float targets are required to be renderable; a framebuffer "
"using one of the formats above still reports GL_FRAMEBUFFER_UNSUPPORTED, "
"which Iris turns into a hard load failure");
} else if (!unusableList.empty()) {
builder.Warn("Three-channel colour attachments",
detail + " - without GL_EXT_color_buffer_float the 32-bit float widening has no "
"renderable target left, so a shaderpack asking for one of the formats "
"above gets GL_FRAMEBUFFER_UNSUPPORTED; the half-float and fixed-point "
"ones above still work");
} else if (!widenedList.empty()) {
builder.Warn("Three-channel colour attachments",
detail + " - the substitution costs the extra alpha channel's memory and is "
"hidden from the application by an ALPHA->ONE swizzle");
} else {
builder.Pass("Three-channel colour attachments", detail);
}
}
// Everything the "MobileGL reported ..." rows need from the GLES device probe. // Everything the "MobileGL reported ..." rows need from the GLES device probe.
struct GlesProbeSummary { struct GlesProbeSummary {
Bool capsValid = false; Bool capsValid = false;
@@ -913,6 +1066,7 @@ namespace MobileGL::MG_Util::SelfTest {
builder.report.formatCapabilities.emplace(); builder.report.formatCapabilities.emplace();
MG_Backend::DirectGLES::PopulateFormatCapabilities( MG_Backend::DirectGLES::PopulateFormatCapabilities(
glesFuncs, caps, builder.report.formatCapabilities.value()); glesFuncs, caps, builder.report.formatCapabilities.value());
ReportThreeChannelColorAttachments(builder, caps, builder.report.formatCapabilities.value());
} while (false); } while (false);
} }
@@ -58,12 +58,99 @@ namespace MobileGL::MG_Util::TextureFormatProcessor {
case GL_R8_SNORM: case GL_R8_SNORM:
applicableOptions |= options & PixelFormatNormalizeOptionBit::NoSnorm8; applicableOptions |= options & PixelFormatNormalizeOptionBit::NoSnorm8;
break; break;
// The rest of the three-channel formats no real ES driver renders to. They have no
// other fallback: none of the driver/forced option bits names them, so before the
// render-target widening existed for ordinary targets an FBO attachment in one of
// them could only ever be answered GL_FRAMEBUFFER_UNSUPPORTED (Complementary
// Reimagined's colortex2 = RGB16F).
//
// GL_RGB9_E5 is deliberately absent: its four-channel sibling would have to be a
// half float, which means unpacking the shared exponent on every transfer, and
// nothing renders to a shared-exponent format on desktop GL either.
case GL_RGB16F:
case GL_RGB32F:
case GL_SRGB8:
case GL_RGB8I:
case GL_RGB8UI:
case GL_RGB16I:
case GL_RGB16UI:
case GL_RGB32I:
case GL_RGB32UI:
applicableOptions |= options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget;
break;
default: default:
break; break;
} }
return applicableOptions; return applicableOptions;
} }
namespace {
// The four-channel sibling a three-channel format is widened to when the target has to
// stay colour-renderable, together with the transfer pair that describes client data for
// it. Kept in one place because all three of NormalizePixelFormat's switches have to agree:
// reporting the widened storage but the original three-channel base format emitted
// inconsistent triples such as (GL_RGBA16F, GL_RGB, GL_BYTE), which is
// GL_INVALID_OPERATION for glTexImage2D on ES. That only ever went unnoticed because the
// bit was reachable for multisample storage alone, and glTexStorage*Multisample takes no
// transfer pair at all.
struct ThreeChannelWidening {
GLenum InternalFormat = GL_UNKNOWN_MGL;
GLenum Format = GL_UNKNOWN_MGL;
GLenum Type = GL_UNKNOWN_MGL;
explicit operator Bool() const { return InternalFormat != GL_UNKNOWN_MGL; }
};
ThreeChannelWidening GetThreeChannelRenderTargetWidening(GLenum internalFormat,
Flags<PixelFormatNormalizeOptionBit> options) {
if (!(options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget)) {
return {};
}
switch (internalFormat) {
// Signed-normalized: matches what the always-on NoRGBA8Snorm fallback already does to
// GL_RGBA8_SNORM, so the two SNORM8 formats land on the same storage.
case GL_RGB8_SNORM:
return {GL_RGBA16F, GL_RGBA, GL_FLOAT};
case GL_RGB16_SNORM:
// A half float loses the low bits of a 16-bit SNORM channel, so keep the
// signed-normalized encoding whenever the driver can render to it.
return (options & PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget)
? ThreeChannelWidening{GL_RGBA16F, GL_RGBA, GL_FLOAT}
: ThreeChannelWidening{GL_RGBA16_SNORM, GL_RGBA, GL_SHORT};
// Unsigned-normalized 16-bit (and the legacy 10/12-bit formats stored as RGB16):
// GL_RGB32F is a legal ES texture format but is not colour-renderable either.
case GL_RGB16:
case GL_RGB10:
case GL_RGB12:
return {GL_RGBA32F, GL_RGBA, GL_FLOAT};
// Floating point.
case GL_RGB16F:
return {GL_RGBA16F, GL_RGBA, GL_HALF_FLOAT};
case GL_RGB32F:
return {GL_RGBA32F, GL_RGBA, GL_FLOAT};
// sRGB: GL_SRGB8_ALPHA8 keeps the sRGB encoding of the colour channels and stores
// the added alpha linearly, which is exactly the three-channel format's semantics.
case GL_SRGB8:
return {GL_SRGB8_ALPHA8, GL_RGBA, GL_UNSIGNED_BYTE};
// Integer.
case GL_RGB8I:
return {GL_RGBA8I, GL_RGBA_INTEGER, GL_BYTE};
case GL_RGB8UI:
return {GL_RGBA8UI, GL_RGBA_INTEGER, GL_UNSIGNED_BYTE};
case GL_RGB16I:
return {GL_RGBA16I, GL_RGBA_INTEGER, GL_SHORT};
case GL_RGB16UI:
return {GL_RGBA16UI, GL_RGBA_INTEGER, GL_UNSIGNED_SHORT};
case GL_RGB32I:
return {GL_RGBA32I, GL_RGBA_INTEGER, GL_INT};
case GL_RGB32UI:
return {GL_RGBA32UI, GL_RGBA_INTEGER, GL_UNSIGNED_INT};
default:
return {};
}
}
} // namespace
void NormalizePixelFormat(GLenum internalFormat, Flags<PixelFormatNormalizeOptionBit> options, void NormalizePixelFormat(GLenum internalFormat, Flags<PixelFormatNormalizeOptionBit> options,
GLenum* outInternalFormat, GLenum* outFormat, GLenum* outType) { GLenum* outInternalFormat, GLenum* outFormat, GLenum* outType) {
#ifdef TRACY_ENABLE #ifdef TRACY_ENABLE
@@ -95,13 +182,6 @@ namespace MobileGL::MG_Util::TextureFormatProcessor {
*outInternalFormat = internalFormat; *outInternalFormat = internalFormat;
break; break;
case GL_RGB16: case GL_RGB16:
if (options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget) {
// GL_RGB32F is a legal ES texture format but is not colour-renderable, so
// glTexStorage2DMultisample rejects it and the attachment ends up with no
// storage at all.
*outInternalFormat = GL_RGBA32F;
break;
}
if ((options & PixelFormatNormalizeOptionBit::NoNorm16) || if ((options & PixelFormatNormalizeOptionBit::NoNorm16) ||
(options & PixelFormatNormalizeOptionBit::NoRgb16)) { (options & PixelFormatNormalizeOptionBit::NoRgb16)) {
*outInternalFormat = GL_RGB32F; *outInternalFormat = GL_RGB32F;
@@ -132,14 +212,6 @@ namespace MobileGL::MG_Util::TextureFormatProcessor {
*outInternalFormat = internalFormat; *outInternalFormat = internalFormat;
break; break;
case GL_RGB16_SNORM: case GL_RGB16_SNORM:
if (options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget) {
// A half float loses the low bits of a 16-bit SNORM channel, so keep the
// signed-normalized encoding whenever the driver can render to it.
*outInternalFormat = (options & PixelFormatNormalizeOptionBit::NoSnorm16RenderTarget)
? GL_RGBA16F
: GL_RGBA16_SNORM;
break;
}
if ((options & PixelFormatNormalizeOptionBit::NoNorm16) || if ((options & PixelFormatNormalizeOptionBit::NoNorm16) ||
(options & PixelFormatNormalizeOptionBit::NoRGB16Snorm) || (options & PixelFormatNormalizeOptionBit::NoRGB16Snorm) ||
(options & PixelFormatNormalizeOptionBit::NoSnorm16)) { (options & PixelFormatNormalizeOptionBit::NoSnorm16)) {
@@ -173,10 +245,6 @@ namespace MobileGL::MG_Util::TextureFormatProcessor {
*outInternalFormat = internalFormat; *outInternalFormat = internalFormat;
break; break;
case GL_RGB8_SNORM: case GL_RGB8_SNORM:
if (options & PixelFormatNormalizeOptionBit::NoThreeChannelRenderTarget) {
*outInternalFormat = GL_RGBA16F;
break;
}
if (options & PixelFormatNormalizeOptionBit::NoSnorm8) { if (options & PixelFormatNormalizeOptionBit::NoSnorm8) {
*outInternalFormat = GL_RGB16F; *outInternalFormat = GL_RGB16F;
break; break;
@@ -615,5 +683,18 @@ namespace MobileGL::MG_Util::TextureFormatProcessor {
break; break;
} }
} }
// Applied last, over whatever the three switches above chose: widening a three-channel
// format to keep a colour attachment renderable outranks every other fallback, because
// the others all pick a three-channel storage the driver still refuses to render to
// (GL_RGB8_SNORM -> GL_RGB16F under NoSnorm8, GL_RGB16 -> GL_RGB32F under NoNorm16).
// All three outputs move together: reporting the widened storage while leaving the
// three-channel base format and its component type in place produced triples like
// (GL_RGBA16F, GL_RGB, GL_BYTE), which ES rejects for glTexImage2D outright.
if (const ThreeChannelWidening widening = GetThreeChannelRenderTargetWidening(internalFormat, options)) {
if (outInternalFormat) *outInternalFormat = widening.InternalFormat;
if (outFormat) *outFormat = widening.Format;
if (outType) *outType = widening.Type;
}
} }
} // namespace MobileGL::MG_Util::TextureFormatProcessor } // namespace MobileGL::MG_Util::TextureFormatProcessor
@@ -20,9 +20,14 @@ namespace MobileGL {
NoRGB16Snorm = 1 << 6, NoRGB16Snorm = 1 << 6,
// The target must be colour-renderable and ES has no renderable three-channel // The target must be colour-renderable and ES has no renderable three-channel
// form of the requested format, so it has to be widened to the four-channel one. // form of the requested format, so it has to be widened to the four-channel one.
// Only meaningful for multisample textures: those can never be uploaded to, only // Set for any colour-attachable target whose native three-channel form the driver
// rendered into, so the extra alpha comes from the draw (1.0 for an RGB source) // refused to render to (multisample storage always, since ES has no three-channel
// and no transfer path has to expand three-channel client data. // multisample format at all; every other target only after its native probe failed).
// The widening is visible to every transfer path, so it also retargets the (format,
// type) pair NormalizePixelFormat reports: the upload has to describe four
// components in the widened storage's component type, the backend has to expand
// three-channel client data with an alpha of 1.0, and sampling/readback has to hide
// the added alpha again (BackendTextureFormatAddsAlpha).
NoThreeChannelRenderTarget = 1 << 7, NoThreeChannelRenderTarget = 1 << 7,
// Pairs with the bit above: the widened four-channel format has to stay renderable AND // Pairs with the bit above: the widened four-channel format has to stay renderable AND
// keep 16-bit signed-normalized precision, which needs both EXT_texture_norm16 and // keep 16-bit signed-normalized precision, which needs both EXT_texture_norm16 and
+6
View File
@@ -63,6 +63,12 @@ The bundled fixtures cover:
- minecraft-1.21.4-fabric-iris-bsl-in-world: captured from Minecraft 1.21.4 Fabric with Sodium, Iris, and BSL - minecraft-1.21.4-fabric-iris-bsl-in-world: captured from Minecraft 1.21.4 Fabric with Sodium, Iris, and BSL
Shaders after entering a singleplayer world. Shaders after entering a singleplayer world.
![Minecraft 1.21.4 Fabric Iris BSL in-world golden](fixtures/minecraft-1.21.4-fabric-iris-bsl-in-world.0000110725.png) ![Minecraft 1.21.4 Fabric Iris BSL in-world golden](fixtures/minecraft-1.21.4-fabric-iris-bsl-in-world.0000110725.png)
- minecraft-1.21.4-fabric-iris-bsl-esc-menu-854: captured on an Android device (Mali-G77, FCL MobileGL capture) from
Minecraft 1.21.4 Fabric with Sodium, Iris, and BSL Shaders, at the pause menu over a BSL-blurred world. The frame
pins glyph rendering: every menu label, the menu title and the tutorial toast must be present. Regressions in the
DirectGLES per-draw texture memo have made the whole text path disappear here while sprites kept rendering, so a
failure that leaves the buttons but empties them is the signature to look for in the diff.
![Minecraft 1.21.4 Fabric Iris BSL ESC menu 854x480 golden](fixtures/minecraft-1.21.4-fabric-iris-bsl-esc-menu-854.0001303534.png)
- minecraft-1.21.4-fabric-iris-makeup-ultrafast-in-world: captured from Minecraft 1.21.4 Fabric with Sodium, Iris, and - minecraft-1.21.4-fabric-iris-makeup-ultrafast-in-world: captured from Minecraft 1.21.4 Fabric with Sodium, Iris, and
MakeUP UltraFast after entering a singleplayer world. MakeUP UltraFast after entering a singleplayer world.
![Minecraft 1.21.4 Fabric Iris MakeUP UltraFast in-world golden](fixtures/minecraft-1.21.4-fabric-iris-makeup-ultrafast-in-world.0000095322.png) ![Minecraft 1.21.4 Fabric Iris MakeUP UltraFast in-world golden](fixtures/minecraft-1.21.4-fabric-iris-makeup-ultrafast-in-world.0000095322.png)
+6
View File
@@ -289,6 +289,12 @@
"golden": "minecraft-1.21.4-fabric-iris-iterationrp-in-world.0000202020.png", "golden": "minecraft-1.21.4-fabric-iris-iterationrp-in-world.0000202020.png",
"target_call": 202020, "target_call": 202020,
"timeout_seconds": 1800 "timeout_seconds": 1800
},
{
"name": "minecraft-1.21.4-fabric-iris-bsl-esc-menu-854",
"trace_archive": "minecraft-1.21.4-fabric-iris-bsl-esc-menu-854.tgz",
"golden": "minecraft-1.21.4-fabric-iris-bsl-esc-menu-854.0001303534.png",
"target_call": 1303534
} }
] ]
} }