[MG_State, MG_Backend, MG_Test, docs] (P7): the reflection archive publishes the storage-block index space, and both magma consumers read it

CONTRACT-P7 §5.3, OQ-8. MagmaProgramSource::EnsureStorageBlocks re-ran SPIRV-Reflect over
every stage module of the program PER DRAW: the class is a borrowed view that
WireDraw.inc's WireProgramSource constructs afresh for each wire draw, so its `mutable`
memo was cold every time it was read. The list it rebuilt is a pure function of the linked
program, so it is now built once at link time and travels in the archive.

LinkArtifacts gains storageBlocks{name, binding, dataSize} with its own VisitFields table,
the codec version goes 2 -> 3, and the schema fingerprint and wireFingerprint move with
it - which is expected, and is what makes a mixed-version pair refuse at the handshake
instead of at the first program.

FILLED IN PHASE A, and that is not where it looks like it belongs. The only function that
sees both finished halves is ProgramSpirvTask's L1 cache insert, but its writes reach the
TRANSLATION CACHE only: phase A's artifacts have already been moved into the ProgramObject
by then. A field filled there would be present on a cache hit and absent on every cold
link. So the fill is glslang-derived, and the equivalence is MEASURED rather than assumed:
ProgramTest.TheArchivesStorageBlockOrderIsTheOneSpirvReflectProduces runs the old
consumer's own algorithm over the real modules and compares element for element, on a
multi-stage program with an arrayed SSBO and an atomic-counter block - the three cases
that could have diverged (the instance array collapses to one entry, the synthesized
counter block IS in the backend's space although GL's enumeration excludes it, and the
cross-stage dedupe walks stages in linked order).

MEASURED, same binary and same window (frames=1 draws=1 acc=18, identical bytes),
DirectVulkan x inproc over the SSBO scenarios: spvReflectCreateShaderModule on the wire
draw path 4 -> 0. The 0 is structural: the call site is gone and MagmaProgramSource.h no
longer includes spirv_reflect.h at all, which is the gate a future edit breaks on purpose.

THE MONOLITH CONSUMER IS SWITCHED TOO, under #if symmetry, so the disaggregated build
retires g_programResourceCaches, ProgramResourceCache, StorageBlockResource,
BufferVariableResource, AddBufferVariablesRecursive, NormalizeDescriptorName,
GetProgramResourceCache and the rehash hazard - the open-addressed-map dangling reference
that was a reproducible segfault in ProgramPipelineScenario's two storage-block cases.
Nothing outside DirectVulkan.cpp read any of it (measured; GL_BUFFER_VARIABLE and
GL_ACTIVE_VARIABLES are answered by ProgramInterface.cpp out of a different index space),
so the two arms differ in exactly the two functions their callers use.

THE PULL BUILD KEEPS ALL OF IT, and the integrator should see why: the archive member is
MOBILEGL_BUILD_DISAGGREGATED-only because one more Vector in LinkArtifacts changes its
size, its implicit destructor and its move constructor, and G1 pins the pull build's .text
byte for byte. So §5.3's deletion is achieved in the disaggregated build and not in the
pull one; retiring the pull half is a one-line change the day that pin is retired or the
member moves. Filed as P13 in notes/p7/magma-c.md.

One improvement rides along: the monolith binding getter now asks the program for a
rebind on every read, over an immutable list, where it used to read a binding patched into
a mutable cache - so a rebind can no longer be lost to a cache rebuild that raced a
state-version bump.

G1: .text 0xa52203 unchanged, 27837 -> 27837 defined symbols, 0 added / 0 removed / 0
resized / 0 renamed, file byte-identical. An unguarded call site caught the guard rule
once mid-slice by failing the pull compile, which is the loud version.

RED-ONCE: publishing nothing reds the order-equivalence case (with the reference list
printed) and 25 of 60 storage-block scenarios - the MONOLITH DirectVulkan arm beside all
three transports, which is the evidence that the monolith consumer really was switched.
Restored: 61/61.
This commit is contained in:
2026-09-22 16:07:59 -04:00
parent f1b04edfc6
commit 0acfa9aab9
11 changed files with 663 additions and 57 deletions
@@ -131,6 +131,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
namespace {
#if !MOBILEGL_BUILD_DISAGGREGATED
// P7 wave 2 package C, OQ-8: THE PULL BUILD'S HALF, VERBATIM. A disaggregated build
// answers GetShaderStorageBlock{Index,Binding} out of LinkArtifacts::storageBlocks
// instead (see those functions below), which retires this whole cache - the map, the
// SPIRV-Reflect rebuild, the teardown clear and the rehash hazard. It cannot be
// retired HERE because the archive member it reads is itself disaggregated-only, for
// G1: one more Vector in LinkArtifacts moves the pull build's .text, which is pinned
// byte-for-byte at 0xa52203.
//
// NOTHING OUTSIDE THIS FILE READS ANY OF IT (measured: BufferVariableResource,
// bufferVariables, activeVariables and StorageBlockResource have no reference anywhere
// else in the tree, and `dataSize` is written and never read - GL_BUFFER_VARIABLE and
// GL_ACTIVE_VARIABLES are answered by MG_Impl/GLImpl/Program/ProgramInterface.cpp out
// of glslang's own reflection, which is a different index space). So the two arms
// below differ in exactly the two functions their callers use, and in nothing else.
struct BufferVariableResource {
String name;
GLuint blockIndex = 0;
@@ -162,6 +177,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Vector<StorageBlockResource> storageBlocks;
Vector<BufferVariableResource> bufferVariables;
};
#endif // !MOBILEGL_BUILD_DISAGGREGATED
struct DrawElementsIndirectCommand {
Uint32 count = 0;
@@ -178,12 +194,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 baseInstance = 0;
};
#if !MOBILEGL_BUILD_DISAGGREGATED
// Keyed by GL program name so the freed-name reuse in IndexGenerator bounds the
// map at the peak-simultaneous-program high-water mark; each slot's ownership is
// checked against the program's lifetime id before it is served (see
// GetProgramResourceCache). Cleared wholesale at EGL teardown via
// ClearProgramResourceCaches.
UnorderedMap<GLuint, ProgramResourceCache> g_programResourceCaches;
#endif
void ClearReadPixelsOutput(GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels) {
if (!pixels || width <= 0 || height <= 0) {
@@ -198,6 +216,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
#if !MOBILEGL_BUILD_DISAGGREGATED
String NormalizeDescriptorName(const SpvReflectDescriptorBinding& binding) {
const char* rawName = binding.name;
if (binding.type_description != nullptr && binding.type_description->type_name != nullptr) {
@@ -348,6 +367,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
return cache;
}
#endif // !MOBILEGL_BUILD_DISAGGREGATED
#if MOBILEGL_BUILD_DISAGGREGATED
// The verb's handles identify server stores. Readback orders GPU-produced
@@ -448,6 +468,67 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} // namespace
#if MOBILEGL_BUILD_DISAGGREGATED
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): THE MONOLITH CONSUMER READS THE ARCHIVE
// TOO, so one published list serves both of this backend's arms.
//
// THE TWO ARMS BELOW ARE NOT TWO ANSWERS. ProgramTest's
// TheArchivesStorageBlockOrderIsTheOneSpirvReflectProduces runs the OLD arm's algorithm -
// SPIRV-Reflect over the real modules, sorted and deduplicated exactly as
// GetProgramResourceCache does it - over a multi-stage program with an arrayed SSBO and an
// atomic-counter block, and asserts element-for-element equality with what the archive
// published. That equivalence is what makes an #if here a build-time SELECTION rather than
// a behavioural fork, and it is the reason the case exists.
//
// WHY AN #if AT ALL, AND WHAT IT COSTS. LinkArtifacts::storageBlocks is
// MOBILEGL_BUILD_DISAGGREGATED-only, because one more Vector member changes the struct's
// size, its implicit destructor and its move constructor in the PULL build whose .text G1
// pins byte-for-byte (0xa52203). So the pull build keeps `g_programResourceCaches`, the
// SPIRV-Reflect rebuild and `ClearProgramResourceCaches`, verbatim; only this build drops
// them - and with them the rehash hazard the ordering comment below the block-binding
// setter documents. Retiring the pull half is the day the member stops needing its guard,
// which is a G1/P13 decision and not a wave-2 one; recorded in notes/p7/magma-c.md.
void ClearProgramResourceCaches() {
// Nothing to clear: there is no cache in this build. Kept as an entry point so
// BackendObject_DirectVulkan's two EGL-teardown call sites stay statement for
// statement what they are in the pull build.
}
GLuint GetShaderStorageBlockIndex(const MG_State::GLState::ProgramObject& program, const String& name) {
const auto& blocks = program.GetLinkReflection().storageBlocks;
const auto find = [&blocks](const String& key) {
return std::find_if(blocks.begin(), blocks.end(),
[&](const MG_State::GLState::StorageBlockReflection& block) { return block.name == key; });
};
auto it = find(name);
if (it == blocks.end()) {
// Archive names are normalised, so an arrayed block that GL enumerates per element
// - "B[0]", "B[1]" - is one entry here, spelled "B". Same second chance the cache
// gave, for the same callers.
const auto bracket = name.rfind('[');
if (bracket == String::npos || name.empty() || name.back() != ']') return GL_INVALID_INDEX;
it = find(name.substr(0, bracket));
if (it == blocks.end()) return GL_INVALID_INDEX;
}
return static_cast<GLuint>(std::distance(blocks.begin(), it));
}
GLuint GetShaderStorageBlockBinding(const MG_State::GLState::ProgramObject& program, GLuint blockIndex) {
const auto& blocks = program.GetLinkReflection().storageBlocks;
if (blockIndex >= blocks.size()) {
return 0;
}
// THE OVERRIDE IS APPLIED ON READ, where the cache used to apply it on rebuild and
// patch it in place on rebind. Same answer by a shorter route: the program is the
// authoritative record of a rebound block (it is what GL_BUFFER_BINDING reports), the
// archive carries the DECLARED binding only, and asking the program here means a
// rebind can no longer be lost to a cache rebuild that happened to race a state-version
// bump.
const Int rebound = program.GetShaderStorageBlockBindingOverride(blocks[blockIndex].name);
if (rebound >= 0) return static_cast<GLuint>(rebound);
return blocks[blockIndex].binding;
}
#else
void ClearProgramResourceCaches() {
// Called from EGL teardown while the backend's m_eglStateMutex is held; GL
// calls are serialized in this codebase (contexts migrate threads but never
@@ -483,6 +564,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
return cache.storageBlocks[blockIndex].binding;
}
#endif
void ClearBufferfi(GLenum buffer, GLint drawbuffer, GLfloat depth, GLint stencil) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::ClearBufferfi called with null VulkanRenderer");
@@ -949,6 +1031,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MakeUnique<GenericErrorInfo>("DirectVulkan", __func__, "Shader storage binding is out of range."));
return;
}
#if MOBILEGL_BUILD_DISAGGREGATED
// P7 wave 2 package C, OQ-8: NOTHING LEFT TO PATCH, and the hazard goes with it.
//
// The frontend has already recorded the new binding on the program, and that record is
// now what GetShaderStorageBlockBinding reads (it asks
// GetShaderStorageBlockBindingOverride on every call, over an IMMUTABLE archive list).
// What stood here was the other half of a mutable cache: resolve the index, take a
// reference into g_programResourceCaches and patch `binding` in place so an
// already-built entry need not be thrown away.
//
// THE HAZARD THAT CAME WITH IT IS GONE TOO, and it is worth naming because it was a
// real crash rather than a theoretical one: GetShaderStorageBlockIndex re-entered
// GetProgramResourceCache, which indexes an OPEN-ADDRESSED map and can therefore
// insert, and a rehash moves entries - so a reference taken before that call dangled.
// Binding one program's storage block while another program's entry was still absent
// from the cache was a reproducible segfault (ProgramPipelineScenario's two
// storage-block cases, in one process). The ordering above was the fix; having no
// cache is the retirement. The pull build below keeps both.
(void)storageBlockBinding;
}
#else
// The frontend already validated that the name denotes an active block, and has
// already recorded the new binding on the program - which is what reseeds this cache
// whenever it is rebuilt. Writing the entry here as well keeps an ALREADY-BUILT cache
@@ -967,6 +1070,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (blockIndex >= cache.storageBlocks.size()) return;
cache.storageBlocks[blockIndex].binding = storageBlockBinding;
}
#endif
void ReadPixels(GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, void* pixels) {
MOBILEGL_ASSERT(pVulkanRenderer, "DirectVulkan::ReadPixels called with null VulkanRenderer");
MOBILEGL_ASSERT(MGB_CTX_LIVE, "DirectVulkan::ReadPixels called with null GL context");
@@ -7,7 +7,11 @@
#include <MG_Pipe/PipeApply.h>
#include <MG_State/GLState/ProgramState/ProgramArtifactsCodec.h>
#include "../DirectVulkanResourceState.h"
#include <spirv_reflect.h>
// P7 wave 2 package C, OQ-8: spirv_reflect.h is GONE from this header, and its absence is the
// gate. Everything this class needed it for - the storage-block index space - now travels in
// the archive (LinkArtifacts::storageBlocks), so the SPIR-V reflector is off the Magma wire
// draw path entirely. A future edit that re-adds the include is re-adding a per-draw reflect,
// because a MagmaProgramSource is constructed once per draw and can memoise nothing.
#endif
namespace MobileGL::MG_Backend::DirectVulkan {
@@ -153,30 +157,49 @@ namespace MobileGL::MG_Backend::DirectVulkan {
GLenum GetTransformFeedbackBufferMode() const {
return IsWire() ? Link().xfbBufferMode : m_frontend->GetTransformFeedbackBufferMode();
}
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): READ OUT OF THE ARCHIVE.
//
// What stood here re-ran spvReflectCreateShaderModule over EVERY stage module of the
// program, and it did so PER DRAW: a MagmaProgramSource is a borrowed view that
// WireDraw.inc's WireProgramSource constructs afresh for each wire draw, so the lazy
// `m_storageBlocksReady` latch could never outlive one. The list it rebuilt is a pure
// function of the linked program, so it is now built once at link time and travels in
// the archive (LinkArtifacts::storageBlocks), which takes the SPIR-V reflector off the
// wire draw path entirely - this header no longer includes spirv_reflect.h at all.
Uint GetShaderStorageBlockIndex(const String& name) const {
if (!IsWire()) return DirectVulkan::GetShaderStorageBlockIndex(*m_frontend, name);
EnsureStorageBlocks();
String key = name;
auto find = [&] {
return std::find_if(m_storageBlocks.begin(), m_storageBlocks.end(),
[&](const StorageBlock& block) { return block.Name == key; });
const auto& blocks = Link().storageBlocks;
// The archive carries NORMALISED names (the array subscript stripped), because
// that is the spelling SPIRV-Reflect's type_name gives and the spelling
// ProgramFactory::ReflectLayout looks up. A caller that still holds a GL-style
// "Foo[2]" gets the same second chance it always had.
const auto find = [&blocks](const String& key) {
return std::find_if(blocks.begin(), blocks.end(),
[&](const MG_State::GLState::StorageBlockReflection& block) {
return block.name == key;
});
};
auto found = find();
if (found == m_storageBlocks.end() && !name.empty() && name.back() == ']') {
auto found = find(name);
if (found == blocks.end() && !name.empty() && name.back() == ']') {
const auto bracket = name.rfind('[');
if (bracket != String::npos) { key = name.substr(0, bracket); found = find(); }
if (bracket != String::npos) found = find(name.substr(0, bracket));
}
return found == m_storageBlocks.end() ? GL_INVALID_INDEX : static_cast<Uint>(found - m_storageBlocks.begin());
return found == blocks.end() ? GL_INVALID_INDEX
: static_cast<Uint>(found - blocks.begin());
}
Uint GetShaderStorageBlockBinding(Uint index) const {
if (!IsWire()) return DirectVulkan::GetShaderStorageBlockBinding(*m_frontend, index);
EnsureStorageBlocks();
if (index >= m_storageBlocks.size()) return 0;
const auto& block = m_storageBlocks[index];
const auto& blocks = Link().storageBlocks;
if (index >= blocks.size()) return 0;
const auto& block = blocks[index];
// THE OVERRIDE STILL WINS, and it still comes off the RECORD rather than the
// archive: glShaderStorageBlockBinding is a post-link mutable and the archive
// carries link-time defaults only (rule on MagmaProgramSource, and the reason
// StorageBlockReflection::binding is documented as the DECLARED binding).
for (const auto& override : m_record->StorageOverrides) {
if (override.Name == block.Name) return override.Binding;
if (override.Name == block.name) return override.Binding;
}
return block.Binding;
return block.binding;
}
private:
@@ -187,48 +210,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return tIndex >= 0 && static_cast<SizeT>(tIndex) < Link().blockReflection.size()
? &Link().blockReflection[tIndex] : nullptr;
}
struct StorageBlock { String Name; Uint Binding; };
void EnsureStorageBlocks() const {
if (m_storageBlocksReady) return;
m_storageBlocksReady = true;
// Match the legacy resource-query index order: stage order, then
// normalized descriptor name/binding, deduplicated across stages.
for (const auto& spirv : GetGeneratedSpirv()) {
if (spirv.empty()) continue;
SpvReflectShaderModule module{};
if (spvReflectCreateShaderModule(spirv.size() * sizeof(Uint), spirv.data(), &module) != SPV_REFLECT_RESULT_SUCCESS) continue;
Uint32 count = 0;
if (spvReflectEnumerateDescriptorBindings(&module, &count, nullptr) == SPV_REFLECT_RESULT_SUCCESS) {
Vector<SpvReflectDescriptorBinding*> bindings(count);
if (spvReflectEnumerateDescriptorBindings(&module, &count, bindings.data()) == SPV_REFLECT_RESULT_SUCCESS) {
auto name = [](const SpvReflectDescriptorBinding* binding) {
const char* raw = binding->name;
if (binding->type_description && binding->type_description->type_name) raw = binding->type_description->type_name;
String result = raw ? raw : "";
const auto suffix = result.find("[0]");
return suffix == String::npos ? result : result.substr(0, suffix);
};
std::sort(bindings.begin(), bindings.end(), [&](const auto* a, const auto* b) {
const String an = a ? name(a) : String(), bn = b ? name(b) : String();
return an != bn ? an < bn : (a && b && a->binding < b->binding);
});
for (const auto* binding : bindings) {
if (!binding || binding->descriptor_type != SPV_REFLECT_DESCRIPTOR_TYPE_STORAGE_BUFFER) continue;
const String key = name(binding);
if (key.empty() || std::any_of(m_storageBlocks.begin(), m_storageBlocks.end(),
[&](const StorageBlock& block) { return block.Name == key; })) continue;
m_storageBlocks.push_back({key, binding->binding});
}
}
}
spvReflectDestroyShaderModule(&module);
}
}
const Program* m_frontend = nullptr;
MG_Pipe::MGPipeHandle m_handle = MG_Pipe::kMGPipeNullHandle;
const MG_Pipe::MGPipeShaderCsoRecord* m_record = nullptr;
mutable Bool m_storageBlocksReady = false;
mutable Vector<StorageBlock> m_storageBlocks;
// P7 OQ-8: the two mutables that memoised the per-draw reflect are gone with it. They
// were `mutable` precisely because this class is a borrowed const view - and that is
// also why they never helped: a view lives for one draw, so the latch was cold every
// time it was read.
};
#else
using MagmaProgramSource = MG_State::GLState::ProgramObject;