[MG_State, MG_Backend, MG_Test, docs] (P7): the reflection archive publishes the storage-block index space, and both magma consumers read it

CONTRACT-P7 §5.3, OQ-8. MagmaProgramSource::EnsureStorageBlocks re-ran SPIRV-Reflect over
every stage module of the program PER DRAW: the class is a borrowed view that
WireDraw.inc's WireProgramSource constructs afresh for each wire draw, so its `mutable`
memo was cold every time it was read. The list it rebuilt is a pure function of the linked
program, so it is now built once at link time and travels in the archive.

LinkArtifacts gains storageBlocks{name, binding, dataSize} with its own VisitFields table,
the codec version goes 2 -> 3, and the schema fingerprint and wireFingerprint move with
it - which is expected, and is what makes a mixed-version pair refuse at the handshake
instead of at the first program.

FILLED IN PHASE A, and that is not where it looks like it belongs. The only function that
sees both finished halves is ProgramSpirvTask's L1 cache insert, but its writes reach the
TRANSLATION CACHE only: phase A's artifacts have already been moved into the ProgramObject
by then. A field filled there would be present on a cache hit and absent on every cold
link. So the fill is glslang-derived, and the equivalence is MEASURED rather than assumed:
ProgramTest.TheArchivesStorageBlockOrderIsTheOneSpirvReflectProduces runs the old
consumer's own algorithm over the real modules and compares element for element, on a
multi-stage program with an arrayed SSBO and an atomic-counter block - the three cases
that could have diverged (the instance array collapses to one entry, the synthesized
counter block IS in the backend's space although GL's enumeration excludes it, and the
cross-stage dedupe walks stages in linked order).

MEASURED, same binary and same window (frames=1 draws=1 acc=18, identical bytes),
DirectVulkan x inproc over the SSBO scenarios: spvReflectCreateShaderModule on the wire
draw path 4 -> 0. The 0 is structural: the call site is gone and MagmaProgramSource.h no
longer includes spirv_reflect.h at all, which is the gate a future edit breaks on purpose.

THE MONOLITH CONSUMER IS SWITCHED TOO, under #if symmetry, so the disaggregated build
retires g_programResourceCaches, ProgramResourceCache, StorageBlockResource,
BufferVariableResource, AddBufferVariablesRecursive, NormalizeDescriptorName,
GetProgramResourceCache and the rehash hazard - the open-addressed-map dangling reference
that was a reproducible segfault in ProgramPipelineScenario's two storage-block cases.
Nothing outside DirectVulkan.cpp read any of it (measured; GL_BUFFER_VARIABLE and
GL_ACTIVE_VARIABLES are answered by ProgramInterface.cpp out of a different index space),
so the two arms differ in exactly the two functions their callers use.

THE PULL BUILD KEEPS ALL OF IT, and the integrator should see why: the archive member is
MOBILEGL_BUILD_DISAGGREGATED-only because one more Vector in LinkArtifacts changes its
size, its implicit destructor and its move constructor, and G1 pins the pull build's .text
byte for byte. So §5.3's deletion is achieved in the disaggregated build and not in the
pull one; retiring the pull half is a one-line change the day that pin is retired or the
member moves. Filed as P13 in notes/p7/magma-c.md.

One improvement rides along: the monolith binding getter now asks the program for a
rebind on every read, over an immutable list, where it used to read a binding patched into
a mutable cache - so a rebind can no longer be lost to a cache rebuild that raced a
state-version bump.

G1: .text 0xa52203 unchanged, 27837 -> 27837 defined symbols, 0 added / 0 removed / 0
resized / 0 renamed, file byte-identical. An unguarded call site caught the guard rule
once mid-slice by failing the pull compile, which is the loud version.

RED-ONCE: publishing nothing reds the order-equivalence case (with the reference list
printed) and 25 of 60 storage-block scenarios - the MONOLITH DirectVulkan arm beside all
three transports, which is the evidence that the monolith consumer really was switched.
Restored: 61/61.
This commit is contained in:
2026-09-22 16:07:59 -04:00
parent f1b04edfc6
commit 0acfa9aab9
11 changed files with 663 additions and 57 deletions
@@ -362,7 +362,15 @@ TEST(ProgramArtifactsCodec, AVersionMismatchIsRefused) {
// no archive owns. A codec arm for it would be a use-after-free waiting for a cache hit.
TEST(ProgramArtifactsCodec, TheTablesVisitEveryMemberExceptTheLiveProgram) {
#if MOBILEGL_PIPE_PUSH
#if MOBILEGL_BUILD_DISAGGREGATED
// P7 OQ-8: 59 members, 58 visited. The 59th is still the live TProgram; the 58th is
// storageBlocks, which exists only in this build (ProgramArtifacts.h guards it so the pull
// build's LinkArtifacts keeps its size, its destructor and its .text).
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<LinkArtifacts>(), 58u);
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<StorageBlockReflection>(), 3u);
#else
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<LinkArtifacts>(), 57u);
#endif
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<SpirvArtifacts>(), 8u);
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<ResourceReflection>(), 14u);
EXPECT_EQ(ProgramArtifactsVisitedFieldCount<XfbVarying>(), 11u);
@@ -381,7 +389,12 @@ TEST(ProgramArtifactsCodec, PortableHeaderUsesWireSchemaRatherThanNativeContaine
Uint64 schema = 0;
std::memcpy(&version, bytes.data(), sizeof(version));
std::memcpy(&schema, bytes.data() + sizeof(version), sizeof(schema));
EXPECT_EQ(version, 2u);
// 3 since P7 OQ-8 (CONTRACT-P7 §5.3): LinkArtifacts gained storageBlocks, so a v2 reader
// would run out of bytes mid-stream rather than notice. The schema word beside it moved
// too - it is derived from the VisitFields tables - and so did `wireFingerprint`, which is
// what makes a mixed-version pair refuse at the handshake instead of at the first program.
EXPECT_EQ(version, kProgramArtifactsCodecVersion);
EXPECT_EQ(version, 3u);
EXPECT_EQ(schema, ProgramArtifactsSchemaFingerprint());
EXPECT_NE(schema, 0u);
EXPECT_NE(schema, sizeof(LinkArtifacts));
@@ -81,7 +81,15 @@ namespace {
EXPECT_EQ(CountFields<TypeFacts>(), 20u);
EXPECT_EQ(CountFields<ResourceReflection>(), 14u);
EXPECT_EQ(CountFields<XfbVarying>(), 11u);
#if MOBILEGL_BUILD_DISAGGREGATED
// P7 OQ-8 added LinkArtifacts::storageBlocks and its own three-field table, in the
// disaggregated build only. The pull/monolith number below is the one G1 measures and
// it has not moved - which is the whole reason the member is guarded.
EXPECT_EQ(CountFields<StorageBlockReflection>(), 3u);
EXPECT_EQ(CountFields<LinkArtifacts>(), 58u);
#else
EXPECT_EQ(CountFields<LinkArtifacts>(), 57u);
#endif
EXPECT_EQ(CountFields<SpirvArtifacts>(), 8u);
}
+171
View File
@@ -380,6 +380,177 @@ void main() {
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
#if MOBILEGL_BUILD_DISAGGREGATED
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): THE ARCHIVE'S STORAGE-BLOCK INDEX SPACE IS
// SPIRV-REFLECT'S, ELEMENT FOR ELEMENT.
//
// This is the equivalence the whole slice rests on. DirectVulkan's resource queries speak an
// index space that used to be DERIVED, by running SPIRV-Reflect over the program's modules -
// per draw on the wire path (MagmaProgramSource) and behind a mutable cache in the monolith
// one (DirectVulkan.cpp's GetProgramResourceCache). It is now PUBLISHED at link time out of
// glslang's reflection instead (ProgramLinkTask::SnapshotStorageBlockIndexSpace). Two
// different producers, one index space: if they ever disagree,
// ProgramFactory::ReflectLayout - which looks blocks up by the name ITS OWN reflect pass
// produced - starts binding storage buffers at the wrong descriptor, silently.
//
// THE REFERENCE IS COMPUTED HERE, not copied as a constant, and it is computed with the same
// algorithm the old consumer used: per module in linked-stage order, STORAGE_BUFFER
// descriptors only, sorted by (normalised name, binding), deduplicated across modules by name.
// Comparing against a hand-written expected list would only pin what the author believed.
//
// THE PROGRAM IS THE ONE THE CONTRACT ASKS FOR: multi-stage (vertex + fragment), with an
// ARRAYED SSBO and an ATOMIC-COUNTER block. Both are the cases where the two producers could
// legitimately have disagreed and neither is exotic:
// * an instance array reflects in glslang as one entry PER ELEMENT ("Blocks[0]",
// "Blocks[1]") and in SPIR-V as ONE descriptor with a count, so the archive has to strip
// the subscript and keep the first element - which is the one whose binding the single
// descriptor reports;
// * an atomic_uint is lowered by the transpiler onto a synthesized gl_AtomicCounterBlock_<N>
// BUFFER block, so SPIRV-Reflect sees an ordinary storage descriptor for it and the
// backend's space has always contained it - even though GL's own GL_SHADER_STORAGE_BLOCK
// enumeration excludes it. The archive must follow the BACKEND's space, not GL's.
TEST_F(ProgramTest, TheArchivesStorageBlockOrderIsTheOneSpirvReflectProduces) {
char infoLog[1024] = "";
const char* vsSrc = R"(#version 460 core
layout(std430, binding = 3) buffer Blocks {
uint value;
} blocks[2];
layout(binding = 0, offset = 0) uniform atomic_uint counter;
layout(std430, binding = 6) buffer VertexOnly {
uint value;
} vertexOnly;
void main() {
atomicCounterIncrement(counter);
blocks[0].value = 1u;
vertexOnly.value = 2u;
gl_Position = vec4(0.0, 0.0, 0.0, 1.0);
}
)";
const char* fsSrc = R"(#version 460 core
layout(std430, binding = 5) buffer FragmentOnly {
uint value;
} fragmentOnly;
layout(std430, binding = 3) buffer Blocks {
uint value;
} blocks[2];
layout(location = 0) out vec4 color;
void main() {
fragmentOnly.value = 3u;
color = vec4(float(blocks[1].value), 0.0, 0.0, 1.0);
}
)";
const GLuint vs = CreateShader(GL_VERTEX_SHADER);
ShaderSource(vs, 1, &vsSrc, nullptr);
CompileShader(vs);
GLint status = GL_FALSE;
GetShaderiv(vs, GL_COMPILE_STATUS, &status);
GetShaderInfoLog(vs, sizeof(infoLog), nullptr, infoLog);
ASSERT_EQ(status, GL_TRUE) << infoLog;
const GLuint fs = CreateShader(GL_FRAGMENT_SHADER);
ShaderSource(fs, 1, &fsSrc, nullptr);
CompileShader(fs);
GetShaderiv(fs, GL_COMPILE_STATUS, &status);
GetShaderInfoLog(fs, sizeof(infoLog), nullptr, infoLog);
ASSERT_EQ(status, GL_TRUE) << infoLog;
const GLuint program = CreateProgram();
AttachShader(program, vs);
AttachShader(program, fs);
LinkProgram(program);
GetProgramiv(program, GL_LINK_STATUS, &status);
GetProgramInfoLog(program, sizeof(infoLog), nullptr, infoLog);
ASSERT_EQ(status, GL_TRUE) << infoLog;
auto programObject = MobileGL::MG_State::pGLContext->GetProgramObject(program);
ASSERT_NE(programObject, nullptr);
// ---- the reference: the old consumer's algorithm, run here over the real modules ----
const auto normalise = [](const SpvReflectDescriptorBinding& binding) {
const char* raw = binding.name;
if (binding.type_description != nullptr && binding.type_description->type_name != nullptr) {
raw = binding.type_description->type_name;
}
String name = raw ? raw : "";
const auto suffix = name.find("[0]");
return suffix == String::npos ? name : name.substr(0, suffix);
};
std::vector<String> reference;
const auto& spirvs = programObject->GetSpirvReflection().generatedSpirv;
ASSERT_GE(spirvs.size(), 2u) << "the case needs a multi-stage program to be a real test of "
"the cross-stage dedupe";
for (const auto& spirv : spirvs) {
if (spirv.empty()) continue;
SpvReflectShaderModule module{};
ASSERT_EQ(spvReflectCreateShaderModule(spirv.size() * sizeof(Uint), spirv.data(), &module),
SPV_REFLECT_RESULT_SUCCESS);
uint32_t count = 0;
ASSERT_EQ(spvReflectEnumerateDescriptorBindings(&module, &count, nullptr),
SPV_REFLECT_RESULT_SUCCESS);
std::vector<SpvReflectDescriptorBinding*> bindings(count);
if (count != 0) {
ASSERT_EQ(spvReflectEnumerateDescriptorBindings(&module, &count, bindings.data()),
SPV_REFLECT_RESULT_SUCCESS);
}
std::sort(bindings.begin(), bindings.end(),
[&](const SpvReflectDescriptorBinding* lhs, const SpvReflectDescriptorBinding* rhs) {
const String lhsName = lhs ? normalise(*lhs) : String();
const String rhsName = rhs ? normalise(*rhs) : String();
if (lhsName != rhsName) return lhsName < rhsName;
return lhs->binding < rhs->binding;
});
for (const SpvReflectDescriptorBinding* binding : bindings) {
if (binding == nullptr ||
binding->descriptor_type != SPV_REFLECT_DESCRIPTOR_TYPE_STORAGE_BUFFER) {
continue;
}
const String name = normalise(*binding);
if (name.empty()) continue;
if (std::find(reference.begin(), reference.end(), name) != reference.end()) continue;
reference.push_back(name);
}
spvReflectDestroyShaderModule(&module);
}
// ---- what the archive published ----
std::vector<String> archived;
for (const auto& block : programObject->GetLinkReflection().storageBlocks) {
archived.push_back(block.name);
}
ASSERT_FALSE(reference.empty())
<< "the reference walk found no storage descriptors at all, so the comparison below "
"would be vacuous - the shaders or the SPIR-V generation changed under this case";
EXPECT_EQ(archived, reference)
<< "LinkArtifacts::storageBlocks is not the index space SPIRV-Reflect produces. "
"GetShaderStorageBlockIndex/Binding hand these indices to ProgramFactory::ReflectLayout, "
"which resolves blocks by the name its own reflect pass produced, so a divergence "
"here binds storage buffers at the wrong descriptor rather than failing loudly.";
// The arrayed block must appear ONCE, under its stripped name: glslang reflects it per
// element and SPIR-V carries one descriptor, and collapsing the two is the whole reason
// the archive normalises.
EXPECT_EQ(std::count(archived.begin(), archived.end(), String("Blocks")), 1)
<< "the arrayed SSBO must collapse to one entry named without its subscript";
// The atomic counter is in the BACKEND's space even though GL's own enumeration excludes it.
EXPECT_NE(std::find_if(archived.begin(), archived.end(),
[](const String& name) {
return name.rfind("gl_AtomicCounterBlock", 0) == 0;
}),
archived.end())
<< "the synthesized atomic-counter block is a STORAGE_BUFFER descriptor in SPIR-V, so "
"DirectVulkan's index space contains it and the archive has to as well";
// And the indices actually resolve, which is what the consumers do with them.
for (Uint i = 0; i < archived.size(); ++i) {
EXPECT_EQ(MG_Backend::DirectVulkan::GetShaderStorageBlockIndex(*programObject, archived[i]), i)
<< "name " << archived[i] << " did not resolve to its own index";
}
EXPECT_EQ(GetError(), GL_NO_ERROR);
}
#endif // MOBILEGL_BUILD_DISAGGREGATED
TEST_F(ProgramTest, DirectVulkanStorageBlockUsesShaderLayoutBinding) {
char infoLog[1024] = "";
const char* csSrc = R"(#version 460 core