mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-18 23:43:23 +09:00
[MG_State, MG_Backend, MG_Test, docs] (P7): the reflection archive publishes the storage-block index space, and both magma consumers read it
CONTRACT-P7 §5.3, OQ-8. MagmaProgramSource::EnsureStorageBlocks re-ran SPIRV-Reflect over
every stage module of the program PER DRAW: the class is a borrowed view that
WireDraw.inc's WireProgramSource constructs afresh for each wire draw, so its `mutable`
memo was cold every time it was read. The list it rebuilt is a pure function of the linked
program, so it is now built once at link time and travels in the archive.
LinkArtifacts gains storageBlocks{name, binding, dataSize} with its own VisitFields table,
the codec version goes 2 -> 3, and the schema fingerprint and wireFingerprint move with
it - which is expected, and is what makes a mixed-version pair refuse at the handshake
instead of at the first program.
FILLED IN PHASE A, and that is not where it looks like it belongs. The only function that
sees both finished halves is ProgramSpirvTask's L1 cache insert, but its writes reach the
TRANSLATION CACHE only: phase A's artifacts have already been moved into the ProgramObject
by then. A field filled there would be present on a cache hit and absent on every cold
link. So the fill is glslang-derived, and the equivalence is MEASURED rather than assumed:
ProgramTest.TheArchivesStorageBlockOrderIsTheOneSpirvReflectProduces runs the old
consumer's own algorithm over the real modules and compares element for element, on a
multi-stage program with an arrayed SSBO and an atomic-counter block - the three cases
that could have diverged (the instance array collapses to one entry, the synthesized
counter block IS in the backend's space although GL's enumeration excludes it, and the
cross-stage dedupe walks stages in linked order).
MEASURED, same binary and same window (frames=1 draws=1 acc=18, identical bytes),
DirectVulkan x inproc over the SSBO scenarios: spvReflectCreateShaderModule on the wire
draw path 4 -> 0. The 0 is structural: the call site is gone and MagmaProgramSource.h no
longer includes spirv_reflect.h at all, which is the gate a future edit breaks on purpose.
THE MONOLITH CONSUMER IS SWITCHED TOO, under #if symmetry, so the disaggregated build
retires g_programResourceCaches, ProgramResourceCache, StorageBlockResource,
BufferVariableResource, AddBufferVariablesRecursive, NormalizeDescriptorName,
GetProgramResourceCache and the rehash hazard - the open-addressed-map dangling reference
that was a reproducible segfault in ProgramPipelineScenario's two storage-block cases.
Nothing outside DirectVulkan.cpp read any of it (measured; GL_BUFFER_VARIABLE and
GL_ACTIVE_VARIABLES are answered by ProgramInterface.cpp out of a different index space),
so the two arms differ in exactly the two functions their callers use.
THE PULL BUILD KEEPS ALL OF IT, and the integrator should see why: the archive member is
MOBILEGL_BUILD_DISAGGREGATED-only because one more Vector in LinkArtifacts changes its
size, its implicit destructor and its move constructor, and G1 pins the pull build's .text
byte for byte. So §5.3's deletion is achieved in the disaggregated build and not in the
pull one; retiring the pull half is a one-line change the day that pin is retired or the
member moves. Filed as P13 in notes/p7/magma-c.md.
One improvement rides along: the monolith binding getter now asks the program for a
rebind on every read, over an immutable list, where it used to read a binding patched into
a mutable cache - so a rebind can no longer be lost to a cache rebuild that raced a
state-version bump.
G1: .text 0xa52203 unchanged, 27837 -> 27837 defined symbols, 0 added / 0 removed / 0
resized / 0 renamed, file byte-identical. An unguarded call site caught the guard rule
once mid-slice by failing the pull compile, which is the loud version.
RED-ONCE: publishing nothing reds the order-equivalence case (with the reference list
printed) and 25 of 60 storage-block scenarios - the MONOLITH DirectVulkan arm beside all
three transports, which is the evidence that the monolith consumer really was switched.
Restored: 61/61.
This commit is contained in:
@@ -90,6 +90,47 @@ namespace MobileGL::MG_State::GLState {
|
||||
using PipeInputReflection = ResourceReflection;
|
||||
using PipeOutputReflection = ResourceReflection;
|
||||
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): THE SHADER STORAGE BLOCKS AS DIRECTVULKAN
|
||||
// INDEXES THEM, published once at link time instead of re-derived per draw.
|
||||
//
|
||||
// WHY A SECOND VIEW OF SOMETHING blockReflection ALREADY HOLDS. `blockReflection` is
|
||||
// glslang's list, in TProgram index order, carrying uniform blocks, storage blocks and the
|
||||
// synthesized atomic-counter blocks together, one entry per ARRAY ELEMENT. DirectVulkan's
|
||||
// resource queries speak a different index space entirely - the one its SPIRV-Reflect walk
|
||||
// produces: storage descriptors only, normalised block name with the array subscript
|
||||
// stripped, deduplicated across stages. `GetShaderStorageBlockIndex(name)` hands that index
|
||||
// out and `GetShaderStorageBlockBinding(index)` reads it back, so the two have to agree
|
||||
// with each other and with the names ProgramFactory::ReflectLayout looks up (which come
|
||||
// from its OWN reflect pass over the same modules). This vector IS that space, built once.
|
||||
//
|
||||
// WHAT IT COSTS AND WHAT IT BUYS. MagmaProgramSource is a BORROWED VIEW constructed afresh
|
||||
// for every wire draw (WireDraw.inc's WireProgramSource), so its lazy
|
||||
// `EnsureStorageBlocks` re-ran spvReflectCreateShaderModule over every stage module of
|
||||
// every program on every draw that touched an SSBO. Reading it out of the archive instead
|
||||
// takes the SPIR-V reflector off the wire draw path entirely.
|
||||
//
|
||||
// `binding` IS THE DECLARED (or IO-mapper-assigned) BINDING AND NOT THE CURRENT ONE.
|
||||
// glShaderStorageBlockBinding travels separately - as the record's StorageOverrides on the
|
||||
// wire and as ProgramObject::GetShaderStorageBlockBindingOverride in the monolith - and
|
||||
// both consumers apply it on read, exactly as they did when this list was rebuilt per
|
||||
// draw. Baking the current binding in would freeze a rebind at link time.
|
||||
//
|
||||
// DISAGGREGATED-ONLY, and that guard is load-bearing for G1: LinkArtifacts is a pull-build
|
||||
// type too, and one more Vector member would change its size, its implicit destructor and
|
||||
// its move constructor in a build whose .text must stay byte-identical.
|
||||
struct StorageBlockReflection {
|
||||
// The NORMALISED name: the SPIR-V block type name with a trailing array subscript
|
||||
// stripped, which is what SpvReflectDescriptorBinding's type_name answers and what
|
||||
// ProgramFactory::ReflectLayout looks blocks up by.
|
||||
String name;
|
||||
Uint32 binding = 0;
|
||||
// GL_BUFFER_DATA_SIZE. Carried because the monolith resource-query consumer reports
|
||||
// it; the wire consumer does not read it today.
|
||||
Int32 dataSize = 0;
|
||||
};
|
||||
#endif
|
||||
|
||||
// Transform feedback (GL 3.0 core: glTransformFeedbackVaryings applies on
|
||||
// the NEXT link; the linked snapshot below is what draws and queries see).
|
||||
struct XfbVarying {
|
||||
@@ -167,6 +208,12 @@ namespace MobileGL::MG_State::GLState {
|
||||
// The owned reflection snapshot. Indexed by TProgram index; see the structs above.
|
||||
Vector<UniformReflection> uniformReflection;
|
||||
Vector<BlockReflection> blockReflection;
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8: DirectVulkan's storage-block index space, filled beside blockReflection in
|
||||
// SnapshotGlslangReflection. See StorageBlockReflection above for what the order is
|
||||
// and why it is not blockReflection's.
|
||||
Vector<StorageBlockReflection> storageBlocks;
|
||||
#endif
|
||||
Vector<PipeInputReflection> pipeInputReflection;
|
||||
Vector<PipeOutputReflection> pipeOutputReflection;
|
||||
// Program-level scalars glslang answers off the linked intermediates.
|
||||
@@ -479,14 +526,32 @@ namespace MobileGL::MG_State::GLState {
|
||||
v("blockMemberElement", a.blockMemberElement);
|
||||
} // 11 fields
|
||||
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8. Three plain fields, so the generic codec arms carry it with no new arm of their
|
||||
// own - which is the whole reason it is a table rather than hand-written bytes.
|
||||
template <class Self, class V>
|
||||
requires std::same_as<std::remove_const_t<Self>, StorageBlockReflection>
|
||||
void VisitFields(Self& a, V&& v) {
|
||||
v("name", a.name);
|
||||
v("binding", a.binding);
|
||||
v("dataSize", a.dataSize);
|
||||
} // 3 fields
|
||||
#endif
|
||||
|
||||
// Every member EXCEPT `program`: it is null for every archived instance by construction
|
||||
// (ProgramTranslationCache.h asserts that at insert) and must never be serialized - it is
|
||||
// the live glslang TProgram that only DoReflection may touch. 57 of the 58 members.
|
||||
// the live glslang TProgram that only DoReflection may touch. 57 of the 58 members in a
|
||||
// pull/monolith build, 58 of 59 in a disaggregated one (P7 OQ-8's storageBlocks).
|
||||
template <class Self, class V>
|
||||
requires std::same_as<std::remove_const_t<Self>, LinkArtifacts>
|
||||
void VisitFields(Self& a, V&& v) {
|
||||
v("uniformReflection", a.uniformReflection);
|
||||
v("blockReflection", a.blockReflection);
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// Beside blockReflection because that is where it is FILLED, and a visitor that walks
|
||||
// the table in declaration order then reads the two together.
|
||||
v("storageBlocks", a.storageBlocks);
|
||||
#endif
|
||||
v("pipeInputReflection", a.pipeInputReflection);
|
||||
v("pipeOutputReflection", a.pipeOutputReflection);
|
||||
v("lastStageIsFragment", a.lastStageIsFragment);
|
||||
@@ -569,7 +634,13 @@ namespace MobileGL::MG_State::GLState {
|
||||
#if defined(__GLIBCXX__) && !defined(_GLIBCXX_DEBUG) && (SIZE_MAX == UINT64_MAX)
|
||||
#define MGL_RESOURCEREFLECTION_SIZE 128
|
||||
#define MGL_XFBVARYING_SIZE 128
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8 added storageBlocks, one more Vector, in the disaggregated build ONLY. The pull
|
||||
// number below is the one G1 measures and it has not moved.
|
||||
#define MGL_LINKARTIFACTS_SIZE 1080
|
||||
#else
|
||||
#define MGL_LINKARTIFACTS_SIZE 1056
|
||||
#endif
|
||||
#define MGL_SPIRVARTIFACTS_SIZE 88
|
||||
#elif defined(_LIBCPP_VERSION) && (SIZE_MAX == UINT64_MAX) && defined(MGL_ARTIFACT_SIZES_LIBCXX_PINNED)
|
||||
// The integrator pins these from the NDK build (brief C.4); until then this branch is inert.
|
||||
|
||||
@@ -61,7 +61,14 @@ namespace MobileGL::MG_State::GLState {
|
||||
// Bumped whenever the byte format changes in a way a previous reader would misread. A
|
||||
// reader that sees a different word REFUSES; it never tries to guess a layout.
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
inline constexpr Uint32 kProgramArtifactsCodecVersion = 2;
|
||||
// 3 since P7 wave 2 package C (OQ-8): LinkArtifacts gained `storageBlocks`, so a v2 reader
|
||||
// would run out of bytes in the middle of the stream rather than notice. The schema
|
||||
// fingerprint beside the version would catch it on its own - it is derived from the
|
||||
// VisitFields tables and therefore moved with the new row - but the version is what a
|
||||
// reader checks FIRST and what the refusal names, and CONTRACT-P7 §5.3 asks for the bump
|
||||
// explicitly. `wireFingerprint` moves with both, which is expected and is what makes a
|
||||
// mixed-version pair refuse at the handshake instead of at the first program.
|
||||
inline constexpr Uint32 kProgramArtifactsCodecVersion = 3;
|
||||
// Derived from the actual VisitFields order/names, container element schemas and scalar
|
||||
// representations. No native container size/offset is included. Also checked in Hello.
|
||||
Uint64 ProgramArtifactsSchemaFingerprint();
|
||||
|
||||
@@ -106,6 +106,29 @@ namespace {
|
||||
return bracket == MobileGL::String::npos ? name : name.substr(0, bracket);
|
||||
}
|
||||
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8. ShaderStage -> EShLanguage, so a reflection record's `stages` mask (which is an
|
||||
// EShLanguageMask) can be tested against the stage of a snapshotted shader. Written here
|
||||
// rather than lifted into a shared header because this is the only caller and
|
||||
// ProgramArtifacts.h's include-closure probe forbids this file's neighbours from growing a
|
||||
// dependency on ShaderStage.h. EShLangCount means "no glslang stage", which the caller
|
||||
// skips rather than guesses at.
|
||||
static EShLanguage ShaderStageToEShLanguage(MobileGL::ShaderStage stage) {
|
||||
switch (stage) {
|
||||
case MobileGL::ShaderStage::Vertex: return EShLangVertex;
|
||||
case MobileGL::ShaderStage::TessControl: return EShLangTessControl;
|
||||
case MobileGL::ShaderStage::TessEval: return EShLangTessEvaluation;
|
||||
case MobileGL::ShaderStage::Geometry: return EShLangGeometry;
|
||||
case MobileGL::ShaderStage::Fragment: return EShLangFragment;
|
||||
case MobileGL::ShaderStage::Compute: return EShLangCompute;
|
||||
case MobileGL::ShaderStage::ShaderStageCount:
|
||||
case MobileGL::ShaderStage::Unknown:
|
||||
break;
|
||||
}
|
||||
return EShLangCount;
|
||||
}
|
||||
#endif
|
||||
|
||||
// Element index of an arrayed interface-block instance: "GOKU[3]" -> 3, "GOKU" -> 0.
|
||||
// Reflection spells arrayed instances exactly this way (glslang expands the instance
|
||||
// array into one TObjectReflection per element), and the subscript it writes is a plain
|
||||
@@ -1735,9 +1758,97 @@ namespace MobileGL::MG_State::GLState {
|
||||
}
|
||||
|
||||
SnapshotGlslangReflection();
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8. Guarded at the CALL as well as at the definition, which is not belt and
|
||||
// braces: LinkArtifacts::storageBlocks does not exist in a pull build, so neither can
|
||||
// the function that fills it, and G1 requires this translation unit to be statement
|
||||
// for statement what it was there.
|
||||
SnapshotStorageBlockIndexSpace();
|
||||
#endif
|
||||
return true;
|
||||
}
|
||||
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 wave 2 package C, OQ-8 (CONTRACT-P7 §5.3): DIRECTVULKAN'S STORAGE-BLOCK INDEX SPACE,
|
||||
// BUILT ONCE HERE INSTEAD OF PER DRAW.
|
||||
//
|
||||
// WHAT ORDER THIS HAS TO PRODUCE, and it is not blockReflection's. The consumers speak the
|
||||
// index space DirectVulkan's SPIRV-Reflect walk produces (DirectVulkan.cpp's
|
||||
// GetProgramResourceCache, and the copy of it MagmaProgramSource used to carry):
|
||||
//
|
||||
// for each SPIR-V module, IN LINKED-STAGE ORDER
|
||||
// take its STORAGE_BUFFER descriptors
|
||||
// sorted by (normalised block name, binding)
|
||||
// appending any name not already present
|
||||
//
|
||||
// Every clause is reproducible from what phase A already owns. `in.shaders` is the linked
|
||||
// shader snapshot and is documented as ALREADY STAGE-SORTED, and it is the same list, in
|
||||
// the same order, that ProgramSpirvTask turns into `generatedSpirv` one module per entry -
|
||||
// so iterating it iterates the modules. `blockReflection[].stages` is an EShLanguageMask,
|
||||
// which is what says whether a given block is one of a given module's descriptors. The
|
||||
// name normalisation is StripArrayElementSuffix, which is the same "[k]" strip
|
||||
// NormalizeDescriptorName does to the SPIR-V type name: glslang expands an INSTANCE ARRAY
|
||||
// into one reflection entry per element ("Foo[0]", "Foo[1]") while SPIR-V carries one
|
||||
// descriptor with a count, so stripping is what makes the two agree on ONE entry named
|
||||
// "Foo".
|
||||
//
|
||||
// ATOMIC-COUNTER BLOCKS ARE INCLUDED, deliberately, and they are the case that makes the
|
||||
// filter `type.isBuffer` rather than IsStorageBlock(). The transpiler lowers every
|
||||
// atomic_uint onto a synthesized gl_AtomicCounterBlock_<binding> BUFFER block, so
|
||||
// SPIRV-Reflect sees an ordinary STORAGE_BUFFER descriptor for it and DirectVulkan's index
|
||||
// space has always contained it. GL's own GL_SHADER_STORAGE_BLOCK enumeration excludes
|
||||
// them (ProgramInterface.cpp's ClassifyBlock does the exclusion there, in the space that
|
||||
// needs it) - these are two different index spaces and this is the backend's.
|
||||
//
|
||||
// `binding` IS THE DECLARED ONE. A rebind travels separately and both consumers apply it
|
||||
// on read; see StorageBlockReflection in ProgramArtifacts.h.
|
||||
void ProgramLinkTask::SnapshotStorageBlockIndexSpace() {
|
||||
artifacts.storageBlocks.clear();
|
||||
const auto alreadyPresent = [this](const String& name) {
|
||||
for (const auto& block : artifacts.storageBlocks) {
|
||||
if (block.name == name) return true;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
for (const LinkShaderInput& shader : in.shaders) {
|
||||
const EShLanguage stage = ShaderStageToEShLanguage(shader.stage);
|
||||
if (stage == EShLangCount) continue;
|
||||
const Uint32 stageBit = 1u << static_cast<Uint32>(stage);
|
||||
|
||||
// Pointers, not copies: blockReflection is stable for the rest of this function and
|
||||
// a BlockReflection carries several strings.
|
||||
Vector<const ProgramObject::BlockReflection*> ofThisStage;
|
||||
for (const ProgramObject::BlockReflection& block : artifacts.blockReflection) {
|
||||
if (!block.type.isBuffer) continue;
|
||||
if ((block.stages & stageBit) == 0) continue;
|
||||
ofThisStage.push_back(&block);
|
||||
}
|
||||
// STABLE, so two entries that normalise to one name keep their declaration order
|
||||
// and the FIRST (element 0, the base binding) is the one that survives the dedupe -
|
||||
// which is the element SPIRV-Reflect's single descriptor reports.
|
||||
std::stable_sort(ofThisStage.begin(), ofThisStage.end(),
|
||||
[](const ProgramObject::BlockReflection* lhs,
|
||||
const ProgramObject::BlockReflection* rhs) {
|
||||
const String lhsName = StripArrayElementSuffix(lhs->name);
|
||||
const String rhsName = StripArrayElementSuffix(rhs->name);
|
||||
if (lhsName != rhsName) return lhsName < rhsName;
|
||||
return lhs->binding < rhs->binding;
|
||||
});
|
||||
for (const ProgramObject::BlockReflection* block : ofThisStage) {
|
||||
const String name = StripArrayElementSuffix(block->name);
|
||||
if (name.empty() || alreadyPresent(name)) continue;
|
||||
ProgramObject::StorageBlockReflection record;
|
||||
record.name = name;
|
||||
record.binding = block->binding < 0 ? 0u : static_cast<Uint32>(block->binding);
|
||||
record.dataSize = static_cast<Int32>(block->size);
|
||||
artifacts.storageBlocks.push_back(Move(record));
|
||||
}
|
||||
}
|
||||
MGLOG_D("ProgramObject %u: Reflection - %zu storage block(s) in the backend index space",
|
||||
in.externalIndex, artifacts.storageBlocks.size());
|
||||
}
|
||||
#endif
|
||||
|
||||
// The last thing DoReflection does, and the thing that lets everything after it stop
|
||||
// caring that a glslang::TProgram ever existed: copy every reflection record the GL query
|
||||
// surface reads into LinkArtifacts' own owned tables.
|
||||
|
||||
@@ -207,6 +207,14 @@ namespace MobileGL::MG_State::GLState {
|
||||
// one and overwritten the qualifier. See the definition for why the invented binding is
|
||||
// deliberately left in place for the backends' own use.
|
||||
void SeedDefaultStorageBlockBindings();
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 wave 2 package C, OQ-8: builds LinkArtifacts::storageBlocks - DirectVulkan's own
|
||||
// storage-block index space - out of the reflection snapshot the call above produced,
|
||||
// so the backends stop re-deriving it with SPIRV-Reflect on every draw. Runs after
|
||||
// SnapshotGlslangReflection because it reads blockReflection. See the definition for
|
||||
// which order it has to reproduce and why each clause of it is derivable here.
|
||||
void SnapshotStorageBlockIndexSpace();
|
||||
#endif
|
||||
Bool ValidateFragmentOutputLocations();
|
||||
Bool ResolveTransformFeedbackVaryings();
|
||||
void ResolveGsTriangleStripCapture(const glslang::TIntermediate* captureIntermediate);
|
||||
|
||||
@@ -48,6 +48,10 @@ namespace MobileGL::MG_State::GLState {
|
||||
using BlockReflection = ResourceReflection;
|
||||
using PipeInputReflection = ResourceReflection;
|
||||
using PipeOutputReflection = ResourceReflection;
|
||||
#if MOBILEGL_BUILD_DISAGGREGATED
|
||||
// P7 OQ-8: DirectVulkan's storage-block index space, carried in LinkArtifacts.
|
||||
using StorageBlockReflection = MobileGL::MG_State::GLState::StorageBlockReflection;
|
||||
#endif
|
||||
using XfbVarying = MobileGL::MG_State::GLState::XfbVarying;
|
||||
using LinkArtifacts = MobileGL::MG_State::GLState::LinkArtifacts;
|
||||
using SpirvArtifacts = MobileGL::MG_State::GLState::SpirvArtifacts;
|
||||
|
||||
Reference in New Issue
Block a user