[Fix] (Magma): make the legacy-memo lever a startup gate for Magma's own bit, bound the {slot, gen} mint, and keep the all-pull arm free of push-only cost

- MOBILEGL_PIPE_LEGACY_MEMOS=0 no longer aborts a draw. D14 spends that lever at
  STARTUP and only on a Track-H subsystem, so MagmaPipeValidateSubsystemConfiguration
  runs once from VulkanRenderer::Initialize and checks bit 6 alone: an Espryt-side
  bitmask cannot kill a Magma run, and bit 0 - which is not Track H and not a memo
  re-key - is out of the lever's scope entirely. MOBILEGL_PIPE_PUSH=0x60 with the
  lever off went from 9/9 aborted to 432/432.
- A pipeline memo with no render-state CSO bound falls back instead of aborting.
  delete_render_state clears the binding, so the null handle is reachable on any
  tree; the fallback is the pre-handle state hash where one is compiled, and the
  client's own MGPipeComputePipelineSubsetHash over the same 396 pipeline bytes
  where it is not - which is what makes -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF a runnable
  configuration (180/432 aborted before, 432/432 now) instead of a build that dies
  on its first draw. The fallback warns once, so a run that never keys on a CSO
  handle says so in its log instead of passing silently.
- The handles are minted by a fixed-capacity, self-recycling identity table in the
  backend, not by MG_Impl's client allocator. Nothing in P2 frees a VertexElementsCso
  or Buffer slot - the frontend has no death notification Magma can hook - so the
  allocator's live Allocate and dead Free grew one SlotState plus one hash-map node
  per object ever created, for the life of the process. The table is 2-way
  set-associative with an LRU victim and a Gen bump on reuse: bounded (32 KB for
  VAOs, 128 KB for buffers), exactly as ABA-proof, and it takes MG_Backend's only
  include of MG_Impl back out.
- Both per-slot memo tables are now a BIJECTION with that mint rather than a masked
  direct map, so two live VAOs cannot share an entry and the eviction decision lives
  once, in the identity table, instead of once per consumer table. The density claim
  the masked tables rested on was false while slots grew monotonically, and the
  masked form had also dropped the second candidate and the frame-serial victim
  choice the address-hashed table used to have.
- snap.vaoHandle is stamped only when bit 6 is on. It was guarded by the compile
  switch alone, so MOBILEGL_PIPE_PUSH=0 - the all-pull control D14 defines as
  reproducing P1 exactly, and the arm D.4.3's T2 is measured on - paid a mint per
  new VAO and a compare per draw for a field that arm never reads.
- Every re-keyed Track-H site now asks the same MagmaPipeTrackHArmIsHandles helper,
  including VertexInputStateFactory::ComputeHash, which decided for itself before
  and could key on the pre-handle identity while its neighbours keyed on the handle.
- The pull build's two pipeline-memo sites keep the base ref's text statement for
  statement: G1 is back to the contract's four resized symbols, 0 added/removed.
This commit is contained in:
2026-09-07 23:18:09 -04:00
parent 3594f03c4e
commit 01179c54d2
5 changed files with 386 additions and 161 deletions
@@ -11,15 +11,17 @@
#include <Config.h>
#if MOBILEGL_PIPE_PUSH
// kMGPipeSubsystem* - the runtime bitmask's named bits - and the client slot allocator that
// mints every MGPipeHandle. Push-only, so the pull build's include graph is unchanged.
#include <MG_Impl/Pipe/SlotAllocator.h>
// kMGPipeSubsystem* - the runtime bitmask's named bits - and MGPipeHandle itself. Both are
// header-only constant/POD declarations, and both are push-only, so the pull build's include
// graph is unchanged (G1).
#include <MG_Pipe/MGPipe.h>
#include <MG_Pipe/MGPipeHandles.h>
#endif
#include <cstdlib>
// Magma's arm selector for the P2 Track H / render-state re-keys (P2 brief D14).
// Magma's arm selector for the P2 Track H / render-state re-keys (P2 brief D14), and the
// bounded {slot, gen} mint the re-keyed sites are written against.
//
// Two switches decide which arm a re-keyed site runs, and they are NOT the same switch:
//
@@ -33,6 +35,11 @@
// backend would otherwise still run the re-keyed code. So a clear bit selects the legacy
// arm, and a run that has explicitly disabled the legacy arm may not fall into it.
//
// D14 spends that last sentence at STARTUP, not per draw: "a Track-H subsystem whose bit is
// clear is a startup Fatal{PipeLegacyMemosDisabled}". Nothing in the draw path aborts, and
// nothing outside Track H consults the legacy-memo lever at all - see
// MagmaPipeValidateSubsystemConfiguration below for both halves of that rule.
//
// The whole header is inert in a pull build: MOBILEGL_PIPE_PUSH is 0 there, every helper
// below is behind it, and the pull build's translation units are byte-identical (G1).
namespace MobileGL::MG_Backend::DirectVulkan {
@@ -43,37 +50,191 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return (MG_Config::Features.PipePush & subsystemBit) != 0;
}
// The legacy arm is about to be entered. Features.PipeLegacyMemos=0 is the operator
// asserting "the pre-handle arm is never entered in this run", which is the lever
// HandleRecycleScenario.Handles pulls (P2 brief D18): entering it anyway would make
// that arm green for the wrong reason, so it is Fatal rather than a fallback.
inline void MagmaPipeRequireLegacyArm(const char* site) {
// ---------------------------------------------------------------------------------
// D14's startup gate
// ---------------------------------------------------------------------------------
//
// Called once from VulkanRenderer::Initialize(), i.e. only when Magma is the backend
// that is actually running. It answers exactly one question and it answers it before the
// first draw: is there an arm for Magma's Track-H subsystem in this configuration?
//
// Three deliberate boundaries, each of which the per-draw shape this replaces got wrong:
//
// * ONLY Magma's own Track-H bit is checked. Espryt's bit 5 is Espryt's business (a
// DirectVulkan run does not execute one line of DirectGLES' re-key), so
// MOBILEGL_PIPE_PUSH=0x20 must not kill a Magma run, and MOBILEGL_PIPE_PUSH=0x40 must
// not kill an Espryt one.
// * bit 0 (kMGPipeSubsystemRenderState) is NOT Track H and is NOT checked. It is not a
// memo re-key at all: it decides where the pipeline memo's STATE KEY comes from, and
// a clear bit there simply means the client is not pushing render-state CSOs in this
// run, which GetOrCreatePipeline answers with its own state hash. D14 labels bits 5
// and 6 "Track H" and labels bit 0 nothing of the sort.
// * it is Fatal at STARTUP, once, not on a draw. A per-draw abort inside
// GetOrCreatePipeline turns a configuration mistake into a mid-frame crash and puts a
// branch nobody needs on the hottest path in the backend.
inline void MagmaPipeValidateSubsystemConfiguration() {
#if MOBILEGL_PIPE_LEGACY_MEMOS
// The pre-handle arm is compiled AND the operator has not forbidden entering it, so a
// clear bit is an ordinary, valid A/B: the site takes the legacy arm.
if (MG_Config::Features.PipeLegacyMemos) return;
MGLOG_F("MGPipe: Fatal{PipeLegacyMemosDisabled} %s wanted the pre-handle arm but "
"MOBILEGL_PIPE_LEGACY_MEMOS=0 forbids entering it",
site);
#else
MGLOG_F("MGPipe: Fatal{PipeLegacyMemosDisabled} %s wanted the pre-handle arm but this "
"build did not compile one (cmake -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF)",
site);
#endif
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) return;
#if MOBILEGL_PIPE_LEGACY_MEMOS
const char* const why = "this run has MOBILEGL_PIPE_LEGACY_MEMOS=0";
#else
const char* const why =
"this build has cmake -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF, which compiles no such arm";
#endif
MGLOG_F("MGPipe: Fatal{PipeLegacyMemosDisabled} Magma's Track-H subsystem "
"(kMGPipeSubsystemMagmaVertexInput, bit 6 of MOBILEGL_PIPE_PUSH) is clear, so the "
"vertex-input cache and the VAO draw memo want the pre-handle arm - but %s. Set "
"bit 6 (MOBILEGL_PIPE_PUSH=0x%llx, or the default 0x%llx), or allow the legacy arm.",
why,
static_cast<unsigned long long>(MG_Config::Features.PipePush |
MG_Pipe::kMGPipeSubsystemMagmaVertexInput),
static_cast<unsigned long long>(MG_Pipe::kMGPipeSubsystemsMigratedAtP2));
std::abort();
}
// The {slot, gen} of a frontend object, minted on first sight and stable for that
// object's whole life (ARCHITECTURE.md 4.2). `lifetimeId` is the client's own identity
// for the object - never a GL name, never a heap address - so a deleted-and-recreated
// object at the same address cannot reproduce a handle, which is precisely the ABA
// HandleRecycleScenario reproduces.
// "Does this Track-H site run the handle arm?" - the ONE question every re-keyed Track-H
// site asks, so that they cannot disagree with each other or with the startup gate.
inline Bool MagmaPipeTrackHArmIsHandles(Uint64 trackHBit) {
#if MOBILEGL_PIPE_LEGACY_MEMOS
return MagmaPipeSubsystemOn(trackHBit);
#else
// No pre-handle arm exists in this build, and MagmaPipeValidateSubsystemConfiguration
// has already made a clear bit a startup Fatal, so the handle arm is the only arm a
// running process can be on.
(void)trackHBit;
return true;
#endif
}
// ---------------------------------------------------------------------------------
// The {slot, gen} mint
// ---------------------------------------------------------------------------------
//
// A VAO is kind VertexElementsCso: that is the gallium-shaped CSO a vertex array
// resolves to, and it is the only kind in MGPipeKind that names vertex-input state.
// Magma acquires the handle itself in P2 because the tracker does not emit object-class
// state yet (P2 emits for dirty bits 0-4 only); when it does, this becomes a read of what
// the client already sent.
// A FIXED-CAPACITY, SELF-RECYCLING identity table: 2-way set-associative, indexed by the
// frontend object's lifetime id, LRU victim within the set, and Gen incremented whenever
// a slot changes owner. It hands out slots in [kMGPipeFirstAllocatableSlot, Count], so a
// consumer's per-slot table is a BIJECTION with this one - one entry per slot, no
// masking, no collision, no probe.
//
// Why not MG_Impl/Pipe/SlotAllocator (the client's allocator, which is what mints handles
// in the finished design)? Because in P2 nothing on this side ever frees one. The tracker
// does not emit object-class state yet (P2 emits for dirty bits 0-4), so no create_*/
// delete_* pair travels for a VAO or a buffer, and the frontend has no death notification
// Magma could hook: BufferBackendOps::OnDestroy is handed a BackendBufferResource, not the
// BufferObject, and fires only for a buffer that ever had one, while VertexArrayObject has
// no hook at all (adding one is D13's explicit-destroy work, and it covers Espryt's six
// kinds, not VertexElementsCso). An allocator with a live Allocate and a dead Free grows
// by one SlotState plus one hash-map node per object EVER created, for the life of the
// process, on a platform with an LMK - and its slot numbers then grow monotonically with
// objects ever created, which is exactly what would make a slot-indexed table collide.
//
// So Magma mints its own, bounded, and says so. This is a P2 STAND-IN either way (the
// client is what mints handles once object-class state travels); what it must not be is a
// leak. Recycling costs the same thing the address-hashed table it replaces cost: a
// colliding pair of live objects evicts each other and re-derives. It is strictly better
// than that table, because the {slot, gen} compare is an exact identity, so an eviction
// can only ever cost a recompute - never the ABA the lifetime-id compare was added for.
//
// Single-threaded, like MGPipeSlots() and like the rest of the renderer.
class MagmaPipeIdentityTable {
public:
explicit MagmaPipeIdentityTable(Uint32 entryCount) : m_entryCount(entryCount) {}
// One entry per slot, so a consumer table sized Count() and indexed by
// MagmaPipeSlotIndex() has exactly one entry per handle this table can hand out.
Uint32 Count() const { return m_entryCount; }
MG_Pipe::MGPipeHandle Acquire(Uint64 lifetimeId) {
if (lifetimeId == 0) return MG_Pipe::kMGPipeNullHandle;
if (m_entries.empty()) m_entries.resize(m_entryCount);
// Lifetime ids are monotonic from 1, so the low bits ARE the dense index: object
// n and object n+1 land in adjacent sets. No mix, because there is no entropy to
// spread - a multiply here would only scatter a sequence that is already perfect.
const Uint32 set = static_cast<Uint32>(lifetimeId) & (SetCount() - 1u);
const Uint32 way0 = set * 2u;
const Uint32 way1 = way0 + 1u;
if (m_entries[way0].LifetimeId == lifetimeId) return Touch(way0);
if (m_entries[way1].LifetimeId == lifetimeId) return Touch(way1);
// Miss. Evict the set's least recently used way - the same victim rule the
// address-hashed VaoDrawMemo table used, kept here so that it lives in ONE place
// instead of once per consumer table.
const Uint32 victim = (m_entries[way0].LastUse <= m_entries[way1].LastUse) ? way0 : way1;
Entry& entry = m_entries[victim];
// The one place Gen may move, and it moves on REUSE: a respecify of the same
// object keeps its {slot, gen} because its lifetime id still matches above.
MOBILEGL_ASSERT(entry.Gen != ~Uint32{0},
"Magma handle generation wrapped on slot %u; {slot, gen} is no longer "
"unique",
victim + MG_Pipe::kMGPipeFirstAllocatableSlot);
++entry.Gen;
entry.LifetimeId = lifetimeId;
return Touch(victim);
}
private:
struct Entry {
Uint64 LifetimeId = 0;
Uint32 Gen = 0;
Uint32 LastUse = 0;
};
Uint32 SetCount() const { return m_entryCount / 2u; }
MG_Pipe::MGPipeHandle Touch(Uint32 index) {
m_entries[index].LastUse = ++m_clock;
return MG_Pipe::MGPipeHandle{index + MG_Pipe::kMGPipeFirstAllocatableSlot,
m_entries[index].Gen};
}
Uint32 m_entryCount = 0;
// Wraps every 2^32 acquisitions. A wrapped clock can only ever pick the wrong victim
// inside one set - a cache decision, never a correctness one.
Uint32 m_clock = 0;
Vector<Entry> m_entries;
};
// The table entry a handle names. Every per-slot table Magma keeps is sized Count() and
// indexed by this, so the index is exact and in range by construction.
inline Uint32 MagmaPipeSlotIndex(const MG_Pipe::MGPipeHandle& handle) {
return handle.Slot - MG_Pipe::kMGPipeFirstAllocatableSlot;
}
// A VAO is kind VertexElementsCso: that is the gallium-shaped CSO a vertex array resolves
// to, and it is the only kind in MGPipeKind that names vertex-input state. 2048 entries
// is what the address-hashed VaoDrawMemo table it replaces held, so the working set this
// covers without eviction is unchanged; at 16 B/entry the table itself is 32 KB.
inline constexpr Uint32 kMagmaVaoIdentityEntries = 2048;
// Buffers are far more numerous than VAOs (Minecraft cycles chunk vertex/index buffers),
// and unlike the VAO table this one feeds a CONTENT hash: an eviction changes the key a
// vertex-input cache entry was built under, so it costs a rebuild rather than a lookup.
// It is only ever consulted when a VAO's configuration version moved (ComputeHash is
// memoised per VAO), so the price is paid per reconfiguration, not per draw - but the
// table is sized four times the VAO one anyway, 128 KB, to keep it rare.
inline constexpr Uint32 kMagmaBufferIdentityEntries = 8192;
inline MagmaPipeIdentityTable& MagmaPipeVaoIdentity() {
static MagmaPipeIdentityTable table(kMagmaVaoIdentityEntries);
return table;
}
inline MagmaPipeIdentityTable& MagmaPipeBufferIdentity() {
static MagmaPipeIdentityTable table(kMagmaBufferIdentityEntries);
return table;
}
// The {slot, gen} of a frontend object. `lifetimeId` is the client's own identity for the
// object - never a GL name, never a heap address - so a deleted-and-recreated object at
// the same address cannot reproduce a handle, which is precisely the ABA
// HandleRecycleScenario reproduces.
inline MG_Pipe::MGPipeHandle MagmaPipeHandleOf(MG_Pipe::MGPipeKind kind, Uint64 lifetimeId) {
return MG_Pipe::MGPipeSlots().Acquire(kind, lifetimeId);
return kind == MG_Pipe::MGPipeKind::Buffer ? MagmaPipeBufferIdentity().Acquire(lifetimeId)
: MagmaPipeVaoIdentity().Acquire(lifetimeId);
}
#endif // MOBILEGL_PIPE_PUSH
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -56,7 +56,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 bufferKey = attr.Buffer ? attr.Buffer->GetLifetimeId() : 0;
#if MOBILEGL_PIPE_PUSH
if (attr.Buffer) {
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
// The SAME arm question the other four re-keyed sites ask, through the same
// helper: a site that decided for itself could silently key on the pre-handle
// identity while its neighbours keyed on the handle.
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
const MG_Pipe::MGPipeHandle handle =
MagmaPipeHandleOf(MG_Pipe::MGPipeKind::Buffer, attr.Buffer->GetLifetimeId());
bufferKey = static_cast<Uint64>(handle.Slot) | (static_cast<Uint64>(handle.Gen) << 32);
@@ -80,21 +83,23 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#if MOBILEGL_PIPE_PUSH
VertexInputStateFactory::VaoBackendMemos& VertexInputStateFactory::MemosFor(
const MG_State::GLState::VertexArrayObject& vao) const {
static_assert(kVaoMemoSlotCount == kMagmaVaoIdentityEntries,
"this table is indexed directly by MagmaPipeSlotIndex, so it has to hold "
"exactly one entry per slot the VAO identity table can mint");
if (m_vaoMemos.empty()) {
m_vaoMemos.resize(kVaoMemoSlotCount);
}
const Uint64 lifetimeId = vao.GetLifetimeId();
if (!m_lastVaoHandleValid || m_lastVaoLifetimeId != lifetimeId) {
m_lastVaoHandle = MagmaPipeHandleOf(MG_Pipe::MGPipeKind::VertexElementsCso, lifetimeId);
m_lastVaoLifetimeId = lifetimeId;
m_lastVaoHandleValid = true;
}
const MG_Pipe::MGPipeHandle handle = m_lastVaoHandle;
VaoBackendMemos& memos = m_vaoMemos[handle.Slot & (kVaoMemoSlotCount - 1)];
const MG_Pipe::MGPipeHandle handle =
MagmaPipeHandleOf(MG_Pipe::MGPipeKind::VertexElementsCso, vao.GetLifetimeId());
// One entry per mintable slot - see the static_assert on kVaoMemoSlotCount - so this
// index is exact and two live VAOs cannot share an entry. There is no probe in front
// of it because the mint itself is one: an array index and at most two Uint64
// compares, which is less than the address hash the pre-handle arm ran.
VaoBackendMemos& memos = m_vaoMemos[MagmaPipeSlotIndex(handle)];
if (!(memos.Owner == handle)) {
// Someone else's entry (a colliding slot, or a slot whose Gen moved because the
// slot was REUSED for a different object). Claim it, contents cleared - never
// inherited, which is the whole point of keying on the generation.
// A slot whose Gen moved because the identity table recycled it for a different
// object. Claim it, contents cleared - never inherited, which is the whole point
// of keying on the generation.
memos = VaoBackendMemos{};
memos.Owner = handle;
}
@@ -105,13 +110,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
#if MOBILEGL_PIPE_PUSH
Bool VertexInputStateFactory::TryGetMemoizedHash(const MG_State::GLState::VertexArrayObject& vao,
Uint64& outHash) const {
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
const VaoBackendMemos& memos = MemosFor(vao);
if (memos.HashConfigVersion != vao.GetConfigVersion()) return false;
outHash = memos.Hash;
return true;
}
MagmaPipeRequireLegacyArm("VertexInputStateFactory::TryGetMemoizedHash");
#if MOBILEGL_PIPE_LEGACY_MEMOS
return vao.GetBackendHashMemo(outHash);
#else
@@ -125,7 +129,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
HashType hash = 0;
#if MOBILEGL_PIPE_PUSH
// P2 D12.5: the same memo, on the backend's side of the boundary.
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
VaoBackendMemos& memos = MemosFor(vao);
if (memos.HashConfigVersion == vao.GetConfigVersion()) {
return memos.Hash;
@@ -135,7 +139,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
memos.HashConfigVersion = vao.GetConfigVersion();
return hash;
}
MagmaPipeRequireLegacyArm("VertexInputStateFactory::GetOrComputeHash");
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
if (!vao.GetBackendHashMemo(hash)) {
@@ -154,7 +157,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// survives the move and is still what stops a stale pointer being dereferenced: the
// POINTEE is a cache entry this factory can erase at a frame boundary, and moving the
// memo does not change that.
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
VaoBackendMemos& memos = MemosFor(vao);
if (memos.StateConfigVersion == vao.GetConfigVersion() && memos.State != nullptr &&
memos.StateEpoch == m_evictionEpoch) {
@@ -164,10 +167,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const BackendVertexInputState& resolved =
GetOrCreateVertexInputState(vao, GetOrComputeHash(vao));
// MemosFor is re-taken: GetOrComputeHash above went through it, and a colliding
// VAO could have claimed the entry in between (it cannot here, since both calls
// name the same VAO, but the reference is not worth keeping live across a call
// that can resize the table).
// MemosFor is re-taken rather than kept live across GetOrCreateVertexInputState:
// the reference is not worth holding across a call that can resize the table.
VaoBackendMemos& stamp = MemosFor(vao);
stamp.State = &resolved;
stamp.StateEpoch = m_evictionEpoch;
@@ -177,12 +178,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// live reader anywhere, so the handle arm retires it rather than moving it.
return resolved;
}
MagmaPipeRequireLegacyArm("VertexInputStateFactory::GetOrCreateVertexInputState");
#endif
#if !MOBILEGL_PIPE_LEGACY_MEMOS
// Unreachable: with no legacy arm compiled, MagmaPipeRequireLegacyArm above aborts.
// Written out rather than left to fall off the end so the function still has a return
// on every path a compiler can see.
// Unreachable: with no legacy arm compiled MagmaPipeTrackHArmIsHandles is a compile-
// time true, so the handle arm above always returns. Written out rather than left to
// fall off the end so the function still has a return on every path a compiler sees.
return GetOrCreateVertexInputState(vao, GetOrComputeHash(vao));
#else
// Per-draw fast path: the VAO carries a pointer to its resolved entry,
@@ -136,9 +136,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// and layoutAuxMasks long ago and its getter has no live reader anywhere in the tree,
// so the handle arm simply stops writing it (D12.5 says delete rather than move).
struct VaoBackendMemos {
// Whose memos these are. A slot is direct-mapped into the table below, so an
// entry can be claimed by a different VAO; the handle compare is what says the
// contents are this object's.
// Whose memos these are. The identity table can recycle a slot for a different
// VAO under LRU pressure, and the handle compare - Gen included - is what says
// the contents are this object's and not its predecessor's.
MG_Pipe::MGPipeHandle Owner = MG_Pipe::kMGPipeNullHandle;
Uint64 Hash = 0;
Uint32 HashConfigVersion = ~0u;
@@ -146,16 +146,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 StateEpoch = 0;
Uint32 StateConfigVersion = ~0u;
};
// Fixed and direct-mapped for the same reason VulkanRenderer's VaoDrawMemo table is
// (P2 m3): nothing frees a VertexElementsCso slot yet, so a grow-on-demand table
// would keep one entry per VAO ever created. 2048 x 48 B is 96 KB.
// Fixed, and a BIJECTION with the identity table that mints the slots
// (MagmaPipeVaoIdentity): entry i is slot i + kMGPipeFirstAllocatableSlot, so the
// index is exact, no two live VAOs can share an entry, and the eviction decision lives
// once - in the identity table's 2-way LRU - instead of once per consumer table.
// Pinned against the mint by a static_assert in VertexInputStateFactory.cpp.
// 2048 x 48 B is 96 KB.
static constexpr Uint32 kVaoMemoSlotCount = 2048; // power of two
mutable Vector<VaoBackendMemos> m_vaoMemos;
// One-entry memo in front of the allocator's lifetimeId -> handle probe, same shape
// and same reason as VulkanRenderer::ResolveVaoHandle.
mutable Uint64 m_lastVaoLifetimeId = 0;
mutable MG_Pipe::MGPipeHandle m_lastVaoHandle = MG_Pipe::kMGPipeNullHandle;
mutable Bool m_lastVaoHandleValid = false;
// The entry belonging to `vao`, claimed (and cleared) if the slot currently holds
// someone else's.
VaoBackendMemos& MemosFor(const MG_State::GLState::VertexArrayObject& vao) const;
@@ -432,6 +432,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
"move the pipeline version, not the parameters version, and the tail would stop " \
"being re-run for it")
// Viewports, DepthRanges and ScissorBoxes are asserted over the WHOLE array while the
// tail reads only element 0. That is deliberately stricter than the reader needs: the
// chunk table has no per-element granularity today, so an array that is dynamic at all
// is dynamic entirely, and asserting the whole of it says so. If a later phase ever
// splits a per-viewport chunk out, this is a build break by design - narrow the assert
// to element 0 then, and say why in the same commit.
MAGMA_TAIL_INPUT_IS_DYNAMIC(Viewports); // ApplyGLViewportState: Viewports[0]
MAGMA_TAIL_INPUT_IS_DYNAMIC(DepthRanges); // ApplyGLViewportState: DepthRanges[0]
MAGMA_TAIL_INPUT_IS_DYNAMIC(BlendColor); // ApplyBlendConstants
@@ -3098,6 +3104,13 @@ void main() {
}
void VulkanRenderer::Initialize() {
#if MOBILEGL_PIPE_PUSH
// P2 D14, and it belongs HERE rather than on a draw: "a Track-H subsystem whose bit is
// clear is a STARTUP Fatal{PipeLegacyMemosDisabled}". Checks Magma's own bit only, and
// only once this backend is the one being brought up, so an Espryt-side bitmask cannot
// kill a Magma run and vice versa.
MagmaPipeValidateSubsystemConfiguration();
#endif
CreateInstance();
CreateSurface();
PickPhysicalDevice();
@@ -3628,22 +3641,21 @@ void main() {
#if MOBILEGL_PIPE_PUSH
// ---- P2 D12.4, the handle arm ----
//
// The slot IS the index. No Fibonacci mix of an address, no two-way probe, no
// frame-serial recycling choice: slots are dense by construction (the allocator has a
// free list plus a high-water mark), so consecutive VAOs land in consecutive entries
// and the collision the address hash existed to spread does not arise below the table
// size. The whole validation is one handle compare, and a handle cannot alias - Gen
// moves on slot REUSE, so a deleted VAO's successor never matches its predecessor's
// entry even at the same address and with a byte-identical configuration.
if (MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
// The slot IS the index, exactly: this table and MagmaPipeVaoIdentity() hold the same
// number of entries and the mint hands out slot i + kMGPipeFirstAllocatableSlot for
// entry i, so the map from live handle to entry is a BIJECTION. No Fibonacci mix of an
// address, no two-way probe here, no frame-serial recycling choice here - not because
// eviction stopped being necessary, but because it happens ONE level down, in the
// identity table's 2-way LRU, where a single decision serves this table and the
// factory's. Two live VAOs cannot land on one entry of this table at all.
//
// The whole validation is one handle compare, and a handle cannot alias: Gen moves
// whenever a slot changes owner, so neither a deleted VAO's successor at the same heap
// address nor a VAO whose slot was recycled under LRU pressure can match a predecessor's
// entry, even with a byte-identical configuration.
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
const MG_Pipe::MGPipeHandle handle = ResolveVaoHandle(*vao);
// Fixed table, so the index wraps rather than growing: nothing in P2 frees a
// VertexElementsCso slot yet (the frontend death notification is Espryt 0b's e2,
// and buffers are the only kind with one today), so a grow-on-demand vector would
// hold one ~1 KB VaoDrawMemo per VAO EVER created. Above the table size this
// degrades to a direct-mapped cache validated by the full {slot, gen}, which is
// strictly better than the address hash it replaces - never wrong, only colder.
const Uint32 index = handle.Slot & (kVaoDrawMemoSlotCount - 1);
const Uint32 index = MagmaPipeSlotIndex(handle);
VaoDrawMemo& entry = m_vaoDrawMemoTable[index];
if (entry.vaoHandle == handle) {
return &entry;
@@ -3660,7 +3672,6 @@ void main() {
entry.bindings.indexBuffer = nullptr;
return &entry;
}
MagmaPipeRequireLegacyArm("LookupVaoDrawMemo");
#endif
// Multiplicative mix of the (16-byte-aligned) address; take high bits, they
// carry the most entropy of a multiply.
@@ -5041,6 +5052,21 @@ void main() {
}
#endif // MOBILEGL_PIPE_LEGACY_MEMOS
#if MOBILEGL_PIPE_PUSH && !MOBILEGL_PIPE_LEGACY_MEMOS
Uint64 VulkanRenderer::ComputePipelineSubsetStateHashFallback() const {
// The client's own hash, over the client's own definition of the pipeline subset - the
// seven pipeline chunks of the P2 chunk table, which is a strict SUPERSET of what
// ComputePipelineStateHash enumerated by hand. The render-pass facts it does not carry
// (colorAttachmentCount, the rasterization sample count, and through them the effective
// sample mask) are exactly the facts entry.renderPassHash separates, which is why the
// CSO handle can key this memo in the first place; this fallback inherits that argument
// unchanged.
//
// Only reached with no render-state CSO bound, and only in a build with no pre-handle
// arm to fall back to instead.
return MG_Pipe::MGPipeComputePipelineSubsetHash(MGB_CTX->GetRenderStateParameters());
}
#endif
// A program that runs a geometry shader AND captures transform feedback. Both halves are
// link-time properties, so this is safe to fold into a pipeline keyed on the program hash.
@@ -5123,32 +5149,29 @@ void main() {
// real handle, the legacy arm a real hash and the null handle, and the probe compares
// both components.
const MG_Pipe::MGPipeHandle renderStateCso = ResolveBoundRenderStateCso();
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
#if MOBILEGL_PIPE_PUSH
if (MG_Pipe::MGPipeHandleIsNull(renderStateCso))
#endif
{
if (!m_pipelineStateHashValid || m_pipelineStateHashVersion != renderStateVersion ||
m_pipelineStateHashColorCount != renderPassEntry.colorAttachmentCount ||
m_pipelineStateHashSampleCount != renderPassEntry.sampleCount) {
m_pipelineStateHash =
ComputePipelineStateHash(renderPassEntry.colorAttachmentCount, renderPassEntry.sampleCount);
m_pipelineStateHashVersion = renderStateVersion;
m_pipelineStateHashColorCount = renderPassEntry.colorAttachmentCount;
m_pipelineStateHashSampleCount = renderPassEntry.sampleCount;
m_pipelineStateHashValid = true;
}
}
#endif
// Exactly one of the two state keys is live per draw, and the ternary short-circuits,
// so a draw on the handle arm neither hashes nor touches the fallback cache.
const Uint64 pipelineStateHash =
#if MOBILEGL_PIPE_PUSH
!MG_Pipe::MGPipeHandleIsNull(renderStateCso) ? 0 :
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
m_pipelineStateHash;
!MG_Pipe::MGPipeHandleIsNull(renderStateCso)
? 0
: ResolveFallbackPipelineStateHash(renderStateVersion,
renderPassEntry.colorAttachmentCount,
renderPassEntry.sampleCount);
#else
0;
// THE PULL BUILD'S TEXT, statement for statement what the base ref has: G1 admits no
// resize of this function, and a helper the compiler merely inlines is not the same
// instruction schedule.
if (!m_pipelineStateHashValid || m_pipelineStateHashVersion != renderStateVersion ||
m_pipelineStateHashColorCount != renderPassEntry.colorAttachmentCount ||
m_pipelineStateHashSampleCount != renderPassEntry.sampleCount) {
m_pipelineStateHash =
ComputePipelineStateHash(renderPassEntry.colorAttachmentCount, renderPassEntry.sampleCount);
m_pipelineStateHashVersion = renderStateVersion;
m_pipelineStateHashColorCount = renderPassEntry.colorAttachmentCount;
m_pipelineStateHashSampleCount = renderPassEntry.sampleCount;
m_pipelineStateHashValid = true;
}
const Uint64 pipelineStateHash = m_pipelineStateHash;
#endif
for (Uint32 i = 0; i < m_pipelineMemoCount; ++i) {
const PipelineMemoEntry& entry = m_pipelineMemo[i];
@@ -6287,7 +6310,7 @@ void main() {
// the two cannot disagree about whether "the VAO moved". The config version stays:
// it answers a different question (did this same object's layout change).
const Bool vaoMoved =
MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)
MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)
? (!(ResolveVaoHandle(vao) == snap.vaoHandle) ||
vao.GetConfigVersion() != snap.vaoConfigVersion)
: (static_cast<const void*>(&vao) != snap.vao ||
@@ -6528,32 +6551,24 @@ void main() {
// fast path's copy of it, and the two must key identically or the fast path would
// hand back a pipeline the full path would not have matched.
const MG_Pipe::MGPipeHandle renderStateCso = ResolveBoundRenderStateCso();
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
#if MOBILEGL_PIPE_PUSH
if (MG_Pipe::MGPipeHandleIsNull(renderStateCso))
#endif
{
if (!m_pipelineStateHashValid || m_pipelineStateHashVersion != renderStateVersion ||
m_pipelineStateHashColorCount != snap.renderPassColorCount ||
m_pipelineStateHashSampleCount != snap.renderPassSampleCount) {
m_pipelineStateHash =
ComputePipelineStateHash(snap.renderPassColorCount, snap.renderPassSampleCount);
m_pipelineStateHashVersion = renderStateVersion;
m_pipelineStateHashColorCount = snap.renderPassColorCount;
m_pipelineStateHashSampleCount = snap.renderPassSampleCount;
m_pipelineStateHashValid = true;
}
}
#endif
const Uint64 pipelineStateHash =
#if MOBILEGL_PIPE_PUSH
!MG_Pipe::MGPipeHandleIsNull(renderStateCso) ? 0 :
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
m_pipelineStateHash;
!MG_Pipe::MGPipeHandleIsNull(renderStateCso)
? 0
: ResolveFallbackPipelineStateHash(renderStateVersion, snap.renderPassColorCount,
snap.renderPassSampleCount);
#else
0;
// The pull build's text, statement for statement (see GetOrCreatePipeline).
if (!m_pipelineStateHashValid || m_pipelineStateHashVersion != renderStateVersion ||
m_pipelineStateHashColorCount != snap.renderPassColorCount ||
m_pipelineStateHashSampleCount != snap.renderPassSampleCount) {
m_pipelineStateHash =
ComputePipelineStateHash(snap.renderPassColorCount, snap.renderPassSampleCount);
m_pipelineStateHashVersion = renderStateVersion;
m_pipelineStateHashColorCount = snap.renderPassColorCount;
m_pipelineStateHashSampleCount = snap.renderPassSampleCount;
m_pipelineStateHashValid = true;
}
const Uint64 pipelineStateHash = m_pipelineStateHash;
#endif
const auto memoTransformFlags =
ProgramFactory::CompileOptionFlags(snap.resolvedTransformFlags);
@@ -6600,7 +6615,13 @@ void main() {
snap.vao = static_cast<const void*>(&vao);
snap.vaoLifetimeId = vao.GetLifetimeId();
#if MOBILEGL_PIPE_PUSH
snap.vaoHandle = ResolveVaoHandle(vao);
// Guarded by the SUBSYSTEM, not only by the build switch: with bit 6 clear the field
// is dead (vaoMoved takes the address/lifetime-id branch), and minting a handle for it
// would put this package's cost inside MOBILEGL_PIPE_PUSH=0 - the all-pull control arm
// D14 defines as reproducing P1 exactly, and the arm D.4.3's T2 is measured on.
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
snap.vaoHandle = ResolveVaoHandle(vao);
}
#endif
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.vaoLayoutHash = vaoLayoutHash;
@@ -7184,7 +7205,11 @@ void main() {
snap.vao = &vao;
snap.vaoLifetimeId = vao.GetLifetimeId();
#if MOBILEGL_PIPE_PUSH
snap.vaoHandle = ResolveVaoHandle(vao);
// Subsystem-guarded for the same reason as the other stamping site: the field
// is dead with bit 6 clear, and MOBILEGL_PIPE_PUSH=0 has to be P1 exactly.
if (MagmaPipeTrackHArmIsHandles(MG_Pipe::kMGPipeSubsystemMagmaVertexInput)) {
snap.vaoHandle = ResolveVaoHandle(vao);
}
#endif
snap.vaoConfigVersion = vao.GetConfigVersion();
snap.drawFbo = drawFbo.get();
@@ -884,23 +884,72 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// The arm is live only when the render-state subsystem is migrated in this run AND the
// client has actually bound a CSO. The second half is not belt and braces: a tree whose
// tracker does not emit create/bind_render_state yet has no handle to key on, and
// keying every draw on the null handle would alias every render state onto one entry.
// delete_render_state clears the binding (MG_Pipe/PipeApply.cpp), so the null handle is
// reachable on any tree. Keying every draw on it would alias every render state onto
// one memo entry, so a null handle means "fall back to a state hash" - never an abort,
// and never a per-draw consultation of the legacy-memo lever: bit 0 is not a Track-H
// subsystem (D14 labels only bits 5 and 6 that), and the lever's Fatal is a STARTUP
// one, in MagmaPipeValidateSubsystemConfiguration.
//
// The fallback is warned ONCE rather than logged at debug, and that is deliberate: a
// silent fallback is what makes "the CSO arm never ran" easy to miss. W is compiled in
// at every shipped log level, _ONCE costs one static bool test, and its ABSENCE from a
// run's log is the positive evidence that every draw keyed on a handle.
//
// Push-only by construction: the pull build does not compile this function at all, so
// its two callers are statement-for-statement what they were (G1).
MG_Pipe::MGPipeHandle ResolveBoundRenderStateCso() const {
if (!MagmaPipeSubsystemOn(MG_Pipe::kMGPipeSubsystemRenderState)) {
MagmaPipeRequireLegacyArm("GetOrCreatePipeline");
return MG_Pipe::kMGPipeNullHandle;
}
const MG_Pipe::MGPipeHandle boundCso = MG_Pipe::MGPipeApplier().BoundRenderStateCso;
if (MG_Pipe::MGPipeHandleIsNull(boundCso)) {
MGLOG_D_ONCE("MGPipe: kMGPipeSubsystemRenderState is on but no render-state CSO is "
"bound; the pipeline memo falls back to the pre-handle state hash");
MagmaPipeRequireLegacyArm("GetOrCreatePipeline");
MGLOG_W_ONCE("MGPipe: kMGPipeSubsystemRenderState is on but no render-state CSO is "
"bound; the pipeline memo is running on a state hash, not on the CSO "
"handle (no tracker on this build, or a draw between "
"delete_render_state and the next bind)");
}
return boundCso;
}
// The memo key's STATE-HASH half, for a draw that has no CSO handle to key on: the
// pre-handle arm, and the fallback of D12.1's handle arm. Cached on the pipeline-state
// version plus the two render-pass facts the hash's inputs depend on, so an unchanged
// (version, colorAttachmentCount, sampleCount) proves the bytes are unchanged.
//
// [deviation from D12.1] The brief deletes this gate and its cached fields outright.
// They cannot go while a no-CSO draw is reachable - and it is, on any tree: a draw
// between delete_render_state and the next bind has no handle. On a tree whose tracker
// binds a CSO these five words are written once and never read again; they retire for
// real when the pull path does, at P13.
Uint64 ResolveFallbackPipelineStateHash(Uint renderStateVersion, Uint32 colorAttachmentCount,
VkSampleCountFlagBits rasterizationSamples) {
if (!m_pipelineStateHashValid || m_pipelineStateHashVersion != renderStateVersion ||
m_pipelineStateHashColorCount != colorAttachmentCount ||
m_pipelineStateHashSampleCount != rasterizationSamples) {
#if MOBILEGL_PIPE_LEGACY_MEMOS
m_pipelineStateHash =
ComputePipelineStateHash(colorAttachmentCount, rasterizationSamples);
#else
m_pipelineStateHash = ComputePipelineSubsetStateHashFallback();
#endif
m_pipelineStateHashVersion = renderStateVersion;
m_pipelineStateHashColorCount = colorAttachmentCount;
m_pipelineStateHashSampleCount = rasterizationSamples;
m_pipelineStateHashValid = true;
}
return m_pipelineStateHash;
}
#endif // MOBILEGL_PIPE_PUSH
#if MOBILEGL_PIPE_PUSH && !MOBILEGL_PIPE_LEGACY_MEMOS
// The same answer as ComputePipelineStateHash, computed from the P2 chunk table
// instead of from a hand-written field list, for the build that compiles no
// pre-handle arm (cmake -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF). It is the CLIENT's own
// hash function - MGPipeComputePipelineSubsetHash over the 396 pipeline bytes - so a
// draw keyed on it and a draw keyed on a CSO handle are keyed on the same equivalence
// class of state, and the render-pass facts stay separated by renderPassHash either
// way. This is what makes the no-legacy build RUNNABLE rather than a configuration
// that aborts on the first draw that arrives without a CSO.
Uint64 ComputePipelineSubsetStateHashFallback() const;
#endif
#if MOBILEGL_PIPE_LEGACY_MEMOS
// THE PRE-HANDLE ARM (P2 brief D12.1 / D14). Hash of every fixed-function GL state the
@@ -924,7 +973,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// the re-key and keeps reading Multisample / SampleMask / SampleMaskValue out of the
// working block.
Uint32 ResolveEffectiveSampleMask(VkSampleCountFlagBits rasterizationSamples) const;
#if MOBILEGL_PIPE_LEGACY_MEMOS
// ResolveFallbackPipelineStateHash's cache. Written once and never read again on a
// build whose client binds a render-state CSO; see that function for why it survives
// the re-key at all.
Uint m_pipelineStateHashVersion = 0;
Uint32 m_pipelineStateHashColorCount = 0;
// The sample count the cached hash was computed at. A pipeline-state input now depends on
@@ -933,7 +984,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkSampleCountFlagBits m_pipelineStateHashSampleCount = VK_SAMPLE_COUNT_1_BIT;
Uint64 m_pipelineStateHash = 0;
Bool m_pipelineStateHashValid = false;
#endif
// GetShaderTransformFlags memo. NOT pure in the pre-transform alone: the
// function also reads whether the bound DRAW framebuffer is the default one
// (only the default framebuffer gets the Y-flip and rotation bits - an FBO
@@ -956,9 +1006,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void InvalidatePipelineMemo() {
m_pipelineMemoCount = 0;
m_pipelineMemoNext = 0;
#if MOBILEGL_PIPE_LEGACY_MEMOS
m_pipelineStateHashValid = false;
#endif
}
UnorderedMap<ProgramFactory::HashType, VkPipeline> m_computePipelines;
UniquePtr<ProgramFactory> m_programFactory;
@@ -1359,32 +1407,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// fixed table also makes every VaoDrawMemo/ResolvedVertexBindings pointer
// stable for the duration of a draw, which the EBO memo handoff
// (m_currentDrawResolvedEntry) relies on.
// [deviation from D12.4] The brief asks for a grow-on-demand Vector. The table is
// FIXED, and under the handle arm it is sized to - and is a BIJECTION with - the
// identity table that mints the slots (MagmaPipeVaoIdentity), so entry i is slot
// i + kMGPipeFirstAllocatableSlot and no two live VAOs can ever share it. Growing on
// demand only makes sense against an allocator that frees, and nothing in P2 frees a
// VertexElementsCso slot; the eviction that has to happen somewhere happens once, in
// the identity table's LRU, instead of twice in two tables that could disagree.
static constexpr Uint32 kVaoDrawMemoSlotCount = 2048; // power of two
Vector<VaoDrawMemo> m_vaoDrawMemoTable;
#if MOBILEGL_PIPE_PUSH
// One-entry memo in front of the slot allocator's lifetimeId -> handle map (P2
// D12.4). Acquiring a handle is a hash probe, and LookupVaoDrawMemo runs per draw, so
// the arm would otherwise have swapped one probe (the address hash it deletes) for
// another. A run of draws over one VAO - the common intra-batch shape - pays a single
// Uint64 compare instead.
//
// A lifetime id is never reused, so a hit can only ever be this same object; the
// valid flag exists rather than a zero sentinel because nothing promises the frontend
// counter starts above zero.
Uint64 m_lastVaoHandleLifetimeId = 0;
MG_Pipe::MGPipeHandle m_lastVaoHandle = MG_Pipe::kMGPipeNullHandle;
Bool m_lastVaoHandleValid = false;
static_assert(kVaoDrawMemoSlotCount == kMagmaVaoIdentityEntries,
"the VAO draw memo table is indexed directly by MagmaPipeSlotIndex, so it has "
"to hold exactly one entry per slot the VAO identity table can mint");
// The VAO's {slot, gen}. An array probe (one mask, at most two Uint64 compares), so
// there is no memo in front of it: the address multiply plus two-way probe it replaces
// cost more than this does, and a cached handle could go stale behind the identity
// table's own eviction, which is a class of bug worth not having.
MG_Pipe::MGPipeHandle ResolveVaoHandle(const MG_State::GLState::VertexArrayObject& vao) {
const Uint64 lifetimeId = vao.GetLifetimeId();
if (m_lastVaoHandleValid && m_lastVaoHandleLifetimeId == lifetimeId) {
return m_lastVaoHandle;
}
const MG_Pipe::MGPipeHandle handle =
MagmaPipeHandleOf(MG_Pipe::MGPipeKind::VertexElementsCso, lifetimeId);
m_lastVaoHandleLifetimeId = lifetimeId;
m_lastVaoHandle = handle;
m_lastVaoHandleValid = true;
return handle;
return MagmaPipeHandleOf(MG_Pipe::MGPipeKind::VertexElementsCso, vao.GetLifetimeId());
}
#endif
// "Is this VAO's content hash already memoized?", asked of whichever side owns the