Commit Graph
11 Commits
Author SHA1 Message Date
swung0x48 77088a681d [Fix] (Magma): drop the per-kind ABA wrapper no build ever evaluates and say why the constexpr table and the knob stand alone 2026-09-08 19:08:12 -04:00
swung0x48 3049dc1d53 [Fix, Test] (Pipe): read the shader composite band's own counters in the composite leak case, arm the ABA flip from anywhere under a backend, match Espryt's exact refusal line and pin the bit-10-requires-bit-11 arm 2026-09-08 19:08:12 -04:00
swung0x48 11044b990f [Test] (Pipe): reproduce the texture, framebuffer, renderbuffer, sampler, view and program handle ABA through public GL and prove the pre-rekey guards are what stops it 2026-09-08 19:08:12 -04:00
swung0x48 a9778eaabe [Fix, Test] (Magma, MG_Test): cover the {slot, gen} generation in a unit test that forces a real slot reuse, and stop claiming the ABA lanes do
- 55d2af9b claimed - in its message, in MagmaPipeArms.h, in VertexInputStateFactory.cpp and
  in MG_IntegrationTest/CMakeLists.txt - that the AbaControlHandles lane defeats the
  GENERATION in {slot, gen}. It does not, and no lane of that shape can. Magma's mint has no
  death notification (nothing in MG_Backend/DirectVulkan consumes NotifyStateObjectDestroyed)
  and returns a slot only through OnFrameBoundary's age sweep, kSweepInterval 256 /
  kRetireAgeBoundaries 1024; HandleRecycleScenario issues five frame boundaries, so the
  replacement VAO acquires against an empty free list and gets a BRAND-NEW slot at Gen 1
  (measured: redVao slot=2 gen=1, greenVao slot=3 gen=1). The knob-off FRESH verdict there is
  decided by the SLOT alone, and deleting ++m_entries[index].Gen leaves all 32 HandleRecycle
  entries green - re-measured this round.
- What the lane does defeat is the object identity that SELECTS the slot, which IS the key the
  handle arm ships, and that is what the three code sites now say. The two requirements are
  mutually exclusive for the pixel-visible memo: a genuine slot reuse needs >= 1024 idle
  boundaries after the dead object's last draw, which necessarily puts the two draws in
  different frames, and ResolvedVertexBindings - the only memo carrying a GPU slice rather
  than a layout - declines across frames by design.
- So the generation is covered where it IS expressible. MG_Test/Pipe/MagmaPipeIdentityTest.cpp
  drives the mint's real retire -> reuse (1280 boundaries, with a keep-alive object holding the
  first allocatable slot so the reuse is not the slot the control aliases onto) and asserts
  four things: the retired slot comes back with Gen+1; with the knob OFF a memo stamped at
  {slot, gen=N} is NOT served at {slot, gen=N+1}; with the knob ON it IS, out of one uncleared
  and unclaimed entry; and a live object keeps its slot, its generation and its memo across two
  sweeps, so the generation cannot be "fixed" by bumping it on every acquisition.
- The claim rule itself moves into MagmaPipeArms.h as MagmaPipeClaimSlotMemos so the suite
  exercises production code rather than a copy of it. VertexInputStateFactory::MemosFor is now
  one call to it and is otherwise unchanged, on both the knob-on and the knob-off path.
- Load-bearing, measured: with ++m_entries[index].Gen commented out, ctest -L unit in
  build-push goes 1563/1566 - three of the four new cases red, one of them naming the inherited
  0xDEAD payload out of the same slot - while ctest -R HandleRecycle stays 32/32. Restored, all
  four pass in build-push and build-verify and skip visibly in the pull build, so the ctest name
  sets stay identical (G2).
2026-09-08 01:08:51 -04:00
swung0x48 55d2af9bd1 [Fix, Test] (Magma, MG_IntegrationTest): make the handle-ABA negative control construct its own collision and defeat the {slot, gen} generation
- MOBILEGL_PIPE_HANDLE_ABA_CONTROL asserted the corruption and saw correct pixels, so
  DirectVulkan.HandleRecycle.AbaControl.*AVertexArray* was RED in an always-on
  integration-gpu lane while every guard it was supposed to be defeating stood. Two
  measured reasons, neither of them the {slot, gen} re-key: (1) D18 spelled the control as
  "hash the raw BufferObject* instead of its lifetime id, and skip the vaoLifetimeId
  compare", which only collides if the allocator hands the freed block back - it does not.
  glGen* recycles the NAME, but a VertexArrayObject is 3920 bytes, past glibc's tcache, so
  its chunk goes to the unsorted bin and is split by the next allocation the replacement
  path makes; four create/delete cycles in one run gave four addresses ~1 MiB apart, and
  the BufferObject behaves the same. (2) The reproducer put a frame boundary between the
  arming draw and the recycled draw, and the only memo that carries a GPU slice rather
  than a layout - ResolvedVertexBindings - declines across frames by design, so no key
  collision whatsoever could have shown up in pixels.
- The control no longer asks the allocator for the collision: on both arms it replaces the
  object identity in DirectVulkan's vertex-input keys with a constant, which is the
  strongest form of "the block came back" and is deterministic. Three sites, all behind
  one question (MagmaPipeAbaControlDefeatsIdentity): the buffer identity leaves
  VertexInputStateFactory::ComputeHash, VertexInputStateFactory::MemosFor claims one entry
  without its Owner compare, and VulkanRenderer::LookupVaoDrawMemo hands one entry back
  uncleared ahead of both arms.
- That is what makes the control cover the key P2 SHIPS. Under MOBILEGL_PIPE_PUSH=0 the
  handle arm is not executed at all, so the old control said nothing about the generation
  in {slot, gen} - the whole of what makes the re-keyed memos ABA-safe. A second lane,
  DirectVulkan.HandleRecycle.AbaControlHandles., runs the handle arm with the knob and
  asserts the same corruption; D18's lane is kept verbatim beside it for the pre-handle arm.
- The reproducer's two draws now share a frame, and both buffers are realised before the
  window, so a moved slice epoch cannot mask the ABA behind a gate that is not about
  identity. Nothing else is relaxed: the frame serial, the slice epochs and the host-map
  check stay in force, so a green arm still means "a replacement object was handed its
  predecessor's resolved vertex bindings because the identity halves of the keys were
  defeated".
- ExpectPixelsFor now prints what it OBSERVED (STALE/FRESH/NEITHER) next to what the arm
  expected, on every arm and whether or not the case passes.
- Knob-off is unchanged and the pull build is untouched: every new branch is
  #if MOBILEGL_PIPE_PUSH, and symbol_report.py --threshold 0 against the pre-P2 baseline
  still reports 0 added / 0 removed / 0 renamed and the same four resized symbols
  (RenderState::RenderState, SetCapability, IsCapabilityEnabled, _GLOBAL__sub_I_DirectGLES.cpp).
2026-09-08 00:19:39 -04:00
swung0x48 e5c032c89e [Fix] (Magma): log the mint's high-water at a level a shipped build keeps
- The live-object high-water mark is the number review v2's MAJOR 1 wants measured on
  minecraft-1.21.4-in-world and ...-sodium-in-world, and no desktop gate can produce it. It
  was emitted at MGLOG_D, which is compiled out of every build that ships and of every build
  P2 measures, so the line existed only in a configuration nobody runs.
- MGLOG_I instead, still only on the allocate-a-new-slot branch and still only at powers of
  two from 1024 up: at most a handful of lines for a whole session, never one on a draw
  (ROADMAP.md:7). Declared as a narrow deviation from D20's "MGLOG_D for anything
  non-critical" in the comment beside it.
2026-09-07 23:18:09 -04:00
swung0x48 a174a06c79 [Fix] (Magma): size the {slot, gen} mint by the live working set instead of by a capacity, and give it to the renderer that uses it
- MagmaPipeIdentityTable was a FIXED 2048/8192-entry, 2-way set-associative LRU. Above
  capacity it evicted LIVE objects, and every memo keyed on the handle died with them: a
  verbatim transcription of the previous Acquire lost 54% of uses' handles at 2500 live VAOs
  against 2048 entries, and 20% at 1024 live VAOs once the lifetime ids are sparse (an app
  that creates and destroys VAOs - the Minecraft chunk shape this exists for).
- Two of the three memos it fed had NO capacity before this package: the content-hash memo
  and the resolved-state memo were unbounded mutable fields on VertexArrayObject. Eviction
  there turns one ComputeHash per VAO reconfiguration into one per DRAW; once the buffer
  table thrashes too, the vertex-input content hash becomes a per-draw value that inserts a
  fresh heap-allocated BackendVertexInputState into an unbounded map on every draw, swept
  only every 256 frame boundaries. That is a worse leak than the one the fixed table was
  introduced to avoid.
- So the mint grows on demand and reclaims by AGE: a lifetime-id map with a one-entry front
  memo, a free list, and an OnFrameBoundary sweep on the same cadence and retirement age as
  the cache entries those slots key. Footprint tracks the live DRAWN working set instead of
  objects ever created, which is the property MG_Impl/Pipe/SlotAllocator cannot have here
  (nothing in P2 can call its Free). Re-run of the same workloads: handle churn is 0.0% at
  512, 1024, 2048, 2500, 3000, 4000, 8192, 10000 and 16000 live objects, consecutive and
  sparse ids alike, at 1 and 5 acquisitions per use.
- VertexInputStateFactory::m_vaoMemos follows the mint with no capacity of its own, through a
  chunked table whose entry addresses never move - which is what the fixed table's only real
  guarantee was, and D12.4's grow-on-demand ask without a relocating Vector.
- VulkanRenderer::m_vaoDrawMemoTable deliberately keeps the base ref's 2048 entries and the
  base ref's older-frameSerial victim rule, and changes only its KEY. It is the one memo of
  the three that had a capacity before P2, a VaoDrawMemo is ~450 B, and losing one costs one
  vertex-binding re-resolve. Measured steady-state miss rate against the base ref's
  address-hashed table: 0.0% vs 6.4% at 512 live VAOs, 0.0% vs 24.0% at 1024, 0.0% vs 60.0%
  at 2048, 36.2% vs 69.5% at 2500, 63.5% vs 79.1% at 3000; both are ~100% at 4096 (2x
  capacity), where an LRU on a cyclic pattern cannot win.
- The two tables are now a MagmaPipeIdentityTables member of VulkanRenderer, handed to its
  VertexInputStateFactory, instead of two function-local statics that outlived every context
  and shared one reclamation clock across two.
- A Gen that reaches 2^32-1 retires its slot for good rather than wrapping. MOBILEGL_ASSERT
  is compiled out of every build P2 runs, and a DEBUG-level build of this tree does not
  compile at all (MG_Util/Types.h uses MOBILEGL_ASSERT before MGLOG_F is declared - untouched
  since the base ref, and not this package's file), so the defence has to be on the release
  path to exist.
- The no-CSO pipeline-memo fallback stops using MGLOG_W_ONCE. MOBILEGL_LOG_ONCE_INTERNAL is
  an unconditional std::atomic_flag::test_and_set - a locked xchg per evaluation, not "one
  static bool test" - and this site is on the per-draw path in exactly the configuration that
  reaches it. A plain per-renderer bool replaces it, and the comment now says what the
  warning's absence does and does not prove (nothing at all while bit 0 is clear).
- MOBILEGL_PIPE_LEGACY_MEMOS=0 with kMGPipeSubsystemRenderState clear still runs the
  pre-handle state hash - there is a correct answer there and bit 0 is not Track H, so it is
  not fatal - but it is no longer silent: the startup gate names the combination.
- The D12.3 static_assert block now names D19's DynamicChunksCoverMagmasDynamicTailKey, whose
  ctest entry lives in package A's file, so the integrator can see which half is missing.
2026-09-07 23:18:09 -04:00
swung0x48 46841ac706 [Fix] (Magma): assert rather than assume that a handle indexed into a per-slot table is non-null
- MagmaPipeSlotIndex subtracted kMGPipeFirstAllocatableSlot unconditionally, so a null
  handle would have indexed a per-slot table at ~0u. It is unreachable - both lifetime-id
  sources start at 1, which is why Acquire's zero guard never fires - but the consequence
  of being wrong about that is an out-of-range write, not a wrong answer, so it is asserted
  and the index is pinned to 0 in a release build.
2026-09-07 23:18:09 -04:00
swung0x48 01179c54d2 [Fix] (Magma): make the legacy-memo lever a startup gate for Magma's own bit, bound the {slot, gen} mint, and keep the all-pull arm free of push-only cost
- MOBILEGL_PIPE_LEGACY_MEMOS=0 no longer aborts a draw. D14 spends that lever at
  STARTUP and only on a Track-H subsystem, so MagmaPipeValidateSubsystemConfiguration
  runs once from VulkanRenderer::Initialize and checks bit 6 alone: an Espryt-side
  bitmask cannot kill a Magma run, and bit 0 - which is not Track H and not a memo
  re-key - is out of the lever's scope entirely. MOBILEGL_PIPE_PUSH=0x60 with the
  lever off went from 9/9 aborted to 432/432.
- A pipeline memo with no render-state CSO bound falls back instead of aborting.
  delete_render_state clears the binding, so the null handle is reachable on any
  tree; the fallback is the pre-handle state hash where one is compiled, and the
  client's own MGPipeComputePipelineSubsetHash over the same 396 pipeline bytes
  where it is not - which is what makes -DMOBILEGL_PIPE_LEGACY_MEMOS=OFF a runnable
  configuration (180/432 aborted before, 432/432 now) instead of a build that dies
  on its first draw. The fallback warns once, so a run that never keys on a CSO
  handle says so in its log instead of passing silently.
- The handles are minted by a fixed-capacity, self-recycling identity table in the
  backend, not by MG_Impl's client allocator. Nothing in P2 frees a VertexElementsCso
  or Buffer slot - the frontend has no death notification Magma can hook - so the
  allocator's live Allocate and dead Free grew one SlotState plus one hash-map node
  per object ever created, for the life of the process. The table is 2-way
  set-associative with an LRU victim and a Gen bump on reuse: bounded (32 KB for
  VAOs, 128 KB for buffers), exactly as ABA-proof, and it takes MG_Backend's only
  include of MG_Impl back out.
- Both per-slot memo tables are now a BIJECTION with that mint rather than a masked
  direct map, so two live VAOs cannot share an entry and the eviction decision lives
  once, in the identity table, instead of once per consumer table. The density claim
  the masked tables rested on was false while slots grew monotonically, and the
  masked form had also dropped the second candidate and the frame-serial victim
  choice the address-hashed table used to have.
- snap.vaoHandle is stamped only when bit 6 is on. It was guarded by the compile
  switch alone, so MOBILEGL_PIPE_PUSH=0 - the all-pull control D14 defines as
  reproducing P1 exactly, and the arm D.4.3's T2 is measured on - paid a mint per
  new VAO and a compare per draw for a field that arm never reads.
- Every re-keyed Track-H site now asks the same MagmaPipeTrackHArmIsHandles helper,
  including VertexInputStateFactory::ComputeHash, which decided for itself before
  and could key on the pre-handle identity while its neighbours keyed on the handle.
- The pull build's two pipeline-memo sites keep the base ref's text statement for
  statement: G1 is back to the contract's four resized symbols, 0 added/removed.
2026-09-07 23:18:09 -04:00
swung0x48 43f8b47088 [Refactor] (Magma): key the vertex-input cache and the VAO draw memo on {slot, gen} instead of a lifetime id and a heap address
- Track H subsystem 4 (P2 brief D12.4, ARCHITECTURE.md 9.5), behind
  kMGPipeSubsystemMagmaVertexInput.
- VertexInputStateFactory::ComputeHash's buffer identity component becomes the buffer's
  {slot, gen} - "lifetimeId -> gen mixed into every server-side content hash". Both are
  equally ABA-proof (the allocator maps one onto the other and bumps Gen only on slot
  REUSE); what changes is that the hash now carries the identity the SERVER will be handed
  once buffers travel as handles, instead of a number only the client can mint.
- LookupVaoDrawMemo becomes a direct slot index: the slot IS the index, and the whole
  validation is one handle compare. Gone with the re-key are the Fibonacci mix of the VAO's
  address, the two-way probe, the frame-serial eviction choice and the (pointer, lifetime
  id) pair - slots are dense by construction, so consecutive VAOs land in consecutive
  entries and the collision the address hash existed to spread does not arise below the
  table size.
- The table stays FIXED at 2048 entries and the slot index wraps, where the brief calls
  for a grow-on-demand vector. Reason, and it is a tree fact the brief does not carry:
  nothing in P2 frees a VertexElementsCso slot. The frontend death notification is Espryt
  0b's e2 and it covers Espryt's six kinds; buffers are the only kind with an OnDestroy
  hook today. A grow-on-demand table would therefore hold one ~1 KB VaoDrawMemo per VAO
  EVER created, which on a chunk-cycling Minecraft frame is tens of megabytes. Above the
  table size this degrades to a direct-mapped cache validated by the full {slot, gen}:
  never wrong, only colder, and strictly better than the address hash it replaces.
  Revisit when object deletion reaches the client allocator.
- SetupDrawSnapshot's VAO identity collapses to the same handle - one compare instead of
  (address, lifetime id) - so the snapshot and the draw memo cannot disagree about whether
  the VAO moved. The config version stays: it answers a different question.
- Handle acquisition sits behind a one-entry memo in the renderer. Acquiring is a hash
  probe into the allocator's lifetimeId -> slot map and LookupVaoDrawMemo runs per draw, so
  without it the arm would have swapped the address hash it deletes for another probe; a
  run of draws over one VAO now pays a single Uint64 compare. Magma acquires the handles
  itself because the tracker does not emit object-class state in P2 (it emits for dirty
  bits 0-4 only); when it does, these become reads of what the client already sent.
- Negative control C (MOBILEGL_PIPE_HANDLE_ABA_CONTROL, brief D18) is implemented here
  because the two guards it defeats live here: it makes ComputeHash hash the raw
  BufferObject* and makes LookupVaoDrawMemo skip the lifetime-id compare - the exact state
  the table was in before the ABA fix. It applies to the PRE-HANDLE arm, which is what
  HandleRecycleScenario.AbaControl runs (MOBILEGL_PIPE_PUSH=0), and it is what proves that
  scenario's reproducer still reproduces instead of passing for the wrong reason.
- Verification on this tree: ctest -L integration-gpu -R DirectVulkan is 432/432 under the
  default bitmask and 432/432 under MOBILEGL_PIPE_PUSH=0, and -L unit is green. Pull build
  symbol_report --threshold 0: 0 added / 0 removed / 0 renamed, 4 resized, all four the
  contract commit's.
2026-09-07 23:18:09 -04:00
swung0x48 96c544514e [Refactor] (Magma): key the pipeline memo on the render-state CSO handle and stop recomputing a hash the client already computed
- P2 D12.1. GetOrCreatePipeline's memo compared a VALUE hash of the pipeline-relevant
  fixed-function state that Magma recomputed itself. After P2 the CLIENT hashes exactly
  those bytes when it mints a content-addressed render-state CSO
  (MGPipeComputePipelineSubsetHash over the seven pipeline chunks), so the bound CSO
  handle IS that key and ComputePipelineStateHash was doing the boundary's work twice.
  The client's pipeline subset is a strict SUPERSET of the 24 members the hash read, so
  the handle discriminates at least as finely as the hash it replaces.
- renderPassHash STAYS in the key, and that is load-bearing rather than conservative:
  ComputePipelineStateHash was never a pure function of RenderStateParameters - its
  signature took colorAttachmentCount and rasterizationSamples, and
  ResolveEffectiveSampleMask reads the latter - so those two render-pass facts have to
  stay separated by something. entry.renderPassHash already separates them (the pass
  hash folds each attachment's sample count and the attachment set), which is why
  collapsing the state half onto a handle loses no discrimination.
  ResolveEffectiveSampleMask is NOT deleted with the hash: it is a payload computation,
  and it keeps reading Multisample / SampleMask / SampleMaskValue out of the working
  block.
- Both memo probes are re-keyed, not just the full path's: TrySetupDrawFastPath carries
  its own copy of the probe, and a fast path that keyed differently from the full path
  would hand back a pipeline the full path would not have matched.
- The arm is chosen at runtime, per D14: kMGPipeSubsystemRenderState in the
  MOBILEGL_PIPE_PUSH bitmask AND a non-null bound CSO. The second half is not belt and
  braces - a tree whose tracker does not emit create/bind_render_state yet has no handle
  to key on, and keying every draw on the null handle would alias every render state onto
  one memo entry. Falling into the pre-handle arm with Features.PipeLegacyMemos=0 is
  Fatal{PipeLegacyMemosDisabled}, so HandleRecycleScenario.Handles cannot go green by
  quietly running the old code.
- ComputePipelineStateHash and its five cached-hash members (m_pipelineStateHash{,Valid,
  Version,ColorCount,SampleCount}) survive only under MOBILEGL_PIPE_LEGACY_MEMOS, which
  a pull build forces ON: they exist purely to avoid re-hashing, and the handle arm never
  hashes. InvalidatePipelineMemo loses them on the same condition.
- New MagmaPipeArms.h holds the two-switch arm selector shared by the P2 Magma re-keys.
- G1, pull build, symbol_report --threshold 0 against ~/w7/p2-before-libMobileGL.so:
  0 added, 0 removed, 0 renamed, 4 resized - and all four are the CONTRACT commit's
  (RenderState::{RenderState,SetCapability,IsCapabilityEnabled} and
  _GLOBAL__sub_I_DirectGLES.cpp). This commit adds none: every edit is inside a
  MOBILEGL_PIPE_PUSH arm and the pre-handle statements are left where they stood, which
  is why ResolveBoundRenderStateCso is push-only rather than a shared helper - an earlier
  shared-helper shape moved 104 bytes of GetOrCreatePipeline around for no behaviour
  change and the gate saw it.
2026-09-07 23:18:09 -04:00