- M2's defer path destroys a wire store MID-FRAME (the serial-0 park, the sweep's prefix walk, the watermark); before it a wire store only died at the boundary that also clears UniformManager's memos (BeginFrame from Present or a drain). Those memos are keyed on the VkBuffer HANDLE - m_descriptorReuseMemo hashes the VkDescriptorBufferInfo words, FastRebindMemo compares uboBuffer - and on the wire arm the direct-bind UBO and the SSBO descriptor name the store's own handle. A store destroyed at a park, its handle value re-minted by the next Create (a heap pointer under lavapipe, LIFO out of glibc's tcache), and a draw resolving the same (handle, range) reused a set whose descriptor still pointed at the dead store's memory: silent wrong bytes, no Fatal (ID-P7-43, the fable review's must-fix).
- VkBufferManager gains m_wireStoreDestroyEpoch, bumped on every path that destroys a wire store's VkBuffer (DeferWireRelease's immediate destroy, SweepDeferredWireReleases once per sweep that freed anything, DestroyAllDeferredWireReleases, Shutdown's record teardown) and never reset, with GetWireStoreDestroyEpoch(). UniformManager mixes it into the descriptor-reuse signature and records it in FastRebindMemo, refusing the fast rebind when it moved; all under MOBILEGL_BUILD_DISAGGREGATED. Deliberately not m_sliceEpochCounter, which every WriteWireBuffer bumps and would defeat the memo on every glBufferSubData. On the wire arm only the signature is reached (SetupWireDraw passes no sampler hint), the FastRebindMemo field is the defensive half.
- Two comments from the same review: SweepDeferredWireReleases no longer claims GetSyncPointSubmitIndex() never decreases - it steps back by one across an abandoned recording (RecreateSwapchain, the minimized Present), and the prefix walk is conservative across that, not wrong; and the header's "floor is not a proof" keeps its decision on its first reason alone, the second was a stale citation of a fix that is on the integration tree.
- Section 9 partition exception for UniformManager.{h,cpp} granted by the integrator for this fix.
Measured on bsl-esc-menu-854 (spawn, lavapipe) with the byte watermark alone: the dead VkBuffers
at the end fell from 25,935 to 32, but one stretch with no submission still parked 12,498 stores
in 39.5 MB - under 64 MiB, so nothing forced a sync. A fixed ceiling
(VkBufferManager::kWireDeferredCountCeiling = 1024, not a knob) now rides the same
WaitForSubmitIndex call; MOBILEGL_IPC_WIRE_DEFERRED_MB=0 still turns both triggers off.
MagmaWireReclaimScenario gains the case: 3000 respecify-and-draw rounds of a 256-byte store in one
frame (750 KB, far below the lane's 8 MiB). Red before this commit on inproc and spawn (live peak
3001 vs a bound of 1027), green after (1025, two forced syncs).
Rule J's red-once for M2. The server role's VkBufferManager publishes four PipeStats gauges
(push builds only, so the pull image is untouched): wire buffer records alive, and the run
maxima of live wire VkBuffers and of parked bytes, plus the watermark's forced-sync count -
maxima because the defect lived inside one frame and a swap-time sample reads after the frame
boundary's own sweep. Two cases, each inside a single frame: 1024 undrawn respecifies of one
4 KiB buffer (live peak <= 4 x wire buffers), and 48 respecify-and-draw rounds of a 1 MiB store
under the lane's MOBILEGL_IPC_WIRE_DEFERRED_MB=8 (parked peak <= budget + one store, every strip
its own colour). Red at base (ec46a550's defer path, counted): 1025 vs 4 live, 49,287,168 vs
9,437,184 bytes and 49 vs 11 live, on both arms; green after: 1 live, 9,437,184 bytes / 9 live
with 5 forced syncs. The knob's negative control (=0) reds the second case with the same
49,287,168 bytes. Inproc and spawn only: the tcp server is the lane-wide fixture supervisor,
whose environment and log the entry cannot reach - named in spawn_lane_parity.py as
MAGMA_NOT_ON_TCP (the ID-P7-14 shape), and PipeStatsTest pins the four short names.
The defer-path sweep frees an orphan once its last submission has retired, but inside one long
frame a store that was drawn from is named by recorded, unsubmitted work, and nothing submits it
until the frame ends. The knob is read once through MG_Config::Ipc like its siblings; above it
the park calls WaitForSubmitIndex(GetSyncPointSubmitIndex(), flush) - the call
WaitForWireBufferHostAccess makes - and sweeps again, which empties the list because no entry can
be tagged past the sync point just waited out. A sync point that cannot complete dies through
the existing Fatal{ResourceUnavailable, "buffer-write-sync"} site, now one helper shared with the
host write (fatal_census: 79 abort sites, unchanged). 0 is the negative control.
RespecifyWireBuffer (every wire glBufferData) and DestroyWireBuffer parked the old VkBuffer in
the per-frame-slot buckets, which only BeginFrame and the idle drain empty. A snapshot-exiting
pbuffer replay delivers one present per run (ID-P7-32), so on bsl-esc-menu-854 the split server
held 25,923 dead stores. The wire arm now keeps its own flat list: a store whose last-use serial
is 0 (never named by a GPU command) is destroyed at the park; any other is tagged with the
renderer's sync-point submit index and destroyed by the sweep every park runs, once that index
has completed or the renderer is idle. The frame-serial floor is deliberately not a proof here
(it cannot move mid-frame, and on this base it is the floor B3 found unsound). The transient
arena's releases and the monolith OnRespecify path are unchanged.
Review ID-P7-34, item 3. 02bb5789 presented the submission term as the fix
and still carried the every-8th-drain mechanism that magma-b3.md §2.4
refutes (the probe saw one successful drain and zero boundary works across
the OpenRA replay). The term is not sound alone: lastUseSubmitIndex is
stamped at AcquireWireSlice, but a mid-draw flush (BindProgramUniformBuffers
-> SyncWireTextureShape's preserve path -> FlushWirePendingCommandsFor-
TextureUpdate) can submit the stamped index without the draw, so the stamp
names S1 while the draw rides S2. The guarantee is the floor in
OnSubmitsCompletedUpTo; the term is a defence and a device probe.
- WriteWireBuffer's block comment, the WireBufferResource field comment,
the AcquireWireSlice stamp comment, the StreamedBufferSubData CMake
comment and the F1WireScenario case comment are rewritten on that
footing; the drain story is gone. The drain probe's comment now says it
refuted the drain as the mechanism.
- WaitForWireBufferHostAccess's early return also requires the submission
term complete, so a caller with the serial term idle cannot skip a wait
the term demands.
- ReadWireBuffer re-stamps lastUseSubmitIndex beside lastUseSerial after
its wait, preserving the pending draw's reservation in both halves.
All disaggregated-only code. OpenRA DirectVulkan SPLIT / SPAWN retraces
ssim 1.000000, 0 px, zero MGWIRE-FLOOR lines; integration-magma-split 90/90,
-spawn 69/69, -tcp 70/70.
WriteWireBuffer answers a resource_subdata either with an ordered vkCmdCopyBuffer or with an
immediate host memcpy, and it chose on `lastUseSerial > GetCompletedSerial()` alone.
GetCompletedSerial is max(m_frameSerial - frameCount, m_completedSerialFloor) and its first term
is PURE COUNTING - it asserts that work from frameCount serials ago has finished with no fence
behind it. On the wire arm m_frameSerial is advanced not only by Present but by
TryDrainFrameTransients -> VkBufferManager::BeginFrame on every 8th drain, which happens
MID-FRAME, and package B established that mid-frame pooled-fence submissions are not covered by
frame.imageInFlightFence. So the predicate could answer "idle" about a buffer that a
recorded-or-in-flight draw still reads; the write was then applied unsynchronised and the draw
executed reading somebody else's vertices - drawing another batch's geometry, or nothing at all,
with no log line and no GL error.
That is OpenRA's frame: ~24 glBufferSubData -> glDrawArrays pairs over the SAME range of one
GL_DYNAMIC_DRAW buffer that is never orphaned, and a picture missing exactly one batch's quads.
The predicate now has a second, fence-backed term: WireBufferResource carries lastUseSubmitIndex,
set at every draw bind and staged copy to GetWireNextSubmitIndex() (m_submitCounter + 1 - what
RetireWireObjects already tags future objects with, and deliberately NOT GetSyncPointSubmitIndex,
which answers m_submitCounter when nothing is recorded yet, exactly the hole a draw acquiring its
vertex slices before BeginCommandRecording falls into), and IsSubmitIndexComplete reports an
unsubmitted index as incomplete. The counting term is kept because it only ever over-reports
busy; the two are OR'd, so the answer is monotonically safer. GetCompletedSerial itself is
untouched - it also serves the monolith VkBufferResource path and changing it would move the pull
build's .text.
MGITEST_MAGMA_FORCE_STALE_BUFFER_SERIAL forces the counting term to answer "idle" (all writes, or
the Nth one alone), which is the state a mid-frame idle drain puts the device into and which no
host lane reaches by itself. It forces only the lie, not a bypass of the ordered copy, so it stays
a valid red-once across this fix.
notes/p7/magma-a.md: the per-site table (retired / declined / kept-fatal with the
reason), every red-once with its captured output, the gate readings, and what is
handed on.
The three places package A's own prose still named `uniform-buffer-byte-tail@P7`
lose the marker. They are comments, not refusal sites, and leaving the marker in
would have made a tree-wide `grep @P7` count them against packages B and C. The
census now reads 15 -> 7, and all seven are real: WireFramebuffer.inc x4 and
VulkanRenderer.cpp x1 (package B), WireDraw.inc x2 (package C).
Three findings worth the integrator's attention, all in the note:
1. Most of this cluster was unreachable, and that is provable rather than
assumed. The frontend validates buffer-range offsets against the very caps the
backend then re-checks - GL_{UNIFORM,SHADER_STORAGE,TEXTURE}_BUFFER_OFFSET_ALIGNMENT
and GL_MAX_UNIFORM_BLOCK_SIZE are all published from the same Vulkan limits -
so five of the eight sites could only ever have fired on a malformed peer. The
two that CAN fire are `uniform-buffer-byte-tail` (GL constrains a range's
offset but not its size) and `unaligned-atomic-buffer-range` (GL gives
GL_ATOMIC_COUNTER_BUFFER no queryable alignment, so the frontend only enforces
offset % 4 while the backend needs minStorageBufferOffsetAlignment).
2. fatal_census does not move for any of this, and no re-baseline is due. Wave 0
already routed these funnels through MGPipeSessionFailHook, so the abort lives
in FatalFunnel.cpp and retiring a WireDescriptorFatal call site changes no
census number: 79 abort sites / 43 family words, unchanged, RC=0. The brief
expected the count to fall because the sites were bare aborts before wave 0.
3. DirectVulkan.Split.Full.IterationRPProgram203Scenario.FixedCompleteInputProducesFixedCompleteGoldenOutput
is red on this worktree's build-split and is NOT ours: it reproduces 3/3 with
package A's backend diff fully reverted, and the mismatch bits are identical
to the S1 exposure texel in magma-two-process-first-run.md. The contract books
that one against the tcp arm, and ~/w7/pipe's own build-split passes it, so
whether it is a split-arm regression between 460a3a72 and 610fa281 or a
configure-dependent S1 needs the integrator.
Gates on the finished branch: integration-magma-split 75/75, -spawn 54/54, -tcp
56/56, -buffers 21/21, full-split 512 entries 454 PASS / 57 SKIP / 1 red (the
pre-existing one above; base was 453/57), ctest -L unit 2408/2408,
spawn_lane_parity RC=0, link_ratchet unchanged at 186, fatal_census RC=0, and G1
pull .text = a52203 with nm --defined-only identical to p7-before/pull-syms.txt
(added=0 removed=0).
`uniform-buffer-byte-tail@P7` is retired. It was the live boundary the Redmi
Minecraft run reported (notes/p5f/magma-inproc-fix.md #5) and, since wave 0, a
named Fatal that took the session with it.
Only the SIZE of a uniform range can land off a word. glBindBufferRange already
forces the offset onto GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT and records
GL_INVALID_VALUE otherwise (GL_Buffer.cpp ValidateBufferRangeOffsetAndSize), and
every device reports that as a multiple of four - but GL 4.6 core 6.1.1 puts no
such rule on size, so a 19-byte bind is legal and 19 % 4 is not 0. The sub-word
start is only reachable protocol-side; the new arm handles it anyway rather than
leaving a second shape to trip over later.
CopyWireBufferSubWordRangeToSlice rounds the read of the APPLICATION's store OUT
to whole words, clamped to the store's end so the widened window can never touch
a byte the application does not own, and lands it in a transient staging slice.
Only the staging -> padded-block shift is sub-word, and that happens inside our
own arena; vkCmdCopyBuffer places no alignment rule on a region's offsets or size
(unlike vkCmdUpdateBuffer / vkCmdFillBuffer), so the shift is a plain legal copy.
The block still starts at the padded slice's own offset, which is what lets the
descriptor keep a minUniformBufferOffsetAlignment-aligned dynamic offset.
red-once (R-16), all three Magma two-process arms, before the backend half:
DirectVulkan.Split.Buffers.F1WireScenario.SubWordUniformBufferRangePadsMissingBytes
DirectVulkan.Spawn.Buffers.F1WireScenario.SubWordUniformBufferRangePadsMissingBytes
DirectVulkan.Tcp.Buffers.F1WireScenario.SubWordUniformBufferRangePadsMissingBytes
Subprocess aborted - MGPipe: Fatal{UnmigratedVerb, "Magma:uniform-buffer-byte-tail@P7"}
and green after it. The case is the sibling of
ShortUniformBufferRangePadsMissingBytes with a 19-byte range; the byte just past
the range is poisoned, so a rounding that leaks it reads tail.x as 0.5f and the
red channel comes back 127 instead of 0.
Lanes: integration-magma-split 74 (was 73), -spawn 53 (52), -tcp 55 (54),
integration-magma-buffers 20 (19), all green. fatal_census unchanged at 79 abort
sites / 43 family words - these refusals stopped being bare aborts in wave 0, so
a retirement moves no census number and needs no re-baseline. link_ratchet
unchanged at 186.
Three follow-ups the first pass needed:
DirectGLES glGetTexImage has to attach a STENCIL_INDEX texture at
GL_STENCIL_ATTACHMENT. It went through the colour branch, which left the scratch
READ framebuffer incomplete for a stencil-only texture, so the readback bailed out
before the new stencil branch could run. ScratchFBOImpl::EnsureDepthAttachment2D
gains the stencil-only point; the shadow remembers which of the three points it
holds so a later depth attach is not deduped against it.
DirectVulkan's aspect staging buffer is recorded into the current frame's command
buffer and outlives the glBlitFramebuffer call that records it, so it retires with
the frame. VkBufferManager::DeferRelease is the manager's own deferred-release
entry point and moves to its public section for that.
Stage updates to adopted GPU-resident buffers on the command timeline instead of overwriting memory that earlier draws still read. Wait for submitted copies as well as pending commands before CPU readback.
Commit coherent mapped writes before Android trace buffer copies, and retire buffer shadows on deletion so copies see current vertices and name reuse cannot leave dangling dirty-shadow entries.
Validated draw ordering, submitted-copy readback, traced buffer copies, and mapped-buffer deletion/name reuse on Redmi Adreno 830. Minecraft 26.3-rc-3 Magma passed movement, HUD, and menu checks and exited normally; the new trace parsed all 64,410 frames without warnings.
Every draw re-resolved its whole vertex binding array: for each enabled binding,
look up the buffer, acquire a slice from the buffer manager, apply the binding's
base offset, fill the VkBuffer and offset arrays, bind. In the Minecraft-shaped
benchmark the same few hundred vertex array objects cycle for the whole run and
each one's answer is stable, so UploadAndBindVertexBuffers was the single largest
cost in the backend at 7.9% of the render thread, with AcquireResidentSlice
another 3.8% underneath it.
The resolved array is now kept per vertex array object and revalidated instead of
rebuilt. Validation is two-tier. The vertex array's own configuration version
already invalidates its backend vertex-input state, so a changed attribute,
format, buffer or base offset yields a different state object - the memo compares
both that object's address and its hash, which mixes the bound buffers and the
whole layout. What that does not cover is the slice moving underneath an
unchanged configuration, so the buffer manager now carries a monotonic epoch that
every writer of slice-deciding state bumps: resident storage creation, respecify,
sub-data, flush of a mapped range, the promotion and demotion between streamed
and resident storage, each fresh arena allocation, and bulk release. The counter
is manager-wide and never reset, so a resource created at a recycled address
cannot reproduce a value some memo still holds.
The miss path was the thing to get right, because the previous attempt in this
area regressed the texture-upload and sampler-churn cases by 60-85%: it added a
verification pass that re-ran the resolution work it was trying to skip, so every
miss paid for it twice. Here a miss is one pointer-keyed lookup and a few stores,
and nothing else runs that the full path would not have run anyway.
ns per draw, DriverBench on a GTX 1660 SUPER: mc_ubo_range 924 -> 767,
mc_vanilla_draw 1346 -> 1227, mc_sampler_churn 1397 -> 1279,
mc_sodium_multidraw 3365 -> 3266. Magma is now 4.1x the native driver on the
per-draw uniform-range case, from 5.4x when this round started. No case
regressed on either backend.
Unit tests 421/421.
A per-draw CPU profile of a real Minecraft frame (perf on the render thread,
which sits at 100% of one core on both backends) said the deficit is translation
overhead, not the GPU, and named where it goes. This removes the largest items
it found, on both backends and in the shared frontend they both feed.
The single biggest one was not translation at all: IsBackendContextCurrentOnThisThread
called eglGetCurrentContext on every invocation, and glvnd answers that with a
getpid() fork check - a real syscall. The predicate sits two and three deep in
every draw (the deferred-release drain, the global-UBO ring availability check,
and the ring allocation), so it accounted for 16.3% of the render thread. EGL is
still the ground truth, but re-verifying it once per thread per frame catches an
external migration at the next frame boundary rather than the next call, which
recovers the same bookkeeping.
Texture uploads now carry a dirty region instead of a per-level flag. Minecraft
animates atlas sprites with 16x16 glTexSubImage2D calls into a 1024x512 atlas
and respecifies the lightmap every frame; a per-level flag turned each of those
into a full-level re-upload - about 3.6 MB a frame of texels nobody changed.
MipmapStorage accumulates the written box, Espryt uploads it with
UNPACK_ROW_LENGTH striding into the level shadow, and Magma stages just that box.
The box is a union, not a range list: repeated writes to one level widen it and
it degrades to exactly the old whole-level upload, which is the honest worst case.
glBufferData(NULL) is the orphaning idiom, and the backend was answering it by
uploading the stale CPU shadow - turning a rename the driver does for free into
a full synchronized upload. BufferObject now records that a NULL respecify leaves
the store undefined, and the upload is skipped until content is actually written.
The rest are smaller and of a kind: the deferred-release queue is probed without
taking its mutex, the UBO ring waits on the frame fence that frees the space it
needs instead of draining the whole pipeline with glFinish at the size cap, VAO
binds go through a shadow so a draw's second bind of the same object does not
reach the driver, the per-draw clean-texture probe short-circuits on the content
version before rebuilding shape info, glUniform drops byte-identical writes
(which otherwise dirty the whole UBO for the next draw), re-binding the texture
or VAO a slot already holds no longer bumps the generation counters a backend
fast path is keyed on, and the texture validators stopped taking shared_ptr by
value.
On Magma: descriptor-set reuse keeps four entries instead of one, because draws
alternating between two programs - the chunk/entity ping-pong - thrashed a single
slot into a full re-allocate and re-write every draw; a DynamicDraw buffer whose
contents survive two frame boundaries is promoted to resident storage instead of
being re-copied into the per-frame arena forever; and sampled-read barriers name
only the shader stages whose device feature is enabled, which also removes a
latent VUID violation (ALL_GRAPHICS names geometry and tessellation stages a
device need not have).
Measured with the Minecraft rig (render distance 32, p50 fps, same machine,
single sample each): vanilla 1.21.1 Espryt 10.8 -> 36.3 and Magma 31.3 -> 44.6;
26.2 snapshot Magma 114.5 -> 210.5. Fabric+Sodium moved inside noise on Magma
(854 -> 766) with the native baseline itself moving 838 -> 1031 between the two
sessions, so treat that cell as unresolved rather than a regression measured.
Unit tests 421/421. The CTS A/B was not run: these numbers and the test suite are
the whole of the evidence, and a conformance regression would not have been
caught here.
Five independent bits of per-draw and per-operation waste in the DirectVulkan
backend, all removing work whose answer was already known.
The per-draw descriptor walk iterated all 256 slots of bindingKinds to find the
one to eight bindings a real GL program declares, because that vector is sized to
the binding cap rather than to the program. Reflection now records the bindings it
actually assigned, and the draw path iterates that. It is built at the end of
ReflectLayout, not where bindingKinds is sized - at that point the vector is only
zero-initialised and the kinds are assigned further down, so a list built there
would be empty. It has to stay ascending: Vulkan consumes pDynamicOffsets in
binding order and the writer pushes them in iteration order, so an unordered list
would silently mis-pair dynamic offsets with their uniform blocks.
Descriptor pools were sized maxSets * the 256-binding cap, declaring 81,920
descriptors per pool and 245,760 across the frames in flight, for sets that hold
what shader reflection found. Sized from eight now; an outlier program is absorbed
by the VK_ERROR_OUT_OF_POOL_MEMORY path that already exists, which works because
pool sizes are aggregate budgets rather than per-set limits.
TrackLiveResource swept the whole live-buffer vector on every insert once it
passed 256 entries, and when the buffers are all live the sweep removes nothing
and the vector grows by one - so creating N live buffers cost about N^2/2
expired() checks. It sweeps on a doubling watermark now, with the same
reclamation semantics.
GenerateMipmap transitioned each destination level individually inside its loop,
but every generated level starts in the same layout and the loop only moves a
level out of TRANSFER_DST after writing it, so the whole range can be prepared in
one barrier - 3(N-1)+1 barrier commands become 2(N-1)+2. Each level is still
transitioned to TRANSFER_SRC before it is read, so the dependency between
consecutive levels is unchanged.
WaitForFrameSerial drained the entire graphics queue, as its own comment admitted.
Every submission records the frame serial it was made under, so it now waits on
the first fence at or past the requested serial. The narrow path deliberately does
not call NotifyDeviceIdle(): that claims every submission has retired, which is
only true after a real drain, so it stays on the fallback.
Verified with an 8213-case A/B (textures, buffers, queries, mipmaps, uniforms and
the whole direct_state_access suite): the Espryt failure list is identical, the
Magma failure list differs by one case, and both crash sets are unchanged on
Magma. That one case, buffer_storage.map_persistent_draw, does not reproduce in
isolation - running the buffer_storage group alone gives byte-identical results on
both builds (the same three failures, not including it), and it reports
NotSupported when run on its own. It is the same ordering-dependent behaviour this
suite shows elsewhere, and the three Espryt crash-set differences are the known
copy_image cluster moving chunk position. Flagging rather than hiding it.
direct_state_access stays at Espryt 370/371 and Magma 371/371; unit tests 421/421.
Second stage of GL 3.0 transform feedback: captured draws now write real
data.
- Device setup enables the VK_EXT_transform_feedback feature when present
and loads the bind/begin/end entry points.
- Captured draws compile an XfbCapture program variant whose last
vertex-processing stage gets XfbBuffer/XfbStride/Offset decorations from
the program's resolved varyings (a new spirv-opt pass). A captured
gl_Position is mirrored into a dedicated output written before every
OpReturn - or before every OpEmitVertex in a geometry stage - ahead of
the position fixup, so the captured value is the shader's own pre-remap
position.
- DrawArrays/DrawElements wrap the draw in Begin/EndTransformFeedbackEXT;
a small counter buffer resumes the append position across draws within
one glBeginTransformFeedback (fresh Begin starts at the bound offsets).
- Capture targets are promoted to persistently-mapped host-coherent GPU
storage (persistent-map storage now also carries the transform feedback
usage), so MapBuffer/GetBufferSubData read the captured bytes after the
fence wait glEndTransformFeedback now performs.
- Draw-mode/feedback-mode validation defers to the geometry shader's
output primitive when one is present, and glGetBooleanv reports
GL_TRANSFORM_FEEDBACK_ACTIVE/PAUSED so dEQP's per-case state reset can
unwind an active capture.
KHR-GL33: transform_feedback capture_vertex_*/capture_geometry_*/
discard_*/draw_xfb and clip_distance.coverage now pass; queries
(PRIMITIVES_WRITTEN) and gl_ClipDistance capture remain.
Introduce a Mesa pipe_resource-style PipeResource that owns a GL buffer's bytes
and its backend GPU resource, abstracting WHERE the authoritative bytes live:
- Shadow mode (non-persistent buffers): a CPU Vector; the backend keeps its own
GPU copy in sync via BufferBackendOps, exactly as before.
- Persistent mode (coherent GL_MAP_PERSISTENT maps): the backend's host-visible,
COHERENT, persistently-mapped GPU memory is the single source of truth. The app
writes into it directly, every reader resolves against it, and NO per-write
backend transfer happens. The CPU shadow is released.
BufferObject no longer owns a raw shadow Vector; it holds a PipeResource and
exposes one accessor, MappedData(), that all readers go through. Every buffer-data
consumer (UBO payload, PBO texture upload, indirect draws, resident/streamed
uploads, both backends) was migrated from GetDataReadOnly()->data() to
MappedData(), so a persistent buffer's readers see GPU memory - not a stale
shadow. That stale-shadow inconsistency is what corrupted rendering (wrong UBOs ->
misplaced/"lost" vertices) in the first zero-copy attempt (625c8a6, reverted in
896cafc); routing every consumer through one accessor makes it structurally
impossible.
Backends provide the map via BufferBackendOps::AcquirePersistentMap:
- DirectVulkan: a HOST_VISIBLE|HOST_COHERENT (required, not just requested),
persistently mapped resident VkBuffer carrying every usage, seeded from the
shadow, never recreated; AcquireResidentSlice binds it directly.
- DirectGLES: EXT_buffer_storage immutable persistent+coherent glMapBufferRange,
falling back to the shadow when the extension is absent.
Fixes the ~7GB GpuMemory OOM + 100%-CPU/ANR running modern Blaze3D Minecraft on
both Magma and Espryt (per-draw whole-buffer re-upload of the coherent persistent
ring buffer), without the coherency/stale-read hazards of the reverted attempt.
BufferTest: zero-copy stress guard (15,360 draws -> 0 per-draw transfers, and every
reader resolves to GPU memory) + a shadow-fallback test. Host suite: 203/203 pass.
Device verification pending.
- Track every graphics-queue submission with a real fence: pooled fences
for mid-frame flushes, the frame slot's fence for Present and readback.
Completion advances a submit counter via vkGetFenceStatus polls,
slot-fence waits, and device-idle points, and raises the buffer-manager
serial floor from the frame serial each submission carried.
- GL sync objects now capture the submission index that will carry the
commands recorded so far; ClientWaitSync honors
GL_SYNC_FLUSH_COMMANDS_BIT with a mid-frame submit (gated on the index
still being unsubmitted so poll loops cannot split the render pass), and
blocking waits flush then vkWaitForFences with the caller timeout.
- FlushPendingCommands retires the submitted command buffer and restarts
recording on a fresh one; retired buffers are freed once the slot fence
is next waited, so an executing buffer is never reset.
- Rewind descriptor-set cursors exactly once per frame in Present (after
the slot-fence wait), plus after the synchronous readback drain,
replacing the ten lazy per-draw-path rewinds.
Verified: host tests 168/168, trace-replay 70/70.
Create 6 / Flywheel 1.0.6 now renders correctly with both flywheel:instancing
and flywheel:indirect on DirectGLES and DirectVulkan (verified in-game on
Adreno 830: waterwheels and cogwheels solid, animated, correct pairing, no
crashes across all four combinations).
- MG_State/MG_Impl: sync explicitly-ranged SSBO bindings of FLUSH_EXPLICIT
persistent maps to the backend before compute dispatches. Flywheel writes
its scatter-copy descriptors into the staging ring's persistent map and
never flushes that span (UB per spec, works on drivers whose maps alias
GPU-visible memory); our maps alias the CPU shadow, so the descriptors
never reached the GPU: the scatter compute copied nothing (GLES: empty
draw commands) or stale garbage (Vulkan: wild indirect commands ending in
VK_ERROR_DEVICE_LOST).
- MG_Impl/MG_Backend: real glFenceSync objects backed by backend fences
(GLES: native ES syncs guarded by context generation and owner thread;
Vulkan: buffer-manager frame serials), replacing always-signaled stubs
that let Flywheel reclaim staging memory the GPU still reads.
- MG_Backend/DirectGLES: compute dispatches now run the same per-program
resource sync as draws (uniform-block bindings and sampler units must be
re-established through the API because layout(binding) is stripped from
transpiled ESSL) and rebind texture units afterwards; the cull shader
used to read a stale _FlwFrameUniforms binding and the depth-pyramid
downsample sampled a stale unit-0 texture, zeroing the Hi-Z pyramid and
occlusion-culling all Flywheel geometry. Image uniforms are excluded from
glUniform1i (ES bakes their unit via layout(binding)); image-unit sync is
clamped to the device limit; eliminated/SSBO-classified uniform blocks
are skipped.
- MG_Backend/DirectGLES: gl_BaseInstance in native indirect draws reads the
GPU-written command buffer through an injected mg_IndirectParams SSBO
view addressed per draw instead of the zero CPU shadow; layout(binding)
is preserved for SSBO/image declarations (ES has no API rebinding for
them); the ES context ownership claim moved to a global atomic owner
thread with an EGL ground-truth check, and deferred buffer op state is
mutex-guarded, so ops cannot silently no-op after context migration.
- MG_Backend/DirectVulkan: new RebaseInstanceIndexPass rewrites vertex
InstanceIndex loads to (InstanceIndex - BaseInstance). glslang's relaxed
Vulkan mode aliases gl_InstanceID to InstanceIndex, which includes
firstInstance, but GL's gl_InstanceID is zero-based - draws with nonzero
baseInstance paired meshes with wrong instance data (cogwheel drawn as a
waterwheel, another wheel collapsed invisible). Gated on the
shaderDrawParameters device feature. Sampled-read barriers additionally
cover the compute stage (the Hi-Z downsample samples the depth
attachment from compute), and short uniform-buffer ranges keep the
existing zero-padding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Support cube map face uploads with cube-compatible images and per-face array layers
- Add uniform texel buffer descriptor support for samplerBuffer bindings
- Cache transient vertex/index buffer uploads per frame to avoid VMA allocation failures