Six more verified defects from the residual memo/cache mechanisms:
- Program resource cache (DirectVulkan reflection): glShaderStorageBlockBinding
deliberately does not bump the backend state version, and the SSO pipeline
composite is unnamed so the by-name in-place patch can never reach its slot -
the composite kept serving pre-rebind SSBO bindings. The cache now keys on the
program's block-binding version; a binding-only change re-applies the overrides
by name instead of re-running spirv-reflect. SetShaderStorageBlockBinding also
gains the equality bail-out its uniform-block sibling has, so the composite
mirror's replay stops churning the version every draw.
- LinkProgram's allowVSOnlyPrograms function-static latch never set its own
initialized flag (dead memo, re-read every call) - and completing it would have
frozen a per-backend capability across re-initialization. Replaced with a fresh
per-link read from the null-checked active backend.
- Query object registry: drained at full library teardown (DestroyAllQueryObjects,
mirroring DestroyAllSyncObjects) - undeleted queries and their backend wrappers
leaked across Destroy/Initialize cycles, stale ids stayed IsQuery == GL_TRUE in
the re-initialized library, and a later delete could hand the old backend's
wrapper to a different backend's DeleteBackendQuery.
- Converted vertex streams and the host-side EBO max-index scan now SyncGpuWrites
before reading the coherent mapping: XFB/SSBO/image writes are merely recorded
at that point, so the conversion read pre-write bytes (the restart-index
rewrite already synced; these two host reads did not).
- Zero-stride converted bindings: both converters rejected stride 0, making the
factory's documented single-element conversion unreachable and silently
dropping every draw using such a binding; the stride is substituted with the
element size for the one-element case.
- DemoteFloat64Pass block relayout: measurement queued into the module eagerly,
so a mid-struct failure left a half-relaid-out block (compacted offsets before
the failing member, 64-bit offsets after) while claiming the block was left
alone. Decoration writes are now collected and committed only when the whole
block measures successfully.
Wave 2 of the advertised-extension conformance campaign.
PROGRAM INTERFACE QUERIES (the load-bearing piece). glGetProgramInterfaceiv
and the five glGetProgramResource* entry points were answered by the
BACKENDS - Espryt asked the real driver about SPIRV-Cross-generated ESSL
whose namespace is not the GL one (default-block uniforms live in
MGL_GLOBAL_UBO there), and Magma kept a second, partial reflection that
hardcoded types and diverged from the frontend. Both are now deleted; a new
frontend resource-model layer (ProgramInterface.{h,cpp}) answers every
interface - uniforms, uniform blocks, atomic-counter buffers (recovered
from glslang's synthesized gl_AtomicCounterBlock_<binding> lowering),
buffer variables, shader-storage blocks (classified by TType storage
qualifier since glslang reflects them as uniform blocks), program inputs/
outputs (built-ins' layoutLocationEnd sentinel mapped to -1), and
transform-feedback varyings including the gl_NextBuffer/gl_SkipComponentsN
pseudo-varyings - from the glslang reflection the frontend already trusts
for glGetActiveUniform. Name/index round-tripping, the "[0]" array
spelling, and the GL 4.6 table 7.2 prop/error matrix live in the new layer
only; GetActiveUniform*/GetActiveAttrib* are untouched.
glShaderStorageBlockBinding now takes the interface-layer index (the one
GetProgramResourceIndex returns, with a range check it never had), records
the binding on the program keyed by block NAME - the one coordinate all
three index spaces agree on - and delegates by name across the backend
boundary. Both backends reseed the recorded bindings on their own program
rebuilds, so an unrelated resync can no longer silently revert a rebound
block, and GL_BUFFER_BINDING reports the live binding, not the declared
one. The Espryt delegate applies only to an already-synced twin and can no
longer trigger SyncToBackend from a getter; the sync path's GL query
out-params are initialized and clamped (a load-dependent stack-garbage
Vector size crash caught by the gate, reproduced 3/50 pre-fix, 100/100
post-fix under saturating load).
ESPRYT RENDER-STATE SHADOW RESET (rides along because it shares
DirectGLES.cpp): the render-state shadow is file-static and survives
MobileGL context switches, so GL_FRAMEBUFFER_SRGB (and the whole synced-cap
class) leaked between contexts - the cross-test leakage class the CTS maps
have carried for a week. MakeCurrent now invalidates the shadow like it
already invalidates the program/FBO/buffer caches, and the resync resolves
a never-set scissor box to the current surface instead of pushing the
(0,0,0,0) sentinel verbatim (which scissored everything away - caught by
the retrace gate, bisected to the exact field via a bitmask probe, and
fixed by resolving like the viewport path rather than reverting).
Frontend riders exposed by the layer: glGetUniformLocation resolves
arrays-of-arrays element addressing ("a[2][1]"); transform-feedback capture
accepts element-addressed varying names ("b[1]") and snapshots the request
verbatim for the interface (Magma's decorate pass logs loudly that element
capture is unimplemented there - follow-up).
KNOWN GAPS, documented in code and tests: the 6 subroutines-* cases
(glslang refuses subroutine for SPIR-V; wave 3), the 5 separate-programs-*
cases (glslang's pipe-I/O reflection cannot see a separable non-vertex
stage's own inputs; needs stage-aware output validation first), and
uniform-block-types' per-instance stage masks (not derivable from the
reflection).
Gate: 593/593 unit at default and kill-switch, x10 each, plus the SSB race
case 100/100 under 20-way CPU load; ext caselist Espryt 77.87% -> 80.42%,
Magma 77.58% -> 79.39% (+212 fixed, 0 newly broken); program_interface_query
2/43 -> 31/43 unique on Espryt, 9/43 -> 31/43 on Magma, backends now
byte-identical; KHR-GL45.direct_state_access 370/371 + 371/371 with the 4
sRGB leak victims recovered in cross-test ordering; KHR-GL33 held at
9884/9886; full 39x2 CI retrace with zero wave-attributable failures (the
3 failing newly-added fixtures are bit-identical on the pristine baseline).
The bug: DirectVulkan.cpp::MultiDrawElements had its entire body
commented out - plain glMultiDrawElements on Magma recorded NOTHING,
no error, no pixels (readback shows the deferred clear never even
materialized). It now shares the tuned base-vertex implementation, and
both plain entries are pixel-proven by a 4-sub-draw harness.
The feature: every CPU-side multi-draw form dispatches through three
tiers after round-9's contiguous-run merge (restructured to merge into
a span BEFORE dispatch, so every tier consumes the shrunken array):
1. VK_EXT_multi_draw: one vkCmdDrawMulti(Indexed)EXT, chunked by
maxMultiDrawCount; per-draw vertexOffset rides in the struct. The
extension is requested only when enumerated and its feature bit
confirmed, entry points via vkGetDeviceProcAddr, demoted if
missing.
2. multiDrawIndirect: the param span uploads DIRECTLY as a transient
INDIRECT-usage buffer - DrawIndexedCmdParam is layout-identical
to VkDrawIndexedIndirectCommand and DrawCmdParam's head is a
legal 24-byte-stride VkDrawIndirectCommand, both static_asserted,
so no repacking - then one vkCmdDraw(Indexed)Indirect per
maxDrawIndirectCount chunk. firstInstance!=0 additionally
requires drawIndirectFirstInstance or the batch drops a tier.
3. The byte-identical unroll.
gl_DrawID: tiers 1-2 are spec-correct (0,1,2,3 across a probe's
sub-draws); the unroll tier keeps the pre-existing always-0 contract.
The default tiers strictly improve DrawID correctness.
Adversarially verified: the five real DirectVulkan retrace images are
BIT-IDENTICAL (md5) across auto/ext/indirect/unroll; zero validation
VUIDs on every tier; a simulated no-EXT device resolves to indirect
and renders the same bytes; the known-red create-indirect fixture
crashes at the identical call before and after (not worse, not fixed).
Unit suite 423/423 on the rebased tree, retrace subset 10/10. Bench:
mc_sodium_multidraw's contiguous shape merges 32->1 before dispatch,
so no bench delta - the tiers' beneficiaries are non-contiguous real
streams (the sodium RETRACE pushes ~58-sub-draw batches, in=out
243101 with zero merges) and mobile drivers. A reproducible +3-4%
code-layout drift on mc_use_program (zero shared code, I-cache
displacement from +400 lines) stays under the action gate and is
booked here rather than hidden.
The per-VAO resolved-bindings map probe was ~45% of
UploadAndBindVertexBuffers' self time, and the aux-memo pointer chase
was the single hottest instruction left in TrySetupDrawFastPath. Both
die together: a fixed 2048-slot two-probe 64B-aligned VaoDrawMemo table
embeds the VAO key, content-hash-validated layout facts and the
bindings payload reordered hot-to-cold. Layout facts hold exactly while
the slot's content hash equals the live VAO's own config-guarded hash;
a recycled VAO address either misses or reproduces a byte-identical
config, for which the facts are correct by construction. Bindings keep
their full per-draw revalidation; recycled slots zero their frame
serials so half-filled entries can never match.
ComputePipelineStateHash, the depth/stencil probe and the
primitive-restart probe now take one bulk GetRenderStateParameters()
fetch instead of ~17 cross-TU accessor calls (verified pure field
reads, identical bit packing). The EBO slice memo gained the same
manager-wide epoch one-compare rescue the vertex half uses.
GetShaderTransformFlags is memoized on pre-transform. Sodium's
MultiDrawElementsBaseVertex hoists GetGLTypeSize out of the
per-sub-draw loop, replaces the division with a shift, and skips
unsupported index types loudly instead of dividing by zero.
Also verified: a GL_BLEND toggle recompiles nothing in steady state -
the glslang frames in earlier state_toggle profiles were startup
contamination.
Quiet-box load-gated 6-round A/B: sodium_multidraw -8.0%, tex_param
-4.1%, use_program -3.3%; steady-state vanilla_draw CPU -20% ns/op at
4096 frames (the 80-frame matrix compresses CPU wins under GPU boost
clocks; profiles confirm UploadAndBindVertexBuffers 6.3% -> 4.4%
including the table probe, and the aux cold-line load gone). The one
matrix flag (pass_switch +7.5%) reversed to -3.2% in 10-pair isolated
re-runs. Unit tests 421/421.
glClearNamedFramebufferiv and glClearNamedFramebufferuiv were stubs, so a clear through
them was silently dropped and the attachment kept whatever it held. Their float siblings
were already implemented, which is what made the gap look like a rendering bug rather than
a missing entry point.
Which buffers they accept is narrower than glClearNamedFramebufferfv and differs between
the two: signed values clear COLOR or STENCIL, unsigned only COLOR (GL 4.6 core 17.4.3.1).
Only the colour buffer is indexed, so a stencil clear naming any drawbuffer other than 0 is
INVALID_VALUE rather than merely ignored, and anything else is INVALID_ENUM. Resolving the
framebuffer by name goes through the same helper the float forms use, which is what reports
INVALID_OPERATION for a name that is neither zero nor an existing framebuffer.
Both backends express them the way they already express the float forms: DirectGLES binds
the named framebuffer and forwards to glClearBuffer*, Magma queues the payload against the
named framebuffer rather than the bound one.
direct_state_access.framebuffers_clear_errors passes on both backends, and
framebuffers_clear passes on Espryt. Magma still fails that one, for a separate reason on
the materialization side rather than in these entry points.
The frontend half of ARB_transform_feedback2 landed for both backends, but Magma's
capture was still written for the one implicit span GL 3.3 has:
- A paused span kept capturing. VK_EXT_transform_feedback's counter buffers already
make consecutive draws append, so pausing is simply "do not wrap this draw" - the
counters keep their values and the next resumed draw carries on where the last
captured one stopped.
- Those counter buffers were context-wide. Transform feedback objects can each hold an
open, paused span at the same time - KHR-GL40.transform_feedback.draw_xfb_test keeps
three - and they were all appending through one set of four slots. Each object now
gets its own group, handed out on first use; past sixteen objects they share group 0,
which only matters for concurrently-paused spans.
- The generation that identifies a span is what a backend keys its append state on, so
it is now part of the per-object state the frontend saves and restores. Without that,
resuming an object that was paused before another one began looked like a new span
and restarted its counters at zero.
GL_PRIMITIVES_GENERATED needed one more thing. It counts what the last vertex
processing stage emitted whether or not anything is being captured, but
VK_QUERY_TYPE_TRANSFORM_FEEDBACK_STREAM_EXT only counts what the capture saw - so a
draw made while the span was paused is invisible to it. The frontend now tallies those
draws, and the Vulkan query adds the delta at result time. The correction lives in the
backend that needs it: an ES driver's GL_PRIMITIVES_GENERATED counts them by itself, and
adding it there too would double them.
transform_feedback* on Magma: 4 failures -> 3. Espryt stays at 38/38.
The TF primitive queries now ride VK_QUERY_TYPE_TRANSFORM_FEEDBACK_STREAM_EXT
pools when the device reports transformFeedbackQueries: each captured draw
is wrapped in a slot (shared between both GL targets when active
together), and results sum the (written, needed) pairs -
GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN from the first,
GL_PRIMITIVES_GENERATED from the second. This is exact through geometry
shaders, so KHR-GL33.transform_feedback.query_geometry_* pass; the CPU
accounting delta remains the fallback for backends without the feature.
GL_SAMPLES_PASSED / GL_ANY_SAMPLES_PASSED(_CONSERVATIVE) now work: every
app draw between Begin/EndQuery is wrapped in a slot of a host-reset
occlusion query pool (precise counts when occlusionQueryPrecise is
granted), and the result flush ends any active render pass before
submitting, waits, sums the slots and recycles them. ANY_* targets
report the boolean form; GL_QUERY_COUNTER_BITS and GL_CURRENT_QUERY
answer for the occlusion targets, and deleting an active query releases
its slot. Draw-time depth/stencil state also honors attachment absence:
a framebuffer without a depth (stencil) attachment behaves as if that
test always passes, even when a packed depth-stencil image is attached
through only one half (verify_partial_attachments.*).
Vulkan has no LINE_LOOP topology and the frontend used to reject the mode
with GL_INVALID_OPERATION, which is itself non-conformant (several KHR-GL33
transform_feedback tests draw line loops and expect no error). DrawArrays,
DrawElements and DrawElementsBaseVertex now rewrite the draw into an
indexed GL_LINE_STRIP whose synthesized uint32 index list revisits the
first vertex, delivered through the client-memory index path (a new
forceClientMemory flag keeps a bound element-array buffer from hijacking
the synthesized pointer). Entry points without the rewrite degrade to an
open line strip instead of a triangle list.
Drops the if (!pVulkanRenderer) { return; } / !MG_State::pGLContext early-return guards across DirectVulkan.cpp in favor of MOBILEGL_ASSERT, matching the pattern already used by the rest of the backend. Legitimate runtime conditions (index bounds, sync/query handle nullness, renderer-generation mismatch, timer-query support) are kept as real checks; only the null-pointer defenses are converted.
Introduce a Mesa pipe_resource-style PipeResource that owns a GL buffer's bytes
and its backend GPU resource, abstracting WHERE the authoritative bytes live:
- Shadow mode (non-persistent buffers): a CPU Vector; the backend keeps its own
GPU copy in sync via BufferBackendOps, exactly as before.
- Persistent mode (coherent GL_MAP_PERSISTENT maps): the backend's host-visible,
COHERENT, persistently-mapped GPU memory is the single source of truth. The app
writes into it directly, every reader resolves against it, and NO per-write
backend transfer happens. The CPU shadow is released.
BufferObject no longer owns a raw shadow Vector; it holds a PipeResource and
exposes one accessor, MappedData(), that all readers go through. Every buffer-data
consumer (UBO payload, PBO texture upload, indirect draws, resident/streamed
uploads, both backends) was migrated from GetDataReadOnly()->data() to
MappedData(), so a persistent buffer's readers see GPU memory - not a stale
shadow. That stale-shadow inconsistency is what corrupted rendering (wrong UBOs ->
misplaced/"lost" vertices) in the first zero-copy attempt (625c8a6, reverted in
896cafc); routing every consumer through one accessor makes it structurally
impossible.
Backends provide the map via BufferBackendOps::AcquirePersistentMap:
- DirectVulkan: a HOST_VISIBLE|HOST_COHERENT (required, not just requested),
persistently mapped resident VkBuffer carrying every usage, seeded from the
shadow, never recreated; AcquireResidentSlice binds it directly.
- DirectGLES: EXT_buffer_storage immutable persistent+coherent glMapBufferRange,
falling back to the shadow when the extension is absent.
Fixes the ~7GB GpuMemory OOM + 100%-CPU/ANR running modern Blaze3D Minecraft on
both Magma and Espryt (per-draw whole-buffer re-upload of the coherent persistent
ring buffer), without the coherency/stale-read hazards of the reverted attempt.
BufferTest: zero-copy stress guard (15,360 draws -> 0 per-draw transfers, and every
reader resolves to GPU memory) + a shadow-fallback test. Host suite: 203/203 pass.
Device verification pending.
Two previously-stubbed GL 3.3 Core entry points.
glMultiDrawArrays: mirrors the existing glMultiDrawElements(BaseVertex)
architecture end to end -- a new MultiDrawArrays backend function-table
slot dispatched from the frontend after program/primitive-mode validation
(plus a drawcount < 0 -> GL_INVALID_VALUE guard).
- DirectGLES: PrepareForDraw once, then loop native glDrawArrays with the
same per-range client-side array upload the single DrawArrays does.
- DirectVulkan: build a MultiDrawCmd payload and hand it to a new
VulkanRenderer::MultiDrawArrays, which does one SetupDraw over the union
of the sub-draw vertex ranges and then a vkCmdDraw per range (mirrors
VulkanRenderer::MultiDrawElements).
glGetBufferSubData: reads a range of the bound buffer's CPU shadow into
client memory via a new BufferObject::DownloadSubData, with the same
validation shape as BufferSubData (INVALID_VALUE for negative/overflowing
range, INVALID_OPERATION for no bound buffer or a non-persistent mapped
buffer). The shadow reflects CPU writes and backend write-backs but not
arbitrary GPU-side writes, which is documented on the method.
Tests: 2 BufferTest cases for glGetBufferSubData (round-trip read of a
middle range and the whole buffer, plus out-of-range/negative/no-buffer
errors). BufferTest 32/32, SanityTest 30/30, VertexArrayTest 42/42;
library builds clean. (The glMultiDrawArrays draw paths are not
runtime-testable on this host and are compile-verified against the tested
MultiDrawElements pattern.)
- Track every graphics-queue submission with a real fence: pooled fences
for mid-frame flushes, the frame slot's fence for Present and readback.
Completion advances a submit counter via vkGetFenceStatus polls,
slot-fence waits, and device-idle points, and raises the buffer-manager
serial floor from the frame serial each submission carried.
- GL sync objects now capture the submission index that will carry the
commands recorded so far; ClientWaitSync honors
GL_SYNC_FLUSH_COMMANDS_BIT with a mid-frame submit (gated on the index
still being unsubmitted so poll loops cannot split the render pass), and
blocking waits flush then vkWaitForFences with the caller timeout.
- FlushPendingCommands retires the submitted command buffer and restarts
recording on a fresh one; retired buffers are freed once the slot fence
is next waited, so an executing buffer is never reset.
- Rewind descriptor-set cursors exactly once per frame in Present (after
the slot-fence wait), plus after the synchronous readback drain,
replacing the ten lazy per-draw-path rewinds.
Verified: host tests 168/168, trace-replay 70/70.
Implements GL timer queries end to end: a frontend query registry
(modeled on the sync module - mutex-guarded objects wrapping opaque
backend handles behind optional function pointers) serving
glGenQueries/glBeginQuery/glEndQuery(GL_TIME_ELAPSED)/glQueryCounter
(GL_TIMESTAMP)/glGetQueryObject*/glGetQueryiv with GL 3.3 error
semantics and a graceful zero-result fallback when a backend cannot
time.
DirectGLES backs spans with GL_EXT_disjoint_timer_query (context-
generation-stamped handles, bounded result waits). DirectVulkan gets a
VkTimerQueryManager: per-frame-in-flight timestamp query pools reset at
command-buffer begin (outside render passes), records harvested by
frame serial before their pool recycles, elapsed = masked tick delta x
timestampPeriod; handles are stamped with a renderer generation that
also now guards fence syncs across renderer recreation. GL_QUERY_
COUNTER_BITS reports 0 unless the live backend can actually time
(dynamic IsTimerQuerySupported hook), and a failed blocking read keeps
the handle alive so the real value stays reachable once the frame
submits.
GL_ARB_timer_query is advertised only when the device supports timing
and MOBILEGL_DISABLE_TIMERQUERY is unset - LWJGL keys Minecraft's F3
'GPU: x%' line off exactly that extension string; verified on device
(Adreno 830) on both backends.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Create 6 / Flywheel 1.0.6 now renders correctly with both flywheel:instancing
and flywheel:indirect on DirectGLES and DirectVulkan (verified in-game on
Adreno 830: waterwheels and cogwheels solid, animated, correct pairing, no
crashes across all four combinations).
- MG_State/MG_Impl: sync explicitly-ranged SSBO bindings of FLUSH_EXPLICIT
persistent maps to the backend before compute dispatches. Flywheel writes
its scatter-copy descriptors into the staging ring's persistent map and
never flushes that span (UB per spec, works on drivers whose maps alias
GPU-visible memory); our maps alias the CPU shadow, so the descriptors
never reached the GPU: the scatter compute copied nothing (GLES: empty
draw commands) or stale garbage (Vulkan: wild indirect commands ending in
VK_ERROR_DEVICE_LOST).
- MG_Impl/MG_Backend: real glFenceSync objects backed by backend fences
(GLES: native ES syncs guarded by context generation and owner thread;
Vulkan: buffer-manager frame serials), replacing always-signaled stubs
that let Flywheel reclaim staging memory the GPU still reads.
- MG_Backend/DirectGLES: compute dispatches now run the same per-program
resource sync as draws (uniform-block bindings and sampler units must be
re-established through the API because layout(binding) is stripped from
transpiled ESSL) and rebind texture units afterwards; the cull shader
used to read a stale _FlwFrameUniforms binding and the depth-pyramid
downsample sampled a stale unit-0 texture, zeroing the Hi-Z pyramid and
occlusion-culling all Flywheel geometry. Image uniforms are excluded from
glUniform1i (ES bakes their unit via layout(binding)); image-unit sync is
clamped to the device limit; eliminated/SSBO-classified uniform blocks
are skipped.
- MG_Backend/DirectGLES: gl_BaseInstance in native indirect draws reads the
GPU-written command buffer through an injected mg_IndirectParams SSBO
view addressed per draw instead of the zero CPU shadow; layout(binding)
is preserved for SSBO/image declarations (ES has no API rebinding for
them); the ES context ownership claim moved to a global atomic owner
thread with an EGL ground-truth check, and deferred buffer op state is
mutex-guarded, so ops cannot silently no-op after context migration.
- MG_Backend/DirectVulkan: new RebaseInstanceIndexPass rewrites vertex
InstanceIndex loads to (InstanceIndex - BaseInstance). glslang's relaxed
Vulkan mode aliases gl_InstanceID to InstanceIndex, which includes
firstInstance, but GL's gl_InstanceID is zero-based - draws with nonzero
baseInstance paired meshes with wrong instance data (cogwheel drawn as a
waterwheel, another wheel collapsed invisible). Gated on the
shaderDrawParameters device feature. Sampled-read barriers additionally
cover the compute stage (the Hi-Z downsample samples the depth
attachment from compute), and short uniform-buffer ranges keep the
existing zero-padding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Enable multiDrawIndirect and shaderDrawParameters device features when
supported (the latter via VkPhysicalDeviceShaderDrawParametersFeatures on
Vulkan 1.1+), so DrawIndex/BaseInstance SPIR-V builtins are valid and
vkCmdDrawIndexedIndirect(Count) may draw more than one command.
- Plain glMultiDrawElementsIndirect no longer requires a GL_PARAMETER_BUFFER
(it previously drew nothing for the standard Flywheel call); it now issues
a native vkCmdDrawIndexedIndirect, with a per-command loop fallback when
the multiDrawIndirect feature is unavailable.
- glDrawElementsIndirect / glDrawArraysIndirect / glMultiDrawArraysIndirect
read the live GPU buffer via native indirect draws instead of the CPU
shadow (which cannot see compute-written commands); the CPU path remains
only for client-memory commands.
- Advertise the same five extensions as DirectGLES for Flywheel's
capability probe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
swung0x48
Swung0x48