First wave of the advertised-extension CTS campaign (targeted caselist: the
glcts groups of every extension both backends advertise, 4867 cases across the
KHR-GL41..46 namespaces). All frontend, shared by both backends:
- Non-square float matrix uniforms actually upload: glUniformMatrix{2x3,3x2,
2x4,4x2,3x4,4x3}fv and the six glProgramUniformMatrix* twins were
validate-only no-ops; they now write column-at-a-time at the global UBO's
16-byte std140 column stride, honouring transpose. glUniformMatrix2fv had
the sibling bug - mat2 written as 4 contiguous floats put column 1 at byte
8 instead of 16. The readback path only ever un-padded mat3, so
glGetUniformfv is fixed for mat2, mat3x2 (previously mis-gathered) and
every non-square shape, with the bounds check widened to the padded span.
- glBindBufferRange validates offset/size at last: size <= 0, offset < 0,
SSBO and UBO offset alignment, transform-feedback offset AND size
multiples of 4 - all before any state write (a negative offset used to
reach Range1D unchecked). glBindBuffersRange inherits per element, with
the ARB_multi_bind up-front [first, first+count) checks added to the
BindBuffersBase/Range and BindSamplers prologues.
- BufferSubData's second, wrong mapped-overlap test deleted (it rejected
every write at or after a mapped range's start, mapped or not); the state
layer's assert relaxed to the same half-open intersection the frontend
checks. BufferStorage error precedence fixed: no-bound-buffer now beats
bad-size/flags.
- glSamplerParameteri accepts the full GL_NEVER..GL_ALWAYS compare-func
range (NEVER/LESS/EQUAL were rejected by a wrong lower bound).
glBindSampler's unit gate uses GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS instead
of the frontend array capacity, shared with glBindSamplers by construction.
- Getters: GL_MAX_SHADER_STORAGE_BLOCK_SIZE in glGetIntegerv; atomic-counter
buffer limits; all 11 per-unit GL_TEXTURE_BINDING_* plus GL_SAMPLER_BINDING
in glGetIntegeri_v; GL_VERTEX_ATTRIB_BINDING/_RELATIVE_OFFSET across the
vertex-attrib query family; glGetFloati_v/glGetDoublei_v implemented (were
stubs); KHR_debug limits raised to spec floors.
- glCreateShader records INVALID_ENUM for an unknown type (it previously
handed out a usable name with no error at all); glCreateShaderProgramv
validates count up front. glDispatchCompute/Indirect validate work-group
counts, offset alignment and indirect-buffer presence.
- glVertexAttribIFormat & friends take a positive integer-type whitelist -
GL_FLOAT/GL_HALF_FLOAT/GL_DOUBLE/GL_FIXED no longer slip through as
integer attributes.
Gate (headless Mesa, default config = async on): 570/570 unit at default and
with the kill switch; ext caselist Espryt 76.29% -> 77.87% (+81 fixed, 6
crashes -> 0, the whole list now runs in one glcts process), Magma 75.94% ->
77.58% (+80 fixed, 0 newly broken); KHR-GL33 full mustpass lost nothing
(9884/9886, the 2 known Mesa-drift failures); retrace smoke clean (the
bsl-GLES miss is the documented golden drift, bit-identical on the pristine
baseline). The 4 DirectGLES direct_state_access.renderbuffers_storage* cases
that turned red are a PRE-EXISTING GL_FRAMEBUFFER_SRGB cross-test leak,
A/B-proven on an unpatched 2e6fc1ff build - wave 1 removed the two accidental
maskers (a crash partition and a failing case whose error path reset the
state). Fixing the leak itself is queued.
A destroyed VertexArrayObject's heap address is handed straight back by the
next allocation of its size, and so is a destroyed BufferObject's. DirectVulkan
keyed its per-VAO draw memo on the VAO POINTER and folded the bound buffer's
ADDRESS into the content hash that validates the memoised bindings, so a
delete/recreate pair under a byte-identical attribute layout reproduced both
the key and its validating hash at once. The successor VAO then inherited the
dead one's resolved bindings and the draw fetched from a destroyed VkBuffer.
Both stated defences failed together, because both reduce to the content hash
and the hash's buffer-identity component was itself a recycled address.
VertexArrayObject and BufferObject now carry a globally-unique, never-reused
GetLifetimeId() - the same contract as ProgramObject's, minted from an atomic
starting at 1 so a zero-initialised slot can never name a live object.
VaoDrawMemo matches on (address, lifetime id) and stores the id on recycle,
SetupDrawSnapshot's "the VAO did not move" test compares the id alongside the
config version, and VertexInputStateFactory::ComputeHash hashes the bound
buffer's id instead of its pointer (0 for client memory).
Proven: the use-after-free reproduces at 100% incidence headless on lavapipe,
including a SEGV whose backtrace is the driver dereferencing a destroyed vertex
buffer inside lvp_queue_submit, and it is gone with the fix. New coverage -
MG_Test/State/ObjectLifetimeIdTest (deterministic, GPU-free, no context: it
waits for the real allocator to repeat an address and asserts the id differs,
and skips loudly rather than passing quietly if it never gets the chance), and
MG_IntegrationTest XfbAfterClipDistanceScenario, registered for DirectGLES,
DirectVulkan, and a third DirectVulkan run with async shader compilation pinned
on because that is a second allocation pattern. Gates: 553/553 unit green at
async=0 and async=1; the scenario 5/5 headless at both flag states; 71/72 CI
trace-replay fixtures over both backends, the one failure a pre-existing
lavapipe crash proven not a regression (identical SIGSEGV at the identical
call number under the pre-fix library).
Pending NVIDIA/X11 confirmation: the KHR-GL{32,40} transform_feedback failures
that opened this investigation never reproduced on lavapipe - the -2/-101
pre-fill signature appears in zero pre-fix runs there - so whether this clears
them is UNPROVEN and must be re-measured on the NVIDIA rig against a freshly
re-run pre-fix baseline. The residual suspect is deliberately untouched here:
m_xfbCounterSlotByObject keys its counter slot on the raw GL transform-feedback
name, so a recycled name whose generation check happens to pass would RESUME
instead of BEGIN. That path was never exercised on lavapipe and is neither
confirmed nor exonerated.
A per-draw CPU profile of a real Minecraft frame (perf on the render thread,
which sits at 100% of one core on both backends) said the deficit is translation
overhead, not the GPU, and named where it goes. This removes the largest items
it found, on both backends and in the shared frontend they both feed.
The single biggest one was not translation at all: IsBackendContextCurrentOnThisThread
called eglGetCurrentContext on every invocation, and glvnd answers that with a
getpid() fork check - a real syscall. The predicate sits two and three deep in
every draw (the deferred-release drain, the global-UBO ring availability check,
and the ring allocation), so it accounted for 16.3% of the render thread. EGL is
still the ground truth, but re-verifying it once per thread per frame catches an
external migration at the next frame boundary rather than the next call, which
recovers the same bookkeeping.
Texture uploads now carry a dirty region instead of a per-level flag. Minecraft
animates atlas sprites with 16x16 glTexSubImage2D calls into a 1024x512 atlas
and respecifies the lightmap every frame; a per-level flag turned each of those
into a full-level re-upload - about 3.6 MB a frame of texels nobody changed.
MipmapStorage accumulates the written box, Espryt uploads it with
UNPACK_ROW_LENGTH striding into the level shadow, and Magma stages just that box.
The box is a union, not a range list: repeated writes to one level widen it and
it degrades to exactly the old whole-level upload, which is the honest worst case.
glBufferData(NULL) is the orphaning idiom, and the backend was answering it by
uploading the stale CPU shadow - turning a rename the driver does for free into
a full synchronized upload. BufferObject now records that a NULL respecify leaves
the store undefined, and the upload is skipped until content is actually written.
The rest are smaller and of a kind: the deferred-release queue is probed without
taking its mutex, the UBO ring waits on the frame fence that frees the space it
needs instead of draining the whole pipeline with glFinish at the size cap, VAO
binds go through a shadow so a draw's second bind of the same object does not
reach the driver, the per-draw clean-texture probe short-circuits on the content
version before rebuilding shape info, glUniform drops byte-identical writes
(which otherwise dirty the whole UBO for the next draw), re-binding the texture
or VAO a slot already holds no longer bumps the generation counters a backend
fast path is keyed on, and the texture validators stopped taking shared_ptr by
value.
On Magma: descriptor-set reuse keeps four entries instead of one, because draws
alternating between two programs - the chunk/entity ping-pong - thrashed a single
slot into a full re-allocate and re-write every draw; a DynamicDraw buffer whose
contents survive two frame boundaries is promoted to resident storage instead of
being re-copied into the per-frame arena forever; and sampled-read barriers name
only the shader stages whose device feature is enabled, which also removes a
latent VUID violation (ALL_GRAPHICS names geometry and tessellation stages a
device need not have).
Measured with the Minecraft rig (render distance 32, p50 fps, same machine,
single sample each): vanilla 1.21.1 Espryt 10.8 -> 36.3 and Magma 31.3 -> 44.6;
26.2 snapshot Magma 114.5 -> 210.5. Fabric+Sodium moved inside noise on Magma
(854 -> 766) with the native baseline itself moving 838 -> 1031 between the two
sessions, so treat that cell as unresolved rather than a regression measured.
Unit tests 421/421. The CTS A/B was not run: these numbers and the test suite are
the whole of the evidence, and a conformance regression would not have been
caught here.
Reading a buffer a compute shader wrote gave zeros: the frontend shadow that MapBuffer
resolves against is only maintained by uploads, and Magma had no path back. Every
KHR-GL40.texture_gather case ends by dispatching a compute shader into an SSBO and
comparing the mapped result, so 66 of 75 failed on it.
Magma needs no readback: EnsureGpuResidentStorage - the same host-visible coherent
adoption the transform feedback capture already uses - makes the shadow BE the memory
the shader writes, so binding a buffer as a shader storage buffer now adopts it. What
coherence does not give is ordering: the writes are visible once they have happened,
and the CPU was reading before the dispatch had retired. The readback op therefore
submits the recorded work and waits.
That exposed a mistake in the frontend flag this rides on: MarkGpuWritten skipped
GPU-resident buffers, reasoning there was no shadow to refresh. True, but the wait is
still needed - "reconcile with the GPU write" is not always "copy it back", and which
of the two it is belongs to the backend. The flag now only says a write is outstanding;
DirectGLES's readback still skips its persistent-mapped buffers when copying.
KHR-GL40.texture_gather on Magma: 66 failures -> 19 (the rest are rectangle textures,
mipmap completeness and tessellation, all still to do). Espryt stays at 75/75.
Buffer contents live in a CPU shadow that every read - MapBuffer, MapBufferRange,
GetBufferSubData, CopyBufferSubData - resolves against, and backend transfer ops only
ever push the shadow outwards. Two paths already knew the GPU can write a buffer on
its own and mirrored the result back by hand (ReadPixels into a pixel-pack buffer,
the transform feedback capture at EndTransformFeedback); a shader storage buffer
written by a draw or a dispatch had no such path at all, so the map handed the
application the bytes from before the dispatch.
Nothing exercised it until now because GL 3.3 has no compute stage. Every
KHR-GL40.texture_gather case ends by dispatching a compute shader that writes its
sampled texel into an SSBO and comparing the mapped result, and all 71 read back the
zero-filled shadow.
Adds the missing direction as a backend op: BufferObject::MarkGpuWritten flags a
buffer the GPU may have moved ahead of the shadow, SyncGpuWrites pulls it back at
every read point, and DirectGLES implements the readback with a plain read map of the
ES buffer. The flag is raised where the storage-buffer points are bound for the
upcoming draw or dispatch, which is the last moment the set of exposed buffers is
known, and cleared by the readback - so a buffer nothing writes costs one bool test
per map. Backends that cannot read their storage back leave the op null and keep
today's behaviour; a GPU-resident (coherent persistent) buffer needs nothing, since
its reads already resolve against the memory the shader wrote.
Drops the texture_gather failures from 71/75 to 25/75 with no crashes left.
Second stage of GL 3.0 transform feedback: captured draws now write real
data.
- Device setup enables the VK_EXT_transform_feedback feature when present
and loads the bind/begin/end entry points.
- Captured draws compile an XfbCapture program variant whose last
vertex-processing stage gets XfbBuffer/XfbStride/Offset decorations from
the program's resolved varyings (a new spirv-opt pass). A captured
gl_Position is mirrored into a dedicated output written before every
OpReturn - or before every OpEmitVertex in a geometry stage - ahead of
the position fixup, so the captured value is the shader's own pre-remap
position.
- DrawArrays/DrawElements wrap the draw in Begin/EndTransformFeedbackEXT;
a small counter buffer resumes the append position across draws within
one glBeginTransformFeedback (fresh Begin starts at the bound offsets).
- Capture targets are promoted to persistently-mapped host-coherent GPU
storage (persistent-map storage now also carries the transform feedback
usage), so MapBuffer/GetBufferSubData read the captured bytes after the
fence wait glEndTransformFeedback now performs.
- Draw-mode/feedback-mode validation defers to the geometry shader's
output primitive when one is present, and glGetBooleanv reports
GL_TRANSFORM_FEEDBACK_ACTIVE/PAUSED so dEQP's per-case state reset can
unwind an active capture.
KHR-GL33: transform_feedback capture_vertex_*/capture_geometry_*/
discard_*/draw_xfb and clip_distance.coverage now pass; queries
(PRIMITIVES_WRITTEN) and gl_ClipDistance capture remain.
Introduce a Mesa pipe_resource-style PipeResource that owns a GL buffer's bytes
and its backend GPU resource, abstracting WHERE the authoritative bytes live:
- Shadow mode (non-persistent buffers): a CPU Vector; the backend keeps its own
GPU copy in sync via BufferBackendOps, exactly as before.
- Persistent mode (coherent GL_MAP_PERSISTENT maps): the backend's host-visible,
COHERENT, persistently-mapped GPU memory is the single source of truth. The app
writes into it directly, every reader resolves against it, and NO per-write
backend transfer happens. The CPU shadow is released.
BufferObject no longer owns a raw shadow Vector; it holds a PipeResource and
exposes one accessor, MappedData(), that all readers go through. Every buffer-data
consumer (UBO payload, PBO texture upload, indirect draws, resident/streamed
uploads, both backends) was migrated from GetDataReadOnly()->data() to
MappedData(), so a persistent buffer's readers see GPU memory - not a stale
shadow. That stale-shadow inconsistency is what corrupted rendering (wrong UBOs ->
misplaced/"lost" vertices) in the first zero-copy attempt (625c8a6, reverted in
896cafc); routing every consumer through one accessor makes it structurally
impossible.
Backends provide the map via BufferBackendOps::AcquirePersistentMap:
- DirectVulkan: a HOST_VISIBLE|HOST_COHERENT (required, not just requested),
persistently mapped resident VkBuffer carrying every usage, seeded from the
shadow, never recreated; AcquireResidentSlice binds it directly.
- DirectGLES: EXT_buffer_storage immutable persistent+coherent glMapBufferRange,
falling back to the shadow when the extension is absent.
Fixes the ~7GB GpuMemory OOM + 100%-CPU/ANR running modern Blaze3D Minecraft on
both Magma and Espryt (per-draw whole-buffer re-upload of the coherent persistent
ring buffer), without the coherency/stale-read hazards of the reverted attempt.
BufferTest: zero-copy stress guard (15,360 draws -> 0 per-draw transfers, and every
reader resolves to GPU memory) + a shadow-fallback test. Host suite: 203/203 pass.
Device verification pending.
Two previously-stubbed GL 3.3 Core entry points.
glMultiDrawArrays: mirrors the existing glMultiDrawElements(BaseVertex)
architecture end to end -- a new MultiDrawArrays backend function-table
slot dispatched from the frontend after program/primitive-mode validation
(plus a drawcount < 0 -> GL_INVALID_VALUE guard).
- DirectGLES: PrepareForDraw once, then loop native glDrawArrays with the
same per-range client-side array upload the single DrawArrays does.
- DirectVulkan: build a MultiDrawCmd payload and hand it to a new
VulkanRenderer::MultiDrawArrays, which does one SetupDraw over the union
of the sub-draw vertex ranges and then a vkCmdDraw per range (mirrors
VulkanRenderer::MultiDrawElements).
glGetBufferSubData: reads a range of the bound buffer's CPU shadow into
client memory via a new BufferObject::DownloadSubData, with the same
validation shape as BufferSubData (INVALID_VALUE for negative/overflowing
range, INVALID_OPERATION for no bound buffer or a non-persistent mapped
buffer). The shadow reflects CPU writes and backend write-backs but not
arbitrary GPU-side writes, which is documented on the method.
Tests: 2 BufferTest cases for glGetBufferSubData (round-trip read of a
middle range and the whole buffer, plus out-of-range/negative/no-buffer
errors). BufferTest 32/32, SanityTest 30/30, VertexArrayTest 42/42;
library builds clean. (The glMultiDrawArrays draw paths are not
runtime-testable on this host and are compile-verified against the tested
MultiDrawElements pattern.)
Create 6 / Flywheel 1.0.6 now renders correctly with both flywheel:instancing
and flywheel:indirect on DirectGLES and DirectVulkan (verified in-game on
Adreno 830: waterwheels and cogwheels solid, animated, correct pairing, no
crashes across all four combinations).
- MG_State/MG_Impl: sync explicitly-ranged SSBO bindings of FLUSH_EXPLICIT
persistent maps to the backend before compute dispatches. Flywheel writes
its scatter-copy descriptors into the staging ring's persistent map and
never flushes that span (UB per spec, works on drivers whose maps alias
GPU-visible memory); our maps alias the CPU shadow, so the descriptors
never reached the GPU: the scatter compute copied nothing (GLES: empty
draw commands) or stale garbage (Vulkan: wild indirect commands ending in
VK_ERROR_DEVICE_LOST).
- MG_Impl/MG_Backend: real glFenceSync objects backed by backend fences
(GLES: native ES syncs guarded by context generation and owner thread;
Vulkan: buffer-manager frame serials), replacing always-signaled stubs
that let Flywheel reclaim staging memory the GPU still reads.
- MG_Backend/DirectGLES: compute dispatches now run the same per-program
resource sync as draws (uniform-block bindings and sampler units must be
re-established through the API because layout(binding) is stripped from
transpiled ESSL) and rebind texture units afterwards; the cull shader
used to read a stale _FlwFrameUniforms binding and the depth-pyramid
downsample sampled a stale unit-0 texture, zeroing the Hi-Z pyramid and
occlusion-culling all Flywheel geometry. Image uniforms are excluded from
glUniform1i (ES bakes their unit via layout(binding)); image-unit sync is
clamped to the device limit; eliminated/SSBO-classified uniform blocks
are skipped.
- MG_Backend/DirectGLES: gl_BaseInstance in native indirect draws reads the
GPU-written command buffer through an injected mg_IndirectParams SSBO
view addressed per draw instead of the zero CPU shadow; layout(binding)
is preserved for SSBO/image declarations (ES has no API rebinding for
them); the ES context ownership claim moved to a global atomic owner
thread with an EGL ground-truth check, and deferred buffer op state is
mutex-guarded, so ops cannot silently no-op after context migration.
- MG_Backend/DirectVulkan: new RebaseInstanceIndexPass rewrites vertex
InstanceIndex loads to (InstanceIndex - BaseInstance). glslang's relaxed
Vulkan mode aliases gl_InstanceID to InstanceIndex, which includes
firstInstance, but GL's gl_InstanceID is zero-based - draws with nonzero
baseInstance paired meshes with wrong instance data (cogwheel drawn as a
waterwheel, another wheel collapsed invisible). Gated on the
shaderDrawParameters device feature. Sampled-read barriers additionally
cover the compute stage (the Hi-Z downsample samples the depth
attachment from compute), and short uniform-buffer ranges keep the
existing zero-padding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
swung0x48
BZLZHH