Compare commits

...
16 Commits
Author SHA1 Message Date
BZLZHH 990e518e33 [Perf] (MG_Backend): give DirectVulkan's draw memo a table that fits in cache lines
The per-VAO resolved-bindings map probe was ~45% of
UploadAndBindVertexBuffers' self time, and the aux-memo pointer chase
was the single hottest instruction left in TrySetupDrawFastPath. Both
die together: a fixed 2048-slot two-probe 64B-aligned VaoDrawMemo table
embeds the VAO key, content-hash-validated layout facts and the
bindings payload reordered hot-to-cold. Layout facts hold exactly while
the slot's content hash equals the live VAO's own config-guarded hash;
a recycled VAO address either misses or reproduces a byte-identical
config, for which the facts are correct by construction. Bindings keep
their full per-draw revalidation; recycled slots zero their frame
serials so half-filled entries can never match.

ComputePipelineStateHash, the depth/stencil probe and the
primitive-restart probe now take one bulk GetRenderStateParameters()
fetch instead of ~17 cross-TU accessor calls (verified pure field
reads, identical bit packing). The EBO slice memo gained the same
manager-wide epoch one-compare rescue the vertex half uses.
GetShaderTransformFlags is memoized on pre-transform. Sodium's
MultiDrawElementsBaseVertex hoists GetGLTypeSize out of the
per-sub-draw loop, replaces the division with a shift, and skips
unsupported index types loudly instead of dividing by zero.

Also verified: a GL_BLEND toggle recompiles nothing in steady state -
the glslang frames in earlier state_toggle profiles were startup
contamination.

Quiet-box load-gated 6-round A/B: sodium_multidraw -8.0%, tex_param
-4.1%, use_program -3.3%; steady-state vanilla_draw CPU -20% ns/op at
4096 frames (the 80-frame matrix compresses CPU wins under GPU boost
clocks; profiles confirm UploadAndBindVertexBuffers 6.3% -> 4.4%
including the table probe, and the aux cold-line load gone). The one
matrix flag (pass_switch +7.5%) reversed to -3.2% in 10-pair isolated
re-runs. Unit tests 421/421.
2026-08-06 14:07:49 -04:00
BZLZHH 25a8f51db5 [Perf] (MG_Backend): make DirectGLES program switches remember their own bindings
mc_use_program cycles programs whose texture bindings never change, yet
every switch re-walked the units. Six fixes, one theme: a switch back to
a known program should find its own state waiting.

Per-program 4-entry resolved-texture-binding memo (round-robin, shadow
memcmp on hit) skips the unit walk when a program returns with its
bindings intact. The whole sampler-uniform pass in
BindCurrentProgramWithResources is memoized per program twin behind
(context, unitBindingsEpoch, samplingGeneration, backendStateVersion,
textureContextGeneration) plus a per-sampled-unit sampler-shadow row
compare, invalidated on relink/backend rebuild; the
BindCurrentUnitSamplers walk sits behind the same keys. Every unit
assignment, sampler-parameter change and bind path was verified to bump
one of those inputs.

UboRingAllocate's common path is now a generation check, a power-of-two
mask, an overrun check and a head bump - the duplicate availability
probe, frame-mark retirement and divisions moved to the wrap slow path.
The per-context framebuffer binding slots (the frontend getter
linear-scans per call) are cached as direct pointers - slots are
by-value members of GLContext, so the pointers are stable by
construction - feeding SyncCurrentFBO, SyncNeccessaryTextures and the
broadcast memo; BindCurrentFBO's per-draw registry hash Find became a
TwinLookupMemo probe. The VAO config-version cold-line load is hoisted
to the top of PrepareForDraw to overlap its miss.

Quiet-box load-gated 6-round order-alternating A/B, all nine cases,
both backends: use_program -27.7%, vanilla_draw -16.2%, ubo_range
-13.4%, pass_switch -11.1%, sampler_churn -10.7%, state_toggle -9.2%,
sodium_multidraw -5.3%, rest flat. No regression on either backend
(magma's one matrix flag disproved by isolated re-runs against
byte-identical DirectVulkan sources). Unit tests 421/421.
2026-08-06 14:07:49 -04:00
BZLZHH 8f2b766b56 [Perf] (MG_Backend): let DirectVulkan trust across frames what it proved once
The draw fast path still paid for its own proofs: the hottest single
load (20% of TrySetupDrawFastPath) was chasing the cold
VertexInputStateFactory heap entry just to answer "same vertex-input
layout?". That answer now comes from the frontend VAO's config-guarded
aux memo (layout hash + attribute masks), and a VAO-cycling stream with
a stable layout skips the pre-flight AND pipeline re-resolution
entirely. The VkProgramObject* is memoized on the snapshot behind a new
ProgramFactory cache-structure epoch (bumped on every insert/erase; use
is re-stamped so the idle sweep can never evict a live entry). A
render-state version move no longer forces the full path: the pipeline
value hash is refreshed in place and the 8-entry memo probed directly
(the GL_BLEND-toggle case).

The resolved-vertex-bindings memo now revalidates all-resident unmapped
entries ACROSS frames via per-binding slice epochs - minted from a
process-lifetime counter so a recycled address can never revalidate,
with every mutation path funnelled through BumpSliceEpoch - while
stamping each resource's GPU-use serial exactly as the skipped acquire
would, preserving the busy-tracking that glBufferSubData's
host-write-vs-staged-copy choice depends on. Resident index buffers get
the same treatment through an EBO slice memo.

The six-part dynamic-state tail (viewport/scissor/blend constants/depth
bias/line width/stencil) is gated behind one render-state-parameters
version + pass-geometry compare per command buffer. GetSlice is inlined;
SampledBindingsUnchanged walks only the program's declared bindings.

Quiet-box 6-round order-alternating A/B (on top of the frontend
VAO-bind commit): vanilla_draw -20.8% (790 -> 626 ns/op, 3.4x native to
2.5x), sampler_churn -28.2%, ubo_range -8.4%, state_toggle -3.8%;
tex_param's matrix flag (+10%) was adjudicated by an isolated
alternating re-run at +1.0% - position bias, not regression. Unit tests
421/421.
2026-08-06 13:18:38 -04:00
BZLZHH b9d8ad0421 [Perf] (MG_Backend): give DirectGLES one epoch that says no buffer moved
Four draw-path costs, one theme: re-proving what nothing invalidated.

A manager-wide buffer-mutation epoch (atomic; bumped with release AFTER
every mutation lands: all six BufferBackendOps via tracking wrappers,
every backend-initiated writeback - XFB readback/scatter, the five
pack-PBO readbacks - registry registration changes, and backend context
destruction; the full site inventory lives in a comment at the accessor)
lets the per-VAO resolved-buffers memo stamp the epoch after one
all-clean probe pass and skip every IsBufferDrawClean probe while it
holds. The IBO keeps its bound-object identity compare - only the probe
is elided. Non-bumping paths are enumerated with why they are safe:
GPU-authoritative writes are ignored by the probe, persistent-mapped
resources are clean by construction, and draws on non-persistent maps
are frontend-rejected GL errors.

GetProgramForDraw is hoisted to one call per PrepareForDraw and handed
to the four consumers that each re-derived it. The enabled-draw-buffers
walk feeding the fragColor broadcast count is memoized on the
(FBO, slot version, object version) trio. The UBO-binding loop probes
IsBufferDrawClean before falling back to EnsureBufferResource.

The texture chain captures (context, maxTouchedUnit, samplingGeneration,
unitBindingsEpoch) once per draw - shared by SyncNeccessaryTextures and
BindCurrentTextures, halving the epoch computations - and an aggregate
gate that is the exact conjunction of the three Sync*ToBackend
early-outs skips the per-texture cross-TU calls.

The t_egl* thread_local verification pair became owner-thread-guarded
atomics reset by MakeCurrent/ReleaseCurrent, removing __tls_get_addr
from the draw loop.

Quiet-box 6-round order-alternating A/B (with the frontend VAO-bind
commit): all NINE Espryt cases improved - sampler_churn -11.0%,
state_toggle -9.3%, ubo_range -9.1%, vanilla_draw -5.4%, pass_switch
-3.5%, the rest -1% to -2.5%. Unit tests 421/421.
2026-08-06 13:16:44 -04:00
BZLZHH f8069c0624 [Perf] (MG_State): stop paying two atomic refcounts for every glBindVertexArray
perf annotate put 94% of VertexArrayState::Bind's 10.5% self time on the
two lock-prefixed shared_ptr refcount RMWs each bind performs. The bound
VAO is now stored as a slot index into m_vertexArrays - no SharedPtr
copy, no atomics on the bind path. The lifetime invariant (the bound
object is kept alive by its slot; any cold path that clobbers a bound
slot - delete-while-bound including slot 0, create-over-bound-slot -
detaches the old object into m_boundDetached so GetBoundVertexArray
keeps answering with it) is enforced in MarkVertexArrayForDeletion /
CreateVertexArrayObject rather than assumed, and documented at the
change. Out-of-range binds and null slots keep their exact old
semantics.

VertexArrayObject also gains two opaque config-version-guarded backend
aux memo words, letting a backend answer "same vertex-input layout?"
from the frontend object instead of chasing its own cold cache entry.

After this change the frontend Bind drops out of the DirectVulkan draw
profile entirely (11.5% -> 0.5%). Measured jointly with the two backend
rounds that land on top: quiet-box 6-round order-alternating A/B,
all nine cases, no case worse than noise on either backend. Unit tests
421/421.
2026-08-06 13:16:02 -04:00
BZLZHH d0aae85da2 [Perf] (MG_Backend): let DirectVulkan's draw fast path survive a VAO swap
TrySetupDrawFastPath declined on its VAO pointer check for every draw of
a 512-VAO cycle - the Blaze3D chunk-render shape - so the fast path was
dead exactly where it mattered: full SetupDraw, per-draw
ResolveSamplerDescriptor, SyncTextureAndGetDescriptor and render-pass
re-fetch, for draws whose only change was the VAO.

Three fixes. A moved VAO now re-runs only the vertex-input pre-flight
and re-resolves the pipeline instead of declining to the full path. That
resolution probes the value-keyed pipeline memo directly off a cached
pipeline-state hash and snapshot render-pass hash, skipping
GetOrCreateRenderPass and its GetPendingRenderbufferClear probes per
draw; a stale cached hash can only miss, never false-hit. And when the
sampler-descriptor hint holds and the program's single dynamic UBO
re-resolves to the same VkBuffer and range - only the dynamic offset
moved, the per-draw glUniform case - the descriptor walk collapses to
one offset recompute and a vkCmdBindDescriptorSets of the same recorded
set with new pDynamicOffsets. The rebind memo is invalidated at
BeginFrame, layout destruction and override walks; the program lifetime
id never repeats, and per-frame descriptor sets are never rewritten
within their frame.

mc_vanilla_draw -36.9% (1260 -> 795 ns/op, 4.6x native to 3.4x),
sodium_multidraw -18.0%, state_toggle -14.9%, sampler_churn -7.8%,
use_program -7.7%, ubo_range -7.3%, tex_param -7.3%. All nine cases on
both backends, interleaved A/B; no attributable regression. Unit tests
421/421.
2026-08-06 12:00:31 -04:00
BZLZHH b904658b10 [Perf] (MG_Backend): stop DirectGLES re-resolving the same VAO's buffers and twins every draw
Four per-draw costs, all lookups that re-answer the same question.

SyncNeccessaryBuffers walked all 32 attribute slots cold and ran
EnsureBufferResource per buffer on every draw. The backend VAO twin now
hosts a resolved-draw-buffers memo: the deduped enabled-attribute buffers
and the index buffer resolve once per VAO config version, and each hit
re-validates every entry with IsBufferDrawClean - a shadow probe mirroring
every no-op branch of EnsureBufferResource (resource identity, context
generation, pending ops, change serial) - falling back to the full path
for just the dirty entries. The IBO entry is checked against the live
bound object each draw, so slot-version wrap cannot false-hit.

The registry hash Finds that resolve state objects to their backend twins
ran several times per draw. TwinLookupMemo - a direct-mapped,
Fibonacci-hashed table (4096 VAO / 256 program slots) with weak-ptr owner
equality against address reuse - answers them in one probe; collisions
fall back to the registry. A live entry's twin is never replaced once
set, so owner equality proves the raw pointer.

SyncCurrentVertexAttributeValues' pending-mask memo was a function-static
single entry that missed every draw once the app cycled VAOs; it now
lives on the twin. CurrentXfb()'s per-draw FastSTL map lookup became a
cached pointer invalidated at every map mutation (open addressing moves
values on any insert/erase/clear).

mc_vanilla_draw -12.6% (3.4x native to 3.0x), ubo_range -9.8%,
sampler_churn -7.5%, pass_switch -6.5%, sodium_multidraw -6.0%,
state_toggle -4.6%. All nine cases measured on both backends, interleaved
A/B; no case regressed. Unit tests 421/421.
2026-08-06 12:00:31 -04:00
BZLZHH 4b3fd11462 [Perf] (MG_Backend): key DirectVulkan's pipeline memo on state values, not a version that never repeats
Two per-draw churn costs, one cause each.

A blend toggle switched pipelines through a memo keyed on a monotonic
pipeline-state version - which never repeats, so flipping GL_BLEND off and back
on produced a "new" key both times, forced the full SetupDraw and rebuilt the
whole pipeline payload for a pipeline the cache already held. The memo now keys
on a value hash of the pipeline-relevant fixed-function state, recomputed only
when the state version moved, and the consecutive-draw fast path re-resolves
just the pipeline through it when nothing but render state changed. Blaze3D
brackets every batch with exactly this toggle; mc_state_toggle drops 36%
(6629 -> 4230 ns/op, 4.8x native to 3.7x).

The sampler-churn cost had the same shape as the Espryt side fixed separately:
glBindSampler bumps the frontend texture-bind generation even when it re-binds
the sampler the unit already holds, so the per-draw fast path died every draw.
The fast path now proves each binding's descriptor inputs unchanged - texture
and sampler lifetime ids, parameter and content sums, the sampling-resolution
generation, image epochs and exact layouts - and reuses the binding's cached
VkDescriptorImageInfo instead of re-running the resolve chain.
mc_sampler_churn drops 30% (1597 -> 1125), and the proof machinery pays for
itself on the uniform-range case too (-17%).

mc_tex_param stays where it is on this backend deliberately: profiling shows its
remaining cost is frontend validation with zero backend work, unreachable from
Renderer/.

All nine cases measured on both backends, interleaved A/B, no case worse than
noise. Unit tests 421/421.
2026-08-06 11:21:26 -04:00
BZLZHH 9be5d95440 [Perf] (MG_Backend): give DirectGLES unit bindings an epoch the sampler churn cannot fake
The texture-binding memos added earlier keyed on the frontend texture-bind
generation, and 26.2-style unit switching defeats them: glBindSampler bumps the
generation even when it re-binds the sampler the unit already carries, so a
frame that cycles active units re-ran the full two-pass, eleven-slot alias
resolution and the unbind walks on every draw. mc_sampler_churn sat at 1674
ns/op against the native driver's 239 - the worst multiplier left on this
backend - with about half the time in two virtual calls per binding slot.

The units now carry an epoch: a snapshot of each touched unit's slot objects and
sampler object, compared by weak_ptr OWNERSHIP rather than raw pointer - a held
weak_ptr pins its control block, so a freed-and-recycled object can never
owner-equal its predecessor, which is the ABA hole a pointer key would have and
the reason version keying was rejected (WithTemporarilyBoundNamedTexture bumps
slot versions without touching the bind generation). The
(context id, bind generation, high-water mark) triple gates the snapshot walk to
at most once per draw; the epoch moves only when a binding really changed. Both
per-draw memos key on the epoch plus the sampling-resolution generation, which
carries what the epoch cannot see: a default texture's image appearing, and
every completeness input. Two smaller memos ride along: the per-unit
sampler-registry lookup (owner-keyed, misses never cached - the backend object
may be created later in the same draw), and the pending-vertex-attribute mask,
whose first version scanned all 32 slots and put +10% on the VAO-cycling case
before being restricted to the program's active locations.

ns per op, DriverBench on a GTX 1660 SUPER, isolated A/B, all nine cases on both
backends: mc_sampler_churn 1673 -> 732, mc_use_program 4513 -> 4279,
mc_state_toggle 2365 -> 2247, everything else within noise and nothing worse.
7.0x native to 3.1x on the churn case.

Unit tests 421/421.
2026-08-06 11:20:52 -04:00
BZLZHH d49d79a64b [Perf] (MG_Backend): pool DirectVulkan's upload staging and batch its submits
Every dirty texture bought itself a fresh staging buffer (vmaCreateBuffer +
vmaMapMemory), a fresh command buffer, a fresh fence, and its own vkQueueSubmit.
A perf profile of the sprite-animation case put 41% of the whole run in the
kernel on the resulting ioctl traffic; the reclaim list already avoided waiting
on the fences, so the cost was the allocation and submission machinery itself,
paid per texture per frame.

Staging now comes from a pool of persistently-mapped blocks (1 MiB minimum,
exact-size beyond that, bump-allocated, 32 MiB idle cap), and uploads record
into one shared batch command buffer from a dedicated command pool, going out as
one submit with one pooled fence per flush. Fences, command buffers and blocks
all recycle through the existing fence-list reclaim instead of being destroyed.
Flush points: before every frame command buffer submission (which is what
preserves the old ordering argument - the batch reaches the queue strictly
before anything that could sample its images), on the glFlush finite-time path,
when a batch would outgrow its staging bound, and eagerly at 128 KiB, which
measured faster because the GPU overlaps the copy with the rest of the frame's
CPU recording. The mid-frame upload-draw-upload-again sequence detects itself
through the batch image list and flushes first, reproducing the old two-submit
granularity exactly; a deferred image release flushes any open batch that still
references the image, because drain proofs only cover submitted work.

ns per op, DriverBench on a GTX 1660 SUPER: mc_tex_stream 9405 -> 5373 (2.3x
the native driver, from 3.9x), atlas_sprite -57%, lightmap -89%, chunk_upload
-10%; draw-path cases unchanged. The suite's sampler-churn number reads a few
percent worse right after the now-much-faster upload case, which was chased to
schedutil downclocking during the newly-blocking-free frames - isolated and
frequency-pinned runs measure parity; noted here so the next person does not
re-chase it.

Unit tests 421/421; Vulkan validation layer clean across draw and upload cases.
2026-08-06 10:36:08 -04:00
BZLZHH f5761ea1f3 [Perf] (MG_Backend): diff only the render-state span that moved, and gate the per-draw walks
Four per-draw costs in DirectGLES, all of the same species: work re-done for an
answer that had not changed.

SyncRenderState was guarded by a single version compare, so one blend toggle -
the way Blaze3D brackets every batch - re-diffed the whole ~40-field render
state block and copied the full struct back into the shadow, every draw. The
parameter struct is now split into three contiguous byte spans, each gated by a
memcmp against the backend shadow; a per-draw blend flip touches only the blend
span. The shadow is byte-cloned after each sync so the span compares stay exact,
padding included. Blocks whose inputs live outside the parameter struct (the
surface-size viewport fallback, the sRGB context capability) stay ungated, and
the dual-source-blend hard-fail still fires every draw because a throwing sync
never stamps the shadow.

SyncMipmapsToBackend gained a first-level clean gate on (context id,
sampling-resolution generation, content version, params version) that skips the
IsComplete walk and the eight-field shape probe outright; every shape mutation
funnels through BumpShapeVersion, which is what makes the gate sound.
SyncToBackend for vertex arrays compares one aggregate config version instead of
three stamps per attribute slot. And SyncNeccessaryTextures memoises the
draw-framebuffer attachment list, keyed the same way the framebuffer sync memo
already is, instead of re-walking attachments per draw.

ns per draw, DriverBench on a GTX 1660 SUPER, isolated A/B: mc_state_toggle
3151 -> 2397, mc_ubo_range 792 -> 579, mc_vanilla_draw 1111 -> 881,
mc_sampler_churn 2019 -> 1676, mc_use_program 5132 -> 4356; every one of the
nine cases improved. Against the native driver Espryt now stands at 3.6x on the
plain draw path, 2.8x on the per-draw uniform-range path and 2.1x on the blend
toggle, from 8.7x / 9.1x / 7.2x when this effort began.

Unit tests 421/421.
2026-08-06 10:35:41 -04:00
BZLZHH b3f774d2c0 [Fix] (CI): name the EGL vendor library the benchmark job runs on
The benchmark job is the only one that brings a real GL context up - DriverBench
dlopens libEGL.so.1 and renders through it - but its apt list only asks for
libegl1, which is glvnd's dispatch layer and nothing more. The vendor library
behind it, libegl-mesa0, has been arriving as a Recommends of libegl1 rather
than because anything asked for it. That is too quiet a dependency for the one
job whose whole purpose is running a driver: a base image change, or
--no-install-recommends turning up anywhere upstream, would leave eglInitialize
with no vendor to dispatch to and fail the job for a reason nothing in the
workflow explains. Name it, next to libgl1-mesa-dri, which is listed for
exactly the same reason.

Verified with a full headless ctest -C Release -L benchmark - no $DISPLAY, no
$EGL_PLATFORM, mesa as the only EGL vendor: SanityBench, ProgramBench,
BufferBench and DriverBench all pass.
2026-08-06 09:44:04 -04:00
BZLZHH d524330032 [Test] (MG_Benchmark, MG_Util): model four more Minecraft frame patterns in the driver bench
The captured traces contain per-frame patterns the bench did not exercise, and
first measurements show two of them are now the worst remaining multipliers -
which is exactly what the missing cases were hiding.

mc_pass_switch: the 26.2 snapshot switches render targets 132 times a frame and
re-declares draw buffers 198 times. Render-target churn is where a Vulkan
backend pays for render-pass breaks and where a tiler pays most on device, and
no case measured it. mc_state_toggle: Blaze3D brackets batches with blend
toggles - 46 enable/disable pairs and 28 blend-func changes per vanilla frame.
mc_tex_param: 26.2 re-sets texture parameters 612 times a frame, almost always
to the value already in place, so this measures redundant-parameter filtering.
mc_use_program: Sodium switches programs 62 times a frame with a mat4 upload on
each, roughly one switch per multi-draw.

All four live in the shared case file at the measured per-frame rates, so the
desktop harness, the on-device harness and the POST screen's Run Bench report
comparable numbers. First desktop measurements (ns/op, native / Espryt / Magma):
pass_switch 8877 / 18502 / 13896, state_toggle 1182 / 8526 / 8305,
tex_param 42 / 102 / 197, use_program 2182 / 10648 / 5096. The state-toggle
multiplier - 7x on both backends - is the largest newly exposed gap and the next
optimization target.

Unit tests 421/421; the Android JNI translation unit compiles against the
extended case set.
2026-08-06 09:39:02 -04:00
BZLZHH f2d210b12d [Perf] (MG_Backend): memoise DirectVulkan's per-draw vertex binding resolution
Every draw re-resolved its whole vertex binding array: for each enabled binding,
look up the buffer, acquire a slice from the buffer manager, apply the binding's
base offset, fill the VkBuffer and offset arrays, bind. In the Minecraft-shaped
benchmark the same few hundred vertex array objects cycle for the whole run and
each one's answer is stable, so UploadAndBindVertexBuffers was the single largest
cost in the backend at 7.9% of the render thread, with AcquireResidentSlice
another 3.8% underneath it.

The resolved array is now kept per vertex array object and revalidated instead of
rebuilt. Validation is two-tier. The vertex array's own configuration version
already invalidates its backend vertex-input state, so a changed attribute,
format, buffer or base offset yields a different state object - the memo compares
both that object's address and its hash, which mixes the bound buffers and the
whole layout. What that does not cover is the slice moving underneath an
unchanged configuration, so the buffer manager now carries a monotonic epoch that
every writer of slice-deciding state bumps: resident storage creation, respecify,
sub-data, flush of a mapped range, the promotion and demotion between streamed
and resident storage, each fresh arena allocation, and bulk release. The counter
is manager-wide and never reset, so a resource created at a recycled address
cannot reproduce a value some memo still holds.

The miss path was the thing to get right, because the previous attempt in this
area regressed the texture-upload and sampler-churn cases by 60-85%: it added a
verification pass that re-ran the resolution work it was trying to skip, so every
miss paid for it twice. Here a miss is one pointer-keyed lookup and a few stores,
and nothing else runs that the full path would not have run anyway.

ns per draw, DriverBench on a GTX 1660 SUPER: mc_ubo_range 924 -> 767,
mc_vanilla_draw 1346 -> 1227, mc_sampler_churn 1397 -> 1279,
mc_sodium_multidraw 3365 -> 3266. Magma is now 4.1x the native driver on the
per-draw uniform-range case, from 5.4x when this round started. No case
regressed on either backend.

Unit tests 421/421.
2026-08-06 09:24:08 -04:00
BZLZHH fd40960f70 [Perf] (MG_Backend): revive DirectGLES's dead framebuffer-sync guard, and stop probing twice
SyncCurrentFBO has an early-out that compares three memos, and it could never
fire. One of the three, g_fboBindVersions, was only ever stamped by
ForceBindCurrentFBO - which runs from glBlitFramebuffer and the DSA
glClearNamedFramebuffer* paths and nowhere else. An application that touches
neither leaves that memo at 0 while the binding slot's version is at least 1 from
its first glBindFramebuffer, so the first term mismatched forever and the guard
was dead code rather than merely too coarse. Every draw therefore re-walked all
40-odd attachment slots and rebuilt the 8-slot snorm/unorm clamp mask for a
framebuffer that had not changed since the previous draw.

SyncCurrentFBO now stamps all three memos itself, through one helper, on every
path that leaves the target synced - including the default-framebuffer
"nothing to do" path, which previously returned without stamping anything. The
memo is renamed to say what it now records (a sync, not a bind). Instrumenting a
throwaway build put it at 539998 hits against 2 misses, the misses being the
first bind of each target; it was 0 hits before.

Skipping the sync also skips the Bind() inside it, so all eleven call sites were
checked: every one issues its own bind afterwards (PrepareForDraw and the
glClearBuffer* paths bind Draw, ReadPixels and the CopyTexSubImage paths bind
Read, BlitFramebuffer binds both, GetTexImage uses its own scoped binder). The
global snorm/unorm clamp masks written inside the sync stay correct because they
can only be stale if a different framebuffer was synced as Draw in between, which
moves the pointer or slot version and forces the re-sync that rewrites them.
InvalidateFramebufferBindingCache now also clears these memos: both its callers
mean the ES context may have been reset, and a live early-out must not survive
that.

Two smaller items in the same pass. StateBackendObjectRegistry kept the backend
twin and its liveness weak_ptr in two maps, so every lookup cost two hash probes
and the draw path does ten to twenty of them; they are one map with one entry
type now, one probe. The weak_ptr check itself is load-bearing and stays -
glDeleteVertexArrays followed by glGenVertexArrays recycles heap addresses
readily. And SyncNeccessaryBuffers ran the full EnsureBufferResource check once
per enabled vertex attribute, which on an interleaved Minecraft-shaped VAO means
four to eight times over the same VBO; it is deduplicated per distinct buffer now.

ns per draw, DriverBench on a GTX 1660 SUPER, A/B against a build differing only
by this diff: mc_vanilla_draw 1403 -> 1113, mc_ubo_range 983 -> 797,
mc_sampler_churn 2309 -> 2003, mc_sodium_multidraw 3232 -> 3023. Against the
native driver Espryt is now 4.3x on both the plain draw and the per-draw
uniform-range case, from 8.7x and 9.1x at the start of this work.

Unit tests 421/421. Also replayed all 38 locally-available DirectGLES trace
fixtures against a baseline library: every one produced bit-identical ssim and
mismatched-pixel counts, including the improved-transparency OIT trace whose
scratch clear framebuffer is exactly the draw-buffer hazard the code comments
warn about.
2026-08-06 09:23:47 -04:00
BZLZHH 49aab57f03 [Perf] (MG_Backend, MG_State): stop re-resolving texture unit bindings on every draw
DirectGLES re-derived the whole texture binding state for every draw: for each
touched unit, two alias-resolution passes over all binding slots, then a third
walk to unbind native targets nothing claimed, then the sampler. With the
Minecraft-shaped bench that was 13.2% of the render thread in BindCurrentTextures
alone, plus 4.6% in SyncNeccessaryTextures deciding which textures to consider.
The answer is identical across a whole terrain batch.

The resolution is now memoised, and what makes replaying it as a no-op legitimate
is that the memo does not merely trust a key: it compares the backend's own bound
texture shadow against the one resolution left behind. Every path that binds a
texture behind this function's back already maintains that shadow - the scratch
bind an upload does on the temp unit, CopyTexSubImage2D and GenerateMipmap
binding on the active unit, the glBindTextures fast path, the scrub a backend
texture performs when it is destroyed or respecified - so a memcmp catches all of
them without having to enumerate them. On top of that the key covers the texture
bind generation, the program that arbitrates aliased targets (pointer, lifetime
id, backend state version, link status), and the ES context generation.

Two invalidation sources had no signal at all and needed one. Mipmap completeness
decides whether a texture is bound in the first place, and it moves with texture
shape and with the effective sampler's filter - so a sampling-resolution
generation now moves with both, routed through single choke points
(TextureObjectBase::BumpShapeVersion, SamplerObject::BumpVersion) so a future
bump site cannot forget it. A texture context id was needed because both
generations restart at zero in a new GLContext, which can land on the old heap
address.

This also closes a pre-existing hole rather than working around it:
glDeleteSamplers unbinds the sampler from every unit straight through
TextureUnit::SetSamplerObject, bypassing the touch bookkeeping, so that setter now
bumps the bind generation on a real change. The sampler bind step itself stays
outside the memo and runs every draw - the program's raw-depth-fetch substitution
rewrites unit samplers immediately afterwards, so a memo there could never hit.

ns per draw, DriverBench on a GTX 1660 SUPER (native / Espryt):
mc_vanilla_draw 253 / 2037->1315, mc_ubo_range 202 / 1684->955,
mc_sodium_multidraw 739 / 3939->3150. Espryt goes from 8.3x to 4.7x the native
driver on the per-draw uniform-range case. Magma is unaffected (the MG_State
additions are counter bumps), and no case regressed.

Unit tests 421/421.
2026-08-06 07:41:10 -04:00
35 changed files with 3841 additions and 713 deletions
+5 -1
View File
@@ -177,9 +177,13 @@ jobs:
uses: lukka/get-cmake@v4.3.3
- name: Install runtime dependencies
# libegl-mesa0 is the EGL vendor library itself: DriverBench brings up a
# real GL context, and libegl1 is only glvnd's dispatch. It normally
# arrives as a Recommends of libegl1, which is too quiet a dependency for
# the one job that needs a working driver.
run: |
sudo apt-get update
sudo apt-get install -y libvulkan1 libegl1 libgles2 libgl1-mesa-dri mesa-vulkan-drivers
sudo apt-get install -y libvulkan1 libegl1 libegl-mesa0 libgles2 libgl1-mesa-dri mesa-vulkan-drivers
- name: Download Linux runtime
uses: actions/download-artifact@v8
File diff suppressed because it is too large Load Diff
+217 -42
View File
@@ -281,6 +281,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
// an older generation hold ids from a dead context.
Uint g_bufferContextGeneration = 1;
// Buffer-mutation epoch backing store (contract, mutation-site list and
// memory-ordering rules: Managers.h at the accessor declarations).
// Starts at 1 so the memo stamps' 0 means "never stamped". Atomic:
// frontend buffer ops may run on non-draw threads while the draw thread
// reads; the release-bump-AFTER-mutation / acquire-read-BEFORE-probes
// pairing makes a stamp taken against stale state impossible to consume.
std::atomic<Uint64> g_bufferMutationEpoch{1};
// Defined next to the indexed-binding shadow below; forward-declared so
// every glDeleteBuffers site in this namespace can scrub stale shadow
// entries (GL resets a deleted buffer's bindings - indexed and pixel
@@ -693,24 +701,71 @@ namespace MobileGL::MG_Backend::DirectGLES {
g_hasDeferredBufferReleases.store(true, std::memory_order_release);
}
// Epoch-tracking wrappers: every op bumps the buffer-mutation epoch AFTER
// its impl returns (release; see Managers.h for why the order matters),
// covering every mutation branch inside - including the early returns
// that only queued pendingRanges or flagged pendingRespecify. Bumping on
// an op that turned out to be a no-op merely re-runs the probes once.
void Ops_RespecifyTracked(BufferObject& bufferObject) {
Ops_Respecify(bufferObject);
BumpBufferMutationEpoch();
}
void Ops_SubDataTracked(BufferObject& bufferObject, SizeT offset, SizeT size) {
Ops_SubData(bufferObject, offset, size);
BumpBufferMutationEpoch();
}
void Ops_FlushMappedRangeTracked(BufferObject& bufferObject, Range1D range,
Flags<BufferMappingAccessBit> appAccess) {
Ops_FlushMappedRange(bufferObject, range, appAccess);
BumpBufferMutationEpoch();
}
void Ops_OnDestroyTracked(SharedPtr<BackendBufferResource>&& resource) {
Ops_OnDestroy(std::move(resource));
BumpBufferMutationEpoch();
}
void* Ops_AcquirePersistentMapTracked(BufferObject& bufferObject) {
void* result = Ops_AcquirePersistentMap(bufferObject);
// Bump even on decline: the frontend still enters a persistent map the
// per-draw probes must start seeing (IsMapped-driven range pushes).
BumpBufferMutationEpoch();
return result;
}
void Ops_ReadbackFromGpuTracked(BufferObject& bufferObject) {
Ops_ReadbackFromGpu(bufferObject);
BumpBufferMutationEpoch();
}
const BufferBackendOps g_glesBufferBackendOps = {
.Respecify = Ops_Respecify,
.SubData = Ops_SubData,
.FlushMappedRange = Ops_FlushMappedRange,
.OnDestroy = Ops_OnDestroy,
.AcquirePersistentMap = Ops_AcquirePersistentMap,
.ReadbackFromGpu = Ops_ReadbackFromGpu,
.Respecify = Ops_RespecifyTracked,
.SubData = Ops_SubDataTracked,
.FlushMappedRange = Ops_FlushMappedRangeTracked,
.OnDestroy = Ops_OnDestroyTracked,
.AcquirePersistentMap = Ops_AcquirePersistentMapTracked,
.ReadbackFromGpu = Ops_ReadbackFromGpuTracked,
};
} // namespace
Uint64 CurrentBufferMutationEpoch() {
return g_bufferMutationEpoch.load(std::memory_order_acquire);
}
void BumpBufferMutationEpoch() {
g_bufferMutationEpoch.fetch_add(1, std::memory_order_release);
}
void RegisterBufferBackendOps() {
MG_State::GLState::SetBufferBackendOps(&g_glesBufferBackendOps);
// Frontend writes issued while ops were unregistered advanced change
// serials with no per-op bump; re-open every draw-clean memo.
BumpBufferMutationEpoch();
}
void UnregisterBufferBackendOps() {
if (MG_State::GLState::GetBufferBackendOps() == &g_glesBufferBackendOps) {
MG_State::GLState::SetBufferBackendOps(nullptr);
}
// From here on frontend writes bypass the tracked ops entirely.
BumpBufferMutationEpoch();
InvalidateArrayBufferBindingCache();
// Pooled ids belong to the dying context too; drop them without glDeleteBuffers.
ClearBufferPool();
@@ -721,8 +776,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
void OnBackendContextDestroyed() {
UnregisterBufferBackendOps();
UnregisterBufferBackendOps(); // also bumps the buffer-mutation epoch
++g_bufferContextGeneration;
// The generation moved AFTER the unregister bump above; re-open the
// memos again so no stamp can predate the generation change.
BumpBufferMutationEpoch();
InvalidateArrayBufferBindingCache();
InvalidateIndexedBufferBindingCache();
InvalidatePixelBufferBindingCaches();
@@ -768,6 +826,25 @@ namespace MobileGL::MG_Backend::DirectGLES {
return static_cast<GLESBufferResource*>(bufferObject->GetBackendResource().get());
}
Bool IsBufferDrawClean(const MG_State::GLState::BufferObject* frontend, const GLESBufferResource* resource) {
// Identity first: a respecify path can hand the frontend a NEW resource; the
// memoed pointer is then stale (and only kept alive by the caller's shadow).
if (!resource || resource != frontend->GetBackendResource().get()) return false;
if (resource->contextGeneration != g_bufferContextGeneration) return false;
if (resource->id == 0) return false;
// Zero-copy coherent persistent store: EnsureBufferResource's own early-out —
// the app writes straight into the mapped GPU storage, nothing to sync.
if (resource->persistentMapped) return resource->persistentPtr != nullptr;
// A live non-zero-copy map may owe a per-draw SyncPersistentMappedRange push
// (persistent maps mutate the shadow without bumping the change serial).
if (frontend->IsMapped()) return false;
if (resource->pendingRespecify || !resource->storageInitialized) return false;
// Same unlocked emptiness probe EnsureBufferResource's replay branch uses.
if (!resource->pendingRanges.empty()) return false;
if (resource->storageSize != frontend->GetSize()) return false;
return resource->syncedChangeSerial.load(std::memory_order_acquire) == frontend->GetChangeSerial();
}
GLESBufferResource* EnsureBufferResource(const SharedPtr<MG_State::GLState::BufferObject>& bufferObject) {
#ifdef TRACY_ENABLE
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
@@ -1093,13 +1170,50 @@ namespace MobileGL::MG_Backend::DirectGLES {
return !g_uboRing.creationFailed;
}
Bool UboRingAllocate(SizeT size, SizeT& outOffset) {
if (size == 0 || !UboRingAvailable()) return false;
// Division-based rounding: the spec doesn't promise a power-of-two
namespace {
// Division-based rounding fallback: the spec doesn't promise a power-of-two
// alignment. Slot offsets stay multiples of the alignment because every
// slot size is, and wrap padding restarts at ring offset 0.
const SizeT alignedSize =
(size + g_uboRing.alignment - 1) / g_uboRing.alignment * g_uboRing.alignment;
inline SizeT UboRingAlignUp(SizeT size, SizeT alignment) {
if ((alignment & (alignment - 1)) == 0) {
return (size + alignment - 1) & ~(alignment - 1);
}
return (size + alignment - 1) / alignment * alignment;
}
Bool UboRingAllocateSlow(SizeT size, SizeT& outOffset);
} // namespace
Bool UboRingAllocate(SizeT size, SizeT& outOffset) {
if (size == 0) return false;
// Fast path: a live ring under the current context with room before both
// the wrap boundary and the in-flight tail. Touches no GL and probes no
// frame marks - the sole caller sits behind UboRingAvailable() in the
// draw preparation, so the context checks have already run this draw.
// `tail` may be stale here (marks are only retired on Present and on the
// slow path); staleness is conservative - the in-flight span reads too
// large, the check fails, and the slow path retires marks and re-tries.
auto& ring = g_uboRing;
if (ring.id != 0 && ring.contextGeneration == g_bufferContextGeneration) {
const SizeT alignedSize = UboRingAlignUp(size, ring.alignment);
// Ring sizes are kUboRingInitialBytes (a power of two) doubled some
// number of times, so the offset modulo reduces to a mask.
static_assert((kUboRingInitialBytes & (kUboRingInitialBytes - 1)) == 0,
"ring offset mask below requires power-of-two ring sizes");
const SizeT offset = static_cast<SizeT>(ring.head & (ring.size - 1));
if (offset + alignedSize <= ring.size &&
ring.head + alignedSize - ring.tail <= ring.size) {
ring.head += alignedSize;
outOffset = offset;
return true;
}
}
return UboRingAllocateSlow(size, outOffset);
}
namespace {
Bool UboRingAllocateSlow(SizeT size, SizeT& outOffset) {
if (!UboRingAvailable()) return false;
const SizeT alignedSize = UboRingAlignUp(size, g_uboRing.alignment);
if (g_uboRing.id == 0 && !CreateUboRingStorage(alignedSize)) {
return false;
}
@@ -1176,6 +1290,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
outOffset = offset;
return true;
}
} // namespace
void* UboRingMappedPtr() { return g_uboRing.mappedPtr; }
Uint UboRingBufferId() { return g_uboRing.id; }
@@ -1348,11 +1463,24 @@ namespace MobileGL::MG_Backend::DirectGLES {
MGLOG_D("Syncing VAO with backend ID %u to backend for state ID %u", m_backendVAOId,
stateVAOObject->GetExternalIndex());
// One compare instead of MAX_VERTEX_ATTRIBS x 3 per draw: the config version
// aggregates every per-attribute version bump (see the member comment), and the
// index-buffer slot version covers the only other thing this function reads. When
// both are clean there is nothing to emit, and the VAO is not even bound here -
// PrepareForDraw's BindCurrentVAO establishes the draw binding regardless.
const Uint32 currentConfigVersion = stateVAOObject->GetConfigVersion();
const Uint16 currentIndexBufferVersion = stateVAOObject->GetIndexBufferBindingSlot().GetVersion();
const Bool attributesDirty = !m_hasSyncedConfigVersion || m_syncedConfigVersion != currentConfigVersion;
const Bool indexBufferDirty = currentIndexBufferVersion != m_syncedIndexBufferVersion;
if (!attributesDirty && !indexBufferDirty) {
return;
}
Bind();
const auto& allAttributeVersions = stateVAOObject->GetAllAttributeVersions();
const auto& allAttributes = stateVAOObject->GetAllAttributes();
for (Uint attribIndex = 0; attribIndex < allAttributes.size(); ++attribIndex) {
for (Uint attribIndex = 0; attribIndex < allAttributes.size() && attributesDirty; ++attribIndex) {
const auto& attrib = allAttributes[attribIndex];
Bool needsSyncSwitch = allAttributeVersions[attribIndex].SwitchVersion !=
m_syncedAttributeVersions[attribIndex].SwitchVersion;
@@ -1407,8 +1535,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
Uint16 currentIndexBufferVersion = stateVAOObject->GetIndexBufferBindingSlot().GetVersion();
if (currentIndexBufferVersion != m_syncedIndexBufferVersion) {
if (indexBufferDirty) {
const auto& indexBufferBinding = stateVAOObject->GetIndexBufferBindingSlot().GetBoundObject();
Bool indexBufferSynced = false;
if (indexBufferBinding) {
@@ -1429,7 +1556,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
m_syncedAttributeVersions = allAttributeVersions;
if (attributesDirty) {
m_syncedAttributeVersions = allAttributeVersions;
m_syncedConfigVersion = currentConfigVersion;
m_hasSyncedConfigVersion = true;
}
}
void BackendVertexArrayObject::SyncClientSideAttributesForDrawArrays(
@@ -1826,6 +1957,23 @@ namespace MobileGL::MG_Backend::DirectGLES {
ZoneScopedC(TRACY_ZONECOLOR_BACKEND);
#endif
// First-level clean gate (see the member comment): three version compares and no
// virtual shape walk. Every mutation the slower probe below would catch bumps one of
// the keys - shape via the context's sampling-resolution generation (coarse: any
// texture's shape churn re-opens every gate, which only costs a fall-through to the
// probe), CPU pixels via the content version, samples/fixed-locations via the params
// version - and backend-side storage resets clear m_isInitialized. Restricted to
// Mipmap storage like the probe fast path: a buffer texture's backing store can move
// without any of these keys noticing.
if (m_isInitialized && m_syncedShapeContextId != 0 && MG_State::pGLContext &&
m_syncedShapeContextId == MG_State::pGLContext->GetTextureContextId() &&
m_syncedShapeGeneration == MG_State::pGLContext->GetSamplingResolutionGeneration() &&
m_syncedContentVersion == stateTextureObject->GetContentVersion() &&
m_syncedShapeParamsVersion == stateTextureObject->GetTextureParamsVersion() &&
stateTextureObject->GetStorageType() == TextureStorageType::Mipmap) {
return;
}
MGLOG_D("Syncing texture mipmaps with backend ID %u to backend for state ID %u", m_backendTextureId,
stateTextureObject->GetExternalIndex());
@@ -1879,6 +2027,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (probe == m_prevTextureInfo) {
MGLOG_D("Texture ID %u already fully synced, skipping scratch bind + upload.",
m_backendTextureId);
// The probe just proved "fully synced" from the real state, so the cheap
// gate may be (re)stamped here: the coarse generation only ever goes stale
// from OTHER textures' churn, and this draw re-validated this one.
if (MG_State::pGLContext) {
m_syncedShapeContextId = MG_State::pGLContext->GetTextureContextId();
m_syncedShapeGeneration = MG_State::pGLContext->GetSamplingResolutionGeneration();
m_syncedShapeParamsVersion = stateTextureObject->GetTextureParamsVersion();
}
return;
}
}
@@ -2412,6 +2568,16 @@ namespace MobileGL::MG_Backend::DirectGLES {
// upload); stamp the version so per-draw re-syncs short-circuit until
// the next CPU-side mutation.
m_syncedContentVersion = stateTextureObject->GetContentVersion();
// Same instant, so the cheap gate's keys describe exactly this synced state.
// Only Mipmap storage may arm it - the gate refuses other storage types anyway,
// but a stale trio must not linger on an object that later switches type.
if (MG_State::pGLContext && stateTextureObject->GetStorageType() == TextureStorageType::Mipmap) {
m_syncedShapeContextId = MG_State::pGLContext->GetTextureContextId();
m_syncedShapeGeneration = MG_State::pGLContext->GetSamplingResolutionGeneration();
m_syncedShapeParamsVersion = stateTextureObject->GetTextureParamsVersion();
} else {
m_syncedShapeContextId = 0;
}
}
void BackendTextureObject::SyncBuiltinSamplerToBackend(
@@ -2740,6 +2906,13 @@ namespace MobileGL::MG_Backend::DirectGLES {
void InvalidateFramebufferBindingCache() {
g_driverFBOBindings = {0, 0};
g_driverFBOBindingKnown = {false, false};
// The per-target "already synced" memo describes work pushed into the ES context
// that is being left or replaced. Both callers (MakeCurrent, DestroyEGLContext)
// mean the context may have been reset under us, so claim nothing is synced:
// a null object never matches a real binding, so SyncCurrentFBO re-pushes.
g_fboSyncedSlotVersions = {0};
g_fboSyncedObjectVersions = {0};
g_fboSyncedObjects = {};
}
void BackendFramebufferObject::InvalidateSyncedState() {
@@ -2768,15 +2941,14 @@ namespace MobileGL::MG_Backend::DirectGLES {
if (attachmentObject.IsTexture()) {
const auto& textureObject = attachmentObject.GetTexture();
SharedPtr<TextureImpl::BackendTextureObject> backendTextureObject;
const auto& backendTextureIt = TextureImpl::g_backendTextureObjects.find(textureObject.get());
if (backendTextureIt == TextureImpl::g_backendTextureObjects.end()) {
auto& backendTextureSlot = TextureImpl::g_backendTextureObjects.GetOrCreate(textureObject);
if (!backendTextureSlot) {
backendTextureSlot = MakeShared<TextureImpl::BackendTextureObject>();
}
backendTextureObject = backendTextureSlot;
if (auto* backendTextureSlot = TextureImpl::g_backendTextureObjects.Find(textureObject.get())) {
backendTextureObject = *backendTextureSlot;
} else {
backendTextureObject = backendTextureIt->second;
auto& newTextureSlot = TextureImpl::g_backendTextureObjects.GetOrCreate(textureObject);
if (!newTextureSlot) {
newTextureSlot = MakeShared<TextureImpl::BackendTextureObject>();
}
backendTextureObject = newTextureSlot;
}
if (!backendTextureObject) {
MGLOG_E("%s: No backend texture found for FBO attachment, cannot bind texture.", __func__);
@@ -2818,18 +2990,17 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
} else if (attachmentObject.IsRenderbuffer()) {
const auto& renderbufferObject = attachmentObject.GetRenderbuffer();
const auto& backendRenderbufferIt =
RenderbufferImpl::g_backendRenderbufferObjects.find(renderbufferObject.get());
SharedPtr<RenderbufferImpl::BackendRenderbufferObject> backendRenderbufferObject;
if (backendRenderbufferIt == RenderbufferImpl::g_backendRenderbufferObjects.end()) {
auto& backendRenderbufferSlot =
RenderbufferImpl::g_backendRenderbufferObjects.GetOrCreate(renderbufferObject);
if (!backendRenderbufferSlot) {
backendRenderbufferSlot = MakeShared<RenderbufferImpl::BackendRenderbufferObject>();
}
backendRenderbufferObject = backendRenderbufferSlot;
if (auto* backendRenderbufferSlot =
RenderbufferImpl::g_backendRenderbufferObjects.Find(renderbufferObject.get())) {
backendRenderbufferObject = *backendRenderbufferSlot;
} else {
backendRenderbufferObject = backendRenderbufferIt->second;
auto& newRenderbufferSlot =
RenderbufferImpl::g_backendRenderbufferObjects.GetOrCreate(renderbufferObject);
if (!newRenderbufferSlot) {
newRenderbufferSlot = MakeShared<RenderbufferImpl::BackendRenderbufferObject>();
}
backendRenderbufferObject = newRenderbufferSlot;
}
backendRenderbufferObject->SyncToBackend(renderbufferObject);
@@ -3189,10 +3360,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
// Verify that the backend object's name and parameters match the frontend attachment state
if (attachmentObject.IsTexture()) {
const auto& textureObject = attachmentObject.GetTexture();
auto backendTextureIt = TextureImpl::g_backendTextureObjects.find(textureObject.get());
MOBILEGL_ASSERT(backendTextureIt != TextureImpl::g_backendTextureObjects.end(),
auto* backendTextureSlot = TextureImpl::g_backendTextureObjects.Find(textureObject.get());
MOBILEGL_ASSERT(backendTextureSlot != nullptr && *backendTextureSlot != nullptr,
"No backend texture found while framebuffer reports texture attachment.");
GLuint backendTexId = backendTextureIt->second->GetBackendTextureId();
GLuint backendTexId = (*backendTextureSlot)->GetBackendTextureId();
MOBILEGL_ASSERT(static_cast<GLint>(backendTexId) == objectName,
"Attachment texture name mismatch between GLES (%d) and backend texture object "
"(%d), frontend texture object ID=%d.",
@@ -3205,12 +3376,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
"Attachment texture level mismatch between GLES and state object.");
} else if (attachmentObject.IsRenderbuffer()) {
const auto& renderbufferObject = attachmentObject.GetRenderbuffer();
auto backendRboIt =
RenderbufferImpl::g_backendRenderbufferObjects.find(renderbufferObject.get());
auto* backendRboSlot =
RenderbufferImpl::g_backendRenderbufferObjects.Find(renderbufferObject.get());
MOBILEGL_ASSERT(
backendRboIt != RenderbufferImpl::g_backendRenderbufferObjects.end(),
backendRboSlot != nullptr && *backendRboSlot != nullptr,
"No backend renderbuffer found while framebuffer reports renderbuffer attachment.");
GLuint backendRboId = backendRboIt->second->GetBackendRenderbufferId();
GLuint backendRboId = (*backendRboSlot)->GetBackendRenderbufferId();
MOBILEGL_ASSERT(static_cast<GLint>(backendRboId) == objectName,
"Attachment renderbuffer name mismatch between GLES and state object.");
}
@@ -3236,7 +3407,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
StateBackendObjectRegistry<MG_State::GLState::FramebufferObject, BackendFramebufferObject>
g_backendFramebufferObjects;
Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboBindVersions = {0};
Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedSlotVersions = {0};
// Tracks the bound FBO's object version (bumped on any attachment/drawbuffer change)
// per target: re-attaching textures or changing draw buffers on an already-bound FBO
// must re-sync it even when the binding-slot version has not moved.
@@ -3588,6 +3759,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
MGLOG_D("Syncing program to backend. State program ID: %u, Backend ID: %u",
stateProgramObject->GetExternalIndex(), m_backendProgramId);
// Every link-derived cache below (incl. m_samplerUniformBindings and its
// lastAssignedUnit/lastAssignedLodBias program-state mirrors) is rebuilt;
// the sampler-pass memo keyed on them must not survive.
m_samplerPassMemo.valid = false;
m_backendProgramUsable = true;
m_snormFallbackClampOutputMask = g_snormFallbackClampOutputMask;
m_unormFallbackClampOutputMask = g_unormFallbackClampOutputMask;
+252 -40
View File
@@ -27,39 +27,54 @@ namespace MobileGL::MG_Backend::DirectGLES {
using StatePtr = SharedPtr<StateObject>;
using StateWeakPtr = std::weak_ptr<StateObject>;
using BackendPtr = SharedPtr<BackendObject>;
using BackendMap = UnorderedMap<StateObject*, BackendPtr>;
using StateRefMap = UnorderedMap<StateObject*, StateWeakPtr>;
// The backend twin and the weak reference that decides whether the raw key still
// names the state object the twin was built for. Both live in one entry: a
// separate liveness map answered nothing the backend probe had not already found
// and cost a second hash lookup on every Find, which the draw path runs ~10 times.
struct Entry {
BackendPtr backend;
StateWeakPtr stateRef;
};
using BackendMap = UnorderedMap<StateObject*, Entry>;
using iterator = typename BackendMap::iterator;
using const_iterator = typename BackendMap::const_iterator;
BackendPtr& GetOrCreate(const StatePtr& stateObj) {
MOBILEGL_ASSERT(stateObj != nullptr, "State object must not be null");
auto* key = stateObj.get();
auto trackedStateIt = m_stateRefs.find(key);
if (trackedStateIt != m_stateRefs.end() && trackedStateIt->second.expired()) {
EraseByKey(key);
auto& entry = m_entries[stateObj.get()];
if (entry.stateRef.expired()) {
// The previous owner of this address is gone and the allocator handed it
// to a new object: its twin describes ids the new state object never made.
entry.backend.reset();
}
m_stateRefs[key] = stateObj;
return m_backendObjects[key];
entry.stateRef = stateObj;
return entry.backend;
}
iterator find(StateObject* stateObj) {
if (!IsAlive(stateObj)) {
EraseByKey(stateObj);
return m_backendObjects.end();
// Null when no live state object owns this key. The result points into the map, so
// it stays valid only until the next GetOrCreate/Find/CollectGarbage on this registry.
BackendPtr* Find(StateObject* stateObj) {
const auto entryIt = m_entries.find(stateObj);
if (entryIt == m_entries.end()) {
return nullptr;
}
return m_backendObjects.find(stateObj);
if (entryIt->second.stateRef.expired()) {
m_entries.erase(entryIt);
return nullptr;
}
return &entryIt->second.backend;
}
const_iterator find(StateObject* stateObj) const {
return const_cast<StateBackendObjectRegistry*>(this)->find(stateObj);
const BackendPtr* Find(StateObject* stateObj) const {
return const_cast<StateBackendObjectRegistry*>(this)->Find(stateObj);
}
iterator begin() { return m_backendObjects.begin(); }
const_iterator begin() const { return m_backendObjects.begin(); }
iterator end() { return m_backendObjects.end(); }
const_iterator end() const { return m_backendObjects.end(); }
iterator begin() { return m_entries.begin(); }
const_iterator begin() const { return m_entries.begin(); }
iterator end() { return m_entries.end(); }
const_iterator end() const { return m_entries.end(); }
void CollectGarbageIfNeeded() {
++m_gcTick;
@@ -73,19 +88,6 @@ namespace MobileGL::MG_Backend::DirectGLES {
void CollectGarbageNow() { CollectGarbage(); }
private:
bool IsAlive(StateObject* stateObj) const {
const auto trackedStateIt = m_stateRefs.find(stateObj);
if (trackedStateIt == m_stateRefs.end()) {
return false;
}
return !trackedStateIt->second.expired();
}
void EraseByKey(StateObject* stateObj) {
m_stateRefs.erase(stateObj);
m_backendObjects.erase(stateObj);
}
void CollectGarbage() {
if (m_isCollecting) {
return;
@@ -94,16 +96,15 @@ namespace MobileGL::MG_Backend::DirectGLES {
m_isCollecting = true;
Vector<StateObject*> staleKeys;
staleKeys.reserve(m_stateRefs.size());
for (const auto& [stateKey, stateWeakRef] : m_stateRefs) {
if (stateWeakRef.expired()) {
staleKeys.reserve(m_entries.size());
for (const auto& [stateKey, entry] : m_entries) {
if (entry.stateRef.expired()) {
staleKeys.push_back(stateKey);
}
}
for (auto* stateKey : staleKeys) {
m_stateRefs.erase(stateKey);
m_backendObjects.erase(stateKey);
m_entries.erase(stateKey);
}
m_isCollecting = false;
@@ -111,8 +112,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
private:
static constexpr Uint32 kGCInterval = 1024;
StateRefMap m_stateRefs;
BackendMap m_backendObjects;
BackendMap m_entries;
Uint32 m_gcTick = 0;
Bool m_isCollecting = false;
};
@@ -120,6 +120,43 @@ namespace MobileGL::MG_Backend::DirectGLES {
namespace BufferImpl {
const GLenum TempBufferTarget = GL_ARRAY_BUFFER;
// --- Buffer-mutation epoch -------------------------------------------------
// Manager-wide monotonic counter: it moves whenever ANY buffer resource may
// have gone from draw-clean to dirty. Draw-path memos read it once per pass
// (CurrentBufferMutationEpoch, acquire), re-run their IsBufferDrawClean
// probes only when it moved, and stamp the PRE-pass value after a pass in
// which every probe came up clean - so a concurrent bump lands strictly
// after the stamped value and forces a re-probe on the next pass no matter
// how the probe interleaved with the mutation. Conservative-correct: a bump
// never skips work, it only re-runs the probes once.
//
// Every clean->dirty transition path bumps it (BumpBufferMutationEpoch,
// release, AFTER the mutation lands so an acquire reader that still sees
// the old epoch cannot have missed the mutation):
// * the frontend BufferBackendOps table - Respecify, SubData,
// FlushMappedRange, AcquirePersistentMap, ReadbackFromGpu, OnDestroy -
// which every frontend change-serial bump and every pending-range
// queueing reaches while ops are registered (upload, orphan/respecify,
// map flush/unmap writeback, persistent-map adoption, delete/pooling);
// * backend-initiated shadow writebacks that bump the frontend change
// serial without an op: transform-feedback capture readback
// (XfbImpl::ReadbackCapturedRanges and the scatter path) and every
// pack-PBO WritebackFromBackend site (glReadPixels/glGetTexImage);
// * RegisterBufferBackendOps/UnregisterBufferBackendOps - while ops are
// unregistered, frontend writes advance serials silently, so both edges
// of that window re-open every memo;
// * OnBackendContextDestroyed - the buffer context generation moved, so
// every previously clean resource is invalid.
// NOT bumped (cleanliness provably unchanged): MarkGpuWritten (the backend
// copy is authoritative; IsBufferDrawClean does not consult it),
// NotifyContentWrite on a GPU-resident buffer (persistent-mapped resources
// are clean by construction), and EnsureBufferResource itself (it only
// repairs toward clean). A non-persistent map (draws on it are GL errors
// the frontend rejects) sets IsMapped without an op; persistent maps reach
// AcquirePersistentMap or (FLUSH_EXPLICIT) publish only via FlushMappedRange.
Uint64 CurrentBufferMutationEpoch();
void BumpBufferMutationEpoch();
// The DirectGLES storage behind one frontend buffer. Owned (refcounted) by
// the frontend BufferObject; immediate BufferBackendOps keep it current, so
// draw-time "sync" reduces to ensuring the storage exists.
@@ -145,6 +182,12 @@ namespace MobileGL::MG_Backend::DirectGLES {
Bool pendingRespecify = false;
VecRange1D pendingRanges;
std::mutex pendingMutex;
// Buffer-mutation epoch (see CurrentBufferMutationEpoch) at which this
// resource last probed IsBufferDrawClean == true, 0 = never (epochs start
// at 1). Written only on the draw thread; per-draw resource consumers
// (the UBO binding walk) skip the probe while their pre-pass epoch read
// matches, exactly like the per-VAO memo stamps.
Uint64 drawCleanEpoch = 0;
// Zero-copy coherent persistent map (EXT_buffer_storage): the GL store is
// immutable, persistently+coherently mapped, and persistentPtr is what the app
// (and the frontend PipeResource) write into directly. While set, draw-time
@@ -170,6 +213,17 @@ namespace MobileGL::MG_Backend::DirectGLES {
GLESBufferResource* EnsureBufferResource(const SharedPtr<MG_State::GLState::BufferObject>& bufferObject);
// Existing resource or nullptr; performs no GL calls.
GLESBufferResource* GetBufferResource(MG_State::GLState::BufferObject* bufferObject);
// True when EnsureBufferResource(frontend) would provably fall straight through
// every branch and do no work — i.e. `resource` is still the frontend's own
// resource, its id belongs to the live ES context, and either it is the
// zero-copy coherent persistent store (draw-time sync is a no-op by design) or
// the storage is initialized at the right size with no pending ops and a synced
// change serial while the buffer is not mapped (an active map may owe a
// per-draw persistent-range push, so it always takes the full path).
// `frontend` must be non-null and alive; the caller guarantees that by holding
// (or shadowing something that holds) a SharedPtr to it. Enables the per-VAO
// resolved-buffers memo to skip EnsureBufferResource on clean static buffers.
Bool IsBufferDrawClean(const MG_State::GLState::BufferObject* frontend, const GLESBufferResource* resource);
// Deletes GL buffers whose owning frontend objects died (possibly on a
// thread without a current ES context). Called from draw-time sync.
@@ -255,11 +309,70 @@ namespace MobileGL::MG_Backend::DirectGLES {
Uint GetBackendVertexArrayId() const { return m_backendVAOId; }
void Bind() const;
// Draw-path memo of SyncNeccessaryBuffers' attribute walk for this VAO: the
// distinct enabled-attribute buffers (deduped) and the index buffer, resolved
// to their backend resources once. Valid while the VAO's config version is
// unchanged — every attach/enable/disable/format mutation bumps it (the same
// invariant SyncToBackend's gate already leans on), and the VAO's attribute
// SharedPtrs pin each memoed frontend buffer for exactly that long, so the raw
// pointers cannot dangle on a hit. Per-buffer cleanliness is NOT memoed here:
// each hit re-checks IsBufferDrawClean (resource identity, context generation,
// pending ops, change serial) and falls back to EnsureBufferResource for just
// the dirty entries via their attribute index. The IBO entry is keyed on the
// slot's bound-object identity instead (its slot version is a wrapping Uint16
// and is not covered by the config version).
struct ResolvedDrawBuffers {
struct Entry {
MG_State::GLState::BufferObject* frontend = nullptr;
BufferImpl::GLESBufferResource* resource = nullptr;
Uint8 attribIndex = 0;
};
Bool valid = false;
Uint32 configVersion = 0;
Uint count = 0;
Array<Entry, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> entries;
MG_State::GLState::BufferObject* iboFrontend = nullptr;
BufferImpl::GLESBufferResource* iboResource = nullptr;
// Buffer-mutation epoch (BufferImpl::CurrentBufferMutationEpoch) at which
// the LAST probe pass found every entry / the IBO clean; 0 = not stamped
// (epochs start at 1). While a stamp matches the pre-pass epoch read, the
// probes are skipped outright: any path that can dirty ANY buffer bumps
// the epoch (the exhaustive site list lives at the epoch declaration).
// The IBO stamp is only trusted together with the bound-object identity
// compare - the VAO's index slot can rebind with no epoch or config move.
Uint64 vboCleanEpoch = 0;
Uint64 iboCleanEpoch = 0;
};
ResolvedDrawBuffers& GetResolvedDrawBuffersMemo() { return m_resolvedDrawBuffers; }
// Memo for SyncCurrentVertexAttributeValues: which of a program's ACTIVE
// attribute locations lack an enabled array in this VAO (those read the
// context's current generic value instead of a buffer). Keyed on the VAO
// config version (enable/disable bumps it) and the program's active-location
// mask. Hosted per twin — the former function-static single entry missed on
// every draw once the app cycled VAOs, re-reading the cold attribute slots.
struct PendingAttribValueMask {
Bool valid = false;
Uint32 configVersion = 0;
Uint32 activeMask = 0;
Uint32 pendingMask = 0;
};
PendingAttribValueMask& GetPendingAttribValueMaskMemo() { return m_pendingAttribValueMask; }
private:
ResolvedDrawBuffers m_resolvedDrawBuffers;
PendingAttribValueMask m_pendingAttribValueMask;
Uint m_backendVAOId = 0;
Array<Uint, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS> m_clientAttributeBufferIds;
Bool m_isInitialized = false;
Uint16 m_syncedIndexBufferVersion = 0;
// Aggregate gate over the per-attribute walk below: the frontend bumps its config
// version on every per-attribute version bump (the three Bump*Version functions are
// its only writers), so an unchanged config version proves every per-attribute
// compare in SyncToBackend would come up clean. The index-buffer slot has its own
// version and is NOT covered. The Bool (not a sentinel value) marks "never synced".
Bool m_hasSyncedConfigVersion = false;
Uint32 m_syncedConfigVersion = 0;
Array<MG_State::GLState::VertexAttributeVersion, MG_State::GLState::VertexArrayObject::MAX_VERTEX_ATTRIBS>
m_syncedAttributeVersions;
};
@@ -373,6 +486,38 @@ namespace MobileGL::MG_Backend::DirectGLES {
void Bind(GLenum target, Uint unit = TempTextureUnit);
Uint GetBackendTextureId() const;
// Aggregate first-level clean gate for the per-draw trio
// SyncTextureParamsToBackend + SyncBuiltinSamplerToBackend +
// SyncMipmapsToBackend: EXACTLY the conjunction of their own early-outs
// (params version == synced params version; builtin-sampler version ==
// synced sampler version; and SyncMipmapsToBackend's cheap gate - stamped
// trio + content version + Mipmap storage). True means each of the three
// would provably return without work, so the caller may skip the calls;
// false only falls through to the three calls, whose own gates re-decide
// individually - this gate must never be MORE permissive than they are.
// `contextId`/`samplingGeneration` are the frontend context's current
// values, hoisted by the caller so a per-draw list walk reads them once
// instead of per texture. `t` must be the live frontend texture.
Bool IsDrawSyncClean(const MG_State::GLState::ITextureObject* t, Uint64 contextId,
Uint64 samplingGeneration) const {
if (!m_isInitialized || m_syncedShapeContextId == 0 || m_syncedShapeContextId != contextId ||
m_syncedShapeGeneration != samplingGeneration) {
return false;
}
const Uint16 paramsVersion = t->GetTextureParamsVersion();
if (m_syncedShapeParamsVersion != paramsVersion || m_syncedTextureParamsVersion != paramsVersion) {
return false;
}
if (m_syncedContentVersion == 0 || m_syncedContentVersion != t->GetContentVersion()) {
return false;
}
const auto& samplerObject = t->GetSamplerObject();
if (!samplerObject || m_syncedSamplerVersion != samplerObject->GetVersion()) {
return false;
}
return t->GetStorageType() == TextureStorageType::Mipmap;
}
private:
void RecreateBackendTexture();
@@ -388,6 +533,21 @@ namespace MobileGL::MG_Backend::DirectGLES {
// clean probe compares this before rebuilding shape info and scanning
// per-level dirty flags; 0 never matches a real version (they start at 1).
Uint64 m_syncedContentVersion = 0;
// First-level clean gate for SyncMipmapsToBackend, checked before even the
// IsComplete()/shape-probe walk. Valid only as a trio with the content and
// texture-params versions: the context's sampling-resolution generation moves on
// EVERY texture-shape mutation (BumpShapeVersion is the only writer of shape and
// unconditionally bumps it), the content version on every CPU pixel mutation, and
// the params version covers SetSamples/SetFixedSampleLocations, which bump neither
// of the other two but feed the shape probe. The context id pins the generation to
// the context that produced it - generations restart at 0 with a new context, and a
// texture is owned by exactly one context (share groups are not implemented), so a
// mutation can never happen under a context this key does not name. 0 = never
// stamped (real context ids start at 1). Backend-side invalidation rides on
// m_isInitialized: RequireImageBindableStorage and RecreateBackendTexture clear it.
Uint64 m_syncedShapeContextId = 0;
Uint64 m_syncedShapeGeneration = 0;
Uint16 m_syncedShapeParamsVersion = 0;
SamplerParameters m_cacheSamplerParameters;
UintVec2 m_cacheLodRange = {0, 1000};
FloatVec4 m_cacheBorderColor = {0.0f, 0.0f, 0.0f, 0.0f};
@@ -484,11 +644,20 @@ namespace MobileGL::MG_Backend::DirectGLES {
// has to apply the clamp itself.
Bool IsFixedPointFallbackReadAttachment();
extern Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboBindVersions;
// What SyncCurrentFBO last pushed for each target, as a (binding, object, revision)
// triple; it re-syncs unless all three still match. Stamped by SyncCurrentFBO and
// ForceBindCurrentFBO, cleared by InvalidateFramebufferBindingCache. The three are
// only meaningful together - see SyncCurrentFBO.
//
// The binding slot's own version, which changes whenever a different object is bound
// to this target. Distinguishes a rebind from an in-place edit, and keeps the raw
// pointer below from matching an address the allocator recycled for a new FBO.
extern Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedSlotVersions;
// Tracks the bound FBO's object version (bumped on any attachment/drawbuffer change)
// per target: re-attaching textures or changing draw buffers on an already-bound FBO
// must re-sync it even when the binding-slot version has not moved.
extern Array<Uint16, SizeT(FramebufferTarget::FramebufferTargetCount)> g_fboSyncedObjectVersions;
// Which object was synced. Raw and never dereferenced: only compared for identity.
extern Array<MG_State::GLState::FramebufferObject*, SizeT(FramebufferTarget::FramebufferTargetCount)>
g_fboSyncedObjects;
@@ -584,6 +753,10 @@ namespace MobileGL::MG_Backend::DirectGLES {
void InvalidatePackStateCache();
} // namespace PixelStoreImpl
namespace SamplerImpl {
class BackendSamplerObject; // for PrgramImpl's sampler-pass memo rows below
}
// Image uniforms take their unit from the layout(binding=N) qualifier baked into
// the transpiled ESSL; unlike samplers they must not (and in ES cannot) be
// assigned through glUniform1i.
@@ -630,6 +803,43 @@ namespace MobileGL::MG_Backend::DirectGLES {
Float lastAssignedLodBias = 0.0f;
};
// Memo of the whole per-draw sampler-uniform pass (glUniform1i unit
// assignments, lod-bias uniform, raw-depth-fetch substitution and the
// per-unit sampler-object binds) in BindCurrentProgramWithResources.
// The pass is a pure function of the keys below, and its only driver-side
// effect is the sampler binding of each sampled unit, so replaying it as
// "do nothing" additionally requires those bindings to still be on the
// driver - the per-entry row compare against g_boundSamplersCache (the
// shadow every sampler bind in this backend already routes through).
//
// Invalidation enumeration:
// * sampler-uniform unit assignment (glUniform1i) and uniform-block
// binding edits -> frontend backendStateVersion;
// * any texture/sampler bind moving on any unit (incl. the high-water
// mark moving) -> unitBindingsEpoch;
// * any sampler parameter (incl. lod bias, compare mode) or texture
// shape/format change -> samplingGeneration;
// * another frontend context -> contextId (never-reused id);
// * ES context recreation -> textureContextGeneration;
// * relink / backend program rebuild -> SyncToBackend resets `valid`
// (it rebuilds m_samplerUniformBindings, whose lastAssignedUnit /
// lastAssignedLodBias dedup state this memo leans on);
// * any other writer moving a sampled unit's sampler binding
// (BindCurrentUnitSamplers on a unit-sampler change, scratch binds)
// -> the row snapshot compare.
struct SamplerPassMemo {
static constexpr SizeT kMaxEntries = 16;
Bool valid = false;
Uint8 count = 0;
Uint64 contextId = 0;
Uint64 unitBindingsEpoch = 0;
Uint64 samplingGeneration = 0;
Uint32 backendStateVersion = 0;
Uint textureContextGeneration = 0;
Array<Uint8, kMaxEntries> units{};
Array<SamplerImpl::BackendSamplerObject*, kMaxEntries> rows{};
};
BackendProgramObjectImpl();
~BackendProgramObjectImpl();
void SyncToBackend(const SharedPtr<MG_State::GLState::ProgramObject>& stateProgramObject);
@@ -658,6 +868,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
// reflected size when the transpiled block pads differently).
Int GetGlobalUboBackendBlockSize() const { return m_globalUboBackendBlockSize; }
BufferImpl::UboRingAllocation& GetGlobalUboRingAllocation() { return m_globalUboRingAllocation; }
SamplerPassMemo& GetSamplerPassMemo() { return m_samplerPassMemo; }
// Frontend link version this backend program (and its resource caches) was
// built from; a mismatch means every link-derived cache here is stale.
Uint32 GetSyncedLinkVersion() const { return m_syncedLinkVersion; }
@@ -686,6 +897,7 @@ namespace MobileGL::MG_Backend::DirectGLES {
Uint32 m_lastUploadedGlobalUboVersion = ~0u;
BufferImpl::UboRingAllocation m_globalUboRingAllocation;
Uint32 m_syncedLinkVersion = ~0u;
SamplerPassMemo m_samplerPassMemo;
};
extern Uint32 g_snormFallbackClampOutputMask;
+5
View File
@@ -1107,6 +1107,11 @@ namespace MobileGL::MG_Backend::DirectGLES {
}
}
}
if (pixelPackBufferObject) {
// WritebackFromBackend bumps change serials with no backend op; re-open
// the buffer draw-clean memos (once for the whole row loop).
BufferImpl::BumpBufferMutationEpoch();
}
return true;
}
} // namespace ReadbackImpl
@@ -16,6 +16,7 @@
#include "MG_Util/Metrics/TextureMetrics.h"
#include "MG_Util/Miscellany/IndexGenerator.h"
#include <atomic>
#include <bit>
#include <cstring>
#include <spirv_reflect.h>
@@ -1450,6 +1451,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
payload.mode = mode;
payload.indexBufferView.indexType = type;
// Loop-invariant: the index type is fixed for the whole multi-draw, so resolve
// its byte size once instead of twice per sub-draw (a cross-TU switch that
// showed up in per-frame profiles of sodium-style 132x32 multi-draws). Index
// sizes are 1/2/4, so the per-sub-draw offset division below reduces to a
// shift - the hardware divide was the hottest instruction of this loop.
const SizeT indexSize = MG_Util::GetGLTypeSize(type);
if (indexSize == 0) {
MGLOG_E("MultiDrawElementsBaseVertex skipped: unsupported index type 0x%x", type);
return;
}
const Uint32 indexSizeShift = static_cast<Uint32>(std::countr_zero(indexSize));
// TODO: allocate draw cmd buf elsewhere
static Vector<DrawIndexedCmdParam> params;
params.clear();
@@ -1464,14 +1477,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
payload.indexBufferView.indexByteOffset = 0;
payload.indexBufferView.indexByteSize =
std::max(reinterpret_cast<SizeT>(indices[i]) + count[i] * MG_Util::GetGLTypeSize(type),
std::max(reinterpret_cast<SizeT>(indices[i]) + count[i] * indexSize,
payload.indexBufferView.indexByteSize);
auto& param = params[i];
param.indexCount = count[i];
param.instanceCount = 1;
param.firstIndex = reinterpret_cast<SizeT>(indices[i]) / MG_Util::GetGLTypeSize(type);
param.firstIndex = reinterpret_cast<SizeT>(indices[i]) >> indexSizeShift;
param.vertexOffset = basevertex[i];
param.firstInstance = 0;
}
@@ -2379,6 +2379,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return it->second;
}
// Structural change: the insert below can move every entry of this
// open-addressing map, so all memoised entry pointers die here.
++m_cacheStructureEpoch;
auto& entry = m_cache[hash];
entry.hash = hash;
entry.lastUsedFrame = m_frameCounter;
@@ -2580,6 +2583,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// erase runs ~VkProgramObject (modules/layouts destroyed); notify after
// so an observer never observes a half-destroyed entry through a lookup.
// Observers only need the handle values to purge their keyed caches.
++m_cacheStructureEpoch; // erase moves/kills entries: memoised pointers die
it = m_cache.erase(it);
if (m_evictionObserver != nullptr) {
m_evictionObserver->OnProgramEvicted(hash, descriptorSetLayout);
@@ -105,8 +105,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// position-invariance quirk (see PipelineFactory::ShouldSuppressDepthWrite).
Bool fragmentReplacesDepth = false;
// Frame-boundary counter value of the last GetOrCreateProgram hit; drives
// cache eviction (see OnFrameBoundary).
Uint64 lastUsedFrame = 0;
// cache eviction (see OnFrameBoundary). Mutable: the draw snapshot's memoised
// entry pointer re-stamps use through a const reference (StampProgramUse).
mutable Uint64 lastUsedFrame = 0;
static inline VkDevice s_device = VK_NULL_HANDLE;
@@ -262,6 +263,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const VkProgramObject& GetOrCreateProgram(
const MG_State::GLState::ProgramObject& program, CompileOptionFlags flags);
// Bumped whenever m_cache's STRUCTURE changes (any insert or erase): the cache is
// an open-addressing map holding entries by value, so both moves existing entries.
// A caller that memoised a VkProgramObject* may keep dereferencing it only while
// this is unchanged; on a bump it must re-run GetOrCreateProgram.
Uint64 GetCacheStructureEpoch() const { return m_cacheStructureEpoch; }
// A memoised entry pointer bypasses GetOrCreateProgram, whose per-lookup stamp is
// what keeps an in-use entry out of OnFrameBoundary's idle sweep - so such a
// caller must re-stamp the entry itself, at least once per frame boundary.
void StampProgramUse(const VkProgramObject& entry) const { entry.lastUsedFrame = m_frameCounter; }
// Observer may be null (no notifications). Not owned.
void SetEvictionObserver(IEvictionObserver* observer) { m_evictionObserver = observer; }
// Frame boundary hook: ages the program cache and evicts long-unused entries
@@ -312,6 +323,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
mutable ProgramLookupCache m_lastLookup;
// Monotonic frame-boundary counter (bumped in OnFrameBoundary) for cache aging.
Uint64 m_frameCounter = 0;
// See GetCacheStructureEpoch(). Starts at 1 so a zero-initialized memo can never match.
Uint64 m_cacheStructureEpoch = 1;
IEvictionObserver* m_evictionObserver = nullptr;
static inline XXH64_state_t* m_hashState = XXH64_createState();
};
@@ -207,11 +207,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (auto& entry : m_descriptorReuseMemo) {
entry.valid = false;
}
m_fastRebindMemo.valid = false;
m_lastBindValid = false;
// Re-fingerprint the bound sampler set fresh this frame so any GL object address
// reuse cannot outlive a single frame (see SamplerResolveMemo).
for (auto& memo : m_samplerResolveMemo) {
memo.valid = false;
memo.infoValid = false;
}
}
@@ -247,6 +249,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (auto& entry : m_descriptorReuseMemo) {
entry.valid = false;
}
// The rebind memo's set may be among the freed ones.
m_fastRebindMemo.valid = false;
MGLOG_D("UniformDescriptorBinder: freed %zu descriptor sets for destroyed layout", purgedSets);
}
}
@@ -254,9 +258,19 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool UniformManager::ResolveSamplerDescriptor(VkCommandBuffer commandBuffer,
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, VkDescriptorImageInfo& outImageInfo) const {
Uint32 binding, VkDescriptorImageInfo& outImageInfo,
Bool trustUnchangedHint) const {
MOBILEGL_ASSERT(m_textureManager != nullptr, "ResolveSamplerDescriptor: texture manager is null");
MOBILEGL_ASSERT(m_samplerManager != nullptr, "ResolveSamplerDescriptor: sampler manager is null");
// The caller proved every input of this binding's resolution unchanged since the
// last full resolve (which also filled the cache), so the whole chain below -
// texture/sampler resolution, completeness probe, sync, layout handling, sampler
// and view lookups - would recompute the identical descriptor.
if (trustUnchangedHint && binding < m_samplerResolveMemo.size() &&
m_samplerResolveMemo[binding].infoValid) {
outImageInfo = m_samplerResolveMemo[binding].info;
return true;
}
MOBILEGL_ASSERT(binding < programObj.samplerNameByBinding.size(),
"ResolveSamplerDescriptor: sampler binding %u name lookup out of range", binding);
// Raw-pointer resolve to skip the SharedPtr atomic refcount churn: the bound texture stays
@@ -430,7 +444,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
.imageView = sampledImageView,
.imageLayout = resource->layout,
};
return outImageInfo.sampler != VK_NULL_HANDLE;
if (outImageInfo.sampler == VK_NULL_HANDLE) {
return false;
}
if (binding < m_samplerResolveMemo.size()) {
m_samplerResolveMemo[binding].info = outImageInfo;
m_samplerResolveMemo[binding].infoValid = true;
}
return true;
}
Bool UniformManager::ResolveSamplerDescriptorOverride(
@@ -808,10 +829,55 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return m_fallbackTexture2D;
}
Bool UniformManager::ResolveSampledBinding(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding,
MG_State::GLState::ITextureObject*& outTexture,
const MG_State::GLState::SamplerObject*& outSampler) const {
// Open-coded ResolveSamplerTextureRaw so the unit is resolved once for both the
// texture and the sampler override - this runs per binding per full-path draw,
// and program-alternating draw streams take the full path on every draw.
MOBILEGL_ASSERT(MG_State::pGLContext != nullptr, "ResolveSampledBinding: GL context is null");
MOBILEGL_ASSERT(binding < programObj.samplerUniformLocationByBinding.size(),
"ResolveSampledBinding: sampler location binding %u out of range", binding);
MOBILEGL_ASSERT(binding < programObj.samplerTextureTargetByBinding.size(),
"ResolveSampledBinding: sampler target binding %u out of range", binding);
const Int location = programObj.samplerUniformLocationByBinding[binding];
const Int unit = ResolveSamplerUnitIndex(program, location, binding);
auto& textureUnit = MG_State::pGLContext->GetTextureUnitObject(unit);
const TextureTarget preferredTarget = programObj.samplerTextureTargetByBinding[binding];
MG_State::GLState::ITextureObject* texture =
textureUnit.GetBindingSlot(preferredTarget).GetBoundObject().get();
// Undefined default texture (name 0, no image) resolves as "unbound", exactly
// like ResolveSamplerTextureRaw reports it.
if (MG_State::GLState::IsUndefinedDefaultTexture(texture)) {
texture = nullptr;
}
if (texture == nullptr) {
// ResolveSamplerDescriptor will substitute the fallback texture for this binding;
// include it in the sampled set so the pre-render-pass sync/transition pass covers
// its first use instead of leaving that work to happen inside an active pass.
if (preferredTarget != TextureTarget::Texture2D &&
preferredTarget != TextureTarget::TextureRectangle) {
return false;
}
texture = GetFallbackTexture(preferredTarget).get();
}
const auto& samplerOverride = textureUnit.GetSamplerObject();
outTexture = texture;
outSampler = samplerOverride ? samplerOverride.get()
: (texture != nullptr ? texture->GetSamplerObject().get() : nullptr);
return true;
}
Bool UniformManager::CollectSampledTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Vector<MG_State::GLState::ITextureObject*>& outTextures) {
Vector<MG_State::GLState::ITextureObject*>& outTextures,
Vector<SampledBindingRecord>* outBindingRecords) {
outTextures.clear();
if (outBindingRecords != nullptr) {
outBindingRecords->clear();
}
const Uint32 bindingCount =
std::min<Uint32>(m_maxBindings, static_cast<Uint32>(programObj.bindingKinds.size()));
@@ -820,17 +886,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
continue;
}
MG_State::GLState::ITextureObject* texture = ResolveSamplerTextureRaw(program, programObj, binding);
if (!texture) {
// ResolveSamplerDescriptor will substitute the fallback texture for this binding;
// include it in the sampled set so the pre-render-pass sync/transition pass covers
// its first use instead of leaving that work to happen inside an active pass.
const TextureTarget preferredTarget = programObj.samplerTextureTargetByBinding[binding];
if (preferredTarget != TextureTarget::Texture2D &&
preferredTarget != TextureTarget::TextureRectangle) {
continue;
}
texture = GetFallbackTexture(preferredTarget).get();
MG_State::GLState::ITextureObject* texture = nullptr;
const MG_State::GLState::SamplerObject* sampler = nullptr;
if (!ResolveSampledBinding(program, programObj, binding, texture, sampler)) {
continue;
}
if (outBindingRecords != nullptr) {
outBindingRecords->push_back({texture != nullptr ? texture->GetLifetimeId() : 0,
sampler != nullptr ? sampler->GetLifetimeId() : 0});
}
auto found = std::find(outTextures.begin(), outTextures.end(), texture);
@@ -841,6 +904,38 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
Bool UniformManager::SampledBindingsUnchanged(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
const Vector<SampledBindingRecord>& previousRecords) const {
SizeT recordIndex = 0;
// Iterate only the bindings this program declares (ascending), exactly like
// BindProgramUniformBuffers: this runs per draw whenever the texture bind
// generation moved, and walking all m_maxBindings slots to find the 1-8 real
// ones dominated it.
for (const Uint32 binding : programObj.activeBindings) {
if (binding >= m_maxBindings) {
break; // ascending, so nothing past the cap can follow
}
if (programObj.bindingKinds[binding] != ProgramFactory::DescriptorBindingKind::CombinedImageSampler) {
continue;
}
MG_State::GLState::ITextureObject* texture = nullptr;
const MG_State::GLState::SamplerObject* sampler = nullptr;
if (!ResolveSampledBinding(program, programObj, binding, texture, sampler)) {
continue;
}
if (recordIndex >= previousRecords.size()) {
return false;
}
const SampledBindingRecord& record = previousRecords[recordIndex++];
if (record.textureLifetimeId != (texture != nullptr ? texture->GetLifetimeId() : 0) ||
record.samplerLifetimeId != (sampler != nullptr ? sampler->GetLifetimeId() : 0)) {
return false;
}
}
return recordIndex == previousRecords.size();
}
Bool UniformManager::CollectStorageImageTextures(
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
@@ -1155,12 +1250,101 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return VK_SUCCESS;
}
Bool UniformManager::ResolveDynamicUboDescriptor(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 binding, Uint32 arrayElement, Uint32 frameIndex,
VkBuffer& outBuffer, VkDeviceSize& outRange,
Uint32& outDynamicOffset) {
UboBindResult ubo{};
const Bool hasPayload = ResolveUniformBufferPayload(program, programObj, binding, arrayElement, ubo);
MOBILEGL_ASSERT(hasPayload && (ubo.directBindable || (ubo.payload != nullptr && ubo.payloadSize > 0)),
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: missing UBO payload on binding %u element %u",
binding, arrayElement);
if (ubo.directBindable) {
// Zero-copy: bind the app's resident VkBuffer directly, no per-draw memcpy.
outBuffer = ubo.buffer;
outRange = ubo.range;
outDynamicOffset = static_cast<Uint32>(ubo.dynamicOffset);
return true;
}
// Global-UBO slice reuse (see GlobalUboSliceMemo): unchanged
// uniform bytes re-use the slice already uploaded this frame.
const Bool isGlobalUbo = programObj.globalUboBinding == static_cast<Int>(binding) && arrayElement == 0;
const Uint64 uboFrameSerial = m_bufferManager->GetFrameSerial();
const Uint64 uboProgramLifetimeId = program.GetLifetimeId();
const Uint32 uboContentVersion = program.GetUBOContentVersion();
if (isGlobalUbo) {
for (const auto& memo : m_globalUboMemo) {
if (memo.buffer != VK_NULL_HANDLE && memo.programLifetimeId == uboProgramLifetimeId &&
memo.frameSerial == uboFrameSerial && memo.uboContentVersion == uboContentVersion &&
memo.range == static_cast<VkDeviceSize>(ubo.payloadSize)) {
outBuffer = memo.buffer;
outRange = memo.range;
outDynamicOffset = static_cast<Uint32>(memo.offset);
return true;
}
}
}
BufferSlice slice{};
if (!m_bufferManager->UploadTransient(BufferKind::Uniform, frameIndex, ubo.payload, ubo.payloadSize,
m_minDynamicOffsetAlignment, slice)) {
MOBILEGL_ASSERT(false,
"UniformDescriptorBinder::ResolveDynamicUboDescriptor failed: UBO upload failed on binding %u element %u",
binding, arrayElement);
return false;
}
outBuffer = slice.buffer;
outRange = ubo.payloadSize;
outDynamicOffset = static_cast<Uint32>(slice.offset);
if (isGlobalUbo) {
m_globalUboMemo[m_globalUboMemoNext] =
GlobalUboSliceMemo{uboProgramLifetimeId, uboFrameSerial, uboContentVersion,
slice.buffer, slice.offset, static_cast<VkDeviceSize>(ubo.payloadSize)};
m_globalUboMemoNext = (m_globalUboMemoNext + 1) % kGlobalUboMemoSize;
}
return true;
}
void UniformManager::BindDescriptorSetDeduped(VkCommandBuffer commandBuffer, VkPipelineBindPoint bindPoint,
VkPipelineLayout pipelineLayout, VkDescriptorSet descriptorSet,
const Vector<Uint32>& dynamicOffsets) {
// Skip the driver call when this exact binding is already live on the
// command buffer (see the bind-dedup shadow in the header).
const Uint32 offsetCount = static_cast<Uint32>(dynamicOffsets.size());
Bool identicalBind = m_lastBindValid && m_lastBindSet == descriptorSet &&
m_lastBindLayout == pipelineLayout && m_lastBindPoint == bindPoint &&
m_lastBindOffsetCount == offsetCount && offsetCount <= kMaxShadowedDynamicOffsets;
if (identicalBind) {
for (Uint32 i = 0; i < offsetCount; ++i) {
if (m_lastBindOffsets[i] != dynamicOffsets[i]) {
identicalBind = false;
break;
}
}
}
if (!identicalBind) {
vkCmdBindDescriptorSets(commandBuffer, bindPoint, pipelineLayout, 0, 1,
&descriptorSet, offsetCount, dynamicOffsets.data());
if (offsetCount <= kMaxShadowedDynamicOffsets) {
m_lastBindValid = true;
m_lastBindSet = descriptorSet;
m_lastBindLayout = pipelineLayout;
m_lastBindPoint = bindPoint;
m_lastBindOffsetCount = offsetCount;
std::copy_n(dynamicOffsets.data(), offsetCount, m_lastBindOffsets);
} else {
m_lastBindValid = false;
}
}
}
Bool UniformManager::BindProgramUniformBuffers(VkCommandBuffer commandBuffer,
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 frameIndex,
VkPipelineBindPoint bindPoint,
const SamplerBindingOverride* samplerBindingOverride) {
const SamplerBindingOverride* samplerBindingOverride,
Bool samplerDescriptorsUnchangedHint) {
auto& frame = m_frames[frameIndex];
if (frame.descriptorPools.empty()) {
MGLOG_E("UniformDescriptorBinder::BindProgramUniformBuffers failed: frame descriptor pools are invalid");
@@ -1170,6 +1354,34 @@ namespace MobileGL::MG_Backend::DirectVulkan {
frame.activeDescriptorPoolIndex = 0;
}
// Dynamic-offset-only rebind (see FastRebindMemo in the header): the last
// cacheable walk of this exact program selected a set whose contents are
// provably still what this walk would write - the hint covers every
// sampler binding, and an unchanged (buffer, range) for the single
// dynamic UBO covers the rest - except the dynamic offset, which rebinding
// the SAME set delivers without any descriptor write.
const Bool cacheable = (samplerBindingOverride == nullptr);
if (cacheable && samplerDescriptorsUnchangedHint && m_fastRebindMemo.valid &&
m_fastRebindMemo.frameIndex == frameIndex &&
m_fastRebindMemo.programLifetimeId == program.GetLifetimeId() &&
m_fastRebindMemo.programHash == programObj.hash) {
VkBuffer uboBuffer = VK_NULL_HANDLE;
VkDeviceSize uboRange = 0;
Uint32 uboDynamicOffset = 0;
if (ResolveDynamicUboDescriptor(program, programObj, m_fastRebindMemo.uboBinding, 0, frameIndex,
uboBuffer, uboRange, uboDynamicOffset) &&
uboBuffer == m_fastRebindMemo.uboBuffer && uboRange == m_fastRebindMemo.uboRange) {
auto& fastOffsets = m_dynamicOffsetsScratch;
fastOffsets.clear();
fastOffsets.push_back(uboDynamicOffset);
BindDescriptorSetDeduped(commandBuffer, bindPoint, programObj.pipelineLayout,
m_fastRebindMemo.set, fastOffsets);
return true;
}
// Any mismatch (arena wrap or growth, direct-bind retarget, upload
// failure) falls through to the full walk, which re-records the memo.
}
// The descriptor set is chosen AFTER the writes are built (below), so a draw
// whose resolved descriptor content matches the previous draw can reuse that
// set and skip both AcquireDescriptorSet and vkUpdateDescriptorSets.
@@ -1201,6 +1413,14 @@ namespace MobileGL::MG_Backend::DirectVulkan {
texelBufferViews.reserve(m_maxBindings);
dynamicOffsets.reserve(programObj.dynamicBindings.size() + uboArrayExtra);
// Eligibility probe for FastRebindMemo, filled by this walk: exactly one
// dynamic-UBO descriptor (no arrayed elements) and otherwise only
// combined-image samplers, so the whole set's content is pinned by the
// sampler hint plus one (buffer, range) compare.
Uint32 dynamicUboDescriptorCount = 0;
Uint32 fastRebindUboBinding = 0;
Bool fastRebindKindsEligible = true;
// Iterate only the bindings this program declares. The old walk covered all 256 slots of
// bindingKinds on every draw to find the 1-8 a real program uses.
for (const Uint32 binding : programObj.activeBindings) {
@@ -1221,67 +1441,18 @@ namespace MobileGL::MG_Backend::DirectVulkan {
binding < programObj.bindingDescriptorCounts.size()
? std::max<Uint32>(1, programObj.bindingDescriptorCounts[binding])
: 1u;
dynamicUboDescriptorCount += descriptorCount;
fastRebindUboBinding = binding;
const SizeT firstBufferInfoIndex = bufferInfos.size();
for (Uint32 element = 0; element < descriptorCount; ++element) {
UboBindResult ubo{};
const Bool hasPayload =
ResolveUniformBufferPayload(program, programObj, binding, element, ubo);
MOBILEGL_ASSERT(hasPayload && ubo.payload != nullptr && ubo.payloadSize > 0,
"UniformDescriptorBinder::BindProgramUniformBuffers failed: missing UBO payload on binding %u element %u",
binding, element);
VkDescriptorBufferInfo bufferInfo{};
// Keep offset 0 (sub-range selected via the dynamic offset) so the hashed bufferInfo
// is stable across draws and the descriptor-set reuse cache keeps hitting.
bufferInfo.offset = 0;
Uint32 dynOffset;
if (ubo.directBindable) {
// Zero-copy: bind the app's resident VkBuffer directly, no per-draw memcpy.
bufferInfo.buffer = ubo.buffer;
bufferInfo.range = ubo.range;
dynOffset = static_cast<Uint32>(ubo.dynamicOffset);
} else {
// Global-UBO slice reuse (see GlobalUboSliceMemo): unchanged
// uniform bytes re-use the slice already uploaded this frame.
const Bool isGlobalUbo =
programObj.globalUboBinding == static_cast<Int>(binding) && element == 0;
const Uint64 uboFrameSerial = m_bufferManager->GetFrameSerial();
const Uint64 uboProgramLifetimeId = program.GetLifetimeId();
const Uint32 uboContentVersion = program.GetUBOContentVersion();
Bool reusedSlice = false;
if (isGlobalUbo) {
for (const auto& memo : m_globalUboMemo) {
if (memo.buffer != VK_NULL_HANDLE &&
memo.programLifetimeId == uboProgramLifetimeId &&
memo.frameSerial == uboFrameSerial &&
memo.uboContentVersion == uboContentVersion &&
memo.range == static_cast<VkDeviceSize>(ubo.payloadSize)) {
bufferInfo.buffer = memo.buffer;
bufferInfo.range = memo.range;
dynOffset = static_cast<Uint32>(memo.offset);
reusedSlice = true;
break;
}
}
}
if (!reusedSlice) {
BufferSlice slice{};
if (!m_bufferManager->UploadTransient(BufferKind::Uniform, frameIndex, ubo.payload,
ubo.payloadSize, m_minDynamicOffsetAlignment, slice)) {
MOBILEGL_ASSERT(false, "UniformDescriptorBinder::BindProgramUniformBuffers failed: UBO upload failed on binding %u element %u",
binding, element);
return false;
}
bufferInfo.buffer = slice.buffer;
bufferInfo.range = ubo.payloadSize;
dynOffset = static_cast<Uint32>(slice.offset);
if (isGlobalUbo) {
m_globalUboMemo[m_globalUboMemoNext] = GlobalUboSliceMemo{
uboProgramLifetimeId, uboFrameSerial, uboContentVersion,
slice.buffer, slice.offset, static_cast<VkDeviceSize>(ubo.payloadSize)};
m_globalUboMemoNext = (m_globalUboMemoNext + 1) % kGlobalUboMemoSize;
}
}
Uint32 dynOffset = 0;
if (!ResolveDynamicUboDescriptor(program, programObj, binding, element, frameIndex,
bufferInfo.buffer, bufferInfo.range, dynOffset)) {
return false;
}
bufferInfos.push_back(bufferInfo);
// Dynamic offsets are consumed in binding order, then array element order,
@@ -1304,6 +1475,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
texelBufferViews.push_back(bufferView);
fastRebindKindsEligible = false;
write.descriptorType = VK_DESCRIPTOR_TYPE_UNIFORM_TEXEL_BUFFER;
write.pTexelBufferView = &texelBufferViews.back();
writes.push_back(write);
@@ -1317,6 +1489,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
bufferInfos.push_back(bufferInfo);
fastRebindKindsEligible = false;
write.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER;
write.pBufferInfo = &bufferInfos.back();
writes.push_back(write);
@@ -1329,6 +1502,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return false;
}
imageInfos.push_back(imageInfo);
fastRebindKindsEligible = false;
write.descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_IMAGE;
write.pImageInfo = &imageInfos.back();
writes.push_back(write);
@@ -1341,7 +1515,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
samplerBindingOverride->sampler != nullptr) {
hasImage = ResolveSamplerDescriptorOverride(*samplerBindingOverride, imageInfo);
} else {
hasImage = ResolveSamplerDescriptor(commandBuffer, program, programObj, binding, imageInfo);
hasImage = ResolveSamplerDescriptor(commandBuffer, program, programObj, binding, imageInfo,
samplerDescriptorsUnchangedHint);
}
if (!hasImage) {
MGLOG_E(
@@ -1372,7 +1547,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// cursor only advances), so its written contents survive; the layout is part of
// the signature so reuse never crosses programs. Sampler overrides (blits)
// bypass and invalidate the cache.
const Bool cacheable = (samplerBindingOverride == nullptr);
Uint64 signature = 0xcbf29ce484222325ULL;
{
const auto mix64 = [&signature](Uint64 word) {
@@ -1435,34 +1609,22 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
// Skip the driver call when this exact binding is already live on the
// command buffer (see the bind-dedup shadow in the header).
const Uint32 offsetCount = static_cast<Uint32>(dynamicOffsets.size());
Bool identicalBind = m_lastBindValid && m_lastBindSet == descriptorSet &&
m_lastBindLayout == programObj.pipelineLayout && m_lastBindPoint == bindPoint &&
m_lastBindOffsetCount == offsetCount && offsetCount <= kMaxShadowedDynamicOffsets;
if (identicalBind) {
for (Uint32 i = 0; i < offsetCount; ++i) {
if (m_lastBindOffsets[i] != dynamicOffsets[i]) {
identicalBind = false;
break;
}
}
}
if (!identicalBind) {
vkCmdBindDescriptorSets(commandBuffer, bindPoint, programObj.pipelineLayout, 0, 1,
&descriptorSet, offsetCount, dynamicOffsets.data());
if (offsetCount <= kMaxShadowedDynamicOffsets) {
m_lastBindValid = true;
m_lastBindSet = descriptorSet;
m_lastBindLayout = programObj.pipelineLayout;
m_lastBindPoint = bindPoint;
m_lastBindOffsetCount = offsetCount;
std::copy_n(dynamicOffsets.data(), offsetCount, m_lastBindOffsets);
} else {
m_lastBindValid = false;
}
// (Re)record the dynamic-offset-only rebind memo. Recording on every
// cacheable walk (allocated or reused set alike - both hold exactly the
// content just computed) keeps the single slot tracking the most recent
// program; a non-cacheable override walk drops it alongside the reuse
// memo above.
if (cacheable && fastRebindKindsEligible && dynamicUboDescriptorCount == 1) {
m_fastRebindMemo = FastRebindMemo{
/*valid=*/true, frameIndex, program.GetLifetimeId(), programObj.hash,
fastRebindUboBinding, bufferInfos[0].buffer,
bufferInfos[0].range, descriptorSet};
} else {
m_fastRebindMemo.valid = false;
}
BindDescriptorSetDeduped(commandBuffer, bindPoint, programObj.pipelineLayout, descriptorSet,
dynamicOffsets);
return true;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -53,18 +53,42 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// caches - a live layout's entry must never be purged (its sets would be
// unreachable pool slots), so there is deliberately no age-based sweep here.
void OnDescriptorSetLayoutDestroyed(VkDescriptorSetLayout descriptorSetLayout);
// One record per visited CombinedImageSampler binding (post fallback substitution,
// in binding order): the resolved texture and effective sampler, as never-reused
// lifetime ids so a freed-and-reallocated object at the same heap address can only
// MISS a comparison, never false-hit it (same ABA rule as SamplerResolveMemo).
struct SampledBindingRecord {
Uint64 textureLifetimeId = 0;
Uint64 samplerLifetimeId = 0;
};
Bool CollectSampledTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Vector<MG_State::GLState::ITextureObject*>& outTextures);
Vector<MG_State::GLState::ITextureObject*>& outTextures,
Vector<SampledBindingRecord>* outBindingRecords = nullptr);
// Shadow-compare for the SetupDraw fast path: re-runs the CollectSampledTextures
// walk and reports whether every visited binding still resolves to the recorded
// (texture, effective sampler) pair. A texture bind generation bump alone (e.g. a
// redundant glBindSampler, which always bumps it) does not prove the sampled set
// moved; this walk does, without rebuilding the set or falling off the fast path.
Bool SampledBindingsUnchanged(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
const Vector<SampledBindingRecord>& previousRecords) const;
Bool CollectStorageImageTextures(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Vector<MG_State::GLState::ITextureObject*>& outTextures) const;
// samplerDescriptorsUnchangedHint: the caller (SetupDraw fast path) proved that
// every input of every combined-image-sampler resolution is unchanged since the
// previous draw's resolve - same (texture, sampler) per binding, texture params
// sum, sampling-resolution generation (sampler params + texture shape), image
// epochs AND per-resource layout values - so the per-binding cached
// VkDescriptorImageInfo may be reused without re-running the resolve chain.
Bool BindProgramUniformBuffers(VkCommandBuffer commandBuffer,
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj,
Uint32 frameIndex,
VkPipelineBindPoint bindPoint = VK_PIPELINE_BIND_POINT_GRAPHICS,
const SamplerBindingOverride* samplerBindingOverride = nullptr);
const SamplerBindingOverride* samplerBindingOverride = nullptr,
Bool samplerDescriptorsUnchangedHint = false);
// Pure format-policy helper kept public for host regression tests. Formatted storage
// images use their shader qualifier; transformed float images use glBindImageTexture's
@@ -114,6 +138,15 @@ namespace MobileGL::MG_Backend::DirectVulkan {
static Bool ResolveSamplerTexture(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
SharedPtr<MG_State::GLState::ITextureObject>& outTexture);
// Shared per-binding resolution for CollectSampledTextures and
// SampledBindingsUnchanged, so membership and comparison can never diverge:
// texture after the fallback substitution (may still be null when no fallback
// exists), effective sampler = unit override else the texture's own sampler.
// False = the binding is skipped (unbound with a non-2D fallback target).
Bool ResolveSampledBinding(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
MG_State::GLState::ITextureObject*& outTexture,
const MG_State::GLState::SamplerObject*& outSampler) const;
// Raw-pointer variant for the per-draw sampled-texture walk (CollectSampledTextures):
// the bound texture stays alive through the draw via GL binding state, so callers that
// only need the pointer skip the SharedPtr copy's atomic refcount churn.
@@ -121,9 +154,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding);
SharedPtr<MG_State::GLState::ITextureObject> GetFallbackTexture(TextureTarget target) const;
// trustUnchangedHint: reuse this binding's cached VkDescriptorImageInfo outright
// (see BindProgramUniformBuffers' samplerDescriptorsUnchangedHint for the proof
// obligations the caller carries).
Bool ResolveSamplerDescriptor(VkCommandBuffer commandBuffer, const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
VkDescriptorImageInfo& outImageInfo) const;
VkDescriptorImageInfo& outImageInfo,
Bool trustUnchangedHint = false) const;
Bool ResolveSamplerDescriptorOverride(const SamplerBindingOverride& samplerBindingOverride,
VkDescriptorImageInfo& outImageInfo) const;
Bool ResolveTexelBufferDescriptor(const MG_State::GLState::ProgramObject& program,
@@ -149,6 +186,21 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool ResolveUniformBufferPayload(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 arrayElement, UboBindResult& out) const;
// Shared resolution of one dynamic-UBO binding element into the
// (buffer, range, dynamicOffset) triple the descriptor consumes: direct
// bind, global-slice reuse, or transient upload. Used by the full walk
// and by the dynamic-offset-only rebind (see FastRebindMemo).
Bool ResolveDynamicUboDescriptor(const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj, Uint32 binding,
Uint32 arrayElement, Uint32 frameIndex, VkBuffer& outBuffer,
VkDeviceSize& outRange, Uint32& outDynamicOffset);
// The vkCmdBindDescriptorSets tail shared by the full walk and the
// dynamic-offset-only rebind: skips the driver call when this exact
// binding is already live on the command buffer (see the bind-dedup
// shadow below), otherwise binds and refreshes the shadow.
void BindDescriptorSetDeduped(VkCommandBuffer commandBuffer, VkPipelineBindPoint bindPoint,
VkPipelineLayout pipelineLayout, VkDescriptorSet descriptorSet,
const Vector<Uint32>& dynamicOffsets);
Bool CreateDescriptorPool(Uint32 maxSets, VkDescriptorPool& outPool) const;
Bool GrowFrameDescriptorPool(FrameResources& frame, Uint32 frameIndex);
VkResult AllocateDescriptorSetsFromActivePool(
@@ -196,6 +248,37 @@ namespace MobileGL::MG_Backend::DirectVulkan {
DescriptorReuseEntry m_descriptorReuseMemo[kDescriptorReuseMemoSize];
Uint32 m_descriptorReuseMemoNext = 0;
// Dynamic-offset-only rebind (see BindProgramUniformBuffers): records the
// descriptor set selected by the last cacheable full walk of a program
// whose active bindings are exactly one dynamic UBO (single descriptor)
// plus combined-image samplers. When the next call proves every sampler
// descriptor input unchanged (samplerDescriptorsUnchangedHint) and the
// UBO re-resolves to the SAME VkBuffer+range - only the dynamic offset
// moved, the per-draw glUniform case - the walk collapses to: resolve one
// offset, rebind the recorded set with new pDynamicOffsets (Vulkan allows
// rebinding the same set with different dynamic offsets).
// Invalidation inventory: BeginFrame clears it (the frame's sets are
// recycled) and the frameIndex field guards cross-frame confusion on top;
// OnDescriptorSetLayoutDestroyed clears it (the set may be freed); a
// sampler-override walk clears it (mirrors m_descriptorReuseMemo); a
// program relink bumps the backend state version and thus programObj.hash
// so the key misses; the program lifetime id is never reused, so a
// deleted-and-recreated program misses; a texture/sampler/binding change
// drops the hint upstream; an arena wrap or growth resolves a different
// VkBuffer and misses. AcquireDescriptorSet's per-frame cursor only
// advances, so the recorded set is never re-written within its frame.
struct FastRebindMemo {
Bool valid = false;
Uint32 frameIndex = 0;
Uint64 programLifetimeId = 0;
ProgramFactory::HashType programHash = 0;
Uint32 uboBinding = 0;
VkBuffer uboBuffer = VK_NULL_HANDLE;
VkDeviceSize uboRange = 0;
VkDescriptorSet set = VK_NULL_HANDLE;
};
FastRebindMemo m_fastRebindMemo;
// vkCmdBindDescriptorSets dedup: consecutive draws with a static uniform
// block resolve to the same set AND the same dynamic offsets, so the
// driver call can be skipped outright. Command-buffer-scope state; reset
@@ -253,6 +336,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
SamplerNumericDomain viewFormatDomain = SamplerNumericDomain::Unknown;
VkFormat viewFormat = VK_FORMAT_UNDEFINED;
Bool viewFormatValid = false;
// Whole resolved descriptor from this binding's last full resolve. Reused
// ONLY under ResolveSamplerDescriptor's trustUnchangedHint, whose caller
// proves every resolve input unchanged; cleared with the per-frame reset
// (the cached VkSampler outlives a frame only via a fresh resolve, which
// also re-stamps it against VkSamplerManager's frame-boundary sweep).
VkDescriptorImageInfo info{};
Bool infoValid = false;
};
mutable Vector<SamplerResolveMemo> m_samplerResolveMemo;
};
@@ -71,6 +71,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
const BackendVertexInputState& entry = GetOrCreateVertexInputState(vao, GetOrComputeHash(vao));
vao.SetBackendStateMemo(&entry, m_evictionEpoch);
// Also mirror the layout identity and the two per-draw masks into the VAO's aux
// memo (pure VALUES derived from the VAO configuration, so config-version
// guarding alone is sound). The draw fast path reads them from the VAO object it
// already touched instead of chasing into this entry - see PackVertexInputAuxMemo.
vao.SetBackendAuxMemo(entry.layoutHash,
PackVertexInputAuxMasks(entry.unsupportedAttribMask, entry.attributeLocationMask));
return entry;
}
@@ -71,6 +71,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
~VertexInputStateFactory() = default;
VertexInputStateFactory(const VertexInputStateFactory&) = delete;
// The VAO aux-memo payload GetOrCreateVertexInputState(vao) stamps: aux0 is the
// entry's layoutHash, aux1 packs (unsupportedAttribMask << 32) | attributeLocationMask.
// Readers that find the aux memo valid can use these without resolving the entry.
static Uint64 PackVertexInputAuxMasks(Uint32 unsupportedAttribMask, Uint32 attributeLocationMask) {
return (static_cast<Uint64>(unsupportedAttribMask) << 32) | attributeLocationMask;
}
HashType ComputeHash(const MG_State::GLState::VertexArrayObject& vao) const;
// Memoized ComputeHash: reuses the VAO's cached hash while its config version
// is unchanged. Use this on per-draw paths.
@@ -258,6 +258,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void VkBufferManager::ReleaseAllLiveResources() {
for (auto& weak : m_liveResources) {
if (auto resource = weak.lock()) {
BumpSliceEpoch(*resource);
resource->buffer.Destroy();
resource->storageSize = 0;
resource->usageFlags = 0;
@@ -272,6 +273,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Bool VkBufferManager::CreateResidentStorage(VkBufferResource& resource, VkDeviceSize size,
VkBufferUsageFlags usage, VkMemoryPropertyFlags requiredFlags) {
// The only place a resident VkBuffer handle is minted, so every resident slice
// change funnels through here (callers release the old handle first).
BumpSliceEpoch(resource);
// Staged range copies write resident storage with vkCmdCopyBuffer.
usage |= VK_BUFFER_USAGE_TRANSFER_DST_BIT;
const Bool created = resource.buffer.Create({
@@ -358,6 +362,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!resource) {
return; // lazy: AcquireResidentSlice performs a full upload on creation
}
// A respecify can change the size, the usage hint (so the resident/streamed
// route), and the contents at once; retire every memo before deciding what to
// do about the storage.
BumpSliceEpoch(*resource);
// Any cached streaming slice refers to the previous contents.
resource->transientFrameSerial = 0;
if (!resource->buffer.IsValid()) {
@@ -390,6 +398,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!resource) {
return;
}
// Drops the streaming memo below and may end in a storage swap or a deferred
// full re-upload, so no memoised slice survives this.
BumpSliceEpoch(*resource);
resource->transientFrameSerial = 0;
if (!resource->buffer.IsValid() || resource->pendingFullUpload) {
return;
@@ -422,6 +433,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
if (!resource) {
return;
}
BumpSliceEpoch(*resource);
resource->transientFrameSerial = 0;
if (!resource->buffer.IsValid() || resource->pendingFullUpload) {
return;
@@ -481,6 +493,13 @@ namespace MobileGL::MG_Backend::DirectVulkan {
TrackLiveResource(resource);
}
// Bumped for the request, not just for the storage it may create. This is the
// one call the frontend makes when a buffer becomes persistently mapped for
// writing (BufferObject::AcquireMemoryRange), and a map the backend declines
// keeps mutating its shadow with no further API call - so it is what lets
// GetSliceEpochCounter stand for "no buffer needs a persistent-map range push".
BumpSliceEpoch(*resource);
// Idempotent: an already-backed buffer returns the same mapped base.
if (resource->persistentMapped && resource->buffer.IsValid() && resource->storageSize == size) {
return resource->buffer.GetMappedData();
@@ -608,8 +627,10 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} else if (resource->transientChangeSerial == changeSerial && resource->transientSize == size &&
resource->transientFrameSerial != 0) {
if (++resource->unchangedStreak >= kStreamedPromotionStreak) {
// Promotion moves the buffer off the arena and onto resident storage.
resource->promotedResident = true;
resource->promotedChangeSerial = changeSerial;
BumpSliceEpoch(*resource);
if (AcquireResidentSlice(kind, bufferObject, outSlice)) {
return true;
}
@@ -619,6 +640,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource->unchangedStreak = 0;
}
// A fresh arena allocation: a different slice than the last call handed back,
// and (below) the point where a promoted buffer's resident storage is released.
// The stable-promotion exit above returns before this, so a buffer the app has
// stopped touching keeps one slice for as long as it keeps its resident storage.
BumpSliceEpoch(*resource);
if (!m_transientUploadArena.Upload(m_currentFrameIndex, bufferObject->MappedData(), size, 16,
outSlice)) {
return false;
@@ -57,6 +57,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// never orphaned or recreated. Draw-time acquire binds it directly, no re-upload.
Bool persistentMapped = false;
// Bumped from a manager-wide counter every time anything that decides which
// BufferSlice an Acquire*Slice call hands back changes: storage created or
// released, a full re-upload becoming due, a promotion/demotion between
// resident and streamed storage, or a new per-frame arena slice. Callers that
// memoise a resolved slice compare this to prove the memo still describes the
// buffer. The counter is manager-wide (never per-resource) so a freshly
// created resource - including one that replaces a destroyed resource at the
// same address - can never reproduce a value some memo already holds. 0 means
// "no slice has ever been handed out", which no memo can match.
Uint64 sliceEpoch = 0;
// Cached transient (streaming) slice for the current frame.
BufferSlice transientSlice{};
Uint64 transientFrameSerial = 0;
@@ -132,6 +143,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void OnResourceDestroyed(SharedPtr<MG_State::GLState::BackendBufferResource>&& resource);
Uint64 GetFrameSerial() const { return m_frameSerial; }
// Highest value handed to any VkBufferResource::sliceEpoch. Unchanged since a
// memo was taken means no buffer this manager owns changed which slice it hands
// back, and none was persistently mapped, in between - so a memo of resolved
// slices needs no per-buffer re-check. See AcquirePersistentMap for the mapping half.
Uint64 GetSliceEpochCounter() const { return m_sliceEpochCounter; }
// Highest frame serial whose GPU work is known complete; serials at or
// below it may be considered signaled. Drives IsResourceBusy and the
// backend GL fence objects.
@@ -158,6 +174,8 @@ namespace MobileGL::MG_Backend::DirectVulkan {
void DestroyAllDeferredReleases();
void TrackLiveResource(const SharedPtr<VkBufferResource>& resource);
void ReleaseAllLiveResources();
// See VkBufferResource::sliceEpoch.
void BumpSliceEpoch(VkBufferResource& resource) { resource.sliceEpoch = ++m_sliceEpochCounter; }
VkBufferManagerInitInfo m_initInfo{};
BufferArena m_transientUploadArena;
@@ -170,5 +188,9 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint32 m_currentFrameIndex = 0;
Uint64 m_frameSerial = 1;
Uint64 m_completedSerialFloor = 0;
// Never reset (not even by Shutdown): a value handed to a resource must stay
// unique for the process, or a memo taken before a re-initialize could match
// a different resource's state after it.
Uint64 m_sliceEpochCounter = 0;
};
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -176,16 +176,4 @@ namespace MobileGL::MG_Backend::DirectVulkan {
return true;
}
BufferSlice VkBufferObject::GetSlice(VkDeviceSize offset, VkDeviceSize size) const {
MOBILEGL_ASSERT(offset <= m_size, "VkBufferObject::GetSlice offset out of range");
const VkDeviceSize resolvedSize = (size == VK_WHOLE_SIZE) ? (m_size - offset) : size;
MOBILEGL_ASSERT(offset + resolvedSize <= m_size, "VkBufferObject::GetSlice range out of bounds");
BufferSlice slice{};
slice.buffer = m_buffer;
slice.offset = offset;
slice.size = resolvedSize;
slice.mapped = (m_mappedData != nullptr) ? static_cast<Uint8*>(m_mappedData) + offset : nullptr;
return slice;
}
} // namespace MobileGL::MG_Backend::DirectVulkan
@@ -48,7 +48,20 @@ namespace MobileGL::MG_Backend::DirectVulkan {
VkBuffer GetHandle() const { return m_buffer; }
VkDeviceSize GetSize() const { return m_size; }
BufferSlice GetSlice(VkDeviceSize offset = 0, VkDeviceSize size = VK_WHOLE_SIZE) const;
// Inline: runs on the per-draw acquire path (a resident buffer bind is a
// GetSlice per binding), where an out-of-line call was measurable.
BufferSlice GetSlice(VkDeviceSize offset = 0, VkDeviceSize size = VK_WHOLE_SIZE) const {
MOBILEGL_ASSERT(offset <= m_size, "VkBufferObject::GetSlice offset out of range");
const VkDeviceSize resolvedSize = (size == VK_WHOLE_SIZE) ? (m_size - offset) : size;
MOBILEGL_ASSERT(offset + resolvedSize <= m_size, "VkBufferObject::GetSlice range out of bounds");
BufferSlice slice{};
slice.buffer = m_buffer;
slice.offset = offset;
slice.size = resolvedSize;
slice.mapped = (m_mappedData != nullptr) ? static_cast<Uint8*>(m_mappedData) + offset : nullptr;
return slice;
}
void* GetMappedData() const { return m_mappedData; }
Bool IsMapped() const { return m_mappedData != nullptr; }
Bool IsValid() const { return m_allocator != nullptr && m_buffer != VK_NULL_HANDLE && m_allocation != nullptr; }
@@ -15,6 +15,7 @@
#include "MG_Util/Converters/MGToVk/TextureEnumConverter.h"
#include <Config.h>
#include <algorithm>
#include <cstdio>
#include <cstdlib>
#include <cstring>
@@ -620,12 +621,34 @@ namespace MobileGL::MG_Backend::DirectVulkan {
TextureResource::s_device = m_device;
TextureResource::s_allocator = m_allocator;
// Own pool for the recycled upload-batch command buffers. Parking a
// dozen reset-but-alive command buffers in the renderer's shared pool
// interleaves their retained chunks with the frame command buffers
// allocated/freed there every frame; isolating them keeps both pools'
// internal allocators dense.
VkCommandPoolCreateInfo uploadPoolInfo{VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO};
uploadPoolInfo.flags = VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT |
VK_COMMAND_POOL_CREATE_TRANSIENT_BIT;
uploadPoolInfo.queueFamilyIndex = initInfo.graphicsQueueFamilyIndex;
VK_VERIFY(vkCreateCommandPool(m_device, &uploadPoolInfo, nullptr, &m_uploadCommandPool),
"vkCreateCommandPool(texture upload batch)");
return true;
}
void VkTextureManager::Shutdown() {
if (m_device != VK_NULL_HANDLE) {
// A still-open (never-submitted) batch is discarded, not submitted:
// the renderer has already drained the device and the data has no
// observer. Submitted batches are waited and recycled, then the
// pools they recycled into are destroyed.
DiscardPendingUploadBatch();
ReclaimCompletedUploads(/*waitAll=*/true);
DestroyUploadPools();
if (m_uploadCommandPool != VK_NULL_HANDLE) {
vkDestroyCommandPool(m_device, m_uploadCommandPool, nullptr);
m_uploadCommandPool = VK_NULL_HANDLE;
}
}
DestroyDeferredReleases();
++m_resourceEraseEpoch; // every memoized resource pointer dies with the map
@@ -1859,6 +1882,11 @@ namespace MobileGL::MG_Backend::DirectVulkan {
resource.syncedTextureParamsVersion = 0;
if (preservedResource) {
// The preserve copy reads the OLD image on its own immediately-
// submitted-and-waited command buffer; a batched upload into that
// image still sitting in the open batch must reach the queue first
// or the copy carries pre-upload texels forward.
FlushPendingUploads();
const Bool preserved = PreserveTextureContentsOnRecreate(
m_device, m_commandPool, m_graphicsQueue, *preservedResource, resource);
MOBILEGL_ASSERT(preserved,
@@ -1869,6 +1897,16 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
void VkTextureManager::DeferResourceRelease(TextureResource&& resource) {
// The deferred-release queues are drained under fence/queue-idle proofs
// that only cover SUBMITTED work; a recorded-but-unsubmitted upload
// batch referencing this image would escape them. Push the batch onto
// the queue first so every later proof covers it. Rare (only recreate/
// erase of an image uploaded this very frame), so the flush is cheap.
if (m_uploadBatchOpen && resource.image != VK_NULL_HANDLE &&
std::find(m_uploadBatchImages.begin(), m_uploadBatchImages.end(), resource.image) !=
m_uploadBatchImages.end()) {
FlushPendingUploads();
}
if (resource.image == VK_NULL_HANDLE && resource.fullView == VK_NULL_HANDLE &&
resource.sampledView == VK_NULL_HANDLE &&
resource.perMipViews.empty() && resource.perMipSampledViews.empty() &&
@@ -1919,14 +1957,211 @@ namespace MobileGL::MG_Backend::DirectVulkan {
} else if (vkGetFenceStatus(m_device, entry.fence) != VK_SUCCESS) {
break;
}
vkDestroyFence(m_device, entry.fence, nullptr);
vkFreeCommandBuffers(m_device, m_commandPool, 1, &entry.commandBuffer);
vmaDestroyBuffer(m_allocator, entry.stagingBuffer, entry.stagingAllocation);
// Recycle, don't destroy: the fence resets into the fence pool,
// the command buffer resets into the CB pool (m_uploadCommandPool
// carries RESET_COMMAND_BUFFER_BIT), and the staging blocks
// return to the block pool for the next batch to bump-allocate.
// This is where the mc_tex_stream win comes from: the per-upload
// fence create/destroy + command-buffer alloc/free ioctl traffic
// was the measured 41%-in-kernel cost, not the submit itself.
if (vkResetFences(m_device, 1, &entry.fence) == VK_SUCCESS) {
m_freeUploadFences.push_back(entry.fence);
} else {
vkDestroyFence(m_device, entry.fence, nullptr);
}
if (vkResetCommandBuffer(entry.commandBuffer, 0) == VK_SUCCESS) {
m_freeUploadCommandBuffers.push_back(entry.commandBuffer);
} else {
vkFreeCommandBuffers(m_device, m_uploadCommandPool, 1, &entry.commandBuffer);
}
for (auto& block : entry.stagingBlocks) {
RecycleUploadStagingBlock(Move(block));
}
entry.stagingBlocks.clear();
}
m_pendingUploadReclaims.erase(m_pendingUploadReclaims.begin(),
m_pendingUploadReclaims.begin() + static_cast<std::ptrdiff_t>(completed));
}
void VkTextureManager::RecycleUploadStagingBlock(UploadStagingBlock&& block) {
if (block.buffer == VK_NULL_HANDLE) {
return;
}
// Bound the idle pool: a one-off giant upload (initial atlas define)
// must not pin its staging memory forever.
constexpr VkDeviceSize kMaxFreeUploadStagingBytes = 32u * 1024u * 1024u;
if (m_allocator == nullptr || m_freeUploadStagingBytes + block.capacity > kMaxFreeUploadStagingBytes) {
vmaDestroyBuffer(m_allocator, block.buffer, block.allocation);
return;
}
block.cursor = 0;
m_freeUploadStagingBytes += block.capacity;
m_freeUploadStagingBlocks.push_back(Move(block));
}
VkCommandBuffer VkTextureManager::EnsureUploadBatchOpen() {
if (m_uploadBatchOpen) {
return m_uploadBatchCommandBuffer;
}
if (!m_freeUploadCommandBuffers.empty()) {
m_uploadBatchCommandBuffer = m_freeUploadCommandBuffers.back();
m_freeUploadCommandBuffers.pop_back();
} else {
VkCommandBufferAllocateInfo allocInfo{};
allocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
allocInfo.commandPool = m_uploadCommandPool;
allocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
allocInfo.commandBufferCount = 1;
VK_VERIFY(vkAllocateCommandBuffers(m_device, &allocInfo, &m_uploadBatchCommandBuffer),
"vkAllocateCommandBuffers(texture upload batch)");
}
VkCommandBufferBeginInfo beginInfo{};
beginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
beginInfo.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
VK_VERIFY(vkBeginCommandBuffer(m_uploadBatchCommandBuffer, &beginInfo),
"vkBeginCommandBuffer(texture upload batch)");
m_uploadBatchOpen = true;
return m_uploadBatchCommandBuffer;
}
Uint8* VkTextureManager::AcquireUploadStagingSpace(VkDeviceSize size, VkBuffer& outBuffer,
VkDeviceSize& outBaseOffset) {
// 16 covers every uncompressed texel size in use (1..16 bytes) and the
// bufferOffset multiple-of-4 rule; per-item offsets inside the span
// keep the pre-batching tight packing.
constexpr VkDeviceSize kUploadStagingAlignment = 16;
constexpr VkDeviceSize kUploadStagingBlockSize = 1u * 1024u * 1024u;
UploadStagingBlock* current = m_uploadBatchBlocks.empty() ? nullptr : &m_uploadBatchBlocks.back();
VkDeviceSize alignedCursor = 0;
if (current != nullptr) {
alignedCursor = (current->cursor + (kUploadStagingAlignment - 1)) & ~(kUploadStagingAlignment - 1);
if (alignedCursor + size > current->capacity) {
current = nullptr;
}
}
if (current == nullptr) {
UploadStagingBlock block;
for (SizeT i = 0; i < m_freeUploadStagingBlocks.size(); ++i) {
if (m_freeUploadStagingBlocks[i].capacity >= size) {
block = Move(m_freeUploadStagingBlocks[i]);
m_freeUploadStagingBytes -= block.capacity;
m_freeUploadStagingBlocks.erase(m_freeUploadStagingBlocks.begin() +
static_cast<std::ptrdiff_t>(i));
break;
}
}
if (block.buffer == VK_NULL_HANDLE) {
VkBufferCreateInfo bufferInfo{};
bufferInfo.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
bufferInfo.size = std::max(kUploadStagingBlockSize, size);
bufferInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
bufferInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
VmaAllocationCreateInfo stagingAllocationInfo{};
stagingAllocationInfo.usage = VMA_MEMORY_USAGE_AUTO_PREFER_HOST;
stagingAllocationInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
VMA_ALLOCATION_CREATE_MAPPED_BIT;
stagingAllocationInfo.requiredFlags =
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
VmaAllocationInfo allocationResult{};
VK_VERIFY(vmaCreateBuffer(m_allocator, &bufferInfo, &stagingAllocationInfo, &block.buffer,
&block.allocation, &allocationResult),
"vmaCreateBuffer(texture upload staging block)");
block.mapped = static_cast<Uint8*>(allocationResult.pMappedData);
block.capacity = bufferInfo.size;
MOBILEGL_ASSERT(block.mapped != nullptr,
"AcquireUploadStagingSpace: staging block is not persistently mapped");
}
block.cursor = 0;
m_uploadBatchBlocks.push_back(Move(block));
current = &m_uploadBatchBlocks.back();
alignedCursor = 0;
}
outBuffer = current->buffer;
outBaseOffset = alignedCursor;
current->cursor = alignedCursor + size;
return current->mapped + alignedCursor;
}
void VkTextureManager::FlushPendingUploads() {
if (!m_uploadBatchOpen) {
return;
}
VK_VERIFY(vkEndCommandBuffer(m_uploadBatchCommandBuffer), "vkEndCommandBuffer(texture upload batch)");
VkFence uploadFence = VK_NULL_HANDLE;
if (!m_freeUploadFences.empty()) {
uploadFence = m_freeUploadFences.back();
m_freeUploadFences.pop_back();
} else {
VkFenceCreateInfo fenceInfo{};
fenceInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO;
VK_VERIFY(vkCreateFence(m_device, &fenceInfo, nullptr, &uploadFence), "vkCreateFence(texture upload)");
}
VkSubmitInfo submitInfo{};
submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &m_uploadBatchCommandBuffer;
VK_VERIFY(vkQueueSubmit(m_graphicsQueue, 1, &submitInfo, uploadFence), "vkQueueSubmit(texture upload batch)");
PendingUploadReclaim reclaim;
reclaim.fence = uploadFence;
reclaim.commandBuffer = m_uploadBatchCommandBuffer;
reclaim.stagingBlocks = Move(m_uploadBatchBlocks);
m_pendingUploadReclaims.push_back(Move(reclaim));
m_uploadBatchCommandBuffer = VK_NULL_HANDLE;
m_uploadBatchOpen = false;
m_uploadBatchBlocks.clear();
m_uploadBatchImages.clear();
m_uploadBatchStagingBytes = 0;
ReclaimCompletedUploads();
// Backstop for pathological upload storms: bound in-flight staging
// memory by blocking on the oldest batch only once the list is deep.
constexpr SizeT kMaxPendingTextureUploads = 16;
if (m_pendingUploadReclaims.size() > kMaxPendingTextureUploads) {
VK_VERIFY(vkWaitForFences(m_device, 1, &m_pendingUploadReclaims.front().fence, VK_TRUE, UINT64_MAX),
"vkWaitForFences(texture upload backstop)");
ReclaimCompletedUploads();
}
}
void VkTextureManager::DiscardPendingUploadBatch() {
if (!m_uploadBatchOpen) {
return;
}
// The batch was never submitted, so the command buffer is in the
// recording state, not pending - freeing it is legal.
vkFreeCommandBuffers(m_device, m_uploadCommandPool, 1, &m_uploadBatchCommandBuffer);
m_uploadBatchCommandBuffer = VK_NULL_HANDLE;
m_uploadBatchOpen = false;
for (auto& block : m_uploadBatchBlocks) {
RecycleUploadStagingBlock(Move(block));
}
m_uploadBatchBlocks.clear();
m_uploadBatchImages.clear();
m_uploadBatchStagingBytes = 0;
}
void VkTextureManager::DestroyUploadPools() {
for (auto& block : m_freeUploadStagingBlocks) {
if (block.buffer != VK_NULL_HANDLE) {
vmaDestroyBuffer(m_allocator, block.buffer, block.allocation);
}
}
m_freeUploadStagingBlocks.clear();
m_freeUploadStagingBytes = 0;
if (!m_freeUploadCommandBuffers.empty()) {
vkFreeCommandBuffers(m_device, m_uploadCommandPool, static_cast<Uint32>(m_freeUploadCommandBuffers.size()),
m_freeUploadCommandBuffers.data());
m_freeUploadCommandBuffers.clear();
}
for (const VkFence fence : m_freeUploadFences) {
vkDestroyFence(m_device, fence, nullptr);
}
m_freeUploadFences.clear();
}
void VkTextureManager::DestroyDeferredReleases() {
for (auto& deferredReleases : m_deferredReleases) {
deferredReleases.clear();
@@ -2271,25 +2506,30 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
// Rare mid-frame hazard, kept at parity with the old per-upload
// submits: this image already has an upload recorded in the OPEN batch
// and has since been referenced by the frame's open recording (drawn).
// Appending here would merge both uploads into the same pre-frame
// submission the old code split into two; flush first so the second
// upload lands in its own later submission, exactly like before.
if (m_uploadBatchOpen && WasTouchedThisRecording(outResource) &&
std::find(m_uploadBatchImages.begin(), m_uploadBatchImages.end(), outResource.image) !=
m_uploadBatchImages.end()) {
FlushPendingUploads();
}
// Bound the staging bytes a single batch can pin before its fence can
// reclaim them.
constexpr VkDeviceSize kMaxBatchStagingBytes = 64u * 1024u * 1024u;
if (m_uploadBatchOpen && m_uploadBatchStagingBytes + stagingSize > kMaxBatchStagingBytes) {
FlushPendingUploads();
}
VkCommandBuffer commandBuffer = EnsureUploadBatchOpen();
VkBuffer stagingBuffer = VK_NULL_HANDLE;
VmaAllocation stagingAllocation = nullptr;
VkBufferCreateInfo bufferInfo{};
bufferInfo.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
bufferInfo.size = stagingSize;
bufferInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
bufferInfo.sharingMode = VK_SHARING_MODE_EXCLUSIVE;
VmaAllocationCreateInfo stagingAllocationInfo{};
stagingAllocationInfo.usage = VMA_MEMORY_USAGE_AUTO_PREFER_HOST;
stagingAllocationInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT;
stagingAllocationInfo.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
VK_VERIFY(vmaCreateBuffer(m_allocator, &bufferInfo, &stagingAllocationInfo, &stagingBuffer, &stagingAllocation, nullptr),
"vmaCreateBuffer(staging texture)");
void* mapped = nullptr;
VK_VERIFY(vmaMapMemory(m_allocator, stagingAllocation, &mapped), "vmaMapMemory(staging texture)");
VkDeviceSize stagingBase = 0;
Uint8* mapped = AcquireUploadStagingSpace(stagingSize, stagingBuffer, stagingBase);
for (const auto& item : uploadItems) {
Uint8* dst = static_cast<Uint8*>(mapped) + item.offset;
Uint8* dst = mapped + item.offset;
if (!item.subRegion) {
std::memcpy(dst, item.source, item.uploadByteSize);
continue;
@@ -2311,21 +2551,6 @@ namespace MobileGL::MG_Backend::DirectVulkan {
}
}
}
vmaUnmapMemory(m_allocator, stagingAllocation);
VkCommandBufferAllocateInfo allocInfo{};
allocInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO;
allocInfo.commandPool = m_commandPool;
allocInfo.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY;
allocInfo.commandBufferCount = 1;
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
VK_VERIFY(vkAllocateCommandBuffers(m_device, &allocInfo, &commandBuffer), "vkAllocateCommandBuffers(texture)");
VkCommandBufferBeginInfo beginInfo{};
beginInfo.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO;
beginInfo.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
VK_VERIFY(vkBeginCommandBuffer(commandBuffer, &beginInfo), "vkBeginCommandBuffer(texture)");
const VkImageAspectFlags aspectMask = GetAspectMaskForFormat(outResource.format);
VkPipelineStageFlags uploadSrcStageMask = VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT;
@@ -2350,7 +2575,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
for (const auto& item : uploadItems) {
const Uint32 depthOrLayers = item.texelSize.z() > 0 ? static_cast<Uint32>(item.texelSize.z()) : 1u;
VkBufferImageCopy copy{};
copy.bufferOffset = item.offset;
copy.bufferOffset = stagingBase + item.offset;
copy.bufferRowLength = 0;
copy.bufferImageHeight = 0;
copy.imageSubresource.aspectMask = aspectMask;
@@ -2383,7 +2608,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
depthCopy.imageSubresource.aspectMask = VK_IMAGE_ASPECT_DEPTH_BIT;
VkBufferImageCopy stencilCopy = copy;
stencilCopy.imageSubresource.aspectMask = VK_IMAGE_ASPECT_STENCIL_BIT;
stencilCopy.bufferOffset = item.offset + static_cast<VkDeviceSize>(texelCount) * 4;
stencilCopy.bufferOffset = stagingBase + item.offset + static_cast<VkDeviceSize>(texelCount) * 4;
const VkBufferImageCopy copies[2] = {depthCopy, stencilCopy};
vkCmdCopyBufferToImage(commandBuffer, stagingBuffer, outResource.image,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 2, copies);
@@ -2406,36 +2631,27 @@ namespace MobileGL::MG_Backend::DirectVulkan {
MOBILEGL_ASSERT(ok, "TransitionImageLayout to sampled read-only layout failed");
outResource.layout = finalLayout;
VK_VERIFY(vkEndCommandBuffer(commandBuffer), "vkEndCommandBuffer(texture)");
VkSubmitInfo submitInfo{};
submitInfo.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO;
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &commandBuffer;
VkFenceCreateInfo fenceInfo{};
fenceInfo.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO;
VkFence uploadFence = VK_NULL_HANDLE;
VK_VERIFY(vkCreateFence(m_device, &fenceInfo, nullptr, &uploadFence), "vkCreateFence(texture upload)");
VK_VERIFY(vkQueueSubmit(m_graphicsQueue, 1, &submitInfo, uploadFence), "vkQueueSubmit(texture)");
// Do NOT wait the fence here: this submit sits behind the previous
// frame's rendering on the queue, so a synchronous wait stalls the CPU
// until the GPU drains - a per-frame vkQueueWaitIdle for any workload
// with animated textures. Ordering against the current frame's draws is
// already guaranteed (its command buffer is submitted later, at
// present), so only the transient objects need to survive execution;
// park them until the fence signals.
m_pendingUploadReclaims.push_back({uploadFence, commandBuffer, stagingBuffer, stagingAllocation});
ReclaimCompletedUploads();
// Backstop for pathological upload storms: bound in-flight staging
// memory by blocking on the oldest upload only once the list is deep.
constexpr SizeT kMaxPendingTextureUploads = 16;
if (m_pendingUploadReclaims.size() > kMaxPendingTextureUploads) {
VK_VERIFY(vkWaitForFences(m_device, 1, &m_pendingUploadReclaims.front().fence, VK_TRUE, UINT64_MAX),
"vkWaitForFences(texture upload backstop)");
ReclaimCompletedUploads();
// Ordering argument (replaces the old immediate per-texture submit):
// this upload is RECORDED into the shared batch command buffer, which
// FlushPendingUploads submits - with one vkQueueSubmit and one pooled
// fence for the whole batch - strictly BEFORE any other submission on
// the same queue whose commands could consume the image: the renderer
// flushes at every frame-command-buffer submit (mid-frame flush,
// readback, Present), and the texture manager flushes before the
// preserve-on-recreate copy and before deferring an image the batch
// references. The frame command buffer therefore still lands behind
// the uploads on the queue, so a texture uploaded and then immediately
// sampled in the same frame sees its data exactly as it did when each
// upload was its own submit. No fence is waited here, for the same
// reason as before: the batch queues behind the previous frame's
// rendering, and a synchronous wait would drain the GPU; the staging
// blocks/command buffer are parked on the reclaim list at flush time
// and recycled once the batch fence signals.
if (std::find(m_uploadBatchImages.begin(), m_uploadBatchImages.end(), outResource.image) ==
m_uploadBatchImages.end()) {
m_uploadBatchImages.push_back(outResource.image);
}
m_uploadBatchStagingBytes += stagingSize;
if (!ok) {
MGLOG_D("%s: texture upload cmd failed", __func__);
@@ -2445,6 +2661,17 @@ namespace MobileGL::MG_Backend::DirectVulkan {
mipmapTexture.MarkStorageDirty(item.target, item.level, false);
}
outResource.layout = finalLayout;
// Large batches flush right away instead of riding until the frame
// submit: a big copy amortizes its own vkQueueSubmit, submitting it
// early lets the GPU overlap the copy with the rest of the frame's
// CPU recording (measurably faster than a frame-tail burst), and the
// frame-tail burst pattern was observed to leave the GPU in a
// latency state that taxes whatever runs next. Small uploads keep
// accumulating, so a lightmap+sprite frame still costs one submit.
constexpr VkDeviceSize kEagerUploadFlushBytes = 128u * 1024u;
if (m_uploadBatchStagingBytes >= kEagerUploadFlushBytes) {
FlushPendingUploads();
}
return true;
}
@@ -65,6 +65,11 @@ public:
VkPipelineStageFlags sampledReadStageMask = VK_PIPELINE_STAGE_VERTEX_SHADER_BIT |
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT |
VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT;
// Family of `graphicsQueue`; the manager creates its own command pool
// on it for the recycled upload-batch command buffers, so their parked
// allocations never sit in (and fragment) the renderer's shared pool
// that frame command buffers churn through every frame.
Uint32 graphicsQueueFamilyIndex = 0;
};
struct TextureResource {
@@ -308,6 +313,14 @@ public:
Bool Initialize(const InitInfo& initInfo);
void Shutdown();
void BeginFrame(Uint32 frameIndex);
// Submits the accumulated texture-upload batch (one command buffer, one
// vkQueueSubmit, one pooled fence) if any uploads are pending. MUST run
// before any other vkQueueSubmit on the shared graphics queue whose
// commands may consume an image the batch writes - the frame command
// buffer submit (mid-frame flush, readback, Present) and the
// preserve-on-recreate copy are the existing callers. No-op when the
// batch is empty.
void FlushPendingUploads();
// Drains every frame slot's deferred image/view releases. Only valid when
// the caller has proven every queue submission complete; used by the
// present-less frame-boundary drain.
@@ -453,6 +466,9 @@ private:
VkPhysicalDevice m_physicalDevice = VK_NULL_HANDLE;
VmaAllocator m_allocator = nullptr;
VkCommandPool m_commandPool = VK_NULL_HANDLE;
// Dedicated pool for the recycled upload-batch command buffers (see
// InitInfo::graphicsQueueFamilyIndex).
VkCommandPool m_uploadCommandPool = VK_NULL_HANDLE;
VkQueue m_graphicsQueue = VK_NULL_HANDLE;
Bool m_imageFormatListSupported = false;
Uint32 m_currentFrameIndex = 0;
@@ -506,15 +522,58 @@ private:
std::unordered_map<VkFormat, VkSampleCountFlags> m_multisampleCountsByFormat;
Vector<Vector<TextureResource>> m_deferredReleases;
Vector<Vector<VkImageView>> m_deferredViewReleases;
// --- Batched upload machinery ---
// Uploads within a frame are recorded into ONE shared command buffer and
// submitted with ONE vkQueueSubmit at FlushPendingUploads (the renderer
// flushes before every frame-command-buffer submit). Staging memory comes
// from a pool of persistently-mapped, reusable blocks instead of a
// vmaCreateBuffer per upload.
struct UploadStagingBlock {
VkBuffer buffer = VK_NULL_HANDLE;
VmaAllocation allocation = nullptr;
Uint8* mapped = nullptr; // persistently mapped for the block's lifetime
VkDeviceSize capacity = 0;
VkDeviceSize cursor = 0; // bump cursor while the block backs the open batch
};
// Opens the batch command buffer lazily (allocates/reuses + begins recording).
VkCommandBuffer EnsureUploadBatchOpen();
// Bump-allocates `size` staging bytes for the open batch, growing onto a
// new/pooled block when the current one cannot fit. Returns the write
// pointer; outBuffer/outBaseOffset locate the space for copy commands.
Uint8* AcquireUploadStagingSpace(VkDeviceSize size, VkBuffer& outBuffer, VkDeviceSize& outBaseOffset);
void RecycleUploadStagingBlock(UploadStagingBlock&& block);
// Drops a recorded-but-unsubmitted batch on the floor. Shutdown only: the
// device is being torn down, so the lost texel data is unobservable.
void DiscardPendingUploadBatch();
void DestroyUploadPools();
Vector<UploadStagingBlock> m_freeUploadStagingBlocks;
VkDeviceSize m_freeUploadStagingBytes = 0;
Vector<VkCommandBuffer> m_freeUploadCommandBuffers;
Vector<VkFence> m_freeUploadFences;
Bool m_uploadBatchOpen = false;
VkCommandBuffer m_uploadBatchCommandBuffer = VK_NULL_HANDLE;
// Blocks whose staging bytes the open batch's copies reference (last =
// the block the bump cursor is currently allocating from).
Vector<UploadStagingBlock> m_uploadBatchBlocks;
// Images the open batch writes; consulted for the rare re-upload-after-
// draw flush and by DeferResourceRelease (an unsubmitted command buffer
// referencing a deferred-released image would escape every fence-based
// destruction proof, so the batch is flushed before the image is parked).
Vector<VkImage> m_uploadBatchImages;
VkDeviceSize m_uploadBatchStagingBytes = 0;
// Texture uploads are submitted out-of-band but NOT waited on (waiting
// behind the queue serialized the CPU against the previous frame's GPU
// work every time an animated atlas re-uploaded). Their transient objects
// are parked here and reclaimed once the upload fence signals.
// work every time an animated atlas re-uploaded). Each flushed batch's
// transients are parked here and RECYCLED (fence reset to the fence pool,
// command buffer reset to the CB pool, staging blocks back to the block
// pool) once the batch fence signals.
struct PendingUploadReclaim {
VkFence fence = VK_NULL_HANDLE;
VkCommandBuffer commandBuffer = VK_NULL_HANDLE;
VkBuffer stagingBuffer = VK_NULL_HANDLE;
VmaAllocation stagingAllocation = nullptr;
Vector<UploadStagingBlock> stagingBlocks;
};
Vector<PendingUploadReclaim> m_pendingUploadReclaims;
};
File diff suppressed because it is too large Load Diff
@@ -531,7 +531,7 @@ namespace MobileGL::MG_Backend::DirectVulkan {
// Per-pipeline provoking-vertex mode. capturesXfbFromGeometryStage must be a LINK-TIME
// property of the program, never the dynamic "is transform feedback active" flag: the
// 8-entry m_pipelineMemo and the SetupDrawSnapshot fast path key on programObj.hash and
// GetRenderStateParametersVersion(), neither of which moves when glBeginTransformFeedback is
// the pipeline-state value hash, neither of which moves when glBeginTransformFeedback is
// called, so a dynamic input here would hand back a stale VkPipeline.
VkProvokingVertexModeEXT SelectProvokingVertexMode(VkPrimitiveTopology topology,
Bool capturesXfbFromGeometryStage) const;
@@ -623,7 +623,12 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 programHash = 0;
Uint64 vertexInputHash = 0;
Uint64 renderPassHash = 0;
Uint renderStateVersion = 0;
// VALUE hash of the pipeline-relevant fixed-function state (see
// ComputePipelineStateHash), not the monotonic pipeline-state version:
// the version never repeats, so a per-draw GL_BLEND toggle would miss
// all entries forever even though the state alternates between two
// values the memo already holds.
Uint64 pipelineStateHash = 0;
ProgramFactory::CompileOptionFlags transformFlags = {};
VkPipeline pipeline = VK_NULL_HANDLE;
};
@@ -631,11 +636,33 @@ namespace MobileGL::MG_Backend::DirectVulkan {
PipelineMemoEntry m_pipelineMemo[kPipelineMemoSize];
Uint32 m_pipelineMemoCount = 0;
Uint32 m_pipelineMemoNext = 0;
// Hash of every fixed-function GL state the pipeline payload reads that the
// memo key's other fields (mode / program / vertex input / render pass /
// transform flags) do not already pin down. Equal hash under an equal rest
// of key => byte-identical PipelineCreatePayload. Cached per pipeline-state
// version: the version is monotonic and bumps on every pipeline-state
// change, so an unchanged (version, colorAttachmentCount) proves the state
// bytes are unchanged and the hash can be reused without re-reading them.
Uint64 ComputePipelineStateHash(Uint32 colorAttachmentCount) const;
Uint m_pipelineStateHashVersion = 0;
Uint32 m_pipelineStateHashColorCount = 0;
Uint64 m_pipelineStateHash = 0;
Bool m_pipelineStateHashValid = false;
// GetShaderTransformFlags(preTransform) memo: a pure function of the swapchain
// pre-transform, re-evaluated only when that value changes (surface rotation).
// No other invalidation input exists.
VkSurfaceTransformFlagBitsKHR m_baseTransformFlagsPreTransform =
VK_SURFACE_TRANSFORM_FLAG_BITS_MAX_ENUM_KHR;
Uint32 m_baseTransformFlagsCache = 0;
Uint32 GetBaseTransformFlagsRaw();
// Drops every memoized pipeline handle. Required at command-buffer
// boundaries and whenever any pipeline may have been destroyed.
// boundaries and whenever any pipeline may have been destroyed. Also drops
// the cached pipeline-state hash: the same boundaries can retire the GL
// context whose monotonic version the cache is keyed on.
void InvalidatePipelineMemo() {
m_pipelineMemoCount = 0;
m_pipelineMemoNext = 0;
m_pipelineStateHashValid = false;
}
UnorderedMap<ProgramFactory::HashType, VkPipeline> m_computePipelines;
UniquePtr<ProgramFactory> m_programFactory;
@@ -707,19 +734,61 @@ namespace MobileGL::MG_Backend::DirectVulkan {
Uint64 renderbufferImageEpoch = 0;
Uint64 sampledContentSum = 0;
Uint64 sampledParamsSum = 0;
// Guards the sampler-descriptor reuse hint: bumped by any sampler-object
// parameter or texture shape change (see GetSamplingResolutionGeneration),
// none of which the sums above cover.
Uint64 samplingResolutionGeneration = 0;
// Render-pass flavor input (DepthTest || StencilTest at snapshot time).
// A pipeline-state change that leaves this equal cannot change which
// render pass GetOrCreateRenderPass would pick, so the fast path may
// re-resolve just the pipeline against the active pass; a change that
// flips it must fall back to the full path's pass selection.
Bool drawUsesDepthStencil = false;
IntVec2 renderPassExtent = {0, 0};
// colorAttachmentCount of the snapshotting draw's render pass: the
// pipeline-state hash input, so the fast path can refresh that hash and
// probe the pipeline memo after a state change without re-fetching the
// render-pass entry (the pass itself is pinned by renderPassHash above).
Uint32 renderPassColorCount = 0;
VkPipeline pipeline = VK_NULL_HANDLE;
// layoutHash of the snapshotting draw's vertex-input state. The pipeline and
// the vertex-input pre-flight depend on the VAO only through this (plus the
// program, pinned separately), so a changed VAO whose aux memo carries the
// same layoutHash re-uses the snapshot's pipeline and pre-flight verdict
// outright - the VAO-cycling case Minecraft chunk rendering hits every draw.
Uint64 vaoLayoutHash = 0;
// Memoised ProgramFactory entry of the snapshotting draw, valid while
// (programLifetimeId, programVersion, resolvedTransformFlags) match - all
// checked above - AND the factory's cache structure epoch is unchanged (the
// cache is open-addressing and holds entries by value, so any insert/erase
// moves them). The fast path must re-stamp use through StampProgramUse when
// it bypasses GetOrCreateProgram, or the idle sweep could evict a live entry.
const ProgramFactory::VkProgramObject* programObj = nullptr;
Uint64 programFactoryEpoch = 0;
};
SetupDrawSnapshot m_setupDrawSnapshot;
// Per-draw scratch buffers (clear keeps capacity) — these paths run for every
// draw call and must not allocate.
Vector<MG_State::GLState::ITextureObject*> m_sampledTexturesScratch;
// Per-binding (texture, effective sampler) lifetime-id records from the same
// CollectSampledTextures walk that filled m_sampledTexturesScratch. The fast
// path shadow-compares against them (SampledBindingsUnchanged) when the
// texture bind generation moved, so a redundant glBindSampler/glBindTexture
// storm that resolves to the same bindings keeps the fast path.
Vector<UniformManager::SampledBindingRecord> m_sampledBindingRecordsScratch;
// Parallel to m_sampledTexturesScratch, refilled by every SetupDraw's
// first sampled-texture loop: the resolved backend resources, so the
// post-transition loop can skip re-resolving textures whose layout is
// already sampleable.
Vector<VkTextureManager::TextureResource*> m_sampledResourcesScratch;
// Layout VALUE of each sampled resource when the snapshot (and so the cached
// sampler descriptors) was built, parallel to m_sampledResourcesScratch. The
// fast path's validity check only proves the layout is still sampleable; the
// descriptor-reuse hint additionally needs it to be the SAME sampleable
// layout (a mid-frame compute dispatch can move a sampled texture from
// READ_ONLY_OPTIMAL to GENERAL, both valid, different descriptor).
Vector<VkImageLayout> m_sampledLayoutSnapshots;
Vector<MG_State::GLState::ITextureObject*> m_storageImageTexturesScratch;
Vector<VkBuffer> m_vertexBuffersScratch;
Vector<VkDeviceSize> m_vertexOffsetsScratch;
@@ -779,6 +848,127 @@ namespace MobileGL::MG_Backend::DirectVulkan {
UnorderedMap<ConvertedVertexStreamKey, ConvertedVertexStream, ConvertedVertexStreamKeyHash>
m_convertedVertexStreams;
// One VAO's resolved vkCmdBindVertexBuffers arguments, reusable by a later draw
// that would resolve them to the same thing. Consecutive draws in a chunk-renderer
// frame keep the program and the vertex layout and only swap the VAO, so a
// per-VAO memo turns the second and later draws through each VAO into a validate
// plus (usually skipped) rebind.
//
// Only whole-buffer bindings are memoised. Client-memory and format-converted
// streams re-upload from a range that depends on the draw's own vertex/index
// range, and synthetic bindings carry glVertexAttrib* values that are not part
// of any key here; a layout using any of them is never stored.
// Field order is hit-path cache locality, hot to cold: the per-draw validate
// reads the scalars and the EBO memo head, then only the first bindingCount
// elements of vkBuffers/vkOffsets; the per-binding revalidation arrays at the
// tail are touched once per frame at most.
struct ResolvedVertexBindings {
// Must equal DynamicStateShadow::kMaxShadowedVertexBindings (static_assert in
// the .cpp): past that width the bind shadow cannot skip a redundant bind
// either, so a wider layout resolves per draw. Minecraft-shaped layouts use four.
static constexpr Uint32 kMaxBindings = 8;
// Frame serial of the last completed resolve OR cross-frame revalidation.
// Zero until a resolve completes, and reset to zero before one starts, so a
// resolve that bails out midway cannot leave a half-filled entry matchable.
// Unlike the original frame-scoped memo, an entry whose buffers are all
// resident and unmapped is revalidated across frames (per-binding slice
// epoch compares) instead of re-resolved - see TryBindResolvedVertexBindings.
Uint64 frameSerial = 0;
// Identity of the resolved Vulkan layout: the VAO's content hash
// (VertexInputStateFactory::GetOrComputeHash - the same value the factory
// keys its entries on) fixes bindings.size(), each binding's base offset,
// which bindings are client/converted, and (through the mixed-in buffer
// addresses) which buffer each binding reads. Compared against the VAO's
// own hash memo on the hit path, so a hit never touches the factory entry.
VertexInputStateFactory::HashType vertexInputHash = 0;
// The program's vertex input layout: decides the synthetic-binding set and
// hence the total binding count.
Uint32 activeAttribMask = 0;
Uint32 bindingCount = 0;
// VkBufferManager::GetSliceEpochCounter() at resolve time. Still equal means
// no buffer anywhere changed its slice or was persistently mapped since, which
// settles every per-binding question below in one compare.
Uint64 sliceEpochCounter = 0;
// Any bound buffer already carrying a host map when the slice was resolved.
// Such a buffer can mutate its shadow with no API call, so it has to be
// re-pushed per draw and the one-compare path above cannot apply.
Bool anyBufferMapped = true;
// Resident element-buffer slice memo (skips the per-draw AcquireResidentSlice
// for the VAO's EBO, which cold-chases 500+ distinct resources in a
// chunk-cycling frame). Self-validating exactly like the bindings above: a hit
// requires the LIVE bound EBO pointer to equal indexBuffer AND either an
// unmoved manager-wide slice-epoch counter (nothing anywhere changed slices
// or gained a host map, the same one-compare rescue the vertex half uses) or
// that buffer's resource still carrying indexSliceEpoch (epochs are minted
// from a process-lifetime counter, so a recycled address can never
// revalidate). Restart-substituted and streamed EBOs are never stored.
// indexFrameSerial tracks the last frame the resource's GPU-use serial was
// stamped through this memo; 0 means no index memo. Independent of the
// vertex half: both are (pointer, epoch)-validated, so neither can serve
// stale state for the other.
const MG_State::GLState::BufferObject* indexBuffer = nullptr;
Uint64 indexSliceEpoch = 0;
// GetSliceEpochCounter() when the resource's epoch was last verified; only
// meaningful while indexFrameSerial matches the current frame serial.
Uint64 indexSliceEpochCounter = 0;
VkBuffer indexVkBuffer = VK_NULL_HANDLE;
VkDeviceSize indexSliceOffset = 0;
Uint64 indexFrameSerial = 0;
// Bound per draw (first bindingCount elements).
VkBuffer vkBuffers[kMaxBindings] = {};
VkDeviceSize vkOffsets[kMaxBindings] = {};
// Per binding: the VAO attribute location its buffer comes from, that buffer,
// and the buffer's VkBufferManager slice epoch when the slice was resolved.
// Only read by the per-frame revalidation and the something-moved fallback.
Uint8 attributeLocations[kMaxBindings] = {};
const MG_State::GLState::BufferObject* buffers[kMaxBindings] = {};
Uint64 sliceEpochs[kMaxBindings] = {};
};
// One direct-mapped slot of the per-VAO draw-memo table below. The key is a
// lookup hint only - a slot is never dereferenced through vaoKey; every fact it
// carries is validated against live state before use:
// - layoutHash/layoutAuxMasks are valid only while contentHash equals the LIVE
// VAO's own hash memo (which the VAO's config version guards), so a config
// change, a buffer rebind, or a recycled VAO address with a different
// configuration all miss. A recycled address with a byte-identical
// configuration AND identical bound buffers reproduces the content hash, and
// then the facts are correct by construction (they are a pure function of it).
// - bindings revalidates per draw exactly as before (frame serial, content
// hash, per-binding live buffer pointers and slice epochs).
struct alignas(64) VaoDrawMemo {
const MG_State::GLState::VertexArrayObject* vaoKey = nullptr;
// The VAO content hash (VertexInputStateFactory::GetOrComputeHash) the two
// layout facts below were derived from; 0 while nothing valid is stored.
Uint64 contentHash = 0;
Bool layoutFactsValid = false;
// The resolved layout identity + packed (unsupported, location) masks -
// the exact values GetBackendAuxMemo used to serve, moved here so the
// per-draw probe stays inside this table's one hot line instead of
// touching a second cold line of every cycled VAO object.
Uint64 layoutHash = 0;
Uint64 layoutAuxMasks = 0;
ResolvedVertexBindings bindings;
};
// Fixed-size, allocated on first use, never rehashed or swept: entries are
// recycled in place on slot collisions (two-slot probe, older frame serial
// evicted), and stale entries self-invalidate through the compares above. A
// fixed table also makes every VaoDrawMemo/ResolvedVertexBindings pointer
// stable for the duration of a draw, which the EBO memo handoff
// (m_currentDrawResolvedEntry) relies on.
static constexpr Uint32 kVaoDrawMemoSlotCount = 2048; // power of two
Vector<VaoDrawMemo> m_vaoDrawMemoTable;
// Finds the slot holding `vao`, or recycles the older of its two candidate
// slots into an empty memo keyed on `vao`. Never returns null.
VaoDrawMemo* LookupVaoDrawMemo(const MG_State::GLState::VertexArrayObject* vao);
// The current draw's memo entry, set by UploadAndBindVertexBuffers and consumed
// by the same draw's UploadAndBindIndexBuffer (the EBO memo lives in the same
// entry). Valid ONLY within that window: the next draw's lookup can recycle the
// slot. Null when the draw's layout is not memoisable.
ResolvedVertexBindings* m_currentDrawResolvedEntry = nullptr;
void CreateInstance();
VkResult SetupDebugMessenger();
VkResult DestroyDebugMessenger();
@@ -809,10 +999,25 @@ namespace MobileGL::MG_Backend::DirectVulkan {
const MG_State::GLState::ProgramObject& program,
const ProgramFactory::VkProgramObject& programObj);
// The per-draw dynamic-state tail (viewport, scissor, blend constants, depth
// bias, line width, stencil), gated behind one render-state-parameters-version
// compare per command buffer - see the gate fields in DynamicStateShadow.
void ApplyDynamicDrawStateTail(FrameContext::FrameData& frame, const IntVec2& extent, Bool isDefaultFbo);
Bool UploadAndBindVertexBuffers(VkCommandBuffer commandBuffer, const MG_State::GLState::VertexArrayObject& vao,
const ProgramFactory::VkProgramObject& programObj,
const DrawCmdParam& drawParams,
const IndexBufferView* pIndexBufferView);
// Binds `entry`'s memoised buffers when every input it was resolved from is
// still live and unchanged, else returns false and leaves nothing bound.
// vaoContentHash is the VAO's memoised content hash (GetBackendHashMemo), which
// pins the layout AND the bound buffers without resolving the factory entry.
// Non-const entry: a cross-frame revalidation refreshes its serial/epoch stamps.
Bool TryBindResolvedVertexBindings(VkCommandBuffer commandBuffer,
const MG_State::GLState::VertexArrayObject& vao,
ResolvedVertexBindings& entry,
Uint64 vaoContentHash,
Uint32 activeAttribMask, Uint64 frameSerial);
Bool UploadAndBindIndexBuffer(FrameContext::FrameData& frame,
const MG_State::GLState::VertexArrayObject& vao,
const IndexBufferView* pIndexBufferView = nullptr);
+63 -16
View File
@@ -12,8 +12,10 @@
* libEGL.so.1 for the native driver, or a libMobileGL.so path for either
* MobileGL backend selected with MOBILEGL_BACKEND_TYPE), creates a desktop-GL
* context on a small pbuffer, renders into its own FBO and paces frames with
* glFinish. No window system is required beyond what the provider itself
* needs - see run_driver_bench.sh.
* glFinish. No window system is required: the default display is tried first
* so a desktop run reaches the real driver, and a headless box (CI, a build
* server) falls back to EGL_MESA_platform_surfaceless - see
* run_driver_bench.sh.
*
* Every case models one hot pattern from captured Minecraft traces:
* draw_tiny back-to-back glDrawElements, shared state (chunk batch)
@@ -67,6 +69,7 @@ typedef unsigned int EGLenum;
#define EGL_CONTEXT_MINOR_VERSION 0x30FB
#define EGL_CONTEXT_OPENGL_PROFILE_MASK 0x30FD
#define EGL_CONTEXT_OPENGL_CORE_PROFILE_BIT 0x00000001
#define EGL_PLATFORM_SURFACELESS_MESA 0x31DD
/* ---- GL constants ---- */
#define GL_COLOR_BUFFER_BIT 0x00004000
@@ -89,6 +92,11 @@ typedef unsigned int EGLenum;
#define GL_NEAREST 0x2600
#define GL_NEAREST_MIPMAP_LINEAR 0x2702
#define GL_DEPTH_TEST 0x0B71
#define GL_BLEND 0x0BE2
#define GL_SRC_ALPHA 0x0302
#define GL_ONE_MINUS_SRC_ALPHA 0x0303
#define GL_ONE 1
#define GL_ZERO 0
#define GL_VERTEX_SHADER 0x8B31
#define GL_FRAGMENT_SHADER 0x8B30
#define GL_COMPILE_STATUS 0x8B81
@@ -133,6 +141,9 @@ static void* g_provider;
GLF(void, glClear, (unsigned))
GLF(void, glClearColor, (float, float, float, float))
GLF(void, glEnable, (GLenum))
GLF(void, glDisable, (GLenum))
GLF(void, glBlendFuncSeparate, (GLenum, GLenum, GLenum, GLenum))
GLF(void, glDrawBuffers, (GLsizei, const GLenum*))
GLF(void, glViewport, (GLint, GLint, GLsizei, GLsizei))
GLF(const unsigned char*, glGetString, (GLenum))
GLF(GLenum, glGetError, (void))
@@ -280,7 +291,21 @@ static void run_case(const char* name, case_fn body, long a, long b, long opsPer
if (glGetError() != GL_NO_ERROR) fprintf(stderr, "WARN: GL error after %s\n", name);
}
/* a = draws per frame */
/* A display that needs no window system. eglGetPlatformDisplay is EGL 1.5
* core and eglGetPlatformDisplayEXT is the EGL_EXT_platform_base spelling
* older loaders ship; both are client entry points, so they resolve before
* any display exists. Only the attribute-list types differ between the two
* and this passes none, so one cast covers both. */
static EGLDisplay surfaceless_display(void) {
void* fn = dlsym(g_provider, "eglGetPlatformDisplay");
if (!fn) fn = g_eglGetProcAddress("eglGetPlatformDisplay");
if (!fn) fn = dlsym(g_provider, "eglGetPlatformDisplayEXT");
if (!fn) fn = g_eglGetProcAddress("eglGetPlatformDisplayEXT");
if (!fn) return NULL;
return ((EGLDisplay(*)(EGLenum, void*, const void*))fn)(EGL_PLATFORM_SURFACELESS_MESA,
EGL_DEFAULT_DISPLAY, NULL);
}
/* ---- EGL bootstrap: one provider library, pbuffer, desktop-GL context ---- */
static int boot_egl(void) {
const char* libpath = getenv("DRIVERBENCH_EGL_LIB");
@@ -304,14 +329,30 @@ static int boot_egl(void) {
ESYM(eglGetError)
g_eglGetProcAddress = (void* (*)(const char*))p_eglGetProcAddress;
EGLDisplay dpy = ((EGLDisplay(*)(void*))p_eglGetDisplay)(EGL_DEFAULT_DISPLAY);
if (!dpy) { fprintf(stderr, "FAIL: eglGetDisplay\n"); return 1; }
EGLint (*getError)(void) = (EGLint(*)(void))p_eglGetError;
EGLBoolean (*initialize)(EGLDisplay, EGLint*, EGLint*) =
(EGLBoolean(*)(EGLDisplay, EGLint*, EGLint*))p_eglInitialize;
/* The default display first: it is the one a windowed app would get, and
* on a desktop it is the one that reaches the real GPU - which is the
* driver this bench exists to measure. It does need a window system,
* though; Mesa's default platform is X11, so with no $DISPLAY (CI, a
* build server, ssh without forwarding) eglInitialize fails. Fall back to
* EGL_MESA_platform_surfaceless rather than give up: every case draws into
* the FBO built by build_resources(), so no window is needed for any of
* the work being timed. */
EGLint maj = 0, min = 0;
if (!((EGLBoolean(*)(EGLDisplay, EGLint*, EGLint*))p_eglInitialize)(dpy, &maj, &min)) {
fprintf(stderr, "FAIL: eglInitialize (0x%x)\n", ((EGLint(*)(void))p_eglGetError)());
return 1;
const char* how = "default display";
EGLDisplay dpy = ((EGLDisplay(*)(void*))p_eglGetDisplay)(EGL_DEFAULT_DISPLAY);
if (!dpy || !initialize(dpy, &maj, &min)) {
dpy = surfaceless_display();
how = "surfaceless display";
if (!dpy || !initialize(dpy, &maj, &min)) {
fprintf(stderr, "FAIL: eglInitialize (0x%x)\n", getError());
return 1;
}
}
fprintf(stderr, "EGL %d.%d via %s\n", maj, min, libpath);
fprintf(stderr, "EGL %d.%d via %s (%s)\n", maj, min, libpath, how);
// Desktop GL first (that is what MobileGL exposes and what the cases are
// written against), GLES 3 second so the same binary can measure a device's
@@ -348,7 +389,10 @@ static int boot_egl(void) {
EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT, EGL_NONE};
ncfg = 0;
if (!chooseConfig(dpy, esCfgAttribs, &cfg, 1, &ncfg) || ncfg < 1) {
const EGLint relaxed[] = {EGL_SURFACE_TYPE, EGL_PBUFFER_BIT, EGL_RED_SIZE, 8, EGL_NONE};
// EGL_SURFACE_TYPE 0 matches any config: a stack that offers no
// pbuffer at all is still usable through the surfaceless context
// path below.
const EGLint relaxed[] = {EGL_SURFACE_TYPE, 0, EGL_RED_SIZE, 8, EGL_NONE};
if (!chooseConfig(dpy, relaxed, &cfg, 1, &ncfg) || ncfg < 1) {
fprintf(stderr, "FAIL: eglChooseConfig\n");
return 1;
@@ -358,20 +402,22 @@ static int boot_egl(void) {
ctx = createContext(dpy, cfg, EGL_NO_CONTEXT, esCtxAttribs);
}
if (ctx == EGL_NO_CONTEXT) {
fprintf(stderr, "FAIL: eglCreateContext (0x%x)\n", ((EGLint(*)(void))p_eglGetError)());
fprintf(stderr, "FAIL: eglCreateContext (0x%x)\n", getError());
return 1;
}
/* The pbuffer only exists to have something to make current - nothing is
* ever drawn to it. Where there is no pbuffer config, EGL_NO_SURFACE is
* exactly what EGL_KHR_surfaceless_context takes, so the same call covers
* both. */
const EGLint pbAttribs[] = {EGL_WIDTH, 64, EGL_HEIGHT, 64, EGL_NONE};
EGLSurface surf = ((EGLSurface(*)(EGLDisplay, EGLConfig, const EGLint*))p_eglCreatePbufferSurface)(
dpy, cfg, pbAttribs);
if (surf == EGL_NO_SURFACE) {
fprintf(stderr, "FAIL: eglCreatePbufferSurface (0x%x)\n", ((EGLint(*)(void))p_eglGetError)());
return 1;
}
if (surf == EGL_NO_SURFACE)
fprintf(stderr, "no pbuffer (0x%x), using a surfaceless context\n", getError());
if (!((EGLBoolean(*)(EGLDisplay, EGLSurface, EGLSurface, EGLContext))p_eglMakeCurrent)(dpy, surf,
surf, ctx)) {
fprintf(stderr, "FAIL: eglMakeCurrent (0x%x)\n", ((EGLint(*)(void))p_eglGetError)());
fprintf(stderr, "FAIL: eglMakeCurrent (0x%x)\n", getError());
return 1;
}
@@ -385,6 +431,7 @@ static int boot_egl(void) {
if (!name) { fprintf(stderr, "FAIL: resolve %s\n", #name); return 1; } \
} while (0)
RESOLVE(glClear); RESOLVE(glClearColor); RESOLVE(glEnable); RESOLVE(glViewport);
RESOLVE(glDisable); RESOLVE(glBlendFuncSeparate); RESOLVE(glDrawBuffers);
RESOLVE(glGetString); RESOLVE(glGetError); RESOLVE(glFinish); RESOLVE(glFlush);
RESOLVE(glGenBuffers); RESOLVE(glBindBuffer); RESOLVE(glBufferData); RESOLVE(glBufferSubData);
RESOLVE(glGenVertexArrays); RESOLVE(glBindVertexArray); RESOLVE(glEnableVertexAttribArray);
@@ -34,6 +34,9 @@ static GLuint g_uboRing;
static GLint g_uboAlign = 256;
static size_t g_uboSlot = 256;
static GLuint g_sampler;
/* Two small offscreen targets for the 26.2-style render-pass churn case. */
static GLuint g_passFbo[2];
static GLuint g_passColor[2];
static float g_mvp[16] = {0.002f, 0, 0, 0, 0, 0.002f, 0, 0, 0, 0, -0.001f, 0, -1.f, -1.f, 0.f, 1.f};
/* Minecraft chunk vertex: pos 3f, color 4ub, uv 2f, packed light 2s -> 32 B */
@@ -162,10 +165,13 @@ static void setup_vao(GLuint vao, GLuint vbo, GLuint ibo) {
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, ibo);
}
static GLuint g_mainFbo;
static void build_resources(void) {
/* offscreen render target: 1280x720 RBO FBO, like CTS fbo surface mode */
GLuint fbo, rboColor, rboDepth;
glGenFramebuffers(1, &fbo);
g_mainFbo = fbo;
glBindFramebuffer(GL_FRAMEBUFFER, fbo);
glGenRenderbuffers(1, &rboColor);
glBindRenderbuffer(GL_RENDERBUFFER, rboColor);
@@ -257,6 +263,21 @@ static void build_resources(void) {
glBufferData(GL_UNIFORM_BUFFER, 4 * 1024 * 1024, g_scratch, GL_DYNAMIC_DRAW);
glBindBuffer(GL_UNIFORM_BUFFER, 0);
for (int i = 0; i < 2; ++i) {
glGenFramebuffers(1, &g_passFbo[i]);
glBindFramebuffer(GL_FRAMEBUFFER, g_passFbo[i]);
glGenRenderbuffers(1, &g_passColor[i]);
glBindRenderbuffer(GL_RENDERBUFFER, g_passColor[i]);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8, 256, 256);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, g_passColor[i]);
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) {
bench_gl_failed("pass FBO incomplete", "");
return;
}
}
/* back to the main offscreen target the harness set up */
glBindFramebuffer(GL_FRAMEBUFFER, g_mainFbo);
glGenSamplers(1, &g_sampler);
glSamplerParameteri(g_sampler, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glSamplerParameteri(g_sampler, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
@@ -516,6 +537,72 @@ static void case_mc_sampler_churn(int frame, long a, long b) {
}
/* 26.2 switches render targets constantly: 132 glBindFramebuffer and 198
* glDrawBuffers per frame. Pass switching is where a Vulkan backend pays for
* render-pass breaks, so this case is the one to watch on Magma. a = passes. */
static void case_mc_pass_switch(int frame, long a, long b) {
(void)frame; (void)b;
static const GLenum kColor0[1] = {GL_COLOR_ATTACHMENT0};
glBindVertexArray(g_vao[0]);
for (long i = 0; i < a; ++i) {
glBindFramebuffer(GL_FRAMEBUFFER, g_passFbo[i & 1]);
glDrawBuffers(1, kColor0);
glViewport(0, 0, 256, 256);
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
}
glBindFramebuffer(GL_FRAMEBUFFER, g_mainFbo);
glViewport(0, 0, 1280, 720);
}
/* Blaze3D toggles blend around batches: 46 glEnable/glDisable pairs and 28
* glBlendFuncSeparate per vanilla frame. a = toggle pairs. */
static void case_mc_state_toggle(int frame, long a, long b) {
(void)frame; (void)b;
glBindVertexArray(g_vao[0]);
for (long i = 0; i < a; ++i) {
glEnable(GL_BLEND);
glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE, GL_ZERO);
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
glDisable(GL_BLEND);
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
}
}
/* 26.2 re-sets texture parameters relentlessly - 612 glTexParameteri per frame,
* almost always to the value already in place. Measures redundant-param
* filtering. a = parameter writes. */
static void case_mc_tex_param(int frame, long a, long b) {
(void)frame; (void)b;
glBindVertexArray(g_vao[0]);
glBindTexture(GL_TEXTURE_2D, g_texAtlas);
for (long i = 0; i < a; i += 4) {
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST_MIPMAP_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
}
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
}
/* Sodium switches programs mid-frame far more than vanilla: 62 glUseProgram and
* 60 mat4 uploads per frame. a = program switches. */
static void case_mc_use_program(int frame, long a, long b) {
(void)frame; (void)b;
glBindVertexArray(g_vao[0]);
for (long i = 0; i < a; ++i) {
if (i & 1) {
glUseProgram(g_progEntity);
glUniformMatrix4fv(g_uMvpEntity, 1, 0, g_mvp);
} else {
glUseProgram(g_progChunk);
glUniformMatrix4fv(g_uMvpChunk, 1, 0, g_mvp);
}
glDrawElements(GL_TRIANGLES, g_quadsPerSection * 6, GL_UNSIGNED_INT, 0);
}
glUseProgram(g_progChunk);
}
/* ---- the case table both harnesses iterate --------------------------------
* a/b are the case's own knobs; opsPerFrame is what one bench frame is
* normalised by, so ns_per_op compares across renderers. The mc_* rates are
@@ -536,6 +623,10 @@ static const BenchCaseDesc kBenchCases[] = {
{"mc_tex_stream", case_mc_tex_stream, 95, 0, 95},
{"mc_uniform_lookup", case_mc_uniform_lookup, 41, 0, 41},
{"mc_sampler_churn", case_mc_sampler_churn, 306, 0, 306},
{"mc_pass_switch", case_mc_pass_switch, 132, 0, 132},
{"mc_state_toggle", case_mc_state_toggle, 46, 0, 46},
{"mc_tex_param", case_mc_tex_param, 612, 0, 612},
{"mc_use_program", case_mc_use_program, 62, 0, 62},
{"draw_tiny", case_draw_tiny, 2048, 0, 2048},
{"draw_uniform", case_draw_uniform, 2048, 0, 2048},
{"draw_multi_vao", case_draw_multi_vao, 2048, 0, 2048},
+12
View File
@@ -120,6 +120,18 @@ namespace MobileGL {
// per-draw sampled-texture set.
Uint64 GetTextureBindGeneration() const { return m_textureState.GetTextureBindGeneration(); }
void BumpTextureBindGeneration() { m_textureState.BumpTextureBindGeneration(); }
// Monotonic counter bumped whenever a texture's shape or a sampler object's
// parameters change, i.e. whenever a bound texture's mipmap-completeness (and so
// whether a backend binds it at all) can have flipped without any bind moving;
// see TextureState::GetSamplingResolutionGeneration.
Uint64 GetSamplingResolutionGeneration() const {
return m_textureState.GetSamplingResolutionGeneration();
}
void BumpSamplingResolutionGeneration() { m_textureState.BumpSamplingResolutionGeneration(); }
// Never-reused id of this context, for backend memos keyed on the two counters
// above: both restart at 0 in a new context, and a recreated context can land on
// the old heap address. See TextureState::GetContextId.
Uint64 GetTextureContextId() const { return m_textureState.GetContextId(); }
Bool ValidateTextureName(Uint index) const;
Bool ValidateTextureObject(Uint index) const;
Int GetActiveTextureUnit() const;
@@ -8,6 +8,8 @@
#include "SamplerObject.h"
#include <MG_State/GLState/Core.h>
#include <atomic>
namespace MobileGL {
@@ -22,81 +24,91 @@ namespace MobileGL {
SamplerObject::SamplerObject(Uint externalIndex)
: m_externalIndex(externalIndex), m_lifetimeId(AllocateLifetimeId()) {}
void SamplerObject::BumpVersion() {
++m_version;
// Every setter early-outs on an unchanged value, so this only runs on a real
// parameter change. The generation is bumped for ALL parameters, not just filter
// ones that feed mipmap-completeness: a backend memo of the resolved per-unit
// bindings must never miss an invalidation, and over-invalidating on a wrap-mode
// write costs one re-resolve.
if (pGLContext) pGLContext->BumpSamplingResolutionGeneration();
}
void SamplerObject::SetWrapS(SamplerWrapMode mode) {
if (mode == m_samplerParameters.wrapS) return;
m_samplerParameters.wrapS = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetWrapT(SamplerWrapMode mode) {
if (mode == m_samplerParameters.wrapT) return;
m_samplerParameters.wrapT = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetWrapR(SamplerWrapMode mode) {
if (mode == m_samplerParameters.wrapR) return;
m_samplerParameters.wrapR = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetMinFilter(SamplerFilterMode mode) {
if (mode == m_samplerParameters.minFilter) return;
m_samplerParameters.minFilter = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetMagFilter(SamplerFilterMode mode) {
if (mode == m_samplerParameters.magFilter) return;
m_samplerParameters.magFilter = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetMipmapMode(SamplerMipmapMode mode) {
if (mode == m_samplerParameters.mipmapMode) return;
m_samplerParameters.mipmapMode = mode;
++m_version;
BumpVersion();
}
void SamplerObject::SetLodRange(Float minLod, Float maxLod) {
if (minLod == m_samplerParameters.minLod && maxLod == m_samplerParameters.maxLod) return;
m_samplerParameters.minLod = minLod;
m_samplerParameters.maxLod = maxLod;
++m_version;
BumpVersion();
}
void SamplerObject::SetLodBias(Float bias) {
if (bias == m_samplerParameters.lodBias) return;
m_samplerParameters.lodBias = bias;
++m_version;
BumpVersion();
}
void SamplerObject::SetMaxAnisotropy(Float maxAnisotropy) {
if (maxAnisotropy == m_samplerParameters.maxAnisotropy) return;
m_samplerParameters.maxAnisotropy = maxAnisotropy;
++m_version;
BumpVersion();
}
void SamplerObject::SetSamplerCompareFunc(SamplerCompareFunc func) {
if (func == m_samplerParameters.compareFunc) return;
m_samplerParameters.compareFunc = func;
++m_version;
BumpVersion();
}
void SamplerObject::SetCompareMode(SamplerCompareMode mode) {
if (mode == m_samplerParameters.compareMode) return;
m_samplerParameters.compareMode = mode;
++m_version;
BumpVersion();
}
SamplerWrapMode SamplerObject::GetWrapS() const {
@@ -153,7 +165,7 @@ namespace MobileGL {
m_samplerParameters.borderColorUI =
UintVec4(static_cast<Uint32>(color.x()), static_cast<Uint32>(color.y()),
static_cast<Uint32>(color.z()), static_cast<Uint32>(color.w()));
++m_version;
BumpVersion();
}
void SamplerObject::SetBorderColorI(const IntVec4& color) {
@@ -166,7 +178,7 @@ namespace MobileGL {
m_samplerParameters.borderColor =
FloatVec4(static_cast<Float>(color.x()), static_cast<Float>(color.y()),
static_cast<Float>(color.z()), static_cast<Float>(color.w()));
++m_version;
BumpVersion();
}
void SamplerObject::SetBorderColorUI(const UintVec4& color) {
@@ -179,7 +191,7 @@ namespace MobileGL {
m_samplerParameters.borderColor =
FloatVec4(static_cast<Float>(color.x()), static_cast<Float>(color.y()),
static_cast<Float>(color.z()), static_cast<Float>(color.w()));
++m_version;
BumpVersion();
}
const FloatVec4& SamplerObject::GetBorderColor() const {
@@ -128,6 +128,12 @@ namespace MobileGL {
private:
static Uint64 AllocateLifetimeId();
// The ONLY way m_version may move. Besides marking this object's parameters
// dirty for the backends it bumps the context-wide sampling-resolution
// generation: MIN_FILTER decides whether a lookup reads the mip chain, which
// decides whether a bound texture is mipmap-complete, which decides whether a
// backend binds it on its unit at all.
void BumpVersion();
const Uint m_externalIndex;
const Uint64 m_lifetimeId;
@@ -25,6 +25,18 @@ namespace MobileGL {
return s_nextTextureLifetimeId.fetch_add(1, std::memory_order_relaxed);
}
void TextureObjectBase::BumpShapeVersion() {
++m_shapeVersion;
// Shape is what mipmap-completeness is computed from, and completeness decides
// whether a backend binds this texture on its unit at all. Nothing else tells a
// backend memo of the resolved per-unit bindings that the answer moved - no bind
// changed and the texel content may be untouched. Proxy textures (used only to
// answer PROXY queries) are never bound, so their shape churn costs a memo
// invalidation for nothing; that is accepted rather than filtered, because a
// missed bump renders wrong pixels while a spare bump only costs one re-resolve.
if (pGLContext) pGLContext->BumpSamplingResolutionGeneration();
}
TextureObjectBase::TextureObjectBase(TextureTarget target, Uint externalIndex)
: m_externalIndex(externalIndex), m_lifetimeId(AllocateLifetimeId()), m_target(target) {
m_sampler = MakeShared<SamplerObject>(0);
@@ -85,7 +97,7 @@ namespace MobileGL {
}
m_internalFormat = format;
++m_shapeVersion;
BumpShapeVersion();
++m_textureParamsVersion;
}
@@ -197,7 +209,7 @@ namespace MobileGL {
m_levelRange.y() = m_levelRange.x();
}
++m_textureParamsVersion;
++m_shapeVersion;
BumpShapeVersion();
}
void TextureObjectBase::SetMaxLevel(Uint maxLevel) {
@@ -208,7 +220,7 @@ namespace MobileGL {
m_levelRange.y() = maxLevel;
++m_textureParamsVersion;
++m_shapeVersion;
BumpShapeVersion();
}
Bool TextureObjectBase::IsImmutable() const {
@@ -291,12 +303,12 @@ namespace MobileGL {
void TextureObjectWithOneMipmap::AllocateStorage(TextureUploadTarget uploadTarget, Uint mipmapLevel,
MipmapInput input) {
++m_shapeVersion;
BumpShapeVersion();
m_textureStorage.AllocateLevel(GetIndexOfTextureUploadTarget(uploadTarget), mipmapLevel, input);
}
void TextureObjectWithOneMipmap::TruncateMipmapLevels(TextureUploadTarget uploadTarget, Uint levelCount) {
++m_shapeVersion;
BumpShapeVersion();
m_textureStorage.TruncateToLevelCount(GetIndexOfTextureUploadTarget(uploadTarget), levelCount);
}
@@ -121,6 +121,12 @@ namespace MobileGL::MG_State::GLState {
protected:
static Uint64 AllocateLifetimeId();
// The ONLY way m_shapeVersion may move. Besides invalidating this object's own
// completeness memo it bumps the context-wide sampling-resolution generation, which is
// what a backend memo of the resolved per-unit bindings watches: completeness decides
// whether a bound texture reaches its native target at all, and a shape change is
// otherwise invisible to such a memo (no bind moved).
void BumpShapeVersion();
const Uint m_externalIndex;
const Uint64 m_lifetimeId;
@@ -28,12 +28,12 @@ namespace MobileGL {
void TextureObject2DCube::AllocateStorage(TextureUploadTarget uploadTarget, Uint mipmapLevel,
MipmapInput input) {
++m_shapeVersion;
BumpShapeVersion();
m_textureStorage.AllocateLevel(GetIndexOfTextureUploadTarget(uploadTarget), mipmapLevel, input);
}
void TextureObject2DCube::TruncateMipmapLevels(TextureUploadTarget uploadTarget, Uint levelCount) {
++m_shapeVersion;
BumpShapeVersion();
m_textureStorage.TruncateToLevelCount(GetIndexOfTextureUploadTarget(uploadTarget), levelCount);
}
@@ -7,6 +7,8 @@
// End of Source File Header
#include "TextureState.h"
#include <atomic>
#include "Defines.h"
#include "TextureEnum.h"
#include "TextureObject.h"
@@ -18,6 +20,12 @@
#include "TextureObjectStubs.h"
namespace MobileGL::MG_State::GLState {
static std::atomic<Uint64> s_nextTextureStateContextId = 1;
Uint64 TextureState::AllocateContextId() {
return s_nextTextureStateContextId.fetch_add(1, std::memory_order_relaxed);
}
static SharedPtr<ITextureObject> MakeTextureObjectForTarget(Uint index, TextureTarget target) {
switch (target) {
case TextureTarget::Texture1D:
@@ -50,7 +58,7 @@ namespace MobileGL::MG_State::GLState {
}
}
TextureState::TextureState() : m_indexGenerator(1024, 1) {
TextureState::TextureState() : m_contextId(AllocateContextId()), m_indexGenerator(1024, 1) {
// GL 3.3 core 3.8: each target owns one default texture object (name 0) per context,
// shared across all texture units, and it is the initial binding of every unit/target
// slot. It is created outside m_textureObjects so name-based paths (glIsTexture,
@@ -84,8 +84,33 @@ namespace MobileGL::MG_State::GLState {
Uint64 GetTextureBindGeneration() const { return m_textureBindGeneration; }
void BumpTextureBindGeneration() { ++m_textureBindGeneration; }
// Sibling of the bind generation for everything that changes WHICH native texture a
// backend ends up putting on a unit WITHOUT any binding moving. Two families feed it:
// a texture's SHAPE (internal format, stored level set, level range - all that
// mipmap-completeness is computed from) and any sampler object's parameters (MIN_FILTER
// decides whether the mip chain is read at all, and an incomplete-for-the-filter texture
// is deliberately left unbound so it samples as (0,0,0,1)). Deliberately coarse - ANY
// texture, ANY sampler - so that no mutation can slip past a per-unit binding memo; the
// setters that feed it all early-out when the value is unchanged, so the redundant
// glTexParameteri calls applications issue every frame do not churn it. Kept separate
// from the bind generation because the sampled texture SET is unaffected by these, and
// the Vulkan backend's set memo keys on that one.
Uint64 GetSamplingResolutionGeneration() const { return m_samplingResolutionGeneration; }
void BumpSamplingResolutionGeneration() { ++m_samplingResolutionGeneration; }
// Globally-unique, never-reused id of THIS texture state, i.e. of the context that owns
// it. Both generations above restart at 0 with a new context, so a backend memo keyed on
// them alone would accept a destroyed-and-recreated context whose counters happen to line
// up - and the heap address is no help, since a context freed and remade lands on it
// again (the unit tests do exactly that between cases).
Uint64 GetContextId() const { return m_contextId; }
private:
static Uint64 AllocateContextId();
const Uint64 m_contextId;
Uint64 m_textureBindGeneration = 0;
Uint64 m_samplingResolutionGeneration = 0;
Int m_maxTouchedUnit = -1;
Int m_activeTextureUnit = 0;
Array<TextureUnit, MAX_TEXTURE_IMAGE_UNITS> m_textureUnits;
@@ -8,6 +8,8 @@
#include "TextureUnit.h"
#include <MG_State/GLState/Core.h>
namespace MobileGL::MG_State::GLState {
TextureUnit::TextureUnit() : m_sampler(nullptr) {
for (int i = 0; i < (int)TextureTarget::TextureTargetCount; ++i) {
@@ -24,7 +26,17 @@ namespace MobileGL::MG_State::GLState {
}
void TextureUnit::SetSamplerObject(const SharedPtr<SamplerObject>& sampler) {
if (m_sampler == sampler) return;
m_sampler = sampler;
// Which sampler object a unit carries is part of "what is bound at this unit": it
// overrides the texture's own sampler state, so it selects the filter that decides a
// bound texture's mipmap-completeness. glBindSampler already bumps the generation
// through NoteUnitTouched, but glDeleteSamplers unbinds the deleted object from every
// unit straight through here (GLContext::MarkSamplerObjectForDeletion) and would
// otherwise leave a backend memo of the resolved per-unit bindings replaying the
// deleted sampler.
if (pGLContext) pGLContext->BumpTextureBindGeneration();
}
const SharedPtr<SamplerObject>& TextureUnit::GetSamplerObject() const {
@@ -149,6 +149,25 @@ namespace MobileGL {
m_backendStateMemoVersion = m_configVersion;
}
// Backend-owned aux memo: two opaque VALUE words (no pointee, so unlike the
// state memo above they need no eviction-epoch guard), valid while the config
// version matches. They live next to m_configVersion, which every per-draw
// path already loads, so a backend can re-read small derived facts about this
// VAO's configuration (e.g. a layout hash and attribute masks) without
// chasing into its own cache's heap entry - that chase is a guaranteed cache
// miss when an app cycles hundreds of VAOs per frame.
Bool GetBackendAuxMemo(Uint64& outAux0, Uint64& outAux1) const {
if (m_backendAuxMemoVersion != m_configVersion) return false;
outAux0 = m_backendAuxMemo0;
outAux1 = m_backendAuxMemo1;
return true;
}
void SetBackendAuxMemo(Uint64 aux0, Uint64 aux1) const {
m_backendAuxMemo0 = aux0;
m_backendAuxMemo1 = aux1;
m_backendAuxMemoVersion = m_configVersion;
}
private:
void BumpAttributeFormatVersion(Uint index);
void BumpAttributeBufferVersion(Uint index);
@@ -185,6 +204,9 @@ namespace MobileGL {
mutable const void* m_backendStateMemo = nullptr;
mutable Uint64 m_backendStateMemoEpoch = 0;
mutable Uint32 m_backendStateMemoVersion = ~0u;
mutable Uint64 m_backendAuxMemo0 = 0;
mutable Uint64 m_backendAuxMemo1 = 0;
mutable Uint32 m_backendAuxMemoVersion = ~0u;
};
} // namespace GLState
} // namespace MG_State
@@ -9,20 +9,26 @@
#include "VertexArrayState.h"
namespace MobileGL::MG_State::GLState {
namespace {
// Shared "nothing bound" answer for GetBoundVertexArray/GetVertexArrayObject.
// Function-local statics carry a guard check per access; this one is
// constant-initialized and lives on the hot per-draw path.
const SharedPtr<VertexArrayObject> kNullVertexArrayObject = nullptr;
} // namespace
VertexArrayState::VertexArrayState() : m_indexGenerator(1024, 1) {
// Generate default VAO at index 0, which is not valid in core profile, but still remains for
// compatibility reasons.
m_indexGenerator.Insert(0);
auto defaultVAO = MakeShared<VertexArrayObject>(0);
m_vertexArrays.push_back(defaultVAO);
m_boundVertexArray = defaultVAO;
m_boundIndex = 0;
}
const SharedPtr<VertexArrayObject>& VertexArrayState::GetVertexArrayObject(Uint index) {
if (index >= m_vertexArrays.size()) {
// FIXME: report a GL error here
static SharedPtr<VertexArrayObject> nullVertexArrayObject = nullptr;
return nullVertexArrayObject;
return kNullVertexArrayObject;
}
return m_vertexArrays[index];
@@ -34,11 +40,20 @@ namespace MobileGL::MG_State::GLState {
}
void VertexArrayState::Bind(Uint index) {
const auto& vertexArray = GetVertexArrayObject(index);
// Re-binding the already-current VAO is a per-batch habit of Blaze3D-style renderers;
// skip the shared_ptr store (two atomic refcount ops) when nothing changes.
if (vertexArray == m_boundVertexArray) return;
m_boundVertexArray = vertexArray;
// Per-draw-batch hot path (Blaze3D-style renderers rebind a different VAO before
// every draw): store only the slot index - no SharedPtr copy, no refcount atomics.
// The bound object's lifetime is guaranteed by its slot (see the invariant note on
// m_boundIndex in the header).
if (m_boundDetached) [[unlikely]] {
// A cold path displaced the previously bound object out of its slot; this Bind
// supersedes it, exactly like the old SharedPtr member being overwritten.
m_boundDetached = nullptr;
}
// Match the previous semantics exactly: binding an out-of-range name, or a name
// whose slot holds no object, left the old SharedPtr member null - resolve that
// NOW, so a slot created later does not silently become bound.
m_boundIndex =
(index < m_vertexArrays.size() && m_vertexArrays[index] != nullptr) ? index : kUnboundIndex;
}
const SharedPtr<VertexArrayObject>& VertexArrayState::CreateVertexArrayObject(Uint index) {
@@ -48,14 +63,36 @@ namespace MobileGL::MG_State::GLState {
m_vertexArrays.resize(index + 1, nullptr);
}
auto& vao = m_vertexArrays[index];
if (index == m_boundIndex && vao != nullptr && !m_boundDetached) {
// Replacing the bound slot's live object: keep the OLD object alive and bound
// (that is what the previous SharedPtr member provided) until the next Bind.
// Unreachable through the GL entry points today - bind-time creation only fills
// empty slots and generated names are never in use - but the invariant is
// enforced here, not assumed.
m_boundDetached = std::move(vao);
}
vao = MakeShared<VertexArrayObject>(index);
return vao;
}
void VertexArrayState::MarkVertexArrayForDeletion(Uint index) {
if (m_indexGenerator.IsValid(index)) {
if (m_boundVertexArray && m_boundVertexArray->GetExternalIndex() == index) {
m_boundVertexArray = GetVertexArrayObject(0);
// "Deleting the bound VAO rebinds the default VAO" needs the same answer the old
// SharedPtr compare gave: either the live bound slot is the one being deleted, or
// the bound object is a detached one that carries this external index.
const Bool deletingBound = m_boundDetached
? m_boundDetached->GetExternalIndex() == index
: (m_boundIndex == index && m_boundIndex != kUnboundIndex);
if (deletingBound) {
m_boundDetached = nullptr;
m_boundIndex = 0; // the default VAO's slot always exists
if (index == 0) {
// Deleting slot 0 while it is bound (unreachable via the GL entry
// points, which filter name 0): the old SharedPtr member kept the
// object alive and bound across the slot null-out below; detach it
// to preserve that.
m_boundDetached = m_vertexArrays[0];
}
}
if (ValidateVertexArrayObject(index)) {
@@ -76,7 +113,16 @@ namespace MobileGL::MG_State::GLState {
}
const SharedPtr<VertexArrayObject>& VertexArrayState::GetBoundVertexArray() {
return m_boundVertexArray;
// NOTE: like GetVertexArrayObject, the returned reference is a slot reference and
// must not be held across CreateVertexArrayObject (vector growth) - existing
// callers bind/create first and only then take the reference.
if (m_boundDetached) [[unlikely]] {
return m_boundDetached;
}
if (m_boundIndex < m_vertexArrays.size()) {
return m_vertexArrays[m_boundIndex];
}
return kNullVertexArrayObject;
}
Vector<SharedPtr<VertexArrayObject>>& VertexArrayState::GetAllVertexArrays() {
@@ -29,9 +29,32 @@ namespace MobileGL {
Vector<SharedPtr<VertexArrayObject>>& GetAllVertexArrays();
private:
// "Nothing bound" (an out-of-range or never-created name was bound). Distinct from
// being bound to a live slot so that a slot filled AFTER such a bind does not
// retroactively become the bound VAO.
static constexpr Uint kUnboundIndex = ~static_cast<Uint>(0);
Vector<SharedPtr<VertexArrayObject>> m_vertexArrays;
IndexGenerator<Uint> m_indexGenerator;
SharedPtr<VertexArrayObject> m_boundVertexArray;
// The bound VAO is represented as an INDEX into m_vertexArrays, not as an owning
// SharedPtr copy. Chunk-style renderers rebind a different VAO before every draw,
// and the SharedPtr store this replaces cost two atomic refcount ops per bind -
// the single largest line of a vanilla-Minecraft draw profile (the lock-prefixed
// refcount RMWs serialize the store buffer in the middle of command recording).
//
// LIFETIME INVARIANT this relies on (and which the cold paths below enforce
// rather than assume): the object GetBoundVertexArray() refers to is kept alive
// by its own slot in m_vertexArrays. Every path that clears or replaces a slot
// either (a) rebinds index 0 first when it targets the bound slot
// (MarkVertexArrayForDeletion), or (b) detaches the displaced object into
// m_boundDetached (CreateVertexArrayObject), which then owns it and keeps
// GetBoundVertexArray() answering with the OLD object - exactly what the previous
// SharedPtr member did - until the next Bind drops it.
Uint m_boundIndex = 0;
// Cold-path ownership backstop, see above. Null in the steady state; Bind clears
// it (one predictable branch on the hot path).
SharedPtr<VertexArrayObject> m_boundDetached;
};
} // namespace GLState
} // namespace MG_State
@@ -41,6 +41,7 @@
#include <MG_Impl/GLImpl/Getter/GL_Getter.h>
#include <MG_Impl/GLImpl/Program/GL_Program.h>
#include <MG_Impl/GLImpl/RenderState/GL_RenderState.h>
// Disable/BlendFuncSeparate live in GL_RenderState.h; DrawBuffers in GL_Framebuffer.h - both already included.
#include <MG_Impl/GLImpl/Sampler/GL_Sampler.h>
#include <MG_Impl/GLImpl/Sync/GL_Sync.h>
#include <MG_Impl/GLImpl/Texture/GL_Texture.h>
@@ -75,6 +76,9 @@
#define glDrawElements MobileGL::MG_Impl::GLImpl::DrawElements
#define glDrawElementsBaseVertex MobileGL::MG_Impl::GLImpl::DrawElementsBaseVertex
#define glEnable MobileGL::MG_Impl::GLImpl::Enable
#define glDisable MobileGL::MG_Impl::GLImpl::Disable
#define glBlendFuncSeparate MobileGL::MG_Impl::GLImpl::BlendFuncSeparate
#define glDrawBuffers MobileGL::MG_Impl::GLImpl::DrawBuffers
#define glEnableVertexAttribArray MobileGL::MG_Impl::GLImpl::EnableVertexAttribArray
#define glFenceSync MobileGL::MG_Impl::GLImpl::FenceSync
#define glFramebufferRenderbuffer MobileGL::MG_Impl::GLImpl::FramebufferRenderbuffer