mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-07 01:53:24 +09:00
d3ec9b81791c4885384c672f2b6893cd3b8c0fc0
100
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 BZLZHH
|
d3ec9b8179 | [Docs] (docs/Performance): add a skill for reading Mali GPU counters | ||
|
BZLZHH
|
05dfff9100 |
[Perf] (MG_Backend/DirectVulkan, MG_IntegrationTest/Scenarios): defer window clears while a window pass is open
- A whole-window clear with a window pass open ends that pass and folds into the next one, instead of running a load-clear-store pass of its own - A present drops the depth/stencil half of a pending window clear |
||
|
BZLZHH
|
6c59b0391c |
[Perf] (MG_Backend/DirectVulkan): defer window clears into the next window pass
- A whole-window glClear with no pass open on the window waits for the next window pass, which loads it with LOAD_OP_CLEAR; other uses run it first - The first window pass after a present loads undefined content as DONT_CARE |
||
|
BZLZHH
|
d6fd101d2a | [Build] (CMakeLists.txt, docs/Disaggregated): default to the split in-process build | ||
|
BZLZHH
|
15ba86ce58 | [CI] (scripts/ci): re-pin revision 6 after the comment-only schema edit | ||
|
BZLZHH
|
8084a93d85 | [Fix] (android-plugin/app): match the announcement wording the supervisor prints | ||
|
BZLZHH
|
6d3b00e820 | [Chore] (docs, MobileGL, scripts, tools, .github): use US spelling in identifiers and file names | ||
|
BZLZHH
|
4f50bedde0 | [Docs, Chore] (docs, tools, MobileGL, scripts, CONTRIBUTING.md, .gitignore): translate the documentation and use US spelling | ||
|
BZLZHH
|
47892afddc | [CI, Docs] (AGENTS.md, CONTRIBUTING.md, scripts): let wide commits summarize their scope | ||
|
BZLZHH
|
de7b7ea21b | [CI, Docs] (CONTRIBUTING.md, scripts): allow an abbreviated scope on wide commits | ||
|
BZLZHH
|
6e36bf2c1b |
[Chore, Docs] (...): drop the archived notes and move the skill under tools
- the archive freeze rule goes with the directory - code comments lose citations to the removed notes |
||
|
BZLZHH
|
8302dd30bf | [Docs] (Anland): the lock screen behind the black desktop, and where the Magma text gap is not | ||
|
BZLZHH
|
a6d788191f | [Docs] (Anland): record the Magma verification and its missing glyphs | ||
|
BZLZHH
|
0d89db36dd | [Docs] (Anland): the D1100 + Mali changes and why each one was needed | ||
|
BZLZHH
|
b7a6e6c538 | [Docs] (Anland): the D1100 + Mali, kernel 4.14 bring-up and its three fixes | ||
|
BZLZHH
|
f44888ca01 |
[Fix] (Espryt): present through eglSwapBuffers when the driver refuses the damage entry point
A vendor EGL can export eglSwapBuffersWithDamageEXT and then answer EGL_FALSE on a window surface without setting an error (measured on an Android 4.14 vendor kernel with a Mali driver; the surface is current, the context is current, the ANativeWindow reports its full extent, and the plain entry point presents the same surface happily). Trusting that pointer kept the frame: the window's buffer queue never saw a buffer, the compositor's own window stayed a solid colour on screen, and every client of the session fell back to wl_shm because the compositor could not sample what it could not present. A refused damage swap now falls back to eglSwapBuffers and says so once. A driver without the extension presents through the plain call as before. |
||
|
BZLZHH
|
8245221755 |
[Fix] (Remote): name a shared image by extent and format when the kernel gives descriptors one identity
(st_dev, st_ino) is how an incoming dma-buf descriptor names the image it was exported from, and it is the only identity the registry had. Some kernels do not keep those apart: on a 4.14 vendor kernel every descriptor of every image reports dev 12 ino 10658 with st_size 0, and /proc/self/fdinfo carries nothing else, so the registry correctly detected the ambiguity and then refused every import - which is every zero-copy window a compositor ever tries to sample, leaving the whole stack on the wl_shm fallback. The identity stays the kernel's to give. When it is ambiguous, the import's own extent and format select among the live images exported under that identity, in allocation order and each claimed once, which is the order a producer allocates and hands them over in. Identify() takes the extent and the fourcc the import record already carries; nothing else changes. |
||
|
BZLZHH
|
32c79f4135 |
[Feature] (EGL): Wayland window surfaces, presented through wl_shm
A Wayland client's window is its own wl_surface, and what the compositor shows is the wl_buffer attached to it - but the pixels of a MobileGL frame are drawn where the backend is, on a split build in another process on another OS. So far a wl_egl_window was taken for an X11 window and its client-side address sent to the Android server (EGL_BAD_NATIVE_WINDOW at best), and under glvnd a Wayland display never reached this vendor at all. A window on a Wayland display - asked for as EGL_PLATFORM_WAYLAND, or a wl_display recognised by its first word being wl_display_interface - is now a window surface in the EGL state at the wl_egl_window's size with a pbuffer of that size behind it in the backend, whatever MOBILEGL_IPC_SURFACE says. At every swap the frame is read back out of the default framebuffer through this library's own glReadPixels (the application's read framebuffer, pack buffer and pack state put back as they were), its rows flipped and swizzled into a wl_shm buffer, and attached, damaged and committed on the application's wl_surface. Three buffers rotate on their release events, dispatched on an event queue of the surface's own. libwayland-client is loaded at run time, so neither the build nor the library depends on it. EGL_KHR_platform_wayland and EGL_EXT_platform_wayland are announced on Linux. Measured: glmark2-wayland runs its build scene at 13 FPS (400x300) and the window shows the frame under kwin. A window resize is not followed yet; the surface keeps its creation size. |
||
|
BZLZHH
|
6bccf98d7f | [Perf] (MG_Remote): Frame time is accounted on both sides, down to the socket reads and the client's waits. | ||
|
BZLZHH
|
3a6c5c7f04 | [Perf] (MG_Remote): The create window ships at 10, and the server now reports where its own frame time goes. | ||
|
BZLZHH
|
b017ec1c1f | [Test] (MG_IntegrationTest/Scenarios): The P4a white-box applier reads fence on a readback. | ||
|
BZLZHH
|
102f2ca7d5 | [Perf] (MG_Remote/Client): The create window's shipped depth is 6, and its ceiling 8. | ||
|
BZLZHH
|
1f1ece8a33 | [Test] (MG_IntegrationTest/Scenarios): Fence the object-death gates on a readback instead of a clear. | ||
|
BZLZHH
|
677689454a |
[Test] (P12): the window's gates move onto the arm it ships on - a real stream session with an answering peer, and it goes red by name when the store is a bounded ring
THE STRUCTURAL FIX, not just one more case. Three rounds of this feature were green in
the lanes while being inert on the device, because every window-level gate in the tree
runs in RemoteClientControls, whose session is TransportMode::InProcess - ShmLink - and
since P12 the window is OFF there by design. So the lanes and the device ran different
transports, and nothing in the tree could tell.
StreamClientPublicationTest.EveryDeferredCreateAnswerComesBackOnTheShippingArm runs on a
real StreamLink with a real ClientSession and a peer that ANSWERS: a scripted servo pops
each record, replies with its seq, advances and flushes the watermark - i.e. exactly what
the drain's WaitForApplied oracle needs, and nothing more (nothing applies a record).
IT IS PROVED TO GO RED, and by the right name. Reverting the store to "keep the last 8
arrivals" makes it die on the diagnostic added with the store:
MGPipe: Fatal{ReplyMissing, "create window"} - seq 1 was declared and APPLIED, and
its answer is no longer in the link's store; the window cannot advance past it and
nothing recovers a lost answer
AND THE FIRST VERSION OF THIS GATE WAS WORTHLESS, which is the part worth keeping: with
24 creates and nothing else it passed against the ring as well, because between a push
and its read only a couple of answers arrive. What killed the device was TRAFFIC - the
server is behind and its late answers to earlier records fill a bounded buffer first - so
the case now emits twelve MORE records whose answers are wanted (declared
SetTextureParams rows) between the first create and the rest, waits for the peer to
answer them, and only then lets the drain come back for the create. Two things were
needed to make that deterministic: the settlement wait (otherwise it is a race the ring
can win by luck) and glFlush(), because a published record reaches the peer on a flush
and not on PublishAndNotify - the neighbouring case in the same file is built on that.
ALSO IN THIS COMMIT: the assertion is the CONSERVATION LAW, not "taken == creates".
Nothing drains the window when no further create arrives, so the last few answers stay
held - taken + pending == creates, with pending <= window, is the honest contract, and
taken == creates would be asserting a drain on a path the row does not have.
318 tests green across five lanes (172 + 15 + 22 + 103 + 6).
|
||
|
BZLZHH
|
3a2555805a |
[Perf] (P12): the create window lands as the default - declared-answer retention replaces the ring, and the load frame goes 28.66 s -> 16.52 s (11.02 s at window 4)
The three things that made the deferral inert are all fixed, in the order they were
found: the reply ring could not hold an answer across the ~1,000 records between two
creates; a residency budget could not help because what displaces an answer is what
the SERVER sends, not what the client emits; and so the mechanism had to be retention.
WHAT LANDED
* ILink::DeclareReplyRead + LinkCapabilities::RetainsReplies. The wire declares a
seq will be read BEFORE it publishes, so the declaration cannot race its own
answer; a link with the capability false ignores it, and the window does not
defer on one (CreateWindowEffective() is gated on it).
* StreamLink's ring of the last 8 answers becomes a declared-answer store: a
monotone whitelist plus a cursor, answers kept until their reader TAKES them
(take-and-delete under one lock, bytes swapped into the scratch), and a capacity
of 24 that is FATAL to exceed rather than evicting. An evicted wanted answer is
unrecoverable - re-latching by sending another create wipes the server's record
(PipeApply.cpp:2033) and its PendingUploads (PipeApply.h:332) while the client
cleared its dirty flags at emission (PipeApply.h:298-304) - so the capacity is
sized so it cannot be reached and reaching it says its own name.
* An answer nobody declared is DROPPED, which is what keeps the store bounded by the
client's deferral depth instead of by the server's answer rate. Retain-on-arrival
grows ~11.3 entries per create against a server that ignores the no-reply bit.
* A read that fails after WaitForApplied has returned Reached is now Fatal{ReplyMissing}
naming the seq: on the stream arm the reply frame strictly precedes the progress
frame that publishes appliedSeq, so that combination is a broken invariant, and
retrying it (what the row used to do) wedges the window for the rest of the
process while leaving every deferred object latched on a provisional accept.
* The residency budget is deleted (WireTables.cpp keeps its history), the window
retirement branch drains once before falling back - it could otherwise strand the
answers it was holding - and Detach clears the store.
MEASURED ON THE DEVICE, load frame, both ends rebuilt, frame-sent identical per op
(create 4,536, records 103,097) throughout:
window 1 (R-5's shape) 4,536 takes 28.66 s
window 2 explicit 2,500 17.27 s
window 2 AS THE SHIPPED DEFAULT 2,563 16.52 s
window 4 1,644 11.02 s
window 2 against an OLD SERVER 2,663 16.38 s (zero Fatals)
The last row is the version-skew case the design review named as a must: a server
built before the no-reply bit answers ~55,903 records a frame, the whitelist drops
every answer nobody declared, and the deferral still reads its own back.
GATES (172 + 15 green), and the important one is proved to go red:
* ADeclaredAnswerSurvivesFarMoreInterveningAnswersThanAnyFixedBuffer - reverting the
store to "keep the last 8 arrivals" fails it immediately with "the declared answer
did not survive 64 intervening ones". That is the line between the ring-1024
stand-in and a bounded mechanism.
* AnAnswerNobodyDeclaredIsNotKeptAtAll - red if everything is retained.
* TheWindowRefusesToDeferOnALinkThatCannotRetainAnswers - on the in-process (ShmLink)
arm the knob at 4 must still block every create, red if the capability gate goes.
* The refused-create case now reads the repair that exists on THAT arm (the latch),
and asserts the window's own books are empty there.
NOT DONE, and named: the window has no unit gate on the STREAM arm yet - only the
device runs. Three rounds of this feature were green in the lanes while inert on the
device because the lanes and the device run different transports; the transport-level
retention gates close that for the store, and the window-level one should be built on
StreamClientPublicationTest's real stream session next.
|
||
|
BZLZHH
|
1ef7439625 |
[Perf] (P12): with the budget off and a deep reply buffer the window pays - 28.66 s -> 11.45 s at W=4, and the round trip does NOT grow, so the ~5 ms is a wakeup the server batches away
The second one-line experiment, and the one that decides whether the retention design is worth building. Ring 1024 plus the residency budget DISABLED (its constant raised out of reach), W=2 and W=4: arm waits rtt_mean by_op 2 wall W=1 (today's default) 4,543 5,672 us 4,536 28.66 s budget on + ring 8 9,078 2,598 us 9,071 27.04 s ring 1024 + budget 7 3,450 5,367 us 3,443 23.04 s ring 1024 + budget OFF W=2 2,560 4,916 us 2,553 16.80 s ring 1024 + budget OFF W=4 1,691 4,224 us 1,684 11.45 s rtt_mean does NOT grow (4,916 / 4,224 us, both BELOW the 5,672 us baseline) while the waits fall by 2x and 2.7x and the wall follows. So the ~5 ms is a WAKEUP round trip and the server drains a batch per wakeup - which is exactly what makes W pay, and it is the question the audit listed as undecidable from the repo and the watershed for whether this is worth doing at all. It is. AND IT BEATS THE HANDOFF'S OWN FLOOR: handoff-2 section 0.4 put the window's landing at 15-20 s, treating the client's ~13.4 s CPU as ADDITIVE with the waits. This frame turns out to be wait-dominated, so amortising four creates per wakeup lands at 11.45 s, 3.5 s below that estimate. 1024 IS A STAND-IN, NOT THE DESIGN: it works only because 1024 happens to be deep enough for this workload, and 1024 retained answers is ~2 GiB worst case (a ReadPixels answer can be 2 MiB). Retention gives the same property - the answer is still there - with a bound tied to the client's own deferral depth, which is why the implementation the audit specified is what gets built, not "make the ring 1024". Also corrects the note's section 7 path 1 (audit F15): on the stream arm SEG_REPLY and the Hello's ReplySlotCount have nothing to do with answers - they travel as Reply frames and StreamLink::ReadReply reads the client's own store - so "deepen the pool" does not address the arm that was measured, and should not be paid for. Reverted (ring 8, budget 7, probes deleted); 172 + 13 green. |
||
|
BZLZHH
|
82de3f6826 |
[Docs] (P12): the ring-1024 experiment - every deferred answer comes back (readfail 8,872 -> 0), so eviction was the whole cause and the residency budget is now the brake
One-line experiment before writing the retention table: StreamLink's kReplyRing 8 -> 1024,
nothing else changed, same load frame on the same device.
ring = 8 (today) ring = 1024
readfail 8,872 0
taken 63 4,499 / 4,500
blocked (fell back) 4,435 0
by_op 2 9,071 3,443
wall 27.9 s 23.04 s
frame-sent create 4,536, records 103,097 identical per op
TWO CONCLUSIONS. First, eviction was the ONLY cause: with a buffer deep enough, every
deferred answer was read back and not one create fell through to the blocking path.
The wedge (want=310) and the useless budget (budget=4,451) were two faces of "eight
slots is not enough". Second, what brakes now is the BUDGET, not the buffer: by_op 2 =
3,443 matches the number of times the budget forced an early read (3,395), i.e. every
one of those is a real blocking wait. Its constant is 7, sized for the 8-slot pool, so
against a 1,024-deep buffer it became the only throttle and the wall only moved 27.9 ->
23.0 s.
WHAT IT MEANS FOR THE DESIGN. Once the answers are RETAINED until read, correctness no
longer depends on how long one survives in a bounded buffer, so the budget must be
sized to the RETENTION CAPACITY rather than to the pool - or be replaced by "retained
means safe". by_op 2 = records/W then puts W=2 at ~2,268 waits and the wall at ~15 s,
which is the goal's own success criterion.
Experiment reverted (ring back to 8, probes deleted); 172 + 13 + 22 + 103 green.
|
||
|
BZLZHH
|
b1ef484116 |
[Perf] (MG_Remote): the residency budget does not make the window usable either - the answer is gone before the client next looks, so a deferral needs RETENTION, not a budget
Steps 1 and 2 of the plan the design review set out, landed and measured on the
device. Step 1 answers the question step 2 depended on; step 2 answers the one the
whole line depended on, and the answer is that the window's shape is wrong.
STEP 1 - CAN AN ANSWER BE LOST IN DELIVERY? NO. ReplyPool now counts what the link
took (PostedReplies) and what it refused (FailedReplies), and a non-OK result is
NAMED instead of swallowed: it checked only BUFFER_TOO_SMALL and let
TRANSPORT_CLOSED vanish with no line and no counter, so "the server never sent it"
and "it was sent and the buffer overwrote it" were the same symptom with opposite
fixes. Measured: posted=4501 failed=0. The twenty-answer gap between the server's
tally and the client's was a sampling difference, not a drop - and the server's
m_answered counts ATTEMPTS (incremented before PostReply), which is why it could
never have settled it.
STEP 2 - DOES A RESIDENCY BUDGET FIX THE WINDOW? NO.
Wire_ResourceCreate now measures how many reply-owning records the client has
emitted since the oldest window entry was pushed (ClientSession::ReplyPostings, a
new clock) and takes the oldest answer back as soon as that approaches the reply
buffer's depth, instead of deferring past the point where the answer can still
exist. On the device:
P12win call=4501 cfg=2 eff=2 pending=2 taken=63 pushed=65 blocked=4435
budget=4451 readfail=8872 posts=54377
kind=frame-op frame=14 wait_replies=9078 by_op=2=9071 wall 27.87 s
budget=4451: it fires on essentially EVERY create, the window stops deferring
(pushed=65), and the drain still reads nothing back. THE REASON IS STRUCTURAL, not
a constant:
* the window only gets control WHEN A CREATE ARRIVES. Between two creates this
client emits on the order of a thousand records - at the FIRST failed read the
reply ring already spanned [1175..1504] while the head was seq 310 - and nothing
looks at the window in that gap;
* what displaces a deferred answer is not what this client emits, it is WHAT THE
SERVER ANSWERS. The server is the one behind (one answer per ~5 ms wakeup while
the client runs ahead), so the replies arriving during the gap are its LATE
ANSWERS TO THE EARLIER CREATES - they fill the 8 slots first;
* so the check fires too late by construction. Tightening it does not fix that; it
only removes the deferral entirely, which is what pushed=65 is.
WHAT THAT LEAVES. A deferral is readable only if the client KEEPS the answers it
deferred. The buffer is bounded by COUNT and the client cannot bound how many
answers the server will post between two of its creates, so hoping the buffer still
holds one is a race the client loses - which is also why the earlier two diagnoses
(reply-pool churn, then FIFO-with-no-way-out) were both describing symptoms of the
same thing. The shape that can work is retention: the reader keeps the answers for
the seqs still outstanding (a handful), drops them when read, and reports a LOSS
when a read asks for one it no longer has. That is a transport change - StreamLink's
ring becomes a retention map with a cap and a lost counter - and it is the next step.
BOTH STEPS STAY: step 1 is a net gain on its own, and step 2 is the safe direction
(it can only degrade the window to no-deferral, i.e. the pre-window shape) and its
clock is the one the retention design needs. MOBILEGL_IPC_CREATE_WINDOW stays at 1.
172/172 and 13/13 after the probes were removed. The measurements and the revised
conclusion are in notes/p12/CREATE-WINDOW-MEASURED.md section 9.
|
||
|
BZLZHH
|
4e48702f26 |
[Feat] (MG_Remote): a record can tell the server its answer will never be read - the bit works and cuts a frame's replies by 92%, and it proves the window's real blocker is the FIFO drain, not the pool depth
The create window could not collect its deferred answers. The previous commit
diagnosed that as the reply pool being 8 slots deep against 55,903 replies in one
frame, and left a no-reply bit as the way out. This lands that bit, measures it on
the device, and reports that the diagnosis was half wrong.
WHAT LANDED. ClientSession::EmitAndWaitTails takes a second question beside
wantReply: willReadReply. They are not the same question and the window is the case
that separates them - it emits with wantReply=false and READS the answer later, so
its records must still be answered. kRecNoReply (RingRecordFlags bit 5, which
aliases kOptional in the call-flag space; the flag-space asserts were extended and
the alias named) is stamped when willReadReply is false, and the decoder skips
PostReply while keeping every acceptance counter - "nobody reads the answer" is not
"nothing happened". The two illegal combinations are Fatal by name.
Three call sites say it: SetTextureParams for a confirmed object, the respecify
row's fire-and-forget half, and sub-data where MGPipeSubDataWantsItsReply decides.
MEASURED ON THE DEVICE (aikjs4s8cys8ba7l, rebuilt APK + client, both halves new):
server: skipped=50,001 answered=4,521 (a frame's replies: 55,903 -> 4,543)
client: Reply frames received seen=4,501, latest seq=97,181
frame-sent: identical per op, records untouched
create row: by_op 2 = 9,070 (was 4,536) - because the drain's blocking take is
now counted, so the row's number stays honest instead of reading as
"create stopped waiting" while the waiting merely moved
AND THE WINDOW STILL READ BACK NOTHING: readfail 8,951 -> 8,872, i.e. -0.9%.
THE REAL CAUSE, from the miss probe:
P12miss n=1 want=310 replySeq=1504 ring=[1175..1504]
P12miss n=9001 want=310 replySeq=102915 ring=[102866..102915]
want=310 every time. The drain is STRICTLY FIFO and a failed read has no way out -
it retries the same head for ever. Any reply buffer is bounded by design (8 pool
slots, or the 8-frame ring the stream link now keeps), so with 4,536 deferred
answers at least one is overwritten before it is read, and that one wedges the
window permanently. Cutting the churn by two orders of magnitude does not help: it
turns a certainty into a certainty. That is the constraint, not the depth.
THE FIX IS NAMED AND NOT TAKEN (the goal this work ran under stops here on purpose).
On a failed read of the head, abandon it: MGPipeNoteHandleUnpublished +
NoteCreateSuspect + pop - the DECLINE path reused for "unknown". The window then
self-heals through the blocking path and the object merely gets slower. It needs its
own gate: a failed read must let the window ADVANCE.
Gates, all green (172 + 13 + 22 + 103):
PipeWireCodecTest.TheNoReplyBitSkipsTheAnswerAndKeepsTheAcceptanceRecord - the skip,
the kept tally, the answered/skipped split, and the negative control that a record
WITHOUT the bit is still answered (a server that stopped answering everything
would otherwise pass)
RemoteClientControls.ACreateWindowDefersTheAnswersAndTheWindowOffDoesNot - now also
asserts the answers CAME BACK (answers_taken=11, outstanding=1, fallbacks=0 against
12 records). RED-ONCE PROVED: giving the window's creates the literal
"fire-and-forget => skip" predicate makes this case fail.
RingTest - the sixth and last flag alias, and the hand-written exhaustiveness list
extended (with a note that it is only as exhaustive as that list).
Server side: AnsweredRecords()/SkippedAnswers() - the number that made this
debuggable, since a wire change whose effect is invisible is undebuggable.
Client-first is safe: an old server ignores bit 5 and replies as before, so the two
cross-version directions degrade to today's behaviour - this is "both ends must
upgrade to take EFFECT", not "to avoid breaking". Hit that in practice first.
|
||
|
BZLZHH
|
7c822cdc85 |
[Docs] (P12): the no-reply flag reviewed - the arithmetic holds and the flag is cheap, but the predicate has to be "this answer will never be read", not "fire-and-forget"
An independent review of the one design door the measurement left open (stop sending
the ~51,000 answers nobody reads, so the 8-slot pool stops churning and the create
window becomes collectable). Verdict: feasible, worth doing, and the proposal AS
WRITTEN would disable the thing it exists to enable.
THE CATCH. The windowed create is wantReply=false AND its answer is read -
WireTables.cpp's drain reads exactly those seqs. A bit keyed on "fire-and-forget"
therefore stops the server answering the creates the window is waiting for: the
window goes from "cannot be collected" to "there is nothing to collect", and the
readfail counter that is the only way to diagnose it disappears with it. It fails
silently, not loudly. The predicate has to be per-record "this answer will never be
read" - true of params / respecify-escape / sub-data, false of a windowed create -
carried as a separate input from wantReply (willReadReply), with
willReadReply && !rowCarriesReplySlot rejected rather than ignored.
VERIFIED HERE, not taken on trust (the reviewer's file:line claims, checked):
- the flag space is RingRecordFlags (Ring.h:281-285), bits 0-4 taken, BIT 5 FREE,
and the header's own table (PipeWire.inc:35-65) says "bit 5 kOptional vs (nothing)
no ring counterpart" - so a new ring bit aliases kOptional, which is exactly what
the assert at PipeWireCodec.cpp:142-145 refuses;
- the three static_asserts that will trip are real and deliberate
(PipeWireCodec.cpp:106, :142-145, RingTest.cpp:740);
- WireFingerprint mixes payload-layout, catalogue, render-state digests + OpCount,
none of which a ring bit moves - so no protocol revision bump is required, and
both cross-version directions degrade to today's behaviour. That makes this
"must upgrade both ends to take EFFECT", not "must upgrade both ends not to
BREAK" - one notch cheaper than handoff-2 section 3.2's protocol surface;
- StreamLink.cpp:564-571 is the independent progress flush (>= 64 records or
>= 1 ms) and :577 is PostReply piggybacking SendProgress on the reply - so
skipping replies coarsens progress granularity. Bounded, but unmeasured.
AND IT CORRECTED TWO OF MY OWN NUMBERS. The frame's reply count is 55,903, not
59,673: op 4 ResourceDestroy carries kNone in the catalogue (PipeWire.inc:179) and
PostReply Fatals on a row with no kReplySlot, so it never answers - I had added its
3,770 records to the total. The derived figures move with it (12.3 replies per create,
0.65 of a create of survival, 6,988 pool churns), and every copy is fixed here:
Config.h, ConfigLoader.cpp, WireTables.cpp's section comment, CONTRACT-P5E.md,
CREATE-WINDOW-MEASURED.md.
TWO MORE THINGS THE FOLLOW-UP MUST CARRY, both recorded in the note's section 7:
- LinkMetricsBeginReply counts ownsReplySlot, so once the flag lands by_op 2 falls
to ~0 - and section 1 of the note tells the reader to use by_op 2 to check whether
the window engaged. That check becomes a FALSE GREEN; the flag needs a
server-side "how many times did this op answer" counter beside it.
- the ~112,000 sendmsg/frame the flag removes are on the apply thread, so the
flag's own value is a measurable claim that nobody has measured.
172/172 and 13/13 after the comment corrections.
|
||
|
BZLZHH
|
7205509a9a | [Docs] (P12): the create window is dead by arithmetic, and the two debts it exposed (the applier can no longer say it dropped a record; the picture still has no real control) | ||
|
BZLZHH
|
d011395939 |
[Docs] (MG_Test, P12): the dropped-records repair is gone, not merely narrowed - the first-use work removed the last row that read an applier refusal, and the case that pins the debt
handoff-2 section 3.3 asks for a gate on the one scope that empties the applier's
object records while the frontend objects live on (MGPipeApplierReleaseObjectRecords,
"latent" because its only caller in the tree is MG_Test). The gate was expected green
for a CHANGED descriptor: the respecify is refused, RespecifyOnce turns the refusal
into a create plus one retry. It is not green.
MEASURED:
dropped-records probe: creates=0 applier_records=0 published=1
Nothing is emitted at all. The refusal can only reach the emitter through an answer
the row READS, and P12 made both rows answer themselves for a "confirmed" object:
the respecify is fire-and-forget (MGPipeResourceRespecifyWantsItsReply) and the
params row writes a PROVISIONAL kStatusOk (Wire_SetTextureParams). "Confirmed" is
MGPipeHandleIsPublished, which answers "did a create GO OUT" - and it stays true
after the records are dropped, which is the scope's whole definition. The only row
left that reads the real answer is create, and create is only sent when the latch is
clear - the one thing that stops being true.
So section 3.1's "clearing the latch is not enough" is the smaller half: nothing
clears the latch either, and nothing can until the applier can say "I hold nothing for
this handle" - section 3.2's third option, the reverse channel, i.e. the protocol
surface the handoff hoped to avoid paying for. That hope was priced on the trigger
being unreachable; it is still unreachable (no production caller), so the call stands,
but the cost of a production caller is now known to be the whole thing.
THE CASE IS WRITTEN AS A DEBT RATHER THAN A SPEC, and says so in its name and its
comment: it pins today's three numbers, so whoever lands the signal sees THIS case go
red and knows the repair started working. Removing the heal from RespecifyOnce would
not make it red - nothing reaches the heal - which is exactly why the assertion is on
the applier's table and the latch rather than on a count (handoff-2 section 3.3: "a
count cannot show this class of error").
Also rewritten: notes/p12/UNPUBLISH-GUARD-SIGNALS.md sections 4 and 6 now carry this
as 4.2 (sections 2 and 3 stand: neither caps generation nor FreshlyPrimed is a signal).
172/172.
|
||
|
BZLZHH
|
d2efd1d79d |
[Perf] (MG_Remote/Client): the create window lands OFF - the deferral cannot be collected, 8 reply slots against 59,673 replies in one frame - measured on the device at 4,536 blocking creates either way
The window was the handoff's remaining item: let up to N resource_create answers be
outstanding, read a provisional accept, collect the answers in order, and un-latch an
object whose late answer is DECLINED. It is a sound mechanism and the gates below pin
it - and on the device it does nothing at all, which is why the default is 1.
MEASURED, same device, same frame, same 101,421,396 B staged, same binary:
MOBILEGL_IPC_CREATE_WINDOW=4 by_op 2 = 4,536 wall 28.64 s
MOBILEGL_IPC_CREATE_WINDOW=1 by_op 2 = 4,536 wall 28.64 s
(the handoff's pre-window baseline: 31.3 s)
by_op 2 counts the creates that took the BLOCKING path (LinkMetricsBeginReply is
called with ownsReplySlot = rowCarriesReplySlot && wantReply), so it not moving means
no create deferred. A temporary counter in the row said why: pushed=24, blocked=4,476,
took=20, readfail=8,951, pending pinned at 4 - the drain READ the answers back and
failed every time.
WHY, IN THE PROTOCOL'S OWN ARITHMETIC. An answer is readable only until its pool slot
is reused, and the pool is kDefaultReplySlotCount = 8. The server answers EVERY
reply-owning record, not only the rows that wait, because it cannot know which those
are (ClientSession.cpp:1897-1900: "The server posts either way"). This frame posts
59,673 of them: create 4,536 + respecify 20,633 + params 18,056 + sub-data 12,671 +
the rest. 8 slots against 59,673 replies churns 7,459 times per frame, i.e. 13.2
replies between two creates, so an answer is overwritten after 8 replies - 0.61 of a
create. The deferral is not slow, it is UNREADABLE, and no window depth fixes that.
AND IT IS NOT NEUTRAL: with the window on, the provisional accept latches an object
the applier may have refused, and the un-latch that corrects it lives in that same
unreadable drain. A refusal would become an object the client believes is published -
the silent divergence the suspect list exists to prevent. So the machinery is kept
(one experiment away from usable: a deeper pool, or a "do not answer me" bit, both
wire changes) and the default is off.
Worktree note: the window implementation itself was uncommitted in the tree at
|
||
|
BZLZHH
|
a2c1f5f292 |
[Fix] (MG_Remote/Transport): StreamLink::ReadReply answers by seq instead of "is it the newest" - the contract ShmLink already kept, and the reason a deferred answer could never be read back
ILink::ReadReply takes a SEQ (ILink.h:285) and ShmLink implements it against a
ReplySlotPool, so any answer the pool still holds is readable. StreamLink kept ONE
reply - replySeq/replyStatus/reply - and then asked the wrong question of it:
x.replyReady.wait(lock, [&]{ return closed || x.replySeq >= seq || appliedSeq >= seq; });
if (x.replySeq != seq) return ... MOBILEGL_ERR_PROTOCOL_MISMATCH;
The wait is written as "at least this seq"; the check is "exactly this seq". Every
answer older than the newest was therefore unreadable, which nothing noticed because
every caller read the answer it had just waited for - i.e. always the newest one. It
is the transport half of the P12 create window's failure: the window defers up to
four answers and collects them in order, and this link could not hand one back.
THE RING, AT THE POOL'S OWN DEPTH. kReplyRing = kDefaultReplySlotCount (8), because
that is the pool this link stands in for: an answer a reader has not taken by the time
eight newer ones have arrived is one the shm arm has already overwritten too, so the
two transports now lose the same answers and no more. Vectors are sized per reply, so
an idle link pays eight empty ones.
This does NOT make the create window work - on the device it changed nothing, because
the pool itself is 8 slots against 59,673 replies in one frame (see the next commit).
It is kept because the two transports disagreeing about ILink's contract is a bug in
its own right, and because the single slot silently failed any read that raced a newer
reply.
Gate: StreamLinkTest.AnOlderReplyIsStillReadableByItsSeq posts three replies and reads
them oldest-first - the order the window's drain takes them in.
|
||
|
BZLZHH
|
82ed1df90e |
[Perf] (MG_Pipe, MG_Remote/Client): a texture respecify for a CONFIRMED object stops waiting - the row pays its round trip once, at first use - and the frame drops from 130.9 s to 31.3 s
- The predicate is one function again: MGPipeResourceRespecifyWantsItsReply (PipeRoute.h) is read by the wire emitter, so the two halves of this row cannot disagree about whether an answer is owed. The buffer half never owes one (both call sites discard the Bool, PipeFill.cpp:1105/:1142); the texture half owes one exactly until MGPipeHandleIsPublished, and that latch is set only by an ACCEPTED create - which EmitResourceRespecify already sends, blocking, when it is clear (TextureEmit.h:869-879). So the object pays once, at first use, and every record after it is fire-and-forget. - MEASURED on the device, same frame, same 101,421,396 B staged, stock power policy: waits 25,154 -> 4,543 and wall 130.9 s -> 31.3 s. The op split is the proof of what changed and what did not: waits by_op 3=20611 -> absent (only 2=4536, 5=6, 9=1 remain), while the RECORDS sent are unchanged - frame-sent still shows 3=20633 and 103,094 records total. The round trips went away; the work did not. - It also lands on an independent prediction: an audit run beside this work (mgl-device/scratch/ scheme-review.md) computed 4,543 remaining waits and a 24-37 s credible range (midpoint ~30 s) from the archived logs, a host-inproc round trip of 35.4 us and a measured 43.1 MiB/s link. Measured: 31.3 s, at the midpoint. - Cumulative on that frame since this work started: 43,232 waits / 221.5 s -> 4,543 / 31.3 s (7.1x). What is left is the create row (4,536 waits, ~23 s) - the confirmation itself, not waste - plus hard floors of ~13.4 s client CPU and ~2.2 s of transfer. Going below that means batching the create row, which keeps real answers. - TWO DEBTS, STATED: (1) the invalidation guard for the one silent-loss case is NOT wired; the audit confirmed independently that no production path can drop a texture record today, and if one appears the notice it needs is protocol-surface (a tenth reverse callback). (2) No SSIM or visual check has been taken since these rows stopped asking - frame-sent agreement is strong evidence but it is not a picture. - Two stale controls in older cases were fixed rather than left: the item-B gate now names the CREATE row (its old total-based control only moved because a respecify was still waiting), and the by-op measurement asserts records-sent for the rows that deliberately stopped waiting. RemoteClientTest 169/169; red-once for the predicate is forcing it back to "always wait". |
||
|
BZLZHH
|
f2b55b9d2d |
[Perf] (MG_Remote/Client): the buffer half of resource_respecify stops waiting - its answer is discarded at both call sites - and the measurement says the texture half is where the other 20,611 waits live
- The two halves of this row are not the same row. PipeFill.cpp:1105 and :1142 both call MGPipeRouteResourceRespecify(desc, nullptr) and drop the Bool, so for a buffer the wait bought a value that was thrown away. It goes fire-and-forget with the provisional-accept shape resource_subdata has had since item B. - The texture half keeps its answer ON PURPOSE: there the Bool is the self-heal trigger AND the gate on the acceptance mirror, and TextureEmit.h:885-888 (ID-18 M3) says advancing that mirror on an unconfirmed answer is how the next identical call gets suppressed "against a record that was never stored". That half needs the provisional/really-accepted distinction plumbed through the reply mailbox before it can stop waiting; it is not a flag flip. - MEASURED, same frame, same 101,421,396 B staged: waits 25,176 -> 25,154 and by_op 3=20633 -> 3=20611. So the buffer half is worth 22 waits (0.1%) and the remaining 20,611 are texture respecifies - which is what says the next step is the mirror design, not another flag. Wall 134.4 s -> 130.9 s. - The ERROR-is-not-a-decline case moves to the texture half: a row that reads no answer cannot fold ERROR into a decision, so the rule it exists for belongs to the half that still waits (the transport fault is caught at the next row that does). New case ABufferRespecifyTakesNoReplyWhileATextureOneStillDoes pins both directions; red-once with the branch removed is exit 155. RemoteClientTest 169/169. |
||
|
BZLZHH
|
5ad8370b37 | [Docs] (P12): the first-use confirmation measured on the device - 43,232 waits to 25,176, 221.5 s to 134.4 s, op 47 gone and every other row identical | ||
|
BZLZHH
|
b62709604d |
[Perf] (MG_Impl/Pipe, MG_Remote/Client): a parameter change for an object whose existence is confirmed takes no reply - the round trip moves to the object's FIRST USE
- The device run put 18,056 of one atlas frame's 43,232 round trips on this row, at ~5 ms each, for records carrying a few hundred bytes. The acceptance it waited for says exactly one thing - "the applier holds a record for this handle" - and the client already keeps its own copy of that fact in MGPipeHandleIsPublished, which is set when the create was ACCEPTED and is never cleared while the record lives (the applier keeps texture records across a make-current, PipeFill.cpp:3756-3759). The wait was the client asking the server a question it had already answered. - SO THE CONFIRMATION MOVES TO THE FIRST USE. EmitTextureParams now checks the latch BEFORE it sends: an unpublished object gets HealUnpublishedRecord - the same create-with-no-storage, the same storage respecify, the same one re-send the refusal branch has always run, extracted so both callers share it - and that create is the ONE blocking record the object owes. Every parameter record after it is emitted fire-and-forget and the caller reads a provisional accept, exactly as resource_subdata has done since item B. Wire_SetTextureParams reads the same latch to decide, so the two halves cannot disagree about whether a wait is owed. - THE FALLBACK IS THE OLD PATH, unchanged: an object whose create was refused stays unpublished, so its next parameter record still takes the real answer and the existing one-retry heal still runs. That is also the red-once: with the unconditional EmitAndWait put back, the new case fails - RemoteClientControls.AParameterChangeForAConfirmedObjectTakesNoReply exits 153, having asserted that the eight parameter records still go OUT while none of them takes a reply. - WHAT IS GIVEN UP, stated rather than implied: a loss the client believes is published (a bug, or a served-context teardown whose only caller today is a test) is now silent on the CLIENT - it stays counted and named on the server, which is the same residue item B accepted for the upload row. - The measurement case's "the counter is live" control moves to create/respecify, because this row's zero is now the fix rather than a dead counter. 168/168 in RemoteClientTest; the unit lane is 2617/2617. Tracks: docs/Disaggregated/notes/p12/DEVICELOST-AND-UPLOAD-WAITS.md. |
||
|
BZLZHH
|
c05e702d8c |
[Docs] (P12): the server's spin budget is the portable half of the idle-exit fix - 221.5 s -> 182.8 s on the same frame, no root and no code change
- 2.9 established that a record's ~5 ms is mostly the DEVICE waking out of a deep idle state. That points at a knob that is already wired: how long the server's apply thread spins before parking. Raised from 50 us to 2000 us (via the Activity's existing `env` extra, which ServerEnvironment.apply parses as KEY=VALUE;KEY=VALUE and which does not scrub MOBILEGL_IPC_SPIN_US), the same 43,232-record frame goes 221.5 s -> 182.8 s (-17%), with rtt_mean 5,040 -> 4,145 us. Pinning the device and disabling its deep idle states is worth 166.5 s (-25%) but needs root; the spin needs neither root nor a rebuild. - The server log proves the knob landed (`mgl-srv-apply started ... spin 2000 us`) rather than leaving the reader to trust an env var, and the note carries the exact am start line. - So the load stall now has three ordered levers with numbers behind them: batch the waits (client-only, not built), keep the server hot (this commit, usable today, trades device CPU for wall clock on a frame that is 94% waiting), and pin the device for measurement discipline. |
||
|
BZLZHH
|
7c72dfde2b |
[Docs] (P12): the ~5 ms per round trip is the DEVICE's deep-idle exit, not the link and not the client - which is what the handoff measured and could not explain
- Three one-variable experiments on the same device, same workload (the 43k-record frame): a 40x larger client spin moves the per-wait mean by ~10% (so it is not the client park/wake); the same frames over adb forward - a link whose raw RTT is several times lower - by ~25-30% (so it is not the link, which is what the handoff saw in 3.6 and could not account for); and pinning the device (governor performance on all eight cores, cpuoff_l / clusteroff_l / mcusysoff / s2idle disabled) by 26% per wait and 25% of the whole frame: 221.5 s -> 166.5 s for 43,218 records and 100.5 MB staged, against 43,232 and 101.4 MB in the baseline. - The mechanism: the server's apply thread parks between records, the SoC drops into a deep idle state, and every record pays the exit latency - milliseconds, once per record, which is the entire shape of the frame. It also matches the server profile the handoff took (65% socket syscalls, ~15% doorbell spin: waiting and waking, not working). - Stated where it matters: the handoff's 249 s was taken on an UNPINNED device, so its absolute numbers carry this idle tax - the project has a pinning discipline for exactly this reason. - The lever list is now ordered: batch the waits (K <= 8 by the reply-slot pool, client-only), keep the server hot between records (its own spin budget, which the APK does not wire yet), and measure split numbers on a pinned device. The device was restored to stock afterwards. |
||
|
BZLZHH
|
6e7c0ce710 | [Docs] (P12): the per-wait 5 ms is not the link - the same frames over adb forward move it by only a quarter, which is what the handoff measured and could not explain | ||
|
BZLZHH
|
f99415a493 | [Docs] (P12): the atlas window's exact composition, and the negative result that kills the record-dedup idea before anyone builds it | ||
|
BZLZHH
|
1d363eb109 |
[Perf] (MG_Remote): the client counts RECORDS SENT per op beside the waits each row PAID, and the pair says item B stopped the waiting without stopping the sending
- A wait count alone cannot say whether a frame is slow because it sends too much or because it waits too often. The device run made that the next question: one atlas frame paid 43,232 waits while the upload row paid none. - LinkMetricsNoteRecord is called at the encoder's one commit point (PipeWireCodec.cpp, beside the existing CallClass::WireRecords count, for the comment's reason: after chunking no emitter knows how many records its call produced). LinkMetricsRecordsFor reads it back, and every frame that emitted something now logs `P65LinkMetrics kind=frame-sent frame=N records=M by_op=...`. - The host measurement now prints and asserts both halves for the same workload. An atlas-shaped load (64 sub-uploads into one level) sends 128 ResourceSubData records and waits for ZERO of them, while ResourceCreate 76/76, ResourceRespecify 130/130 and SetTextureParams 130/130 are one wait per record. That is item B stated exactly: the records still go out; the round trips do not. It also says what the next thing to fix is - the record COUNT of those three rows, not their waits, which the contract rules they must keep. |
||
|
BZLZHH
|
d057e1cb8a | [Docs] (P12): the first device run of A and B - the crash path walked twice without a crash, and the atlas frame says item B is necessary but not sufficient | ||
|
BZLZHH
|
170b789e7f |
[Perf] (MG_Remote/Transport): the per-op reply-wait split prints with every frame, because a whole-session total cannot name the rows behind one frame
- The device run that motivated this made the point: 43,232 of a session's 43,629 reply waits were in ONE atlas frame, and the exit summary never appeared at all - a SIGTERM does not take the session's clean stop path, so LinkMetricsEnd is never reached. The split added one commit ago was therefore invisible on exactly the run that needed it. - Emit now writes a second line for any frame that bought a wait: `P65LinkMetrics kind=frame-op frame=N wait_replies=M by_op=<id>=<count>,...`. The formatting is factored into FormatOpSplit and shared with the exit line. Frames that bought nothing stay quiet, so a steady-state session does not double its log. - Measured on the device (MC 26.2 into a world over TCP, MOBILEGL_PIPE_STATS=1): the steady state is one wait per frame and it is op 9 (Present); the upload-heavy startup frames are `2=50,3=68,47=62` - ResourceCreate / ResourceRespecify / SetTextureParams, with ResourceSubData (48) absent, which is item B working on the wire it was written for. |
||
|
BZLZHH
|
e734880cf2 | [Docs] (P12): the device page names the by_op ids, and the note says where the per-op line comes from | ||
|
BZLZHH
|
618395529f |
[Perf] (MG_Remote/Transport): the per-op reply-wait split also lands in the exit summary, because a spawn client never prints a frame line
- The tally added beside this is only useful if somebody can read it where the problem is. The handoff says why that was hard: under spawn the client does not advance the frame window, so the per-frame P65LinkMetrics lines never print, and a SIGKILL takes the exit-time dump with it - "only the dump at exit, and SIGKILL does not get it". - LinkMetricsEnd writes the split on its own line from the one place a client that ends the ordinary way always reaches: P65LinkMetrics kind=per-op wait_replies=N by_op=<id>=<count>,... . Ids stay raw (this file includes only <cstdint> and friends, not the op table) and the reader names them: 2 ResourceCreate, 3 ResourceRespecify, 47 SetTextureParams, 48 ResourceSubData. The mapping is written down in mgl-device/README.md, which is the page the device run reads. |
||
|
BZLZHH
|
ce0235b190 |
[Docs] (P12): the three rows that keep their wait get a ruling with evidence behind it, and the handoff's open op-split question gets its numbers
- What was open: the handoff's 4.3 called ResourceCreate and SetTextureParams "a second problem of the same kind, needing its own analysis" and left it there; 5.4 said the per-op split of the 55,428 waits was never quantified. The measurement committed beside this answers both. - The ruling (CONTRACT-P5E.md 2.5): ResourceCreate, ResourceRespecify and SetTextureParams stay kWaitReply, because each answer carries a fact about the SERVER's record table that the client's own tracker provably cannot know. RespecifyOnce says it outright - "the applier's REFUSAL is the only signal that says 'I hold nothing for this handle'", which is the scope MGPipeApplierReleaseObjectRecords leaves for a served context whose frontend objects live on; EmitTextureParams uses its own refusal as the same self-heal trigger; PublishCreate may not take the publication latch on a refused create (D-I1/c0b) or the death path emits a resource_destroy for a record that does not exist. Unlike the upload half these are not "the emission gate already answered it" rows, so removing their waits removes the signal. - The numbers, both shapes a stitcher produces: an atlas stitch goes 64 -> 0 reply waits; a texture per sprite goes 384 -> 320 with ResourceSubData now zero and the remainder all three of those rows. So item B hits exactly the shape it was aimed at, and what is left is per-resource / per-call rather than per-chunk. - notes/p12/DEVICELOST-AND-UPLOAD-WAITS.md gains 2.4 with the table and the ruling, and its "second problem of the same kind" entry is closed rather than carried; MEASUREMENTS.md records the two shapes beside the 1 -> 0 single-upload number. |
||
|
BZLZHH
|
fa8b500ba8 |
[Perf] (MG_Remote/Transport): the reply-wait metric gains a per-op tally, and the measurement says which rows actually buy the waits
- The handoff measured the total (55,428 reply waits in one Minecraft frame) and left "which rows" open: a total is the same number whether the waits are uploads, creates or parameter sets, and those three have different fixes - so knowing the split is what says whether P12 item B is the whole story or a fraction of one. - LinkMetricsBeginReply now takes the op and Window carries a per-op tally, so LinkMetricsReplyWaitsFor(op) reads back what the P65LinkMetrics line prints as `wait_replies` split by row. Same activation as everything else in that file (MOBILEGL_PIPE_STATS=1), one array in the window, and a value past the table lands on the last slot so the per-op numbers still SUM to the total. - RemoteClientControls.ReplyWaitsByOpForAnAtlasShapedLoad drives the two shapes a stitcher produces and prints the table: an atlas stitch (64 glTexSubImage2D into one level) goes from 64 reply waits to ZERO, and a texture per sprite (64 textures, 256 glTexParameteri) from 384 to 320 with the remainder all ResourceCreate 76 / ResourceRespecify 130 / SetTextureParams 130. - It prints before it asserts: the table is the point of the case, and a gate that fired first would hide the numbers on exactly the run that needs them - the red one. The assertions are the structural ones that can be wrong about the mechanism (upload rows zero, the other rows not, the per-op tally summing to the total); the numbers themselves are recorded, not gated. - The A/B is a real run: with the wire-arm arm test removed the case goes red with the full table, which is where the 64 and 384 above come from. |
||
|
BZLZHH
|
bfd6392668 |
[Docs] (P12): the P5E contract records the texture half's paid-off wait, and the round's note carries the measured G1
- CONTRACT-P5E.md 2.5 said "the TEXTURE half keeps its wait because DrainTextureSubData clears the level's dirty flag on the accepted reply (D-D5; trailing)". The trailing is paid, so the clause now states the three facts that make that answer DERIVABLE on the wire arm (the emission gate IS the consumer question the applier's own acceptance belt asks; what remains is a loud bug class; a client-cancelled record still answers "not accepted"), where the wait moves to (SEG_STAGE), that MGPipeSubDataWantsItsReply stays the one place that decides, and the gate with its red-once. ResourceCreate and SetTextureParams are named as analyzed separately rather than swept in. - notes/p12/DEVICELOST-AND-UPLOAD-WAITS.md: 2.3 now carries the G1 A/B this machine ran - .text 13872002 bytes on both sides, 31746 symbols, 0 added / 0 removed, no per-symbol difference - instead of deferring it to CI, and 3 trades its "G1 not run" item for the device tooling (mgl-device/: launch-mc.py, gl-aliases.sh, start-phone-server.sh) the next round needs. |
||
|
BZLZHH
|
3b9f536dd2 |
[Docs] (P12): the device-lost reply tolerance and the texture-upload wait granularity get their own record - what landed, the gates that can go red, and what is still owed
- notes/p12/DEVICELOST-AND-UPLOAD-WAITS.md: the two items, their root causes with file:line anchors, the five cases that pin them, the red each one produced when the fix was taken back out, and the four things this round did NOT do (device end-to-end, local G1, the ResourceCreate / SetTextureParams pair, and the sandbox's missing /dev/dri - which reds the two EventForfeitPeer lanes for reasons that have nothing to do with the change). - p12/README.md: a line in the summary and the new file in this directory's table. - MEASUREMENTS.md: the P12 conclusion and row now carry the number the gate pins - one reply wait per pure texture upload becomes zero - with the device wall clock named as the measurement still owed rather than left implied. |
||
|
BZLZHH
|
33e02f4bcf |
[Perf] (MG_Pipe, MG_Remote/Client): on the wire arm a texture upload stops owning a reply - the wait moves from every record to the SEG_STAGE window
- MGPipeSubDataWantsItsReply is the ONE place that decides whether a resource_subdata record waits
for its answer (WireTables.cpp:370 and PipeRoute.h:328 both read it, and the comment beside it
says that a second spelling is how the client comes to wait for an answer the emitter told the
server not to bother with). The buffer half has answered "no" since P5e; the texture half now
does too, ON THE WIRE ARM ONLY.
- THE ANSWER IS DERIVABLE THERE, which is the argument - not that nobody reads it:
* The emission is already gated on the answer's content. A texture record is produced only where
FamilyIsLive(kMGPipeSubsystemTextureResources, ...) holds, and under split that gate IS
CapsMirrorInstance().ServerConsumes(bit 10) (PipeFill.cpp:1719-1740) - the same published fact
the applier's own acceptance belt reads (PipeApply.cpp:1408-1419, NoP4aConsumer). So the one
refusal this acceptance exists to carry, "no consumer" (ID-39 / D-D5 / ID-18 M3, which is why
the texture half ever asked), cannot be the answer to a record this client emitted.
* What remains is a bug class, and it is loud already: a record that names no texture, declares
no texels or carries no bytes is an MGP_TRIP_WIRE_REPORT (PipeApply.cpp:1201-1218, :2245-2252),
and a handle the applier does not hold is counted and named (ResolveResourceIn, :721-728) with
the client's own tracker preventing it from arising (PipeFill.cpp:1045-1051).
* Every answer the client can still get for free, it still gets: the emitter reads this route's
Bool exactly as before, so a record the CLIENT cancelled (the device-lost latch, an exhausted
stage) still answers "not accepted" and the level stays dirty and is retried - D-D5's safe
direction, unchanged.
- THE WAIT MOVES TO THE STAGE. A run that does not fit SEG_STAGE reclaims retired bytes and parks
on the stage doorbell (PipeWireCodec.cpp StageAllocate / ReclaimStagedBytes) - the one wait a
run-ahead client owes, and it is proportional to the WINDOW rather than to the call count. On
Minecraft 26.2's atlas frame that is the handoff's 55,428 reply waits at ~4.4 ms each: 96 MiB of
uploads through a 32 MiB window.
- THE GATE IS THE NUMBER ITSELF. LinkMetricsReplyWaits is added so a case can read back the same
`wait_replies` the per-frame P65LinkMetrics line prints - LinkMetrics.cpp:76-80 counts exactly
the records that own a reply slot. TextureUploadsStopOwningAReplyOnTheWireArm asserts that a pure
glTexSubImage2D crosses the wire and moves it by ZERO, while a resource_create row - kWaitReply,
and it always was - still moves it, so the zero cannot be "nothing was counted". Red-once run:
with the arm test removed the child exits 142.
- Monolith and every non-wire arm are untouched, and that is pinned twice: the existing
RemoteRunAhead.OnlyTheTextureHalfOfResourceSubDataWantsItsReply still reads true with no arm
installed, and the new test sits inside MOBILEGL_BUILD_DISAGGREGATED with the original expression
as its fall-through, so a pull build preprocesses to the line it had (G1).
- Verification debt, stated rather than implied: the host gate is the COUNT, not a wall clock.
In-process control sessions run lockstep (the caps peer publishes no kCapRunAheadApply), so the
verb barrier still waits there; the end-to-end wall-clock win is the device measurement the
handoff describes and it has not been taken yet.
|
||
|
BZLZHH
|
096fa1a8c1 |
[Fix] (MG_Remote/Client): a DECLINED reply is an answer, not a corruption - the fence and query consumers stop folding status 1 into the abort, and the uncarried-bytes self-check spares a session that has latched device-lost
- ReadFenceReply (EmitTables.cpp) and ReadQueryReply (QueryEmit.inc) tested `status != kStatusOk`,
which is two values in one branch. ReplySink's own header says "DECLINED IS A REAL ANSWER, not
a failure" (PipeWireCodec.h:457-460), and the escape rows are held to the other half of the same
rule - "ERROR IS NOT A DECLINE ... an escape may not fold 2 into `false`" (WireTables.cpp:446-450).
- BOTH VALUES OF 1 ARE REACHABLE WITHOUT A BYTE OF CORRUPTION: the server declines these five rows
by design whenever it cannot answer them (PipeWireCodec.cpp:1820-1830 for the two fence rows,
:1841-1861 for the three query rows - a zero-length answer to a kWaitReply row, R-5), and the
client produces the same status itself once the session stops carrying records
(ClientSession.cpp:1837 the device-lost latch, :2025 a doorbell that died during teardown, where
the comment says outright that "the verb did not happen" is reported as DECLINED and not ERROR).
- WHAT THE FOLD COST: after a clean device loss Minecraft 26.2's next glClientWaitSync - the
chunk-upload fences - died of Fatal{ProtocolCorruption, "Fence.reply"} (rc=134) instead of taking
the no-op the design promises ("GL calls become no-ops", design/08-runtime-and-platform.md:33).
The query rows had the same shape waiting for them.
- The no-op is the one this frontend already gives for a sync it cannot wait on: GL_Sync.cpp:103-106,
PipeApplier.cpp:175-177 and DirectGLES.cpp:16489 all answer GL_ALREADY_SIGNALED. A query row
answers its own "no answer yet" value - availability false, Produced 0 (which the caller already
returns as false with the out-parameter untouched), timestamp 0.
- ERROR and a short OK keep the named Fatal they always had: a transport fault is not the server
saying no, and an OK answer that arrived empty is a reply nobody can read. ReadPixels keeps its
deliberate Fatal{ReadbackDeclined}: unlike these rows a blocking readback cannot succeed short of
complete pixels, and that ruling is untouched.
- PipeFill.cpp's UncarriedInitialBytes self-check now spares a session that has latched device-lost.
The resource_subdata it counts is DECLINED by design there, so "it emitted nothing" is the latch
working rather than bytes this client forgot to carry - and there is no server left for them to
exist on. P12's exit gate (a) is that killing the server gives a CLEAN device-lost latch, not a
corpse at the next glBufferData that carries data.
- Four cases in RemoteClientControls.inc drive the production consumers over a live session whose
peer declines them. Red-once run for all three Fatals (Fence.reply, Query.reply,
UncarriedInitialBytes), and ErrorFenceReplyStillRefusesByName is the control that keeps the
tolerance from becoming blanket. RemoteClientTest 166/166; the host unit lane 2610/2610.
- Split only: both consumer files compile under MOBILEGL_BUILD_DISAGGREGATED and the PipeFill hunk
sits inside its `#if`, so a pull build preprocesses to exactly what it had (G1).
|
||
|
BZLZHH
|
c984d95205 |
[Fix] (MG_Remote): CapsSnapshot.apiVersion carries the backend's own API version string, so a split client's GL_RENDERER names the real device instead of the target GL version (split build only; the pull build is unchanged, G1)
- PublishCapsSnapshot filled that field with info.RendererGLInfo.TargetGLVersion.toString(). The client reads it back through BackendObject_Remote::GetBackendAPIVersionString (CapsMirror's ApiVersion) and prints it in the parentheses of GL_RENDERER, which is the only place a real device is named; a monolith build fills the same parentheses from the backend's own GetBackendAPIVersionString. A split client therefore answered "Espryt (MobileGL Core) (4.6.0)" where monolith answered "Espryt (MobileGL Core) (Adreno (TM) 750, OpenGL ES 3.2)". - The target version still crosses the wire inside the renderer-info blob (CapsCodec), which is where GL_VERSION reads it, so no reader loses it. - Content only: the field already existed and the blob layout is untouched, so the wire fingerprint needs no bump, and the file compiles only under MOBILEGL_BUILD_DISAGGREGATED. - Verified with a PC client against a phone server: Minecraft and Sodium both log "Espryt (MobileGL Core) (Adreno (TM) 750, OpenGL ES 3.2)". |
||
|
BZLZHH
|
d8d7530011 |
[Fix, Test] (MG_Backend/DirectGLES, MG_Util, MG_Impl): widen three-channel render targets wherever the driver refuses them
Complementary Reimagined would not load through Espryt on Mali: Iris got GL_FRAMEBUFFER_UNSUPPORTED building its composite framebuffer, because colortex1 is RGB8_SNORM and colortex2 is RGB16F - three-channel formats that no real ES driver can render to (EXT_render_snorm covers R/RG/RGBA only, and the float extensions exclude the RGB forms). The frontend's probe cache diagnosed this correctly and then had nothing to offer: the NoThreeChannelRenderTarget widening machinery existed but was gated to multisample targets alone. llvmpipe turns out to refuse most of the same attachments - CI retrace stayed green only because a replay never branches on glCheckFramebufferStatus - so this was never a desktop-vs- device split, just an unlit path. The widening now applies to every color-attachable image, renderbuffers included, riding the driver-probe branch so the native format is still tried first and substituted only on refusal. One ThreeChannelWidening table owns the widened (internalformat, format, type) triple per source format - the previous per-case branches disagreed with each other and could emit an unuploadable (RGBA16F, GL_RGB, GL_BYTE) combination or widen into another three-channel format the driver refuses just the same. Uploads repack three-component client data to four with the format's own one in the alpha channel (127 is not 1 for RGB8I - the integer arms carry integer ones); readback drops the synthetic alpha, derived from the actual image being read, not the bound framebuffer, so glGetTexImage through a scratch FBO cannot be confused by an unrelated widened attachment. Stored alpha on a widened attachment is now an invariant 1.0 rather than an accident: the color-mask sync clears the alpha bit per draw buffer (glColorMaski for MRT mixes), and clears route through glClearBufferfv with alpha substituted on widened slots only - scissored clears inherit the discipline for free, integer color buffers keep their explicit integer-clear path, and glGet still answers the application's own mask. GL_DST_ALPHA blending, blits and readback therefore all see 1.0 without further interception. DriverPost grows the rows this bug earned: EXT_color_buffer_float detection (previously unreferenced anywhere) with a FAIL row when absent, the missing EXT_render_snorm row, and a three-channel- attachment row that reports one representative per widening class - graded so a half-float-only driver warns about the 32-bit float gap instead of being declared unsupported. Gates: 606/606 unit at default and with the async kill switch; full retrace, both backends - the complementary fixtures now run with the widening ACTIVE on llvmpipe and pass with a slightly better SSIM than before; ext caselist DirectGLES holds 3914/4867 with zero set drift while 54 cases move from NotSupported to genuinely passing; on the Mali-G77 device, Complementary Reimagined builds its pipeline and renders in-world through Espryt (md5-verified build), BSL still green. A new ThreeChannelAttachmentScenario pins the frontend answer - COMPLETE where it used to say UNSUPPORTED - on the real driver. |
||
|
BZLZHH
|
0f394fa46f |
[Test] (tools/trace_replay): pin the Iris glyph-death bug with a BSL pause-menu fixture
minecraft-1.21.4-fabric-iris-bsl-esc-menu-854: Minecraft 1.21.4 Fabric + Sodium + Iris + BSL 10.1.3 through Espryt on Mali-G77, paused at the ESC menu over the BSL-blurred world, captured at 854x480 through the FCL trace-capture skill. The trimmed trace keeps the whole session preamble deliberately: the |
||
|
BZLZHH
|
107669b3db |
[Fix, Test] (MG_Impl, MG_Backend/DirectGLES): DSA by-name texture calls corrupted borrowed-slot memo pairings - process-wide glyph death under Iris
Field report: on Espryt with a BSL Iris pipeline built, every glyph in the game died - HUD, menu labels, even the vanilla title screen after leaving the world - while sprites kept rendering. Captured on-device (FCL apitrace rig), reproduced headlessly on llvmpipe, and pinned with a three-way replay: the same trace renders full text on raw Mesa desktop GL and on Magma, so the stream was intact and the execution was Espryt's. MECHANISM. WithTemporarilyBoundNamedTexture implements the by-name (DSA) texture entry points by binding the named texture onto the active unit's real slot, running the bound-texture code, and restoring - without moving the texture bind generation on either edge. DirectGLES's per-draw texture sync memo keys on that generation and BORROWS the slot pointer, so a memo built for texture A kept passing every key while a by-name call had texture B sitting in the slot: A's backend twin was driven with B's frontend object, and SyncMipmapsToBackend re-specified A's storage with B's shape. In the trace, a by-name upload to a BSL 2048x2048 map while the 16x16 lightmap was bound re-specified the lightmap's GL texture 2048x2048-NULL and back 16x16-NULL. The lightmap exists only as render output - no glTexSubImage2D ever touches it - so it stayed zero forever, and rendertype_text (vertexColor = Color * texelFetch(lightmap, ...)), alpha-discards every glyph. Background quads never sample the lightmap, which is why only text died. FIX, class-level, two layers: - Frontend (shared, closes the same hole for DirectVulkan's generation- keyed memos): the temporary bind and the restore each bump the texture bind generation (only when the slot actually changed), and the restore is an RAII scope guard so a throwing body can no longer leak the temporary binding - a second latent bug of the same class. Deliberately a generation bump and not a touched-unit note: the high-water mark must not chase by-name calls, and a completed bind/restore pair leaves the content epoch unchanged, so the cost is an owner-compare re-walk, not a memo rebuild. - DirectGLES defense in depth: both borrowed-pair memos (g_unitTextureSyncList, g_fboTextureSyncList) record which frontend texture each backend twin was paired with and re-check it before any replay (last in the key conjunction, behind the context-id compare). A stale pairing now costs a list rebuild instead of silent cross-texture storage corruption. Tests, both red with their own layer reverted: TextureTest.NamedTextureCallKeepsUnitBindingAccountingCoherent (the accounting contract) and DirectGLESTextureSync.UnitMemoRefusesToDriveA- TwinFromAnotherTexture (the corrupting sequence shape against a mock GLES table, asserting the resident texture's storage is never re-specified). 595/595 unit at default and with the async kill switch. Replay evidence: the captured BSL ESC-menu trace renders all text through Espryt post-fix, byte-comparable to the Mesa-direct and Magma replays; the no-shaderpack control is unchanged. A trace fixture wiring this scene into CI follows in a separate commit. |
||
|
BZLZHH
|
3e0460e472 |
[Feat, Fix] (MG_Impl, MG_State, MG_Backend): program interface queries from frontend reflection; Espryt state-shadow reset
Wave 2 of the advertised-extension conformance campaign.
PROGRAM INTERFACE QUERIES (the load-bearing piece). glGetProgramInterfaceiv
and the five glGetProgramResource* entry points were answered by the
BACKENDS - Espryt asked the real driver about SPIRV-Cross-generated ESSL
whose namespace is not the GL one (default-block uniforms live in
MGL_GLOBAL_UBO there), and Magma kept a second, partial reflection that
hardcoded types and diverged from the frontend. Both are now deleted; a new
frontend resource-model layer (ProgramInterface.{h,cpp}) answers every
interface - uniforms, uniform blocks, atomic-counter buffers (recovered
from glslang's synthesized gl_AtomicCounterBlock_<binding> lowering),
buffer variables, shader-storage blocks (classified by TType storage
qualifier since glslang reflects them as uniform blocks), program inputs/
outputs (built-ins' layoutLocationEnd sentinel mapped to -1), and
transform-feedback varyings including the gl_NextBuffer/gl_SkipComponentsN
pseudo-varyings - from the glslang reflection the frontend already trusts
for glGetActiveUniform. Name/index round-tripping, the "[0]" array
spelling, and the GL 4.6 table 7.2 prop/error matrix live in the new layer
only; GetActiveUniform*/GetActiveAttrib* are untouched.
glShaderStorageBlockBinding now takes the interface-layer index (the one
GetProgramResourceIndex returns, with a range check it never had), records
the binding on the program keyed by block NAME - the one coordinate all
three index spaces agree on - and delegates by name across the backend
boundary. Both backends reseed the recorded bindings on their own program
rebuilds, so an unrelated resync can no longer silently revert a rebound
block, and GL_BUFFER_BINDING reports the live binding, not the declared
one. The Espryt delegate applies only to an already-synced twin and can no
longer trigger SyncToBackend from a getter; the sync path's GL query
out-params are initialized and clamped (a load-dependent stack-garbage
Vector size crash caught by the gate, reproduced 3/50 pre-fix, 100/100
post-fix under saturating load).
ESPRYT RENDER-STATE SHADOW RESET (rides along because it shares
DirectGLES.cpp): the render-state shadow is file-static and survives
MobileGL context switches, so GL_FRAMEBUFFER_SRGB (and the whole synced-cap
class) leaked between contexts - the cross-test leakage class the CTS maps
have carried for a week. MakeCurrent now invalidates the shadow like it
already invalidates the program/FBO/buffer caches, and the resync resolves
a never-set scissor box to the current surface instead of pushing the
(0,0,0,0) sentinel verbatim (which scissored everything away - caught by
the retrace gate, bisected to the exact field via a bitmask probe, and
fixed by resolving like the viewport path rather than reverting).
Frontend riders exposed by the layer: glGetUniformLocation resolves
arrays-of-arrays element addressing ("a[2][1]"); transform-feedback capture
accepts element-addressed varying names ("b[1]") and snapshots the request
verbatim for the interface (Magma's decorate pass logs loudly that element
capture is unimplemented there - follow-up).
KNOWN GAPS, documented in code and tests: the 6 subroutines-* cases
(glslang refuses subroutine for SPIR-V; wave 3), the 5 separate-programs-*
cases (glslang's pipe-I/O reflection cannot see a separable non-vertex
stage's own inputs; needs stage-aware output validation first), and
uniform-block-types' per-instance stage masks (not derivable from the
reflection).
Gate: 593/593 unit at default and kill-switch, x10 each, plus the SSB race
case 100/100 under 20-way CPU load; ext caselist Espryt 77.87% -> 80.42%,
Magma 77.58% -> 79.39% (+212 fixed, 0 newly broken); program_interface_query
2/43 -> 31/43 unique on Espryt, 9/43 -> 31/43 on Magma, backends now
byte-identical; KHR-GL45.direct_state_access 370/371 + 371/371 with the 4
sRGB leak victims recovered in cross-test ordering; KHR-GL33 held at
9884/9886; full 39x2 CI retrace with zero wave-attributable failures (the
3 failing newly-added fixtures are bit-identical on the pristine baseline).
|
||
|
BZLZHH
|
33ff177bb2 |
[Fix, Test] (MG_Impl, MG_State): advertised-extension conformance wave 1 - uniforms, validators, getters
First wave of the advertised-extension CTS campaign (targeted caselist: the
glcts groups of every extension both backends advertise, 4867 cases across the
KHR-GL41..46 namespaces). All frontend, shared by both backends:
- Non-square float matrix uniforms actually upload: glUniformMatrix{2x3,3x2,
2x4,4x2,3x4,4x3}fv and the six glProgramUniformMatrix* twins were
validate-only no-ops; they now write column-at-a-time at the global UBO's
16-byte std140 column stride, honouring transpose. glUniformMatrix2fv had
the sibling bug - mat2 written as 4 contiguous floats put column 1 at byte
8 instead of 16. The readback path only ever un-padded mat3, so
glGetUniformfv is fixed for mat2, mat3x2 (previously mis-gathered) and
every non-square shape, with the bounds check widened to the padded span.
- glBindBufferRange validates offset/size at last: size <= 0, offset < 0,
SSBO and UBO offset alignment, transform-feedback offset AND size
multiples of 4 - all before any state write (a negative offset used to
reach Range1D unchecked). glBindBuffersRange inherits per element, with
the ARB_multi_bind up-front [first, first+count) checks added to the
BindBuffersBase/Range and BindSamplers prologues.
- BufferSubData's second, wrong mapped-overlap test deleted (it rejected
every write at or after a mapped range's start, mapped or not); the state
layer's assert relaxed to the same half-open intersection the frontend
checks. BufferStorage error precedence fixed: no-bound-buffer now beats
bad-size/flags.
- glSamplerParameteri accepts the full GL_NEVER..GL_ALWAYS compare-func
range (NEVER/LESS/EQUAL were rejected by a wrong lower bound).
glBindSampler's unit gate uses GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS instead
of the frontend array capacity, shared with glBindSamplers by construction.
- Getters: GL_MAX_SHADER_STORAGE_BLOCK_SIZE in glGetIntegerv; atomic-counter
buffer limits; all 11 per-unit GL_TEXTURE_BINDING_* plus GL_SAMPLER_BINDING
in glGetIntegeri_v; GL_VERTEX_ATTRIB_BINDING/_RELATIVE_OFFSET across the
vertex-attrib query family; glGetFloati_v/glGetDoublei_v implemented (were
stubs); KHR_debug limits raised to spec floors.
- glCreateShader records INVALID_ENUM for an unknown type (it previously
handed out a usable name with no error at all); glCreateShaderProgramv
validates count up front. glDispatchCompute/Indirect validate work-group
counts, offset alignment and indirect-buffer presence.
- glVertexAttribIFormat & friends take a positive integer-type whitelist -
GL_FLOAT/GL_HALF_FLOAT/GL_DOUBLE/GL_FIXED no longer slip through as
integer attributes.
Gate (headless Mesa, default config = async on): 570/570 unit at default and
with the kill switch; ext caselist Espryt 76.29% -> 77.87% (+81 fixed, 6
crashes -> 0, the whole list now runs in one glcts process), Magma 75.94% ->
77.58% (+80 fixed, 0 newly broken); KHR-GL33 full mustpass lost nothing
(9884/9886, the 2 known Mesa-drift failures); retrace smoke clean (the
bsl-GLES miss is the documented golden drift, bit-identical on the pristine
baseline). The 4 DirectGLES direct_state_access.renderbuffers_storage* cases
that turned red are a PRE-EXISTING GL_FRAMEBUFFER_SRGB cross-test leak,
A/B-proven on an unpatched
|
||
|
BZLZHH
|
2e6fc1ffc0 |
[Feat] (MG_Util, MG_Test): enable asynchronous shader compilation by default (P1 stage 7)
kAsyncShaderCompileDefault flips to true, which also advertises
GL_KHR_parallel_shader_compile by default on both backends. Unset
MOBILEGL_ASYNC_SHADER_COMPILE now resolves to ON; =0 remains the complete kill
switch (reverts the threading and withdraws the extension together).
The gate behind the flip (headless Mesa - llvmpipe for Espryt, lavapipe for
Magma - at
|
||
|
BZLZHH
|
c6299f754f |
[Fix] (MG_Backend/DirectVulkan, MG_State): key per-object memos on lifetime ids, not heap addresses
A destroyed VertexArrayObject's heap address is handed straight back by the
next allocation of its size, and so is a destroyed BufferObject's. DirectVulkan
keyed its per-VAO draw memo on the VAO POINTER and folded the bound buffer's
ADDRESS into the content hash that validates the memoised bindings, so a
delete/recreate pair under a byte-identical attribute layout reproduced both
the key and its validating hash at once. The successor VAO then inherited the
dead one's resolved bindings and the draw fetched from a destroyed VkBuffer.
Both stated defences failed together, because both reduce to the content hash
and the hash's buffer-identity component was itself a recycled address.
VertexArrayObject and BufferObject now carry a globally-unique, never-reused
GetLifetimeId() - the same contract as ProgramObject's, minted from an atomic
starting at 1 so a zero-initialised slot can never name a live object.
VaoDrawMemo matches on (address, lifetime id) and stores the id on recycle,
SetupDrawSnapshot's "the VAO did not move" test compares the id alongside the
config version, and VertexInputStateFactory::ComputeHash hashes the bound
buffer's id instead of its pointer (0 for client memory).
Proven: the use-after-free reproduces at 100% incidence headless on lavapipe,
including a SEGV whose backtrace is the driver dereferencing a destroyed vertex
buffer inside lvp_queue_submit, and it is gone with the fix. New coverage -
MG_Test/State/ObjectLifetimeIdTest (deterministic, GPU-free, no context: it
waits for the real allocator to repeat an address and asserts the id differs,
and skips loudly rather than passing quietly if it never gets the chance), and
MG_IntegrationTest XfbAfterClipDistanceScenario, registered for DirectGLES,
DirectVulkan, and a third DirectVulkan run with async shader compilation pinned
on because that is a second allocation pattern. Gates: 553/553 unit green at
async=0 and async=1; the scenario 5/5 headless at both flag states; 71/72 CI
trace-replay fixtures over both backends, the one failure a pre-existing
lavapipe crash proven not a regression (identical SIGSEGV at the identical
call number under the pre-fix library).
Pending NVIDIA/X11 confirmation: the KHR-GL{32,40} transform_feedback failures
that opened this investigation never reproduced on lavapipe - the -2/-101
pre-fill signature appears in zero pre-fix runs there - so whether this clears
them is UNPROVEN and must be re-measured on the NVIDIA rig against a freshly
re-run pre-fix baseline. The residual suspect is deliberately untouched here:
m_xfbCounterSlotByObject keys its counter slot on the raw GL transform-feedback
name, so a recycled name whose generation check happens to pass would RESUME
instead of BEGIN. That path was never exercised on lavapipe and is neither
confirmed nor exonerated.
|
||
|
BZLZHH
|
dcf918b9ee |
[Perf] (MG_State): adopt in-flight compile jobs across shader objects (P1 stage 6)
~21% of a shaderpack's glCompileShader calls hand different shader objects byte-identical source; the P0b cache only helps after one finishes, so under async two workers would run the whole pipeline twice. Now the GL thread consults a per-context (stage, hash, length, envFingerprint) -> weak-node map at enqueue and ADOPTS the in-flight (or completed) node instead of posting a duplicate - a hit is honored only after a full byte comparison (the hash never decides), a cancel-requested or settled-cancelled node is never adopted, and no worker ever waits. Sharing a node makes the unconditional cancel wrong, so release is now adopter-counted: a plain GL-thread Int (every mutation site is a GL entry point; the single-threadedness argument and the terminal-early-out that keeps the count exact are in the header), and the cancel fires only at count zero AND with no pending link pinning the node (the stage-4 MarkLinkReferenced precedence). Adoption also re-points the object's source at the node's snapshot so the layer-1 memo's pointer compare stays armed - without that, an adopter's next glCompileShader would re-enqueue the very duplicate this stage removes. Both guards are negative-control-proven: each removed guard fails exactly its own tests. Count discipline was proven with a temporary hard-abort on underflow/leak across the full suite and retrace corpus - zero hits. 18 new tests (13 GL-surface incl. shared-node re-source/delete/orphan-sweep isolation, shared failure logs, 48-over-6 stress with a deterministic adoption count, flag-off and KHR-suspended zero-adoption guards; 5 direct map cases incl. fingerprint mismatch and cancelled/expired pruning). Gates: 538/538 unit both flag states, async suites x5 no flakes, NVIDIA DirectGLES retrace identical sets both states. Timing: 2-worker (Android-shaped) 1-3% faster consistently on complementary and BSL; 4-worker unchanged - the win this stage exists for lands where CPU is scarce. |
||
|
BZLZHH
|
d98f72447d |
[Feat, Test] (MG_Backend/DirectVulkan, MG_Test): Magma advertisement + the parallel-compile test net
Same gated push as the Espryt commit. The tests ride here because they exercise both backends' advertisement paths and every piece of the extension: ParallelShaderCompileTest (13 unit cases - the held-job proof that GL_FALSE is observable and a second poll still shows outstanding work, program equivalent, untouched objects read TRUE, always-TRUE with async off, unknown pnames still INVALID_ENUM, zero-count join+inline for compiles AND links, nonzero restores while Initialize() does not, clamping and 0xFFFFFFFF and KHR/ARB sharing one state, the getter vs the budget, the string tracking configuration through glGetString AND glGetStringi) and AsyncCompileScenario (5 real-GPU cases per the design: 64-compile polling, forced-join correctness, string/thread-count checks against a live driver, zero-count synchronous settlement, and async-vs-sync frames rendered byte-identical with quadrant signatures so two identically-wrong images cannot pass). Verified 5/5 on NVIDIA in the full 2x2 backend x flag matrix with MOBILEGL_ITEST_REQUIRE_GPU=1. |
||
|
BZLZHH
|
f15cb8900f |
[Feat] (MG_Backend/DirectGLES): advertise GL_KHR_parallel_shader_compile when async is enabled
Gated on the configuration flag on purpose: the string is the one change a retrace can never cover (Iris/Sodium pipeline their submissions differently once they see it), so the kill switch has to withdraw the app-visible behaviour along with the threading. |
||
|
BZLZHH
|
bd0def6133 |
[Feat] (MG_Impl, MG_State, MG_Util): the GL_KHR_parallel_shader_compile surface (P1 stage 5)
GL_COMPLETION_STATUS_KHR in both object getters, reading the non-joining node-direct state - the one query that must never block is asserted never to reach a join gate. glMaxShaderCompilerThreadsKHR/ARB share one implementation: a zero count suspends async FIRST and then joins every outstanding compile and link this context owns (suspend-before-join is the only order whose post-condition is 'nothing in flight'), a nonzero count restores; the suspension is a process latch the extension controls, kept distinct from the configuration flag that gates the ADVERTISEMENT - an app that turned threading off has not made the extension disappear. GL_MAX_SHADER_COMPILER_THREADS_KHR reports the thread count. DriverPost gains the MobileGL-side async row (PASS/INFO naming the env knob) and an informational host-driver row backed by a new GLES capability probe. The extension string itself lands per backend in the two follow-up commits, keeping this one green stand-alone. |
||
|
BZLZHH
|
6f8b7fbc40 |
[Feat] (MG_State, MG_Util): async program linking on the job graph (P1 stage 4)
glLinkProgram with the flag on snapshots its inputs in a GL-thread prologue (stage-sorted shaders with their compile nodes taken without joining, env, explicit locations/fragdata/xfb, draw-buffer count), then runs the whole link body - glslang link/mapIO, SPIR-V, reflection, routing tables - as a ProgramLinkTask that auto-posts when its last compile dependency settles (+1-guarded countdown; no worker ever waits on another job). The publish is one move of the LinkArtifacts block at the join, with the second version bump so nothing memoized during the pending window survives. The consume-once TShader claim moved onto the shared compile node as a CAS: two link jobs racing for one shader resolve to winner-takes-the-parse, loser re-parses the preprocessed source against the node's own env - identical SPIR-V pinned by test for 2 and for 12 sharing programs. Two deliberate corrections to the design's cancel matrix, both test-proven: attach/detach do NOT cancel a pending link (the snapshot isolates it, and glCreateShaderProgramv's link-then-detach would otherwise discard its own result before anyone read it); and a compile node a pending link depends on is pinned against the orphan-name sweep - the ordinary LWJGL teardown compile/attach/link/detach/delete used to cancel the dependency and turn a must-pass link into GL_FALSE. Continuations are now throw-contained per-item (a stage-3 leftover made load-bearing by the first real continuation), and the review's deadlock find is fixed: the dispatch loop no longer cancels a node while holding the pool mutex, since that cancel can run OnDepSettled -> Post -> same mutex. Explicit joins: the draw path (GetProgramForDraw, both the pipeline stage loop and the plain-UseProgram half) and the composite-link site; destroy paths cancel-not-join; COMPLETION_STATUS readers stay non-joining. Gates: 506/506 unit both flag states; AsyncCompile/AsyncLink/AsyncTeardown suites x10 repeats clean both states (teardown with 128 jobs in flight, then re-Initialize); full NVIDIA DirectGLES retrace flag on twice - result sets identical to flag off, zero new deltas. Compile-phase prefix-diff, flag on vs off: complementary-reimagined 5.21s -> 2.16s, BSL 1.72s -> 0.90s - past the design's final acceptance targets before the KHR extension is even advertised. Default remains OFF until stage 5+7. |
||
|
BZLZHH
|
e5fb57f7eb |
[Feat] (MG_State, MG_Util): async shader compilation behind the default-off flag (P1 stage 3)
glCompileShader with MOBILEGL_ASYNC_SHADER_COMPILE=1 snapshots its inputs on the GL thread (source SharedPtr, CompileEnv, cache handle) and runs the whole pure pipeline - preprocess, validators, extractors, glslang parse - as a ShaderCompileTask on the worker pool, returning immediately. Every read of compile-produced state joins through the single Compiled() gate; links stay synchronous this stage and join their attached shaders at the top of the body. Flag off, the path is the same code run inline. Mechanics: the job node owns all its inputs (no back-pointer, no lifetime tie to the shader object), so re-sourcing or deleting a pending shader is cancel-and-drop, never a wait; glslang worker hygiene is a TLS-allocator scope guard plus GL-thread builtin prewarm (gated on the flag, latch reset on Destroy so re-initialization re-warms); worker-side diagnostics defer through the job and replay on the GL thread at the join, enforced by IsPoolThread asserts in RecordError and an empty-deferred-errors tripwire. A body that throws publishes a COMPLETE failed compile (status false, real info log) rather than an abandoned node, and never memoizes away the retry; a failed enqueue (OOM) cancels the node instead of stranding the joiner - including inside the dispatch loop, where the in-flight slot is repaid. The pool StopAndDrains from an atexit sentinel too: workers still inside glslang parse while exit() ran static destructors was a real 2-in-5 SIGSEGV, reproduced and fixed (15/15 clean after). Backend-internal shader objects (default FS, DirectVulkan blit/mipmap) are cache-less and always compile inline - compile-and-read-in-one-breath needs no round trip. Gates: unit suite 488/488 with the flag off AND on (x5); AsyncCompileTest (12 e2e cases: pending re-source/delete/recompile, byte-identical failure logs across modes, 48-compile cache stress) x10 repeats clean both modes; full NVIDIA DirectGLES retrace identical result sets flag off/on (zero new deltas); compile-phase timing flat as designed (links still serial - the parallel win arrives with stage 4's async link + stage 5's KHR_parallel_shader_compile). |
||
|
BZLZHH
|
c93e5fa409 |
[Refactor] (MG_State, MG_Util): join-by-construction link/compile artifacts (P1 stage 2)
Still fully synchronous - EnsureLinkJoined()/EnsureCompileJoined() are empty
inline no-ops (verified to fold away at every one of the ~1200 call sites;
this project builds without LTO) - but every read of link- or compile-produced
state now goes through a private accessor the compiler enforces, so when
stage 4 moves the bodies onto pool workers, 'which reads must join' is a
type-system fact instead of a 400-line audit.
- ProgramObject: the 31 fields ResetLinkArtifacts clears plus the 5 link
outputs it forgot (infoLog, linkedFragData{Location,Index}, the geometry
strip-capture pair) move into a nested LinkArtifacts behind Artifacts().
ResetLinkArtifacts is now a worker-safe pure clear; the link-observable
version bumps (backendState/link/uboContent) move to a GL-thread-only
BumpLinkObservableVersions() called once from Link()'s prologue and from
glProgramBinary's mandated failure - the link body never writes them, so
a stage-4 worker cannot lose an invalidation against the draw path.
- ShaderObject: compile artifacts (TShader, preprocessed source, side-channel
maps, status/log, consume-once flag) behind Compiled(); the P0b layer-1
memo trio deliberately stays outside as the future non-joining
COMPLETION_STATUS_KHR fast path.
- CompileEnv (new): a GL-thread snapshot of everything the compile pipeline
used to read live from the backend mid-parse - compute limits (the
GetIntegeri_v reach-back is gone from the worker path), advertised
extensions, device quirks, TBuiltInResource inputs. Captured lazily per
backend activation; the consume-once re-parse now runs against the same
env as the original parse.
- The GL-thread prologue / worker-body boundary is marked in Link() where
the stage sort ends; everything below is a pure function of the snapshot.
Public getter signatures unchanged - MG_Impl and both backends compile
untouched. Unit 476/476, Program suites 117/117, DirectGLES retrace 38/39 on
llvmpipe (the one failure is the known pre-existing non-CI iterationrp case;
the NVIDIA userspace driver was updated out from under the running kernel
module mid-session, so GLX there is down until a reboot).
|
||
|
BZLZHH
|
8191075133 |
[Feat] (MG_Util): the async-compile pool skeleton behind a default-off flag (P1 stage 1)
Standalone Asio (submodule, asio-1-38-2 @ 8806a680, ASIO_STANDALONE + ASIO_NO_DEPRECATED, header-only - no linked artifact) and the job machinery the async shader pipeline will run on: JobNode (state machine with deferred errors, continuations firing exactly once, dependency counters, cancel semantics split into request vs outcome) and ShaderCompilePool (asio::thread_pool behind a pimpl so no header leaks asio; big-core count via cpufreq at >=85% of peak clamped to [1,4]; lazily constructed, so with the flag off no worker thread ever exists; StopAndDrain leads DestroyImpl). MOBILEGL_ASYNC_SHADER_COMPILE / _THREADS config knobs, default OFF. Nothing in the GL pipeline references the pool yet - grep-verified; the full DirectGLES retrace and compile benches are byte- and time-identical. 25 threaded unit tests, clean across 20x gtest_repeat. |
||
|
BZLZHH
|
d6caed7822 |
[Fix] (MG_Util, MG_State): five latent frontend bugs the async work made load-bearing
- SpvcSession's move constructor and move assignment dropped the parsed metadata, so a moved-to session silently reported empty reflection. - ParseComputeLocalSize used std::stoull, whose std::out_of_range escaped glCompileShader on an oversized local_size literal; now std::from_chars saturating to UINT_MAX, pinned by a regression test that reproduced the escaping exception. - The compute local_size std::regex was rebuilt on every compile; hoisted. - LinkProgram dumped every shader's full source through MGLOG_D per link. - glslang::FinalizeProcess ran before the GL context tore down, leaving the context's live TShaders pointing at freed builtin symbol tables. |
||
|
BZLZHH
|
9152e88734 |
[Perf] (MG_State): dedupe shader compiles by source hash
Iris-style packs hand MobileGL the same source text repeatedly: probed across three shaderpack traces, 28-32% of all glCompileShader work was redundant - ~9% same-object recompiles with byte-identical source, ~21% distinct shader objects sharing identical source (the same common GLSL chunk glued into many program stages). Two layers, both keyed by XXH64 + length with a full byte compare on every hit (correctness never rides on the hash): - Per-object: a successful (or failed) compile remembers its source hash; glShaderSource with byte-identical text keeps the compiled state and glCompileShader on unchanged source returns immediately. Deterministic (stage, source) pipeline makes the memo observationally identical to recompiling; the consume-once TakeShaderForLink re-parse path is untouched. - Cross-object: a per-context bounded cache (ProgramState-owned, declared to outlive every shader object) shares the preprocessed source, both explicit side-channel maps, and the validation verdicts between objects with equal source; only the glslang parse stays per-object. Single-GL-thread today; flagged for a mutex when compiles go async (P1). Interleaved A/B on the iterationrp trace (the recompile-heavy pack): 5.65s -> 5.46s median total replay, every round faster; BSL/complementary stay flat (their duplicate sources are the small common shaders, so calls drop but wall time is parse-bound on unique sources). Full DirectGLES retrace, 445-test unit suite, and dedupe-semantics tests (no-op recompile, invalidation on new source, failed-compile memo, cache bounds) all green. |
||
|
BZLZHH
|
2406e2d219 |
[Perf, Fix] (MG_Util): preprocessing cleanups - dead scanners, quote-mask bug, one version inspection per compile
Three scoped changes to ShaderSourceProcessor, none altering any transform's
output (pinned by a byte-stability test across the legacy-shader anchor path):
- Delete BlankBlockComments and RemoveDefineForIdentifier - dead since their
callers left; the former's newline-terminated quote handling moves into
MaskCommentsAndQuotedText (below) together with its rationale comment.
- Fix MaskCommentsAndQuotedText treating a quote as running past end-of-line.
GLSL has no multi-line literals, but a stray apostrophe in a directive or
comment tail ("#pragma message can't") blanked the REST OF THE FILE for
every masked consumer - the tokenizer, the version inspection, and the P0a
explicit-location/binding extractors silently lost everything after it.
- Inspect the shader language once per PreprocessShaderSource run instead of
up to five times: NormalizeVersionDirective now takes the already-computed
ShaderLanguageInfo, and the two after-version injections share one
AfterVersionAnchor instead of re-running a full masked sweep each
(FindAfterVersionDirective -> InspectShaderLanguage) to find the same spot.
Compile-phase timings hold (BSL 1.848s, complementary-reimagined ~5.7s);
retraces and the 435-test unit suite unchanged.
|
||
|
BZLZHH
|
b228f813c0 |
[Perf] (MG_Util): replace the builtin-shadowing string scans with one tokenize and a SPIR-V OpName pass
RenameBuiltinShadowingFunction probed the whole source ten times per compile (5 names x mask + scan, each a full-text pass) and still had two blind spots: a 5-name list and single-line-definition-only detection. On Complementary-scale packs (4.5MB of sources) that was ~68% of the compile phase. The rename is now split by FAILURE LAYER, both halves sharing one name table header so they cannot drift: - A SPIR-V OpName pass in SanitizeAndOptimizeBinary covers the full ESSL 3.20 builtin set (~146 names). Renaming a function id is safe by construction: builtin calls are GLSL.std.450 instructions and can never resolve to a user OpFunction, overloads are distinct ids (a helper overload delegating to the real builtin keeps working), dead preprocessor branches never reach SPIR-V, and macro-expanded definitions are covered. ESSL 3.x is the only consumer that forbids the redefinitions, and this pass runs before its transpile. - A lexical pass covers only the 5 names whose exact-signature redefinitions glslang's relaxed parse rejects outright (never producing SPIR-V for the backstop): the historical fma/max3/min3/round/tanh. One TokenizeCode pass; definition detection requires brace depth 0, a type-identifier previous token that is neither a statement keyword nor a directive tail, and skips files whose token-level braces do not balance (preprocessor-asymmetric arms) - over-detection is unrecoverable, so every ambiguity falls through to the backstop. Measured on the compile phase (prefix-diff, 3-run medians, Espryt/NVIDIA): complementary-reimagined 20.0s -> 5.5s, BSL 2.14s -> 1.85s. bliss (the pack that ships from-scratch fma/tanh helpers) stays at SSIM 0.999962. Tests: end-to-end ESSL assertions for the multiline-definition and new-overload shapes, the three adversarial-review reproductions (statement- keyword call under asymmetric braces, dead-#if compat shim, overload delegating to the shadowed builtin), and a source-level assertion pinning the lexical half specifically. |
||
|
BZLZHH
|
0d0527192a |
[Perf] (MG_State, MG_Util): compile shaders with a single relaxed parse
glCompileShader used to parse every source twice: once under the GL client (reflection only) and once under the relaxed Vulkan client (SPIR-V + the plain-uniform global UBO), with GenerateBinary re-preprocessing, re-parsing and re-linking every attached shader on every glLinkProgram. The GL-client pass is gone: Compile() performs the one link-compatible relaxed parse and the linked TProgram serves reflection and codegen both. Measured on the BSL shaderpack compile phase: Espryt 2.80s -> 2.14s, Magma 3.78s -> 3.07s. What the relaxed parse cannot provide is restored explicitly: - explicit layout(location/binding) qualifiers on default-block uniforms and samplers are extracted lexically at Compile() (the relaxed parse strips them) and merged per link with cross-stage conflict checks; - uniforms the relaxed parse sweeps into MGL_GLOBAL_UBO but no stage reads are filtered from the GL reflection surface through GL<->TProgram index translation maps (dead uniforms stay inactive, the synthesized block stays hidden, builtins reflect under their GL spellings); - SPIR-V is generated BEFORE buildReflection touches the program (its live-variable analysis perturbs GlslangToSpv output - generated modules stay bit-identical to the old pipeline's), while the glUniform*-to-scratch routing tables are built strictly AFTER reflection, whose results size and key them; - a TShader feeds exactly one link (mapIO mutates the intermediate); relinks and multi-program attachments re-parse the stored preprocessed source. Validated: DirectGLES retrace suite green (two pre-existing local-driver failures unchanged old vs new), KHR-GL30 877/878 on Espryt/NVIDIA (the one failure pre-exists this change), unit tests green, per-module SPIR-V hashes identical across a full DirectVulkan replay. |
||
|
BZLZHH
|
81bcbd6c14 |
[Fix] (MG_State): allocate program and shader names from one shared name space
GL 3.3 core 2.11 puts program and shader names in one name space: a shader name passed where a program is expected must fail with INVALID_OPERATION, and vice versa. Two independent IndexGenerators handed out colliding names (shader 2 and program 2 could coexist), so CheckProgramNameValidity resolved a shader handle to an unrelated linked program and the error checks in KHR-GL30.get_uniform_tests.get_uniform were silently swallowed - the case only ever passed because the collided program happened to reject the queried location. One shared generator keeps the names disjoint; the per-kind object tables are unchanged. |
||
|
BZLZHH
|
867fe3e0ef |
[Feat] (MG_Util, MG_IntegrationTest): POST rows for the Espryt multi-draw tier, and the scenario that pins it
Three DriverPost rows per the POST rule, since the ladder took on two new driver dependencies: glDrawElementsBaseVertex (WARN when absent - every base-vertex draw then costs a CPU index rewrite and an upload) and compute shaders (INFO - the default tiers never use them). The third names the tier that will actually run, with the full set the driver supports, resolved by the same function the backend calls so the two can not drift. The existing "Multi-draw base vertex" row stopped saying the fallback is a per-draw loop, which is no longer the whole truth. Scenario D asserts the one contract every tier shares: a multi-draw paints exactly what the unrolled single draws paint. The reference side is a loop of glDrawElementsBaseVertex and never enters the emulation, so a tier cannot make itself look right by breaking both sides alike, and a blank-frame pair is rejected outright - drawing nothing is the failure mode this path actually has. Nine cases, chosen for the shapes the Minecraft retraces contain none of: narrow index types, a base vertex past the index type's range, primitive restart inside a strip on two index types, client-memory index arrays, and a batch with zero-count sub-draws (whose prefix sums the flattening tier's binary search has to skip). Each of the six tiers passes all nine on NVIDIA, and ext/auto/compute also pass on Mesa where the ext tier is reachable. The suite is falsifiable, not merely green: rewriting the rebase the way MobileGlues does it - truncate to the source width, no restart passthrough - fails exactly three cases on the drawelements tier (both restart cases and the out-of-range base vertex) and leaves basevertex, which rewrites nothing, passing. That control is also what turned up the restart hole in the flattening tier fixed in the previous commit. |
||
|
BZLZHH
|
0ec487c993 |
[Feat] (MG_Backend): port MobileGlues' multi-draw emulation to DirectGLES as a tier ladder
ES has neither glMultiDrawElements nor glMultiDrawElementsBaseVertex, so both are emulated. DirectGLES had two ways of doing it - one glMultiDrawElementsBaseVertexEXT where the driver has the extension interaction, otherwise a per-draw loop. This adds the five MobileGlues uses (gl/multidraw.cpp), so the ladder is now: one glMultiDrawElementsBaseVertexEXT; one glMultiDrawElementsIndirectEXT over a synthesized command buffer; one glDrawElementsIndirect per command over that same buffer; the base-vertex replay; plain glDrawElements over a CPU-rewritten index stream, for drivers with no base-vertex draw at all; and a compute shader that flattens the whole batch into one rebased index buffer drawn by a single glDrawElements. They live in their own translation unit that owns the entry point outright, preparation included - the compute tier has to dispatch BEFORE PrepareForDraw, or it would have to unpick the program, storage-block and index bindings the preparation just made, and a dispatch inside an open transform-feedback span is not legal at all. The auto ladder is ext -> basevertex -> multiindirect -> indirect -> drawelements, which is NOT MobileGlues' order (it puts the indirect tiers first). Measured on mc_sodium_multidraw, ns/op, median of three: NVIDIA ES 3.2 basevertex 2500 vs multiindirect 5700 and indirect 5800; Mesa llvmpipe ext 19300, basevertex 25200, multiindirect 27600, drawelements 28700, indirect 31000. Ring-allocating the command staging instead of respecifying per batch was tried first and moved the indirect tiers by less than noise, so the cost is the indirect draw path itself, not the upload; only a real multi-draw entry point beats replaying the sub-draws. auto therefore resolves to basevertex on this box - byte for byte the behaviour that shipped - and the new tiers are what a driver with the ext interaction, or without base vertex at all, now gets. compute is never chosen by auto (nor by MobileGlues'): it rewrites the primitive stream rather than replaying it, and it measured slowest here. Four places this deliberately does not follow MobileGlues, each a correctness bug there. A rewritten stream is emitted as GL_UNSIGNED_INT whatever came in, because GL adds baseVertex at full precision and folding it into ushort indices wraps. The restart sentinel is carried across a rebase unrebased, or an enabled primitive restart is lost. The flattening tier declines strip/loop/fan modes, any sub-draw whose count is not a whole number of primitives, and any batch at all while primitive restart is enabled - a restart ends a primitive, so leftover vertices would find a third vertex in the next sub-draw and become a triangle GL never draws. And the indirect tiers decline client-memory index arrays, which have no buffer to address. gl_DrawID gets better rather than worse: the unrolled tiers now feed each sub-draw its index (the spec's value, where the old loop left the uniform untouched), and a program that actually reads it demotes the batched tiers, which can only hold one value for the whole batch. The per-batch cost is nil for the programs that do not read it. Verified: the five DirectGLES retraces are byte-identical (md5) across all six tiers on NVIDIA and on Mesa, each tier proven to have really executed rather than silently demoted, via a per-tier announcement in the log. Unit suite 421/421. The full retrace suite's five failures all reproduce unchanged on a stashed tree, so none are new. |
||
|
BZLZHH
|
23b880c8be |
[Feat] (MG_Config, MG_Util): a tier knob and two capability flags for Espryt multi-draw
MOBILEGL_ESPRYT_MULTIDRAW_MODE=ext|multiindirect|indirect|basevertex| drawelements|compute|auto names the DirectGLES emulation tier for glMultiDrawElements(BaseVertex). Same contract as the Magma knob: a preference, not a demand, clamped at resolution time to what the driver actually has, and invalid values keep auto. Nothing reads it yet. The two capability flags the ladder selects on are new because neither existed in the shape the choice needs. SupportsDrawElementsBaseVertex is the weaker sibling of SupportsMultiDrawElementsBaseVertex - ES 3.2 core or EXT/OES_draw_elements_base_vertex, with no GL_EXT_multi_draw_arrays requirement - and it decides whether a batch can replay its sub-draws with their own base vertices or has to fold them into rewritten indices. SupportsComputeShader is ES 3.1 core plus the dispatch, barrier and shader-object entry points. Both keep the house rule the multi-draw flags already follow: the extension/version check is what proves support, the resolved pointer only confirms it, because eglGetProcAddress may hand back a live-looking stub for a function the context does not implement. |
||
|
BZLZHH
|
ebc5bff9b1 |
[Fix, Feat] (MG_Backend): make DirectVulkan multi-draw actually draw, then pick its best tier
The bug: DirectVulkan.cpp::MultiDrawElements had its entire body
commented out - plain glMultiDrawElements on Magma recorded NOTHING,
no error, no pixels (readback shows the deferred clear never even
materialized). It now shares the tuned base-vertex implementation, and
both plain entries are pixel-proven by a 4-sub-draw harness.
The feature: every CPU-side multi-draw form dispatches through three
tiers after round-9's contiguous-run merge (restructured to merge into
a span BEFORE dispatch, so every tier consumes the shrunken array):
1. VK_EXT_multi_draw: one vkCmdDrawMulti(Indexed)EXT, chunked by
maxMultiDrawCount; per-draw vertexOffset rides in the struct. The
extension is requested only when enumerated and its feature bit
confirmed, entry points via vkGetDeviceProcAddr, demoted if
missing.
2. multiDrawIndirect: the param span uploads DIRECTLY as a transient
INDIRECT-usage buffer - DrawIndexedCmdParam is layout-identical
to VkDrawIndexedIndirectCommand and DrawCmdParam's head is a
legal 24-byte-stride VkDrawIndirectCommand, both static_asserted,
so no repacking - then one vkCmdDraw(Indexed)Indirect per
maxDrawIndirectCount chunk. firstInstance!=0 additionally
requires drawIndirectFirstInstance or the batch drops a tier.
3. The byte-identical unroll.
gl_DrawID: tiers 1-2 are spec-correct (0,1,2,3 across a probe's
sub-draws); the unroll tier keeps the pre-existing always-0 contract.
The default tiers strictly improve DrawID correctness.
Adversarially verified: the five real DirectVulkan retrace images are
BIT-IDENTICAL (md5) across auto/ext/indirect/unroll; zero validation
VUIDs on every tier; a simulated no-EXT device resolves to indirect
and renders the same bytes; the known-red create-indirect fixture
crashes at the identical call before and after (not worse, not fixed).
Unit suite 423/423 on the rebased tree, retrace subset 10/10. Bench:
mc_sodium_multidraw's contiguous shape merges 32->1 before dispatch,
so no bench delta - the tiers' beneficiaries are non-contiguous real
streams (the sodium RETRACE pushes ~58-sub-draw batches, in=out
243101 with zero merges) and mobile drivers. A reproducible +3-4%
code-layout drift on mc_use_program (zero shared code, I-cache
displacement from +400 lines) stays under the action gate and is
booked here rather than hidden.
|
||
|
BZLZHH
|
231d5c90e4 |
[Feat] (MG_Config, MG_Util): a preference knob and POST rows for Magma's multi-draw tiers
MOBILEGL_MAGMA_MULTIDRAW_MODE=ext|indirect|unroll|auto selects the DirectVulkan multi-draw dispatch tier, clamped to what the device supports with one INFO line when it falls back; auto (and unset) picks the best supported tier. Invalid values keep auto. Magma-only: the variable has no effect on DirectGLES. Note for the escape hatch: mode=unroll also forces the GL indirect multi-draw paths onto their per-command loop, where gl_DrawID reads 0 for every sub-draw - Flywheel-style content that keys on flw_drawId renders accordingly. Three DriverPost rows per the POST rule: VK_EXT_multi_draw (PASS/INFO), the multiDrawIndirect feature (WARN downgraded to INFO - there is always a fallback tier), and the resolved dispatch tier with the full chain. drawIndirectFirstInstance gains a row too, since the indirect tier's legality check now relies on it. |
||
|
BZLZHH
|
d5f5e6405b |
[Perf] (MG_Backend): batch DirectGLES multi-draw base-vertex where the driver really has it
When SupportsMultiDrawElementsBaseVertex is true, glMultiDrawElements- BaseVertex issues one glMultiDrawElementsBaseVertexEXT instead of a per-draw loop; the fallback loop is byte-identical otherwise. The local NVIDIA ES driver lacks GL_EXT_multi_draw_arrays, so the batch cannot engage here and no local win is claimed (counter-proven: batched=0 / fallback=264329 across a sodium retrace). On Mesa llvmpipe, which implements the full interaction, the batch engages (batched=4566, ~58 sub-draws per call) and is pixel-identical to a forced-fallback control (same SSIM to the last digit). The beneficiaries are mobile drivers advertising the interaction - the Sodium chunk path collapses 32 driver entries into one - and the DriverPost row shows which side any device falls on. A/B on both backends: every case inside the 5% bar. Unit suite 423/423, retrace subset 10/10. |
||
|
BZLZHH
|
ac3a83b207 |
[Fix] (MG_Util, MG_Test): never take a non-null eglGetProcAddress result as support
On GLVND Linux eglGetProcAddress returns a non-NULL trampoline for ANY name - including a fabricated one - so pointer-nullness can never signal driver support. The three EXT multi-draw entry points were registered as required (spurious error logs on drivers without them) and their pointers were trusted; the NVIDIA ES driver hands back a stub for glMultiDrawElementsBaseVertexEXT that SILENTLY DROPS draws, which once made a "77% faster" multi-draw batch that rendered nothing. The three entries are optional now, and two extension-derived capability flags follow the established Supports* pattern - each is an extension-string check AND a resolved pointer, so a flag alone is sufficient at a call site: SupportsMultiDrawIndirect: GL_EXT_multi_draw_indirect + both entry points resolved. SupportsMultiDrawElementsBaseVertex: (GL_EXT or GL_OES_draw_elements_base_vertex) + GL_EXT_multi_draw_arrays + the entry point resolved. The multi_draw_arrays conjunct is the registry fact the stub exploited: glMultiDrawElementsBaseVertexEXT exists only in interaction with GL_EXT_multi_draw_arrays, and this NVIDIA driver advertises everything else EXCEPT that one - so the entry point is genuinely unsupported while eglGetProcAddress still "resolves" it. Two DriverPost rows report both capabilities (INFO when absent - a fallback always exists). Unit tests pin the stub shape, the exact NVIDIA shape, the supported shape and extension-without-pointer. Proven load-bearing: forcing the old pointer-only condition on the NVIDIA ES driver reproduces the silent drop exactly (sodium retrace SSIM 1.000000 -> 0.329522, no crash, no GL error); with the gate the same run is a literal 1.000000. Unit suite 423/423 (two new tests), retrace subset 10/10, integration suite 52/52. |
||
|
BZLZHH
|
335f2decbd |
[Perf] (MG_Backend): stop DirectVulkan re-proving sampler sets and re-walking render passes
Two per-draw costs from the round-10 profiles. A per-program sampled-set epoch inside UniformManager skips the per-binding descriptor proof walk when no texture or sampler API ran since that program's previous draw - the mc_sampler_churn/mc_tex_param pattern. The pass-switch path stops re-deriving render-pass state that its own value hash already pins. Load-gated 6-round order-alternating A/B (medians): magma tex_param -13.3%, pass_switch -10.9%, state_toggle -5.4%; espryt untouched and unmoved. The two matrix flags (sodium +7.5%, tex_stream +6.2%) reversed under 10-pair isolated alternating re-runs (-5.5% and +2.5%) - the same position-bias artifact every previous round's flags showed. Unit tests 421/421; retrace subset and the 52-entry integration suite pass. Landing note: this diff was authored by a round-10 agent whose session died before adjudication; the A/B data survived (r10bmag_ab_raw.csv) and the flags were adjudicated before landing. Its relink also exposed the pre-existing exit-teardown SIGSEGV fixed in the previous commit. |
||
|
BZLZHH
|
fb1ad96c04 |
[Fix] (MG_Backend): stop DirectGLES twin destructors calling a dead driver at exit
The static twin registries destroy their backend objects from __run_exit_handlers, and a twin destructor then jumps through g_GLESFuncs into a driver library that exit() may already have torn down - a latent SIGSEGV that DriverBench has been dumping core with on every exit, and that any relink shuffling static destructor order can hand to the trace-replay binary (a byte-perfect replay then "fails with status Segmentation fault"). A process-teardown flag now short-circuits the program, VAO and texture twin destructors: past exit() the driver reclaims every GPU object anyway, so the skip is a deliberate leak of nothing. The flag is set by a std::atexit handler registered lazily on first registry use - by then every static everywhere has finished constructing, so the handler runs BEFORE any static destructor. A registry-destructor hook was tried first and is wrong: tests and cache resets destroy temporary registry instances mid-run, which latched the flag while the process was alive (caught by DirectGLESBackendTexture.DestructorDeletesIdAndScrubsBindingCache). 421/421 unit tests, the retrace subset exits cleanly on both backends, and the 52-entry integration suite passes. |
||
|
BZLZHH
|
313b75a7c0 |
[Test] (MG_IntegrationTest): pin the two shipped memo bugs with rendered pixels
Both |
||
|
BZLZHH
|
d7976326fa |
[Fix] (MG_Backend): two DirectVulkan draw memos trusted more than they proved
Two correctness holes from the round-7/8 fast-path work, found by bisecting the retrace matrix after corruption reports on device. Cross-frame slice trust: the vertex-binding and EBO memos skipped the acquire - the frame's content-sync point - whenever their recorded slice epochs still matched, trusting the BumpSliceEpoch inventory to cover every way a buffer's GPU copy can go stale. At least one mutation path escapes it: journeymap and common-mods retraces shipped visibly corrupted, and Sodium on an Adreno device rendered random triangles from stale vertex data. A memo recorded in an earlier frame now declines, so the first draw of each (VAO, frame) re-runs the full acquire; the same-frame paths (layout memo, factory-chase elimination, one-compare rescue) are untouched. The cross-frame idea can return once the bump-site inventory is proven complete against exactly these traces. Transform-flags memo key: GetShaderTransformFlags reads the swapchain pre-transform AND whether the bound draw framebuffer is the default one - only a presenting pass gets the Y-flip/rotation bits. The memo declared it pure in the pre-transform, so after any render-to-texture pass the next default-framebuffer pass inherited the FBO's unflipped flags: 1.17-main-menu retraced as a perfectly rendered, perfectly upside-down frame (SSIM 0.052, deterministic), and cloud passes flickered on device. The memo now keys on (preTransform, isDefaultFbo). DirectVulkan retraces for 1.17-main-menu, journeymap, common-mods, sodium and xaero-world-map all pass on lavapipe; unit tests 421/421. |
||
|
BZLZHH
|
72ee7c439c |
[Perf] (MG_Backend): merge DirectVulkan's contiguous sub-draws, remember four programs
61% of mc_sodium_multidraw's steady-state CPU sat inside the driver encoding one vkCmdDrawIndexed per sub-draw. MultiDrawElements now collapses contiguous runs: merge only when the topology is a list (POINTS/LINES/TRIANGLES), the accumulated count sits on a primitive boundary, primitive restart is off, baseVertex/instanceCount/ firstInstance are identical and firstIndex is adjacent, with a count-overflow guard - the bench's 132x32 sub-draws become 132x1. Dangling-index discard semantics for list topologies are what the GL spec already mandates per draw. No new Vulkan feature, so no DriverPost gate; VK_EXT_multi_draw stays a gated follow-up. The draw fast path's single SetupDraw snapshot died on every program ping-pong (use_program's A/B pattern sent every other draw down the full path, CollectSampledTextures alone 6.2% self). A 4-entry program-keyed snapshot table (MRU by program lifetime id, per-entry sampled-set copies, per-entry invalidation on decline or full-path start, all entries still cleared at command-buffer boundary, pipeline age-out and swapchain recreate) keeps all cycling programs hot. Load-gated 6-round order-alternating A/B, sha1-fingerprinted pair: sodium_multidraw -41.3%, use_program -23.9%, pass_switch -12.7%, tex_param -3.8%, vanilla -2.1%; the one flag (tex_stream +5.4%) reversed to -0.5% across 10 isolated alternating pairs. Espryt untouched and unmoved. Unit tests 421/421. |
||
|
BZLZHH
|
cdea275227 |
[Perf] (MG_Backend): stage only the rects DirectVulkan actually dirtied
Consume MipmapStorage's new dirty-rect list: pack each rect tightly into the staging block and issue ONE vkCmdCopyBufferToImage with N regions instead of staging the whole union box. Offsets are computed identically in the pack and copy loops; disjoint rects mean no overlapping copy destinations; the combined depth-stencil and RGB-expand/depth-convert paths keep their single-box route (gated to the color-aspect, no-conversion case). 54% of mc_tex_stream's steady-state CPU was the one shadow->staging memmove of the union box; staged bytes drop to 4.8% (~2MB -> ~95KB per frame) and the case improves ~-49% (5945 -> 3048 ns/op, ~2.2x native to ~1.2x). Zero validation-layer findings on the 95-region copy. Unit tests 421/421. |
||
|
BZLZHH
|
6a02c5fea0 |
[Perf] (MG_Backend): upload only the rects DirectGLES actually dirtied
Consume MipmapStorage's new dirty-rect list: when a level offers a profitable rect list, the sync path issues one glTexSubImage2D/3D per rect under a single UNPACK_ROW_LENGTH set/reset instead of one call covering the union box. Striding is the exact scheme the single-box path already uses (UNPACK_ALIGNMENT pinned to 1 by ScopedDefaultUnpackState, so every bpp is stride-exact); levels without a profitable list take the old path unchanged. On the atlas-streaming case this trades one ~2MB upload for ~95 small ones totalling ~95KB - roughly a wash in driver-call overhead on desktop NVIDIA GL (mc_tex_stream ~-3%), a clear byte-volume win for tiled/mobile GLES where the driver shadow-copies every upload. Unit tests 421/421. |
||
|
BZLZHH
|
7db5b35a3e |
[Perf] (MG_State): remember every dirty rect, not just their union
A Minecraft frame updates ~95 scattered 16x16 sprites in a 1024x512 atlas; MipmapStorage's single union dirty box turned ~95KB of changed texels into a ~2MB upload on every backend. The storage now keeps a bounded (96-slot) list of pairwise-disjoint dirty rects BEHIND the untouched union box: rects cascade-merge on touch or overlap, overflow folds the pair with minimum enlargement and re-cascades, whole-level dirties and respecifies just clear the list (empty list = "union box tells all"). GetDirtyRects hands the list out only when it has 2+ rects, fits the caller's capacity, and its summed area is under 75% of the union box - fewer driver calls beat equal bytes - so consumers can never stage more than the union box did. The list is maintained inside the same four mutation funnels every texel writer already goes through (MarkDirty, MarkDirtyRegion, AllocateLevel, TruncateToLevelCount - callers enumerated at the declaration), so list and union box cannot disagree. Backends OPT IN: the union-box API and its update order are byte-identical, and an unmodified backend keeps rendering exactly as before. 96 slots is measured, not guessed: on the bench's 95-sprite lattice a 16-slot list collapses to >93% of the union box, 96 slots reach 4.8% (~2MB -> ~95KB staged per frame). Verified by a 2859-check fuzz run against a reference dirty bitmap (union exactness, full coverage, disjointness, bounds, profitability). Unit tests 421/421. |
||
|
BZLZHH
|
990e518e33 |
[Perf] (MG_Backend): give DirectVulkan's draw memo a table that fits in cache lines
The per-VAO resolved-bindings map probe was ~45% of UploadAndBindVertexBuffers' self time, and the aux-memo pointer chase was the single hottest instruction left in TrySetupDrawFastPath. Both die together: a fixed 2048-slot two-probe 64B-aligned VaoDrawMemo table embeds the VAO key, content-hash-validated layout facts and the bindings payload reordered hot-to-cold. Layout facts hold exactly while the slot's content hash equals the live VAO's own config-guarded hash; a recycled VAO address either misses or reproduces a byte-identical config, for which the facts are correct by construction. Bindings keep their full per-draw revalidation; recycled slots zero their frame serials so half-filled entries can never match. ComputePipelineStateHash, the depth/stencil probe and the primitive-restart probe now take one bulk GetRenderStateParameters() fetch instead of ~17 cross-TU accessor calls (verified pure field reads, identical bit packing). The EBO slice memo gained the same manager-wide epoch one-compare rescue the vertex half uses. GetShaderTransformFlags is memoized on pre-transform. Sodium's MultiDrawElementsBaseVertex hoists GetGLTypeSize out of the per-sub-draw loop, replaces the division with a shift, and skips unsupported index types loudly instead of dividing by zero. Also verified: a GL_BLEND toggle recompiles nothing in steady state - the glslang frames in earlier state_toggle profiles were startup contamination. Quiet-box load-gated 6-round A/B: sodium_multidraw -8.0%, tex_param -4.1%, use_program -3.3%; steady-state vanilla_draw CPU -20% ns/op at 4096 frames (the 80-frame matrix compresses CPU wins under GPU boost clocks; profiles confirm UploadAndBindVertexBuffers 6.3% -> 4.4% including the table probe, and the aux cold-line load gone). The one matrix flag (pass_switch +7.5%) reversed to -3.2% in 10-pair isolated re-runs. Unit tests 421/421. |
||
|
BZLZHH
|
25a8f51db5 |
[Perf] (MG_Backend): make DirectGLES program switches remember their own bindings
mc_use_program cycles programs whose texture bindings never change, yet every switch re-walked the units. Six fixes, one theme: a switch back to a known program should find its own state waiting. Per-program 4-entry resolved-texture-binding memo (round-robin, shadow memcmp on hit) skips the unit walk when a program returns with its bindings intact. The whole sampler-uniform pass in BindCurrentProgramWithResources is memoized per program twin behind (context, unitBindingsEpoch, samplingGeneration, backendStateVersion, textureContextGeneration) plus a per-sampled-unit sampler-shadow row compare, invalidated on relink/backend rebuild; the BindCurrentUnitSamplers walk sits behind the same keys. Every unit assignment, sampler-parameter change and bind path was verified to bump one of those inputs. UboRingAllocate's common path is now a generation check, a power-of-two mask, an overrun check and a head bump - the duplicate availability probe, frame-mark retirement and divisions moved to the wrap slow path. The per-context framebuffer binding slots (the frontend getter linear-scans per call) are cached as direct pointers - slots are by-value members of GLContext, so the pointers are stable by construction - feeding SyncCurrentFBO, SyncNeccessaryTextures and the broadcast memo; BindCurrentFBO's per-draw registry hash Find became a TwinLookupMemo probe. The VAO config-version cold-line load is hoisted to the top of PrepareForDraw to overlap its miss. Quiet-box load-gated 6-round order-alternating A/B, all nine cases, both backends: use_program -27.7%, vanilla_draw -16.2%, ubo_range -13.4%, pass_switch -11.1%, sampler_churn -10.7%, state_toggle -9.2%, sodium_multidraw -5.3%, rest flat. No regression on either backend (magma's one matrix flag disproved by isolated re-runs against byte-identical DirectVulkan sources). Unit tests 421/421. |
||
|
BZLZHH
|
8f2b766b56 |
[Perf] (MG_Backend): let DirectVulkan trust across frames what it proved once
The draw fast path still paid for its own proofs: the hottest single load (20% of TrySetupDrawFastPath) was chasing the cold VertexInputStateFactory heap entry just to answer "same vertex-input layout?". That answer now comes from the frontend VAO's config-guarded aux memo (layout hash + attribute masks), and a VAO-cycling stream with a stable layout skips the pre-flight AND pipeline re-resolution entirely. The VkProgramObject* is memoized on the snapshot behind a new ProgramFactory cache-structure epoch (bumped on every insert/erase; use is re-stamped so the idle sweep can never evict a live entry). A render-state version move no longer forces the full path: the pipeline value hash is refreshed in place and the 8-entry memo probed directly (the GL_BLEND-toggle case). The resolved-vertex-bindings memo now revalidates all-resident unmapped entries ACROSS frames via per-binding slice epochs - minted from a process-lifetime counter so a recycled address can never revalidate, with every mutation path funnelled through BumpSliceEpoch - while stamping each resource's GPU-use serial exactly as the skipped acquire would, preserving the busy-tracking that glBufferSubData's host-write-vs-staged-copy choice depends on. Resident index buffers get the same treatment through an EBO slice memo. The six-part dynamic-state tail (viewport/scissor/blend constants/depth bias/line width/stencil) is gated behind one render-state-parameters version + pass-geometry compare per command buffer. GetSlice is inlined; SampledBindingsUnchanged walks only the program's declared bindings. Quiet-box 6-round order-alternating A/B (on top of the frontend VAO-bind commit): vanilla_draw -20.8% (790 -> 626 ns/op, 3.4x native to 2.5x), sampler_churn -28.2%, ubo_range -8.4%, state_toggle -3.8%; tex_param's matrix flag (+10%) was adjudicated by an isolated alternating re-run at +1.0% - position bias, not regression. Unit tests 421/421. |
||
|
BZLZHH
|
b9d8ad0421 |
[Perf] (MG_Backend): give DirectGLES one epoch that says no buffer moved
Four draw-path costs, one theme: re-proving what nothing invalidated. A manager-wide buffer-mutation epoch (atomic; bumped with release AFTER every mutation lands: all six BufferBackendOps via tracking wrappers, every backend-initiated writeback - XFB readback/scatter, the five pack-PBO readbacks - registry registration changes, and backend context destruction; the full site inventory lives in a comment at the accessor) lets the per-VAO resolved-buffers memo stamp the epoch after one all-clean probe pass and skip every IsBufferDrawClean probe while it holds. The IBO keeps its bound-object identity compare - only the probe is elided. Non-bumping paths are enumerated with why they are safe: GPU-authoritative writes are ignored by the probe, persistent-mapped resources are clean by construction, and draws on non-persistent maps are frontend-rejected GL errors. GetProgramForDraw is hoisted to one call per PrepareForDraw and handed to the four consumers that each re-derived it. The enabled-draw-buffers walk feeding the fragColor broadcast count is memoized on the (FBO, slot version, object version) trio. The UBO-binding loop probes IsBufferDrawClean before falling back to EnsureBufferResource. The texture chain captures (context, maxTouchedUnit, samplingGeneration, unitBindingsEpoch) once per draw - shared by SyncNeccessaryTextures and BindCurrentTextures, halving the epoch computations - and an aggregate gate that is the exact conjunction of the three Sync*ToBackend early-outs skips the per-texture cross-TU calls. The t_egl* thread_local verification pair became owner-thread-guarded atomics reset by MakeCurrent/ReleaseCurrent, removing __tls_get_addr from the draw loop. Quiet-box 6-round order-alternating A/B (with the frontend VAO-bind commit): all NINE Espryt cases improved - sampler_churn -11.0%, state_toggle -9.3%, ubo_range -9.1%, vanilla_draw -5.4%, pass_switch -3.5%, the rest -1% to -2.5%. Unit tests 421/421. |
||
|
BZLZHH
|
f8069c0624 |
[Perf] (MG_State): stop paying two atomic refcounts for every glBindVertexArray
perf annotate put 94% of VertexArrayState::Bind's 10.5% self time on the two lock-prefixed shared_ptr refcount RMWs each bind performs. The bound VAO is now stored as a slot index into m_vertexArrays - no SharedPtr copy, no atomics on the bind path. The lifetime invariant (the bound object is kept alive by its slot; any cold path that clobbers a bound slot - delete-while-bound including slot 0, create-over-bound-slot - detaches the old object into m_boundDetached so GetBoundVertexArray keeps answering with it) is enforced in MarkVertexArrayForDeletion / CreateVertexArrayObject rather than assumed, and documented at the change. Out-of-range binds and null slots keep their exact old semantics. VertexArrayObject also gains two opaque config-version-guarded backend aux memo words, letting a backend answer "same vertex-input layout?" from the frontend object instead of chasing its own cold cache entry. After this change the frontend Bind drops out of the DirectVulkan draw profile entirely (11.5% -> 0.5%). Measured jointly with the two backend rounds that land on top: quiet-box 6-round order-alternating A/B, all nine cases, no case worse than noise on either backend. Unit tests 421/421. |
||
|
BZLZHH
|
d0aae85da2 |
[Perf] (MG_Backend): let DirectVulkan's draw fast path survive a VAO swap
TrySetupDrawFastPath declined on its VAO pointer check for every draw of a 512-VAO cycle - the Blaze3D chunk-render shape - so the fast path was dead exactly where it mattered: full SetupDraw, per-draw ResolveSamplerDescriptor, SyncTextureAndGetDescriptor and render-pass re-fetch, for draws whose only change was the VAO. Three fixes. A moved VAO now re-runs only the vertex-input pre-flight and re-resolves the pipeline instead of declining to the full path. That resolution probes the value-keyed pipeline memo directly off a cached pipeline-state hash and snapshot render-pass hash, skipping GetOrCreateRenderPass and its GetPendingRenderbufferClear probes per draw; a stale cached hash can only miss, never false-hit. And when the sampler-descriptor hint holds and the program's single dynamic UBO re-resolves to the same VkBuffer and range - only the dynamic offset moved, the per-draw glUniform case - the descriptor walk collapses to one offset recompute and a vkCmdBindDescriptorSets of the same recorded set with new pDynamicOffsets. The rebind memo is invalidated at BeginFrame, layout destruction and override walks; the program lifetime id never repeats, and per-frame descriptor sets are never rewritten within their frame. mc_vanilla_draw -36.9% (1260 -> 795 ns/op, 4.6x native to 3.4x), sodium_multidraw -18.0%, state_toggle -14.9%, sampler_churn -7.8%, use_program -7.7%, ubo_range -7.3%, tex_param -7.3%. All nine cases on both backends, interleaved A/B; no attributable regression. Unit tests 421/421. |
||
|
BZLZHH
|
b904658b10 |
[Perf] (MG_Backend): stop DirectGLES re-resolving the same VAO's buffers and twins every draw
Four per-draw costs, all lookups that re-answer the same question. SyncNeccessaryBuffers walked all 32 attribute slots cold and ran EnsureBufferResource per buffer on every draw. The backend VAO twin now hosts a resolved-draw-buffers memo: the deduped enabled-attribute buffers and the index buffer resolve once per VAO config version, and each hit re-validates every entry with IsBufferDrawClean - a shadow probe mirroring every no-op branch of EnsureBufferResource (resource identity, context generation, pending ops, change serial) - falling back to the full path for just the dirty entries. The IBO entry is checked against the live bound object each draw, so slot-version wrap cannot false-hit. The registry hash Finds that resolve state objects to their backend twins ran several times per draw. TwinLookupMemo - a direct-mapped, Fibonacci-hashed table (4096 VAO / 256 program slots) with weak-ptr owner equality against address reuse - answers them in one probe; collisions fall back to the registry. A live entry's twin is never replaced once set, so owner equality proves the raw pointer. SyncCurrentVertexAttributeValues' pending-mask memo was a function-static single entry that missed every draw once the app cycled VAOs; it now lives on the twin. CurrentXfb()'s per-draw FastSTL map lookup became a cached pointer invalidated at every map mutation (open addressing moves values on any insert/erase/clear). mc_vanilla_draw -12.6% (3.4x native to 3.0x), ubo_range -9.8%, sampler_churn -7.5%, pass_switch -6.5%, sodium_multidraw -6.0%, state_toggle -4.6%. All nine cases measured on both backends, interleaved A/B; no case regressed. Unit tests 421/421. |
||
|
BZLZHH
|
4b3fd11462 |
[Perf] (MG_Backend): key DirectVulkan's pipeline memo on state values, not a version that never repeats
Two per-draw churn costs, one cause each. A blend toggle switched pipelines through a memo keyed on a monotonic pipeline-state version - which never repeats, so flipping GL_BLEND off and back on produced a "new" key both times, forced the full SetupDraw and rebuilt the whole pipeline payload for a pipeline the cache already held. The memo now keys on a value hash of the pipeline-relevant fixed-function state, recomputed only when the state version moved, and the consecutive-draw fast path re-resolves just the pipeline through it when nothing but render state changed. Blaze3D brackets every batch with exactly this toggle; mc_state_toggle drops 36% (6629 -> 4230 ns/op, 4.8x native to 3.7x). The sampler-churn cost had the same shape as the Espryt side fixed separately: glBindSampler bumps the frontend texture-bind generation even when it re-binds the sampler the unit already holds, so the per-draw fast path died every draw. The fast path now proves each binding's descriptor inputs unchanged - texture and sampler lifetime ids, parameter and content sums, the sampling-resolution generation, image epochs and exact layouts - and reuses the binding's cached VkDescriptorImageInfo instead of re-running the resolve chain. mc_sampler_churn drops 30% (1597 -> 1125), and the proof machinery pays for itself on the uniform-range case too (-17%). mc_tex_param stays where it is on this backend deliberately: profiling shows its remaining cost is frontend validation with zero backend work, unreachable from Renderer/. All nine cases measured on both backends, interleaved A/B, no case worse than noise. Unit tests 421/421. |
||
|
BZLZHH
|
9be5d95440 |
[Perf] (MG_Backend): give DirectGLES unit bindings an epoch the sampler churn cannot fake
The texture-binding memos added earlier keyed on the frontend texture-bind generation, and 26.2-style unit switching defeats them: glBindSampler bumps the generation even when it re-binds the sampler the unit already carries, so a frame that cycles active units re-ran the full two-pass, eleven-slot alias resolution and the unbind walks on every draw. mc_sampler_churn sat at 1674 ns/op against the native driver's 239 - the worst multiplier left on this backend - with about half the time in two virtual calls per binding slot. The units now carry an epoch: a snapshot of each touched unit's slot objects and sampler object, compared by weak_ptr OWNERSHIP rather than raw pointer - a held weak_ptr pins its control block, so a freed-and-recycled object can never owner-equal its predecessor, which is the ABA hole a pointer key would have and the reason version keying was rejected (WithTemporarilyBoundNamedTexture bumps slot versions without touching the bind generation). The (context id, bind generation, high-water mark) triple gates the snapshot walk to at most once per draw; the epoch moves only when a binding really changed. Both per-draw memos key on the epoch plus the sampling-resolution generation, which carries what the epoch cannot see: a default texture's image appearing, and every completeness input. Two smaller memos ride along: the per-unit sampler-registry lookup (owner-keyed, misses never cached - the backend object may be created later in the same draw), and the pending-vertex-attribute mask, whose first version scanned all 32 slots and put +10% on the VAO-cycling case before being restricted to the program's active locations. ns per op, DriverBench on a GTX 1660 SUPER, isolated A/B, all nine cases on both backends: mc_sampler_churn 1673 -> 732, mc_use_program 4513 -> 4279, mc_state_toggle 2365 -> 2247, everything else within noise and nothing worse. 7.0x native to 3.1x on the churn case. Unit tests 421/421. |
||
|
BZLZHH
|
d49d79a64b |
[Perf] (MG_Backend): pool DirectVulkan's upload staging and batch its submits
Every dirty texture bought itself a fresh staging buffer (vmaCreateBuffer + vmaMapMemory), a fresh command buffer, a fresh fence, and its own vkQueueSubmit. A perf profile of the sprite-animation case put 41% of the whole run in the kernel on the resulting ioctl traffic; the reclaim list already avoided waiting on the fences, so the cost was the allocation and submission machinery itself, paid per texture per frame. Staging now comes from a pool of persistently-mapped blocks (1 MiB minimum, exact-size beyond that, bump-allocated, 32 MiB idle cap), and uploads record into one shared batch command buffer from a dedicated command pool, going out as one submit with one pooled fence per flush. Fences, command buffers and blocks all recycle through the existing fence-list reclaim instead of being destroyed. Flush points: before every frame command buffer submission (which is what preserves the old ordering argument - the batch reaches the queue strictly before anything that could sample its images), on the glFlush finite-time path, when a batch would outgrow its staging bound, and eagerly at 128 KiB, which measured faster because the GPU overlaps the copy with the rest of the frame's CPU recording. The mid-frame upload-draw-upload-again sequence detects itself through the batch image list and flushes first, reproducing the old two-submit granularity exactly; a deferred image release flushes any open batch that still references the image, because drain proofs only cover submitted work. ns per op, DriverBench on a GTX 1660 SUPER: mc_tex_stream 9405 -> 5373 (2.3x the native driver, from 3.9x), atlas_sprite -57%, lightmap -89%, chunk_upload -10%; draw-path cases unchanged. The suite's sampler-churn number reads a few percent worse right after the now-much-faster upload case, which was chased to schedutil downclocking during the newly-blocking-free frames - isolated and frequency-pinned runs measure parity; noted here so the next person does not re-chase it. Unit tests 421/421; Vulkan validation layer clean across draw and upload cases. |