mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-20 00:11:56 +09:00
[Docs] (Disaggregated): P6's plan and contract draft; the working notes move from a local directory into the repository
Two things land together: the plan and contract draft P6 needs before it starts, and the
working notes that until now existed on exactly one machine.
1. P6's plan and contract draft (neither started; not one line of code)
docs/Disaggregated/P6-SPAWN-PLAN.md the package plan
docs/Disaggregated/P6-CONTRACT-DRAFT.md the contract draft (moves to
MG_Remote/CONTRACT-P6.md when c6 lands)
The three findings that mattered, all written into the documents:
a. P6 is SMALLER than its roadmap cell suggests. The transport primitives are already in
the tree and already tested: SocketDoorbell including hangup detection and the two
lost-wakeup fences (Doorbell.cpp:211-360, four cases in FdPassingTest), SCM_RIGHTS fd
passing, ShmSegment::Adopt, and the SurfaceOp / WindowKind schema already defined in
protocol.fbs. inproc uses a real second shm mapping rather than an alias, and
SessionRings.h:22-28 says that was done so today is not the first time it runs. P6 is
assembly plus a control plane plus a process, not "write a socket transport".
b. The evidence for "P6 is only a transport swap" has EXPIRED. It was bought by P5c, whose
audit found 59 direct memory accesses in a tree that already looked clean. P5e changed
the tree again. So the first package, a6, is a read-only audit that writes no code, and
every contract row that depends on it is marked PENDING a6 rather than decided in
advance.
c. P6 does not get a game running on the phone. ANativeWindow* is meaningless in the child,
and protocol.fbs:186-196's own comment says Android transfers the window out of band -
that band is P12. P6 lands pbuffer / surfaceless / offscreen, which is what HeadlessGL,
the split-equivalent lane and trace replay need, and which is exactly what P6's own exit
gate measures. A real window reaching the server is a named refusal. This gets its own
section because "P6 is done" would otherwise read as "the game runs now".
The contract adds rule G: a role may not name a process-local handle. It extends P5c's
rule E (a role may not name the other role's memory) from memory to identity - EGLDisplay /
EGLSurface / EGLContext / ANativeWindow* / fd number / pid / mapped address either cross as
a token this contract defines or they do not cross, and a descriptor crosses only through
SCM_RIGHTS, where the kernel renumbers it.
What P5e left for P6 gets its own section. The latch semantics are already designed
(ARCHITECTURE.md:504) and are copied unchanged. What is new is that after run-ahead the
client no longer waits while the peer process can DIE: the client holds records that were
published and will never be applied, and Android's freezer suspends the child without
closing anything. So slow and dead must be distinguishable WITHOUT a timeout threshold.
That is P6's only new design; everything else is carriage. It shares a predicate with E1's
outstanding re-specification (ID-122), and the two should be designed together.
Two gates carried over from P5e's lessons: G2 requires integration-spawn's test-name set to
match integration-split's name for name, because a differing set means the lane is quietly
running less; and the arm-proof gate requires every entry to record the child pid and
transport=spawn in its OWN private log - ID-124 established that this kind of false green
really happens, and a spawn lane that actually ran monolith would pass every other gate.
S2's shape must be settled before it is written, and the document says why: E1 is the
cautionary case.
2. The working notes move into the repository (docs/Disaggregated/notes/, 415 .md files)
ROADMAP / CURRENT_STAGE_PROGRESS / MEASUREMENTS cite ruling numbers, audit line numbers and
the provenance of gate figures from ~/w7/notes/, a directory that exists on one machine and
is under no version control. A record cited hundreds of times that lives only locally is not
a record, so it moves in whole, with its directory structure intact.
Only .md moves. Scripts, run logs, census and gate result JSON, patches, APKs and nm dumps
stay where they are: they are evidence DATA rather than documents - large, regenerated per
run, and mostly carrying absolute paths and device serials that mean something only on the
machine that produced them. The mapping and this split are written in
docs/Disaggregated/notes/README.md.
Two exceptions are recorded there too: P6's BRIEF has been promoted to a first-class
document and keeps no copy under notes/, so the two cannot drift apart; and recovered/ holds
material salvaged from failed workflow runs, which may duplicate the like-named phase
directories and is kept because a few of those were the only complete version at the time.
CI's citation lint takes docs/Disaggregated/*.md only (non-recursive), and notes/ is
deliberately outside it: those notes' file:line citations were written against their own
code heads, and resolving them against today's HEAD would fail wholesale - which is not an
error, it is their correct state as a historical record. To check one, use the revision it
declares (--rev).
3. Inconsistencies fixed in passing
- README.md's status line still said P5e was in progress while the stage table already said
closed.
- CURRENT_STAGE_PROGRESS's next-step item 1 still listed the exit gate as pending, though
(a) and (b) were run in the previous commit; rewritten to closed plus the one item that
actually remains (E1).
- ROADMAP's P6 row still had the pre-phase plan in three of its cells; it now carries the
package order, the scope boundary and the two new gates.
Checked with check_doc_citations.py: 125 citations across docs/Disaggregated/*.md, 0 problems.
This commit is contained in:
@@ -0,0 +1,237 @@
|
||||
# P5d round 3 brief: inproc performance, the profile-directed cuts
|
||||
|
||||
Repo: MobileGL (branch `feat/disaggregated`, head `85362a85`). You work in an isolated git worktree
|
||||
of it (your cwd). Read `docs/Disaggregated/P5D-INPROC-PERFORMANCE.md` (rounds 1-2), the relevant
|
||||
parts of `MobileGL/MG_Remote/CONTRACT-P5C.md` (role guards, rule E) and `docs/Disaggregated/ROADMAP.md`
|
||||
"通用纪律" before touching code.
|
||||
|
||||
## Why this round exists (measured, not guessed)
|
||||
|
||||
Workload: FCL + Minecraft 26.3-rc-3, view distance 12, Redmi 2f7cbe2e (Adreno 830), DirectGLES.
|
||||
Split (`MOBILEGL_TRANSPORT=inproc`) 63-65 fps vs monolith 110-118 fps, ~852 draws/frame.
|
||||
simpleperf (cpu-cycles, whole process, 15 s in world) at head 56a77348, symbolized:
|
||||
|
||||
Process split: GL/client thread (`Thread-17`) 39.5% of cycles, apply thread (`mgl-srv-apply`) 36.9%.
|
||||
Monolith: GL thread 52.6% of a 39% smaller total.
|
||||
|
||||
### Apply thread (mgl-srv-apply), self %
|
||||
- `std::mutex::unlock` 13.6, `std::mutex::lock` 12.4, `pthread_mutex_lock` 12.1, `pthread_mutex_unlock` 11.0
|
||||
-> ~49% of the thread. Call graph: 58-60% of `mutex::lock/unlock` samples have
|
||||
`ServerLoop::ApplyThreadMain()` as the DIRECT caller; 99.8% of `pthread_mutex_lock` via `mutex::lock`
|
||||
is ApplyThreadMain. These are NOT the backend `pendingMutex` sites the round-2 report guessed.
|
||||
They are the idle poll: `for(;;){ PumpControlRequest(); DrainRing(); if(ready()) continue; bell.Wait(...) }`
|
||||
where `PumpControlRequest()` takes `m_controlMutex` every iteration and the `ready` lambda calls
|
||||
`ControlIsPending()` which ALSO takes `m_controlMutex` (ServerLoop.cpp:371-384, 452-475).
|
||||
- `ApplyThreadMain` self 7.95, `@plt` 6.1 (31% of plt from ApplyThreadMain), `__emutls_get_address` 7.4
|
||||
- Inclusive: `ApplyThreadMain` 78.4%, `DrainRing` only 29.0% (= the real apply work: OnDrawVbo 22.7%,
|
||||
PrepareForDraw 16.3%, SyncNeccessaryTextures 8.6%, SyncMipmapsToBackend 3.3%, HandleOf<ITextureObject> 3.0%,
|
||||
`MGPipeRefuseAllocatorFromApplyThread` 2.4% incl.)
|
||||
- `MGPipeServerStampVerbBoundary` 1.1 self (63-field loop per verb, PipeInputs.cpp:103-130)
|
||||
- `__aarch64_swp2_rel`/`cas2_acq`/`ldadd8` ~2.5 (outlined LSE atomics; leave alone)
|
||||
|
||||
### Client thread (Thread-17), self % / inclusive %
|
||||
- `SessionProducer::WaitForApplied` 27.9 self / 34.2 incl (the R-1 barrier spin; `EmitAndWaitTails` 36.3 incl)
|
||||
- persistent-map push: `PushDrawConsumers` 1.96 self / 22.5 incl; `PushBlocksForChecked` 5.57 self / 21.5 incl;
|
||||
`XXH3_hashLong_64b_default` 6.51 + `XXH3_64bits` 2.02 (callers: `PushBlocksForChecked` via PushDrawConsumers 98%);
|
||||
`BufferObject::MappedData()` 1.16 self / 3.86 incl; `IsMapped` 0.58; `GetMappedRange` 0.43; `PushIsArmed` 0.47
|
||||
- `MGPipeValidateForVerb` 4.11 self / 10.9 incl (65% called from `DrawElementsInstancedBaseVertex_Backend`);
|
||||
inside it `MGPipeTracker::Update` 1.46, `MGPipeFillAccess::CopyField` 0.64 self / 2.54 incl
|
||||
(45% of `BufferState::GetBindingSlot` 1.73 comes from CopyField in the draw fill)
|
||||
- clock: `steady_clock::now()` 2.34 self / 5.91 incl, `__kernel_clock_gettime` 2.37, `clock_gettime` 1.94
|
||||
-> ~6.6% of the thread is reading the clock. `Doorbell::Wait` (Doorbell.h:123-178) reads the clock at
|
||||
least twice per wait (start + spinEnd) and once per 64 spins; ~900+ waits/frame at 64 fps.
|
||||
- `ServerLoop::OnApplyThread()` 2.66 self / 3.64 incl on the CLIENT thread (role guards on hot paths:
|
||||
`RefuseLegacyBufferArmFromApplyThread` in `BufferObject::MappedData/IsMapped/...`,
|
||||
`PersistentMapTracker::OnServerRole`, `RefusePipeInputsTouchWhileApplierOwnsIt`, `RunsAsTheServerRole`,
|
||||
`MGPipeRefuseAllocatorFromApplyThread`). `OnApplyThread` = `ServerLoopInstance()` (function-static guard)
|
||||
+ `m_running` atomic + `m_applyThreadId` atomic<std::thread::id> + `std::this_thread::get_id()` +
|
||||
thread::id compare (pthread_equal call). `pthread_self` shows 0.56 self.
|
||||
- `MarkWritableImageBufferTextures` 2.15 self / 3.59 incl (GpuWritePending.cpp:85-101: sweeps all
|
||||
`MAX_TEXTURE_IMAGE_UNITS` image units per draw; `GetImageTextureBinding` 1.37)
|
||||
- `__emutls_get_address` 0.46 client; on MONOLITH's GL thread it is 7.7% (top symbol!) with callers
|
||||
`MGPipeFrontendKeyedRegistryScope` ctor 18.75% / dtor 13.95%, `MGPipeReverseAnnouncementScope` ctor/dtor 8.4%
|
||||
(SlotAllocator.cpp:41-75: `thread_local Uint32` depth counters; the scope is constructed at ~20 sites in
|
||||
DirectGLES.cpp, e.g. inside `StateBackendObjectRegistry::HandleOf`). On the apply thread emutls is 7.4%.
|
||||
|
||||
### The shape of the frame (why the client-side cuts matter)
|
||||
Lockstep (R-1): every kCtxVerb (draw/clear/blit/dispatch/readback) and every reply-slot row waits for
|
||||
`appliedSeq == seq`. Frame time ~= client work + server apply time (waited for) + handoff latency.
|
||||
Nothing in this round removes the barrier (that is P11's versioned inputs / P3b-P4b twin tables). This
|
||||
round removes overhead on BOTH threads: every cycle cut on the apply thread's real work shortens the
|
||||
client's wait one-for-one; every cycle cut on the client thread shortens the frame directly.
|
||||
|
||||
## Packages (one agent each, independent, non-overlapping files where possible)
|
||||
|
||||
### T - transport: apply loop idle poll, wait spin, park counters
|
||||
Files: `MobileGL/MG_Remote/Server/ServerLoop.{h,cpp}`, `MobileGL/MG_Remote/Transport/Doorbell.{h,cpp}`,
|
||||
`MobileGL/MG_Remote/Transport/Ring.cpp`, `SessionRings.h`, `MobileGL/MG_Remote/Client/ClientSession.cpp`,
|
||||
`MobileGL/MG_Util/PipeStats*` (find the "MGPipe stats:" line producer).
|
||||
1. `ControlIsPending()` and `PumpControlRequest()` must not take `m_controlMutex` when nothing is posted:
|
||||
add `std::atomic<Bool> m_controlPosted` (set release under the lock by the poster in
|
||||
`RunOnApplyThread`, cleared by the pump under the lock when it takes the request). The mailbox
|
||||
handshake semantics (C2 critical section, NOT_INITIALIZED answer at exit, kMaxControlFramesPerPump)
|
||||
must stay exactly as documented in the file. The `ready` lambda reads only atomics.
|
||||
2. `DrainRing()` on an empty ring must be a couple of loads, no lock, no clock. Check.
|
||||
3. `Doorbell::Wait`: the spin phase must not read the clock at all in the common case. Replace
|
||||
"spin until steady_clock passes spinEnd, clock read every 64 iterations" with a calibrated iteration
|
||||
budget: once per process (or per Doorbell), measure how many `ready()`+`CpuRelax()` iterations fit in
|
||||
`spinUs` (or simply how many CpuRelax fit in 1 us) and spin that many iterations clock-free; only the
|
||||
park phase (rare) computes deadlines. Keep `timeoutMs`/`kWaitForever` semantics, the parked-flag fence
|
||||
protocol and the Dead() handling byte-for-byte in meaning. Document the overshoot bound.
|
||||
`WaitForApplied`'s common path (server not yet there) currently pays 2+ clock reads per wait; after
|
||||
this it pays zero unless it parks.
|
||||
4. Counters for the measurement: server-side `m_parks` already exists (`ParkCount()`); add a client-side
|
||||
park counter (SessionProducer::Park -> count parks that reach `Doorbell::Park`) and a wait counter,
|
||||
and print both (server parks, client parks, waits) in the periodic `MGPipe stats:` line (find how the
|
||||
line is assembled; keep it one line; add a `wait[...]` group). If the stats line producer lives in a
|
||||
place that cannot see MG_Remote symbols, expose the counters through a small accessor registered by
|
||||
the client session (no new include of MG_Remote into MG_Util if that violates the include-closure
|
||||
gate `scripts/check_include_closure.py` - read its rules first).
|
||||
Tests: existing `MG_Test/Wire/ServerLoopTest.cpp`, `DoorbellTest`/`RingTest` must keep passing by
|
||||
inspection; add one small test for "a posted control request is visible to ControlIsPending without
|
||||
the pump" and one for the calibrated spin (e.g. Wait returns true once ready flips within the spin
|
||||
budget, and parks otherwise) if the existing fixtures make that cheap. Do not build.
|
||||
|
||||
### D - role guards and TLS: make every hot-path guard nearly free
|
||||
Files: `MobileGL/MG_Remote/Server/ServerLoop.{h,cpp}` (OnApplyThread only - coordinate: package T also
|
||||
edits ServerLoop.cpp; keep your hunk confined to `OnApplyThread`, the thread-identity member(s) and
|
||||
`ApplyThreadMain`'s first lines where the identity is stored), `MobileGL/MG_Impl/Pipe/SlotAllocator.cpp`
|
||||
(+ its header), `MobileGL/MG_State/GLState/BufferState/BufferObject.cpp` (RefuseLegacyBufferArmFromApplyThread),
|
||||
`MobileGL/MG_Remote/Client/PersistentMapTracker.cpp` (OnServerRole / IsLivePersistentMap guard order),
|
||||
`MobileGL/MG_Remote/Client/ClientSession.cpp` (RefusePipeInputsTouchWhileApplierOwnsIt: order the
|
||||
early-outs cheapest-first: Transport==Monolith, BatchWaits!=0, then the rest),
|
||||
`MobileGL/MG_Remote/Client/WireTables.cpp` (RunsAsTheServerRole), `MobileGL/MG_State/GLState/TextureState/MipmapStorage.cpp`.
|
||||
1. `ServerLoop::OnApplyThread()`: store the apply thread's identity as a plain integer key readable
|
||||
with a relaxed atomic load and compare it against a register-cheap identity of the calling thread:
|
||||
on aarch64 and x86_64 with clang/gcc use `__builtin_thread_pointer()` (verify availability per
|
||||
target with `__has_builtin`; NDK clang 18/19 supports aarch64; check x86_64 support and fall back to
|
||||
`pthread_self()` (POSIX) or `std::this_thread::get_id()` (MSVC) where the builtin is unavailable).
|
||||
Keep the semantics: false when no apply thread is running (m_running false / identity 0), true only
|
||||
on the running apply thread. The function-static `ServerLoopInstance()` guard on every call must go
|
||||
(a namespace-scope atomic key is enough; the ServerLoop object still owns it logically).
|
||||
2. `MGPipeFrontendKeyedRegistryScope` / `MGPipeReverseAnnouncementScope` (SlotAllocator.cpp:39-75):
|
||||
the depth counters are only ever READ by `MGPipeRefuseAllocatorFromApplyThread`, which returns early
|
||||
unless `OnApplyThread()`. So the counters only matter on the apply thread. Replace the `thread_local`
|
||||
with a plain (non-TLS) counter incremented/decremented only when `OnApplyThread()` is true (cheap after
|
||||
item 1); on the GL thread the ctor/dtor become one predicate call. Prove in a comment why no data race
|
||||
exists (only the apply thread writes and reads; monolith never reads). Keep `Active()` semantics on the
|
||||
apply thread exact. If you find a reader other than the apply-thread guard, stop and report instead of
|
||||
changing semantics.
|
||||
3. `RefuseLegacyBufferArmFromApplyThread` (BufferObject.cpp:42-52) and `PersistentMapTracker::OnServerRole`,
|
||||
`IsLivePersistentMap`: with item 1 these are ~free; make sure `PushIsArmed()` (a config read) is not
|
||||
itself expensive (it reads `MG_Config::Transport` - fine) and that no guard does more than one
|
||||
OnApplyThread call per accessor.
|
||||
4. Look for any other per-draw `thread_local` reads on the client and apply paths (`t_activeDecoder` in
|
||||
PipeWireCodec.cpp:359, `g_reply` in PipeRoute.cpp:61, `g_inBarrierWait` in ClientSession.cpp:117,
|
||||
`g_textureLegacyArmScopeDepth` MipmapStorage.cpp:73, `tl_initializing`) and report which are on the
|
||||
hot path; fix the cheap ones the same way ONLY if the single-writer argument holds; otherwise report.
|
||||
Monolith equivalence: the pull build (`MOBILEGL_BUILD_DISAGGREGATED=OFF`) must compile and its behaviour
|
||||
must be identical; G1 (pull-build symbol/.text identity) changes must be named in your report.
|
||||
|
||||
### B - persistent-map push: no edge hashing, page-granular shadows
|
||||
Files: `MobileGL/MG_State/GLState/BufferState/PipeResource.h` (the aligned allocator), the shadow
|
||||
container in `BufferObject.{h,cpp}` / `PipeResource.*`, `MobileGL/MG_Remote/Client/PersistentMapTracker.{h,cpp}`
|
||||
(TrackWriteMap ~259-330, the SIGSEGV handler ~200-240, PushBlocksForChecked 492-685, PushDrawConsumers 704-818).
|
||||
Today: the shadow allocator gives 4096-byte ALIGNMENT but not a page-multiple SIZE, so the last page of
|
||||
a shadow can hold foreign bytes; TrackWriteMap therefore aligns the protected interior INWARD and the
|
||||
partial-page edges (head edge when the mapped offset is not page-aligned; tail edge for every buffer
|
||||
whose mapped end is not page-aligned - i.e. nearly every buffer) are hashed with XXH3 on EVERY draw for
|
||||
every live map with edges (the epoch-skip fast path pushes edges of ALL live maps, not just consumers).
|
||||
That is 8.5% of the client thread in XXH3 plus the MappedData/IsMapped/GetMappedRange guards around it.
|
||||
Goal: zero per-draw hashing in steady state.
|
||||
1. Make every shadow allocation page-granular: round the allocation SIZE up to a multiple of
|
||||
SHADOW_ALLOCATION_ALIGNMENT in the allocator (verify the container: what `Bytes()` returns, whether
|
||||
capacity vs size matters, whether any shadow can be backed by something else - adopted GPU stores are
|
||||
excluded from tracking by IsLivePersistentMap; confirm). Then every byte of
|
||||
`[base, base + RoundUp(size, 4096))` belongs to this shadow and no other allocation.
|
||||
2. TrackWriteMap: when the shadow base is page-aligned and the allocation is known page-granular, align
|
||||
the protected range OUTWARD to page boundaries (clamped to the shadow's own page-rounded extent), so
|
||||
there are no edges and `hasEdges` is false; sub-page maps become one protected page. Keep the inward
|
||||
arm as the fallback for any shadow that fails the page-granular test (and keep the hash arm for
|
||||
untracked maps). The SIGSEGV handler's ownership tests (SEGV_ACCERR, inside a live tracked range,
|
||||
bitmap-bit replay discrimination) must hold for the widened range: bytes outside the mapped range but
|
||||
inside the shadow are written by the client's own API paths (SubData/FlushMappedRange memcpy into the
|
||||
shadow) - a fault there is answered like any interior fault; make sure the push clamps to [begin,end)
|
||||
as today and that nothing else (e.g. readback writeback into the shadow on the client thread,
|
||||
`Forget`, resize/respecify that reallocates the shadow, `NoteMapStateChanged`) breaks the
|
||||
"protected pages are re-armed / released" invariant. Re-read the round-1 commit message of cb06538c
|
||||
(git show cb06538c) for the reasons behind inward alignment before you widen anything.
|
||||
3. The epoch-skip fast path (PushDrawConsumers 727-737) then does nothing per member; keep the loop or
|
||||
drop it - measure nothing, reason. First push still ships everything once.
|
||||
4. Report: a table of the per-draw work that remains in PushDrawConsumers when the epoch has not moved.
|
||||
Tests: `MG_Test/Buffer/SplitBufferTest.cpp` and `MG_IntegrationTest/Scenarios/PersistentCoherentMapScenario.cpp`
|
||||
exercise this; add/adjust a unit case for the outward alignment (a sub-page map gets a protected page;
|
||||
an unaligned base falls back to inward+edges). Do not build.
|
||||
Danger notes from cb06538c: a signal-blocked driver/runtime worker thread faulting on a foreign byte
|
||||
inside a protected page kills the process (that was the ctest SmallRing segfault) - the whole point of
|
||||
item 1 is that no foreign byte can be inside a protected page any more; state that argument precisely.
|
||||
|
||||
### C - client verb overhead: validate/fill, image-unit sweep, server stamp loop
|
||||
Files: `MobileGL/MG_Impl/Pipe/PipeFill.cpp` (MGPipeValidateForVerb 2781+, the residual fill / CopyField),
|
||||
`MobileGL/MG_Pipe/PipeInputs*`/`MobileGL/MG_Backend/MGPipe/PipeInputs.cpp` (MGPipeServerStampVerbBoundary),
|
||||
`MobileGL/MG_Remote/Client/GpuWritePending.cpp`, `MobileGL/MG_Remote/Client/EmitTables.cpp` (draw emit path),
|
||||
`MobileGL/MG_State/GLState/TextureState/*` only if an image-unit high-water mark is needed.
|
||||
1. `MarkWritableImageBufferTextures`: replace the 32-unit sweep per draw with a high-water mark
|
||||
(the GLContext already keeps touched-unit marks for texture units and buffer binding points; add the
|
||||
same for image units at the BindImageTexture site if none exists - note this is MG_State (shared with
|
||||
monolith) so keep it minimal and name the G1 impact; or, if `ImageEmit`/the tracker already knows
|
||||
whether any writable image-buffer unit is bound, use that latch).
|
||||
2. `MGPipeValidateForVerb` draw path: find what the 10.9% inclusive is made of beyond tracker.Update
|
||||
(1.5%) and CopyField (2.5%): the residual fill's per-verb copies (which fields does the draw class
|
||||
copy, and which of those are already record-supplied under a live wire - P5c rv skips the value-class
|
||||
rows; check the object-class rows' copy cost, e.g. `GetBindingSlot` from CopyField), the
|
||||
`RefusePipeInputsTouchWhileApplierOwnsIt` guard order (package D reorders it; don't duplicate),
|
||||
`ContextValuesWireLive()` / `RunsAsTheServerRole()` per verb (cache per session if legal),
|
||||
`ParsePoisonOmissionKnob()` per verb (should be once). Cut what is provably redundant; leave the
|
||||
emitters' semantics alone.
|
||||
3. `MGPipeServerStampVerbBoundary`: the 63-field loop per verb on the apply thread computes
|
||||
`answerable(field, verb)` from constant tables - precompute per verb once (a static table of
|
||||
which fields are answerable per verb, built lazily or constexpr) so the per-verb work is a
|
||||
table-driven fill. Keep the withdrawal semantics exactly (FilledGen = serial or 0).
|
||||
4. Anything else on the draw emit path in EmitTables.cpp that is per-draw and redundant (e.g. repeated
|
||||
session lookups, ReadDrawBindings re-reading slots the fill already read). Report what you found and
|
||||
what you left.
|
||||
Monolith: PipeFill.cpp is compiled into the monolith push build too; keep monolith behaviour identical
|
||||
(G1 for the pull build; push-build equivalence by inspection).
|
||||
|
||||
### E - research only (no code): can the draw barrier be deferred by one verb?
|
||||
Deliverable: a design note, not a patch. Question: the client today fills the residual object-class
|
||||
rows of `gPipeInputs` (FieldOwnership.def BARRIER_PULLED: GetBoundVertexArray, GetBufferBindingSlot,
|
||||
GetBufferBindingPoint, GetFramebufferBindingSlot, GetImageTextureBinding, GetTextureUnitObject,
|
||||
GetProgramForDraw, GetProgramForDispatch, GetTransformFeedbackProgram, GetBufferBindingPointCount,
|
||||
GetProgramObject, GetTextureObject, HasOpenTransformFeedbackSpan, ValidateProgramName, RecordError),
|
||||
publishes the draw, and WAITS. Map, for the draw/clear/blit verbs on DirectGLES under inproc:
|
||||
(a) every server-side read that reaches client-owned memory through those rows (what does the server
|
||||
dereference through each pointer: VAO attribute array? texture unit binding slots? UBO binding point
|
||||
table entries + ranges? program object internals?), including the two exemption scopes
|
||||
(`MGPipeFrontendKeyedRegistryScope` sites in DirectGLES.cpp, `MGPipeReverseAnnouncementScope`);
|
||||
(b) which of those reads would be UNSAFE if the client continued past the publish and executed the
|
||||
next draw's GL calls (glBindBufferRange / glBindTexture / glUseProgram / glVertexAttribPointer / ...)
|
||||
before the server applied this draw;
|
||||
(c) the minimal "snapshot at fill" scheme: what would have to be copied per draw into a per-verb
|
||||
snapshot the server reads instead (count the bytes/refcount ops per draw at 852 draws/frame), and
|
||||
where the client would then wait ("before the next fill" instead of "after this publish"), i.e.
|
||||
one-verb pipelining that overlaps the client's inter-draw work with the server's apply;
|
||||
(d) what the existing pushed records already cover (SetSamplerViews / VertexElements CSO /
|
||||
set_vertex_buffers / set_context_values) such that the server COULD stop reading the live tables;
|
||||
(e) a cost/benefit estimate and a recommendation: P5d round 4, or leave to P11 / P3b-P4b.
|
||||
Write it to the report path given below (markdown, <= 400 lines, with file:line citations at head 85362a85).
|
||||
|
||||
## Rules for every package
|
||||
- Do NOT build (no gradle, no cmake, no ndk), do NOT touch adb/devices, do NOT push, do NOT commit
|
||||
in any shared tree. Work only in your isolated worktree; deliver `git diff HEAD -- MobileGL` as a
|
||||
patch FILE at the path given below, plus a short report file. Both must be written before your
|
||||
final message.
|
||||
- No `Co-Authored-By`, no attribution lines anywhere.
|
||||
- No hot-path instrumentation left in (counters that print per frame are fine when behind the
|
||||
existing `MG_Util::PipeStats::Enabled()` / stats period; no per-draw logging).
|
||||
- Prefer changes inside `#if MOBILEGL_BUILD_DISAGGREGATED` or in MG_Remote (split-only). When a change
|
||||
touches code compiled into the pull build (MOBILEGL_BUILD_DISAGGREGATED=OFF, MOBILEGL_PIPE_PUSH=OFF),
|
||||
say so in the report (G1 names every symbol/.text delta) and keep behaviour identical.
|
||||
- Every correctness-bearing change gets a red-once argument: name the existing test (or the one you
|
||||
add) that goes red if the change is reverted or broken. Keep tests light; do not add a test for a
|
||||
refactor with no semantic change.
|
||||
- Keep the file's own comment discipline: explain WHY in the code comment, in the voice of the
|
||||
surrounding code, and do not delete the reasoning comments you replace - update them.
|
||||
- Report format (markdown): what changed (per file), why (tie to the profile numbers above), the
|
||||
red-once test, G1/monolith impact, risks and what you did not do. <= 120 lines.
|
||||
@@ -0,0 +1,66 @@
|
||||
# P5d round 3 - device results log (Redmi 2f7cbe2e, FCL + MC 26.3-rc-3 world "test", VD12, DirectGLES)
|
||||
|
||||
Protocol: CPU pinned big 1958400 / little 1555200 (pin_device.sh), fan level 2 (~14.2k rpm), screen on,
|
||||
cpuss-0-0 50-54 C during runs. GPU: the vendor holds thermal_pwrlevel=1 and resets min_pwrlevel to 12 at
|
||||
every app start, so the GPU is GOVERNED, not pinned (gpuclk 342 MHz in every inproc arm = GPU idle,
|
||||
832 MHz in monolith). All inproc arms are CPU-bound on the client thread; monolith may be slightly GPU-
|
||||
limited. fps = frames/second from the `MGPipe stats:` lines over a 30 s window after 20 s settle (world
|
||||
loads in ~15 s on this device); ms/frame = per-thread utime+stime over the window / frames.
|
||||
Thread affinity: `sched_setaffinity`/`taskset` are accepted and silently ignored by this kernel for
|
||||
app threads (RESOLVED mask 0xff; taskset "new affinity mask: ff"), so core placement is the scheduler's.
|
||||
|
||||
## Session base0 (unpatched head 85362a85 lib, APK with mg_env.txt plumbing) - 2026-09-18 03:18-03:26
|
||||
|
||||
| arm | env | fps mean / p50 / min | client ms/frame (core) | apply ms/frame (core) |
|
||||
|---|---|---|---|---|
|
||||
| base0-inproc | - | 64.8 / 64.3 / 60.9 | 15.03 (cpu5, mid) | 12.42 (cpu7, big) |
|
||||
| base0-mono | monolith | 129.2 / 116.5 / 112.0 | 5.85 (cpu7, big) | - |
|
||||
| spin2000 | MOBILEGL_IPC_SPIN_US=2000 | 72.7 / 72.6 / 68.5 | 14.00 (cpu2) | 13.64 (cpu7) |
|
||||
| nobarrier | MOBILEGL_IPC_VERB_BARRIER=0 | aborted: Fatal{UnmigratedPipeInput, "GetFramebufferBindingSlot@FenceSync"} at the first race - the negative control cannot run a real workload | | |
|
||||
| affoff | MOBILEGL_IPC_SERVER_AFFINITY=off | 65.9 / 65.9 / 57.7 | 14.20 (cpu5) | 12.21 (cpu7) |
|
||||
|
||||
## Session place1 (same lib) - 03:27-03:35
|
||||
|
||||
| arm | env | fps mean / p50 / min | client ms/frame (core) | apply ms/frame (core) |
|
||||
|---|---|---|---|---|
|
||||
| bigclient | taskset c0 on GL thread (ignored) | 64.9 / 65.0 / 57.4 | 15.05 (cpu4) | 12.75 (cpu7) |
|
||||
| bigclient-spin2000 | SPIN_US=2000 (+ignored taskset) | 69.7 / 68.9 / 65.0 | 13.94 (cpu4) | n/a |
|
||||
| bigboth-spin2000 | SPIN_US=2000 (+ignored taskset) | 68.3 / 68.3 / 65.2 | 14.62 (cpu4) | 14.31 (cpu7) |
|
||||
| spin10000 | SPIN_US=10000 | 62.5 / 63.2 / 59.1 | 15.49 (cpu2) | 15.41 (cpu6) |
|
||||
|
||||
Readings:
|
||||
- Split/monolith p50 ratio at the head: 64.3 / 116.5 = 1:1.81 (report said ~1:1.7).
|
||||
- The split's client thread is the critical path: ~15 ms CPU per 15.5 ms frame (97% busy, spin-waits
|
||||
included), and the scheduler puts it on a MID core (cpu2/4/5 at 1555 MHz) while the always-spinning
|
||||
apply thread holds a big core; monolith's render thread sits on cpu7 (big).
|
||||
- Spin 2000 us: +4..+8 fps (3 arms 68-73 vs 64-66) - park/unpark latency is a real, second-order cost.
|
||||
Spin 10000 us: worse (63) - both threads then burn full cores and the client lands on cpu2.
|
||||
- Server affinity knob: no effect (it never took on this kernel).
|
||||
|
||||
## Session r3a (round-3 PRE-review build, patches T/D/B/C as first written) - 03:42-03:49
|
||||
|
||||
| arm | fps mean / p50 / min | client ms/frame (core) | apply ms/frame (core) | wait[] (run totals at window end) |
|
||||
|---|---|---|---|---|
|
||||
| r3-inproc | 112.5 / 114.0 / 88.8 | 8.68 (cpu4) | 6.93 (cpu7) | srv=7.56M srvpark=76k cli=6.70M clipark=17k |
|
||||
| r3-spin2000 | 113.0 / 112.3 / 106.1 | 8.88 (cpu5) | 8.88 (cpu7) | srvpark=1.6k clipark=286 |
|
||||
| r3-mono | 115.4 / 115.7 / 108.4 | 5.70 (cpu5) | - | capped |
|
||||
| r3-spin1000 | 107.6 / 108.5 / 98.0 | 9.27 (cpu5) | 8.63 (cpu7) | |
|
||||
|
||||
## Session r3b/r3c/r3d (round-3 POST-review build = commit 1f8de61b) - 04:04-04:32
|
||||
|
||||
| arm | fps mean / p50 / min / max | client ms/frame (core) | apply ms/frame (core) | note |
|
||||
|---|---|---|---|---|
|
||||
| r3b-inproc | (false "app died": one pidof hiccup; fixed in bench_fcl3+) | | | 14 stats lines, 855 draws/f, no Fatal |
|
||||
| r3b-mono | 246.9 / 266.1 / 123 / 298 | 3.9 (cpu7, 97% busy) | - | UNCAPPED run - vsync not enforced this time |
|
||||
| r3b-inproc2 | 107.4 / 106.7 / 98 / 120 | 9.3 (cpu5) | 7.1 (cpu7) | |
|
||||
| r3b-spin2000 | 109.1 / 109.6 / 102 / 113 | 9.1 (cpu4) | 8.7 (cpu7) | presented (SF timestats) 113 fps |
|
||||
| r3c-mono1 | 166.1 / 205.8 / 110 / 280 | 5.0 (cpu7) | - | uncapped, swinging 110-280 inside the window |
|
||||
| r3c-inproc1 | 103.8 / 106.1 / 91 / 112 | 9.24 (cpu4) | 7.0 (cpu7) | |
|
||||
| r3c-mono2 | 114.6 / 115.3 / 108 / 117 | 6.2 (cpu7) | - | capped at 120 Hz |
|
||||
| r3c-inproc2 | 101.6 / 103.3 / 82 / 111 | 9.2 (cpu5) | 7.4 (cpu7) | |
|
||||
| r3c-inproc3 (csbig spec ignored by v3 driver) | 100.9 / 101.9 / 83 / 109 | 9.1 (cpu5) | 7.3 (cpu7) | |
|
||||
| r3c-mono3 | 114.9 / 115.1 / 109 / 117 | 6.1 (cpu7) | - | capped |
|
||||
| r3d-inproc-csbig (cpuset 6-7 written for the GL tid) | 112.5 / 115.3 / 92 / 122 | 8.63 (cpu5) | 6.98 (cpu7) | thread still on cpu5: cpuset ignored too |
|
||||
|
||||
pmap wire bytes identical pre/post review (364.7 KB/frame); post-review inproc 103-107 vs pre-review 114 is inside run-to-run drift (body 50.8 vs 53-54 C at window start, mid-core placement).
|
||||
Reading: monolith fps is bimodal on this device (115 = 120 Hz vsync cap; 206-266 = uncapped runs); per-thread CPU ms/frame is the comparable metric. Client thread 15.0 -> 8.7-9.2 ms/frame (mid core), apply 12.4 -> 7.0-7.4 (mostly spin), monolith render thread 5.0-6.2 (big core).
|
||||
@@ -0,0 +1,188 @@
|
||||
# P5d round 3, package B: persistent-map push - page-granular shadows, no edge hashing
|
||||
|
||||
Worktree `MobileGL-p5d-B` (branch `p5d-B`, base `85362a85`). Patch: `patches/B.patch` (5 files,
|
||||
+416/-91). Not built, not committed.
|
||||
|
||||
## What changed (per file)
|
||||
|
||||
**`MG_State/GLState/BufferState/PipeResource.h`**
|
||||
- `ShadowAllocationBytesFor(count)` (split-only): rounds an allocation SIZE up to a multiple of
|
||||
`SHADOW_ALLOCATION_ALIGNMENT` (4096). `MapAlignedAllocator::allocate` asks the heap for that many
|
||||
bytes under `#if MOBILEGL_BUILD_DISAGGREGATED`; the pull-build arm of `allocate` is the old text,
|
||||
token for token. Round 2 gave shadows a page-aligned BASE only; the last page could still end
|
||||
mid-way with a neighbour allocation in the remainder.
|
||||
- `PipeResource::ShadowAllocationBytes()` (split-only): `ShadowAllocationBytesFor(m_shadow->capacity())`,
|
||||
0 for an adopted store or an unsized shadow. `capacity()` is exactly the count `std::vector` passed to
|
||||
`allocate()` (libc++/libstdc++/MSVC all record it), and the rounding is the allocator's own function,
|
||||
so the extent the tracker widens into can never exceed what was allocated.
|
||||
- Confirmed: a live member's `MappedData()` is always `m_shadow->data()` - `Bytes()` returns the GPU map
|
||||
only when `m_gpuMapped != nullptr`, and `IsLivePersistentMap`'s second row (`IsBackendPersistentMapped`)
|
||||
excludes exactly that; nothing else backs a shadow. `ResizeShadow` is reserve(bit_ceil)+resize, so the
|
||||
allocation is `capacity()` bytes and `[0, size) ⊆ [0, capacity)`; the shadow is never reallocated while a
|
||||
map is live (Respecify/AllocateImmutableStorage call `ReleaseMemory(false)` first, which untracks;
|
||||
EnsureGpuResidentStorage/TryAdoptLargeStorage refuse while mapped).
|
||||
|
||||
**`MG_State/GLState/BufferState/BufferObject.h`**: `ShadowAllocationBytes()` accessor inside the existing
|
||||
`#if MOBILEGL_BUILD_DISAGGREGATED` block (pull build: no text, no layout change).
|
||||
|
||||
**`MG_Remote/Client/PersistentMapTracker.cpp`**
|
||||
- `TrackWriteMap(..., shadowExtent)`: **outward** alignment when the shadow is page-granular (base
|
||||
page-aligned, extent != 0, extent a page multiple, `rangeEnd <= extent`): `base = RoundDown(begin)`,
|
||||
`end = RoundUp(end)` clamped to `shadow + extent`; `hasEdges = base > rawBegin || end < rawEnd` is then
|
||||
false and a sub-page map is one protected page. Anything failing the test keeps the **inward** arm
|
||||
(fallback, comment rewritten to state both ownership arguments; the round-1 reasoning is kept, not
|
||||
deleted). `static_assert(SHADOW_ALLOCATION_ALIGNMENT % kPageBytes == 0)` pins the precondition.
|
||||
- `InstallWriteFaultHandler`: refuses (once, `g_pageSizeRefused`) when `sysconf(_SC_PAGESIZE) != 4096`.
|
||||
Every ownership claim is made at 4 KB; on a 16 KB-page kernel (Android 15 admits them) a 4 KB
|
||||
`mprotect` covers three foreign quarters - the process-killer. Pre-existing hazard, made deterministic:
|
||||
hash arm on such kernels, never an unsafe protect.
|
||||
- `PushBlocksForChecked`: both XXH3 edge hashes are gated on `tracked->hasEdges`; the `interiorBegin/End`
|
||||
comments no longer claim `>= begin / <= end`; the page loop's clamp to `[begin, end)` is unchanged and is
|
||||
what keeps a head/tail page that starts before `begin` or runs past `end` inside the mapped range.
|
||||
- `PushDrawConsumers` epoch skip: walks the new `m_edgedMembers` set (members whose slot has unprotected
|
||||
edges - inward fallback only) instead of the whole membership; empty in steady state. Keys copied out
|
||||
before the walk because `PushBlocksForChecked` may `Forget()`.
|
||||
- `NoteMapStateChanged` passes `buffer.ShadowAllocationBytes()` and maintains `m_edgedMembers` beside
|
||||
`m_untrackedMembers` (erase on re-note, Forget, ClearForTest).
|
||||
- Test hooks: `MprotectArmAvailableForTest`, `TrackedSlotForTest`, `TrackForTest`, `UntrackForTest`,
|
||||
`FaultEpochForTest`.
|
||||
- SIGSEGV handler: logic untouched. Comment on the every-overlapping-slot loop updated (no two tracked
|
||||
ranges can share a page any more; loop kept as the checkable form).
|
||||
|
||||
**`MG_Remote/Client/PersistentMapTracker.h`**: `TrackedWriteMap` edge-field comments (edges exist only on
|
||||
the inward fallback), `m_edgedMembers`, the five test hooks.
|
||||
|
||||
**`MG_Test/Buffer/SplitBufferTest.cpp`**: two cases + two skip stubs (G2/G14 name-set rule, "eighteen").
|
||||
|
||||
## Why (profile)
|
||||
Client thread at 56a77348: `XXH3_hashLong_64b_default` 6.51 + `XXH3_64bits` 2.02 = 8.5%, 98% via
|
||||
`PushBlocksForChecked` from `PushDrawConsumers`; plus `PushBlocksForChecked` 5.57 self, `MappedData` 1.16,
|
||||
`IsMapped` 0.58, `GetMappedRange` 0.43 around it. Cause: nearly every mapped end is not page-aligned, so
|
||||
nearly every tracked map had `hasEdges` and the epoch-skip fast path hashed its edges for EVERY live map
|
||||
on EVERY draw. With page-granular shadows there are no edges, the fast path touches no member, and steady
|
||||
state has zero per-draw hashing and zero `MappedData/IsMapped/GetMappedRange` calls from this module.
|
||||
|
||||
## The ownership argument, precisely (cb06538c's danger note)
|
||||
A page may be protected only if every byte of it belongs to this shadow. Outward: the heap handed the
|
||||
shadow out as ONE block of `extent` bytes at a 4096-aligned base (`operator new(extent, align_val_t{4096})`,
|
||||
extent a multiple of 4096); allocator metadata lives before the block, never inside; therefore
|
||||
`[shadow, shadow + extent)` contains no byte of any other allocation, and every page intersecting
|
||||
`[shadow+begin, shadow+end)` with `end <= extent` lies inside it. A signal-blocked driver/runtime worker
|
||||
never writes a client shadow (rule E: the server reads its own staged copy). Bytes of a head/tail page
|
||||
outside the mapped range are written only by the client's API paths (SubData/FlushMappedRange/Respecify
|
||||
memcpy, the readback writeback landing in `WaitForApplied`'s event drain on the client thread) - threads
|
||||
that run the handler; such a fault is answered like an interior one, the push clamps to `[begin, end)`,
|
||||
and the byte itself crossed by its own record. `Forget`/unmap/re-note all route through `UntrackWriteMap`
|
||||
(restore writable, then retire) before any reallocation, unchanged.
|
||||
|
||||
## Per-draw work remaining in PushDrawConsumers, epoch unmoved
|
||||
| step | cost |
|
||||
|---|---|
|
||||
| `PersistentMapTracker::Instance()` (3 call sites in EmitTables) | function-static guard load |
|
||||
| `PushIsArmed()` | one `MG_Config::Transport` load + compare |
|
||||
| `OnServerRole()` | `ServerLoop::OnApplyThread()` (package D makes it a relaxed load + compare) |
|
||||
| `m_livePersistentMaps.empty()` | one load |
|
||||
| `g_faultEpoch.load(acquire)` + compare with `m_lastFaultEpoch` | one atomic load |
|
||||
| `m_untrackedMembers.empty()`, `m_edgedMembers.empty()` | two loads |
|
||||
| per member | nothing |
|
||||
First push after a map still ships everything once (all bits set at registration; first fault moves the
|
||||
epoch, the walk pushes every set bit). The loop was kept (over `m_edgedMembers`, not the membership) because
|
||||
the inward fallback still exists; it is not entered in steady state.
|
||||
|
||||
## Red-once
|
||||
- `SplitBufferSet.ASubPageMapIsOneProtectedPageAndAWriteThroughItFaults`: reverting the outward alignment
|
||||
gives `pageCount == 0`, `hasEdges == true`, and a write through the pointer that does not move the fault
|
||||
epoch - three red assertions. Reverting the allocator rounding alone: `ShadowAllocationBytes() % 4096`
|
||||
fails for a 100-byte buffer (capacity 128).
|
||||
- `SplitBufferSet.AShadowThatFailsThePageGranularTestKeepsTheInwardAlignmentAndItsEdges`: an unaligned base,
|
||||
an unknown extent (0) and a range past its extent each register inward (1 page, edges); the true extent
|
||||
widens to 3 pages, no edges, end == extent. Widening without the extent test goes red on the third
|
||||
registration.
|
||||
- Existing: `ThePushCutsTheMappedSpanIntoBlocksAndMovesPmap`, `TheLastBlockIsTheRemainderAndNotAWholeBlock`
|
||||
(64K+7: now one page-rounded span, still 2 blocks / kSize bytes by the clamp), `PersistentCoherentMapScenario`
|
||||
(pixels through the push) keep passing by inspection.
|
||||
|
||||
## G1 / monolith
|
||||
Pull build (`MOBILEGL_BUILD_DISAGGREGATED=OFF`): zero delta. `ShadowAllocationBytesFor`,
|
||||
`PipeResource::ShadowAllocationBytes`, `BufferObject::ShadowAllocationBytes` exist only under the option;
|
||||
`allocate()`'s pull arm is the previous text verbatim. Push monolith build with the option ON but
|
||||
`Transport=Monolith`: allocations of shadows/staging stores round up to 4096 multiples (invisible to the
|
||||
application; capacities >= 4096 are already powers of two, so only sub-4K stores grow), tracker inert as
|
||||
before (`PushIsArmed` false).
|
||||
|
||||
## Risks / not done
|
||||
- Not built (per rules). Watch for: `sysconf`/`unistd.h` already included; `uintptr_t` in the test via
|
||||
`<cstdint>` through the tracker header; `flat_hash_map::value_type::first` for `m_edgedMembers`.
|
||||
- The 16 KB-page refusal is new behaviour on such kernels (hash arm instead of mprotect); on the Redmi
|
||||
(4 KB) it is a no-op. It is the honest answer, but it is a performance cliff there rather than a fix.
|
||||
- Two tracked shadows can no longer share a page; the handler still scans every slot per fault (unchanged).
|
||||
- Not touched: role guards (`RefuseLegacyBufferArmFromApplyThread`, `OnServerRole`) - package D; the hash
|
||||
arm's `MOBILEGL_IPC_PERSISTENT_HASH_SUPPRESS` XXH3 for untracked members - still exists for table-full /
|
||||
handler-unavailable / >256 MB maps; docs (`P5D-INPROC-PERFORMANCE.md`) not updated - integrator's row.
|
||||
|
||||
## Fix round
|
||||
|
||||
Patch regenerated from `MobileGL-p5d-B` (6 files, +654/-149; BufferObject.cpp is new to the set). Not built.
|
||||
|
||||
**Blocker (both reviews): the epoch skip lost the drain the old edge service did by accident.**
|
||||
Agreed, and the diagnosis is exact: `m_lastFaultEpoch = epoch` before a walk that pushed only
|
||||
consumers left a faulted non-consumer marked, writable and unable to move the epoch again. Fix, in
|
||||
`PersistentMapTracker.cpp`:
|
||||
- The walk now DRAINS every tracked member with a marked page, consumer or not, before the binding
|
||||
walk (`AnyPageMarked`, one word load per 64 pages, walk-time only). The skip's proof then holds:
|
||||
the epoch moves only on a mark, and the last walk drained every mark there was. A drained push is
|
||||
bitmap-driven (faulted blocks only), so nothing pays the whole-range rescan the consumer filter
|
||||
exists to avoid. `m_lastFaultEpoch` is still taken before the drain, so a fault landing mid-drain
|
||||
forces the next draw to walk again.
|
||||
- `TrackWriteMap` bumps the epoch after publishing a slot: a fresh slot has every page marked, which
|
||||
is a fault of every page as far as the push is concerned. This closes the second review's
|
||||
"registered and drawn without a fault is never shipped" hole - "first push ships everything once"
|
||||
is now true of the draw path, not only of the read-only verbs' whole push.
|
||||
- The handler's `g_faultEpoch.fetch_add` is now `release` (was relaxed) so the walk's existing
|
||||
`acquire` load actually orders the bit before the epoch across threads (chunk builders write
|
||||
arenas off the GL thread). Cheap: handler-only.
|
||||
- Skip path unchanged in cost: one atomic load, two empties, nothing per member. The per-draw table
|
||||
above still stands. The walk (epoch moved) gained one word-scan per tracked member.
|
||||
- The binding walk moved under `if (ctx != nullptr)` so the drain runs without a context too; the
|
||||
walk body is re-indented, otherwise unchanged.
|
||||
|
||||
**Major (review 1): no red-once for the skip semantic.** Added
|
||||
`SplitBufferSet.AFaultedMapTheDrawDidNotBindStillShipsBeforeTheDrawThatDoes` (+ skip stub; name
|
||||
sets 20/20, "TWENTY"). Two one-page maps A and B, B bound to SSBO point 0, driven through
|
||||
`PushDrawConsumers` itself: draw 1 ships both (2 blocks: fresh A ships although unbound - the
|
||||
registration bump), draw 2 ships nothing, a write through A's pointer faults (asserted via the
|
||||
epoch), draw 3 with only B bound must ship A's block (3), draw 4 binding A ships nothing more (3,
|
||||
`BytesPushed == 3 * 4096`). Reverting the drain: draw 3 stays at 2 and draw 4 skips - red at the
|
||||
"3u" assertion. Reverting the registration bump alone: draw 1 stays at 0 - red at "2u".
|
||||
|
||||
**Minor (review 1): `g_segvInstalled` published before the page-size check.** The check is now
|
||||
`KernelPageIsTheTrackerPage()`, a function-static memo asked BEFORE the CAS; `g_pageSizeRefused`
|
||||
is gone. No caller can pass `TrackWriteMap`'s installed gate while the size is undecided.
|
||||
|
||||
**Minor (review 1): `ShadowAllocationBytes()` unguarded.** Now out of line in `BufferObject.cpp`
|
||||
(under the option) with `RefuseLegacyBufferArmFromApplyThread("ShadowAllocationBytes")`, beside
|
||||
`MappedData`. Pull build: no text (declaration and definition are both under the option).
|
||||
|
||||
**Minor (review 1): outward can exceed `kMaxTrackedPages` by one page where inward fit.** Widening
|
||||
never demotes: if the outward count exceeds the capacity, `TrackWriteMap` takes the inward arm for
|
||||
that map (edges, `m_edgedMembers`) instead of refusing it onto the hash arm, so no map that was
|
||||
tracked before this round becomes an untracked member that vetoes the skip process-wide. Only a
|
||||
~256 MB map on the boundary pays the old edge-hash cost; no test (it would need a 256 MB shadow).
|
||||
|
||||
**Minor (review 2): extent inferred from `capacity()`.** `PipeResource` now records
|
||||
`m_shadowExtent = ShadowAllocationBytesFor(reserved)` in `ResizeShadow` (the count it asked the
|
||||
allocator for) and zeroes it in `AdoptPersistentMap`; `ShadowAllocationBytes()` returns it (0 while
|
||||
GPU-resident). When the reserve did not reallocate, the block behind `Bytes()` is a larger earlier
|
||||
reserve's, so the recorded value is at most the true extent in every case; `size <= reserved`
|
||||
keeps `rangeEnd <= extent`. `Shadow()` has no callers, and every shadow allocation goes through
|
||||
`ResizeShadow`, so nothing else can move the block. Split-only member (G1 layout unchanged).
|
||||
|
||||
**Minor (review 2): the `hasEdges &&` conjuncts were dead.** Dropped both; the comment now says
|
||||
the two geometric conditions are self-gating under outward alignment and that this, not a flag,
|
||||
is what removed the per-draw XXH3. `hasEdges` stays for `m_edgedMembers` (header comment updated).
|
||||
|
||||
**Not changed:** the 16 KB refusal, the outward arithmetic, the allocator rounding, the SIGSEGV
|
||||
handler's ownership tests, `PushAllMembers`. Risks: not built - watch `if (const auto& ctx =
|
||||
MG_State::pGLContext)` (UniquePtr in an if-declaration, same shape as the `vao` line below it),
|
||||
`entry.first` on `UnorderedMap<Uint64, MemberEntry>`, and `MGLOG_W` inside the function-static
|
||||
lambda.
|
||||
@@ -0,0 +1,250 @@
|
||||
# P5d round 3 - package C: client verb overhead (image-unit sweep, validate/fill, server stamp)
|
||||
|
||||
Worktree `MobileGL-p5d-C`, branch `p5d-C`, base `85362a85`. Not built, not committed.
|
||||
Patch: `.../scratchpad/p5d/patches/C.patch` (865 lines, 11 files, after the fix round below).
|
||||
|
||||
## What changed, per file
|
||||
|
||||
### 1. The 32-unit image sweep per draw (brief item 1)
|
||||
|
||||
- `MG_State/GLState/TextureState/TextureState.h` - new `NoteImageUnitTouched(Int)` /
|
||||
`GetMaxTouchedImageUnit()` and an `Int m_maxTouchedImageUnit = -1` member, all inside
|
||||
`#if MOBILEGL_PIPE_PUSH` (the private member joins the two push-only generations already there).
|
||||
A **second** mark, not a widening of `m_maxTouchedUnit` (image units are their own array). It
|
||||
only ever grows, so a walk bounded by it over-approximates the units that hold a binding - the
|
||||
only direction a conservative GPU-write set may fail in.
|
||||
- `MG_State/GLState/Core.h` - the two `GLContext` pass-throughs, same guard.
|
||||
- `MG_Impl/GLImpl/Texture/GL_Texture.cpp` - `BindImageTexture` feeds the mark, same guard.
|
||||
- `MG_Remote/Client/GpuWritePending.cpp` - `MarkWritableImageBufferTextures` walks
|
||||
`unit <= GetMaxTouchedImageUnit()` instead of `unit < MAX_TEXTURE_IMAGE_UNITS`. On the profiled
|
||||
workload (no image binding anywhere) the mark is -1, so the walk is one compare.
|
||||
Profile tie-in: 2.15% self / 3.59% inclusive of the GL thread, `GetImageTextureBinding` 1.37 of it.
|
||||
- `MG_Impl/Pipe/ImageEmit.h` - its comment claimed no such mark could exist without resizing the
|
||||
pull build's object. Updated, not deleted: the mark is push-only, and the emitter deliberately
|
||||
keeps its program-derived window (the narrower "which units can a *shader* read").
|
||||
|
||||
### 2. `MGPipeValidateForVerb` (brief item 2)
|
||||
|
||||
- `MG_Impl/Pipe/PipeFill.cpp` - step 4's seven-term `supplied` conjunction was evaluated 63 times
|
||||
per verb; it reads nothing about the verb. It is now a memo (`SuppliedFieldMask`) keyed on the
|
||||
four facts it does read: the push mask, `ApplierDerivesRenderStateFields()`,
|
||||
`contextValuesWireLive` (passed in, so P5c rv's emission half and fill half still read the ONE
|
||||
answer), and `P4aFamilyHasItsConsumer(kMGPipeP4aFamilySubsystems)` - all four families ride that
|
||||
one signal by that function's own rule. The walk becomes one bit test per field; the expression
|
||||
is moved, not copied.
|
||||
Profile tie-in: `MGPipeValidateForVerb` 4.11% self / 10.9% incl.; `P4aFamilyHasItsConsumer` alone
|
||||
is a `CapsMirror` read taken 63x/verb for one answer under split.
|
||||
- `ParsePoisonOmissionKnob` - the per-verb `String == String` (out-of-line even for two empty
|
||||
strings, which is every shipping configuration) is now a size compare inline, contents only when
|
||||
a non-empty value is involved. Re-parse condition unchanged.
|
||||
- `MG_Impl/Pipe/PipeFill.h` - `MGPipeResidualFillSuppliesField(field, pushMask, applierDerives,
|
||||
contextValuesWireLive)`, exported for the unit gate only, in `MGPipeP4aFamilyEmits`' shape.
|
||||
|
||||
### 3. `MGPipeServerStampVerbBoundary` (brief item 3)
|
||||
|
||||
- `MG_Backend/MGPipe/PipeInputs.cpp` - the per-verb 63-field `answerable` derivation is now a
|
||||
`constexpr` per-verb-class mask table (`AnswerableMaskForClass` / `kAnswerableByClass`), built
|
||||
from the same expression. The loop body is `FilledGen[i] = serial & -bit` - no lookups, no calls,
|
||||
no branch. Withdrawal semantics byte-identical, including the out-of-range verb
|
||||
(`kNoFieldIsAnswerable`, all 63 withdrawn), which is what `FieldIsInVerbClass` answered for it;
|
||||
that helper stays, `MGPipeInputUnfreshRead` still uses it. Profile: 1.1% self of the apply
|
||||
thread, and on a lockstep frame that is client wait time one-for-one.
|
||||
|
||||
## Red-once
|
||||
|
||||
| change | test |
|
||||
|---|---|
|
||||
| stamp table | `FieldOwnershipTest.AServerStampMakesRecordSuppliedFieldsFreshAndWithdrawsTheRest` (existing) - it recomputes `answerable` independently per field and compares. `TheStickyExemptionIsCancelledByTheServerStamp` covers the withdrawal of the seven forwards. |
|
||||
| supplied-set memo | `FieldOwnershipTest.TheResidualFillsSuppliedMemoReKeysOnEveryInputThatMovesAnAnswer` (**added**; renamed and extended in the fix round below, which restates this row) - moves each of the three key inputs that can move an answer on its own and reads the answer back in between, in an order where a memo missing that input must return the stale answer. This is the only thing a memo can get wrong that the expression could not; which fields the fill copies has no other observable, because a push build routes every emission straight into the applier, which writes the same storage. |
|
||||
| image-unit bound | `SplitBufferSet.Row2TheSweepIsBoundedByTheImageUnitHighWaterMark` (**added**) - pins both ends of the bound (mark at -1 walks nothing; a unit at the mark is found). Revert the bound and its second half goes red. |
|
||||
| the mark's producer | `ImageEmit.TheBindFeedsTheImageUnitHighWaterMark` (**added**) - `GL::BindImageTexture` must move the mark, and an unbind must not shrink it. Drop `NoteImageUnitTouched` from `GL_Texture.cpp` and this goes red at the entry point instead of as a missed GPU write. |
|
||||
| (adjusted) | `SplitBufferSet.Row2AWritableImageBufferTextureIsMarkedByADraw` now notes the unit, because it writes the binding straight into the slot and never goes through `glBindImageTexture`. |
|
||||
|
||||
G2/G14 name parity kept: the two new split/push-only names have pull-lane skip twins
|
||||
(`SplitBufferTest.cpp`'s `MGL_SPLIT_ONLY_OR_SKIP` list, `ImageEmitTest.cpp`'s
|
||||
`MGL_IMAGE_EMIT_TEST_LIST`, `FieldOwnershipTest.cpp`'s `#if !MOBILEGL_PIPE_PUSH` block).
|
||||
|
||||
## G1 / monolith impact
|
||||
|
||||
- **Pull build (`MOBILEGL_PIPE_PUSH=OFF`): no change at all.** `PipeFill.{h,cpp}` is appended to
|
||||
`SOURCE_FILES` only under `MOBILEGL_PIPE_PUSH` (CMakeLists.txt:519-523); `PipeInputs.cpp`'s stamp
|
||||
is inside `#if MOBILEGL_BUILD_DISAGGREGATED`; `GpuWritePending.cpp` is MG_Remote. The two MG_State
|
||||
files and the one MG_Impl call site are each `#if MOBILEGL_PIPE_PUSH`. G1 stays 0/0/0/0.
|
||||
- **Push monolith:** `TextureState` grows 4 bytes (`m_maxTouchedImageUnit`) and `glBindImageTexture`
|
||||
gains one compare. Behaviour identical - the member has no reader outside MG_Remote.
|
||||
`PipeFill.cpp`'s memo and the knob fast path are behaviour-identical by inspection.
|
||||
- **Behavioural delta worth naming:** `MG_Remote::Client::ConsumerRefusals()` will count fewer
|
||||
refusals, because the residual fill asks `P4aFamilyHasItsConsumer` once per environment instead of
|
||||
63 times per verb. It is a diagnostic counter; the one-time `MGLOG_W` per refused subsystem still
|
||||
fires, and no test asserts an exact value that the fill contributes to
|
||||
(`SanityTest.…CountsTheRefusal` asserts a delta `> 0` from the mint path;
|
||||
`RemoteClientTest`/`CapsMirrorTest` drive `ServerConsumes` directly).
|
||||
|
||||
## What remains per draw in the validate point / draw emit (brief item 4, surveyed not cut)
|
||||
|
||||
- `wants()` still calls `P4aFamilyHasItsConsumer` per emitted gate (~10/verb) **and tests the dirty
|
||||
bit LAST**, so a clean bit still pays the consumer read. Reordering the dirty test first is a pure
|
||||
win *except* that `ServerConsumes` counts and logs - "a refusal that happened cannot fail to be
|
||||
counted" is R-8's stated rule. **Left alone deliberately**; it needs R-8's owner to say whether
|
||||
the counter may move.
|
||||
- `ContextValuesWireLive()` per verb = three loads (`g_active`, `m_started`, one acquire load). Left.
|
||||
- `RunsAsTheServerRole()` at PipeFill.cpp:3037 is inside the `tracker.FreshlyPrimed()` arm - a
|
||||
make-current edge, not per draw. Not hot. (PipeFill.cpp:992's is the respecify path, also not
|
||||
per draw.) Package D owns the guard itself.
|
||||
- `EmitTables.cpp`: `ReadDrawBindings()` re-resolves the element buffer's handle every draw
|
||||
(`MGPipeResourceTrackerInstance().Find` -> `MGPipeSlots().FindByLifetimeId`, which under split also
|
||||
pays `MGPipeRefuseAllocatorFromApplyThread` - package D's guard). `VertexInputEmit.h:257-274`
|
||||
computed the same handle at validate, but **only when bit `NewIndexBuffer` was dirty**, so there is
|
||||
no already-computed value to reuse on a clean-index draw; a latch keyed on the VAO's index slot +
|
||||
the tracker generation would be needed. **Left** - it is a new latch with its own invalidation
|
||||
story and belongs with package B/D's tracker work, not in a cut described as "provably redundant".
|
||||
`RequireSession` and the `g_dropDrawEmission` ordering are one global read each and E2's control
|
||||
order respectively; nothing else on that path is redundant.
|
||||
|
||||
## Risks / not done
|
||||
|
||||
- **Not built.** No compiler has seen any of this. The constexpr stamp table
|
||||
(`MakeAnswerableMaskTable`, a `constexpr` function with loops over `inline constexpr` generated
|
||||
tables) is the most likely place for a toolchain complaint.
|
||||
- The memo is a file-scope global, like `g_residualDue` / `g_omission` / the verify latches beside
|
||||
it; what keeps a single writer on all of them is the verb barrier. If a future change lets two
|
||||
roles fill concurrently, this global joins the list that has to move - stated in the code comment.
|
||||
- The image-unit mark's safety rests on `glBindImageTexture` being the only writer of
|
||||
`m_imageTextureBindings` that can *add* a binding. I checked: the other writer is
|
||||
`TextureState::MarkTextureObjectForDeletion`, which only unbinds. Tests that poke
|
||||
`GetImageTextureBinding(u)` directly must note the unit - one such test existed
|
||||
(`SplitBufferTest`) and was updated; `MG_IntegrationTest` has none.
|
||||
- Brief item 2 also asked about the residual fill's *object-class* copy cost (`GetBindingSlot` from
|
||||
`CopyField`, 1.73 self / 45% of it from the draw fill). That cost is the copy itself, not the
|
||||
predicate: those rows are BARRIER-PULLED and must be copied until P3b/P4b/P7/P8 give them twins.
|
||||
Nothing cut there; it is package E's question.
|
||||
|
||||
## Fix round
|
||||
|
||||
Same worktree (`MobileGL-p5d-C`), still uncommitted, still not built. Patch regenerated:
|
||||
11 files, 587/64. Every blocker and major addressed; all seven minors addressed too (six were
|
||||
one- or two-line corrections, the seventh is the by-value return).
|
||||
|
||||
### MAJOR - the memo's fourth key input is untested (`FieldOwnershipTest.cpp:405`)
|
||||
|
||||
**The finding is right, and it is stronger than stated: that input is not merely untested, it is
|
||||
UNTESTABLE through this observable, and the suggested fix cannot work as written.**
|
||||
|
||||
The reviewer proposed flipping the consumer signal around `GetFramebufferBindingSlot`,
|
||||
`GetTextureUnitObject`, `GetProgramForDraw` or `GetImageTextureBinding`. Those are exactly the
|
||||
five fields whose emitter belongs to a P4a family - and every one of them is a field
|
||||
`EmittedCallSuppliesTheWholeField()` answers **false** for (PipeFill.cpp:2276-2290: their storage
|
||||
is a frontend heap reference - a `BindingSlot<FramebufferObject>`, an `ImageTextureBinding`, a
|
||||
`TextureUnit`, two `SharedPtr<ProgramObject>` - that no payload can carry, so the fill pulls them
|
||||
whatever the consumer says). So `supplied` is `false` for all five in BOTH consumer states: the
|
||||
consumer conjunct is **dominated**, an assertion pair over those fields would read
|
||||
`EXPECT_FALSE` / `EXPECT_FALSE`, and a memo whose key dropped the signal would pass it - vacuous
|
||||
in the other direction, which is no better.
|
||||
|
||||
What went in instead, in three parts:
|
||||
|
||||
1. **`PipeFill.cpp` - a compile-time trip wire.** `NoP4aFamilyFieldIsWhollySupplied()`, a
|
||||
`constexpr` walk of `kMGPipeFieldEmittedBy` through `SubsystemForEmitter` and
|
||||
`EmittedCallSuppliesTheWholeField`, with a `static_assert` over it. It fires on the day
|
||||
P3b/P4b/P7/P8 give one of those rows a twin the applier can write - which is the day the
|
||||
consumer signal becomes observable - and its message says to go write the pair. This is the
|
||||
durable guard; it sits next to the predicate and cannot rot.
|
||||
2. **`FieldOwnershipTest.cpp` - step 4, which MOVES THE SIGNAL FOR REAL.** Under monolith the
|
||||
signal is `MGPipeGetResourceOps() != nullptr` (PipeFill.cpp:1156-1171), and
|
||||
`MG_Config::Transport` is a `constexpr` Monolith in every lane but a split one, so
|
||||
`MGPipeSetResourceOps(&ops)` / `(nullptr)` moves it in the lane this case runs in, with no new
|
||||
include and no MG_Remote dependency. The case collects the WHOLE 63-field answer set in each
|
||||
state at `kEverySubsystem` (the only mask at which the fill reads the signal at all), asserts
|
||||
the signal really moved, names `GetFramebufferBindingSlot` both ways, and asserts the two
|
||||
answer sets are identical - with a failure message saying that a difference is *correct
|
||||
behaviour* and the instruction to rewrite the step into the move-and-read-back pair, naming
|
||||
`CapsMirrorInstance().Adopt()` and the SanityTest case that already drives it for the split
|
||||
arm. It restores the previous op table on the way out.
|
||||
3. **The name.** `...ReKeysOnEveryInput` becomes **`...ReKeysOnEveryInputThatMovesAnAnswer`**, in
|
||||
both the push case and its pull-lane skip twin (G2/G14 parity kept). The finding that the old
|
||||
name and the red-once table oversold the coverage is accepted.
|
||||
|
||||
Red-once table correction for that row: the memo's case covers **three** of the four key inputs
|
||||
by moving them; the fourth is covered by a domination assertion plus a `static_assert`, because
|
||||
no observable distinguishes it today.
|
||||
|
||||
### MINOR - the consumer read moved to every verb, counters move the wrong way (both reviewers)
|
||||
|
||||
Accepted, and fixed rather than merely documented. `SuppliedFieldMask` now reads the signal only
|
||||
where the walk could reach it:
|
||||
|
||||
```cpp
|
||||
const Bool p4aConsumer = (pushMask & kMGPipeP4aFamilySubsystems) != 0 &&
|
||||
P4aFamilyHasItsConsumer(kMGPipeP4aFamilySubsystems);
|
||||
```
|
||||
|
||||
At a mask with no P4a bit, `(pushMask & subsystem) != 0` short-circuits every P4a field in the
|
||||
rebuild and `P4aFamilyHasItsConsumer` returns a constant `true` for every other subsystem, so no
|
||||
field can reach the mirror - the guard is exact, not conservative. The key stays complete because
|
||||
`pushMask` is itself in the key, so a mask that gains a P4a bit re-keys on the mask. Net effect:
|
||||
`ConsumerRefusals()` and `LastRefusedSubsystem()` are now unmoved at the A/B lanes and in the
|
||||
bring-up window (CallMask 0) where the baseline never touched the mirror, and strictly *fewer*
|
||||
refusals elsewhere - which makes the report's "will count fewer refusals" line true in both
|
||||
directions rather than only one. The reasoning is in the code comment at the read.
|
||||
|
||||
### MINOR - `SuppliedFieldMask` returned a reference into a memo it rebuilds in place
|
||||
|
||||
Accepted. It now returns `MGPipeFieldMask` **by value**; the rebuild fills a local and publishes
|
||||
it into `g_fillPlan` in one assignment at the end, so no caller can observe a partially rebuilt
|
||||
mask and no rebuild can zero storage a caller holds. The call site at the walk takes
|
||||
`const MGPipeFieldMask supplied` (a copy, 16 bytes, read 63 times - cheaper than the indirection
|
||||
it replaces). The comment that cited `g_residualDue` as precedent now says why that precedent
|
||||
covers the file-scope-ness but not the width.
|
||||
|
||||
### MINOR - the stale-memo failure mode is stated wrong (`PipeFill.h:78`, test header)
|
||||
|
||||
Accepted; the reviewer is right. The walk stamps `FilledGen[i] = filled.CurrentVerbSerial`
|
||||
whether or not it copied, so a wrongly-skipped field reads **FRESH with the previous verb's
|
||||
value** - a silent stale binding slot, caught only by the verify lane, not
|
||||
`Fatal{UnmigratedPipeInput}` (poison catches an *unstamped* read, not a stamped-but-uncopied
|
||||
one). Corrected in the test case header and stated explicitly in `PipeFill.h`, where it is now
|
||||
the reason the key case is the only guard.
|
||||
|
||||
### MINOR - "32 slots" / "32 units wide" (`TextureState.h`, `GpuWritePending.cpp`)
|
||||
|
||||
Accepted, in both places. `MAX_TEXTURE_IMAGE_UNITS` is **192**: the replaced sweep was 192
|
||||
`GetImageTextureBinding` reads per draw, so the measured win is 6x larger than the comment
|
||||
claimed, not smaller. `GpuWritePending.cpp` now names the constant and the count;
|
||||
`TextureState.h`'s folding argument now uses real image-unit numbers (a bind of unit 7 pushing
|
||||
the texture scan out to eight slots, the last advertised unit pushing it to 191) instead of the
|
||||
invented 32. The report's own prose never quoted 32, so nothing there needed the same fix.
|
||||
|
||||
### MINOR - the G2 parity count comment is off by one (`SplitBufferTest.cpp:552`)
|
||||
|
||||
Accepted; the block declares eighteen names in each lane (diffed, 18 vs 18). Rather than write
|
||||
`EIGHTEEN` and let it rot a third time - it read `SIXTEEN` against seventeen before this commit
|
||||
and `SEVENTEEN` against eighteen after - the sentence now drops the count, says the list below is
|
||||
the authority, and records that the number has never once matched.
|
||||
|
||||
### MINOR - the new producer case binds image unit 4 (`ImageEmitTest.cpp:582`)
|
||||
|
||||
Accepted. The case now binds **unit 1**, like every neighbouring case, and - because even that
|
||||
depends on the backend advertising two image units - it `ASSERT`s that the binding actually
|
||||
landed (`Ctx().GetImageTextureBinding(1).Texture != nullptr`) before it reads the mark. A bind
|
||||
refused at `unit >= GetAdvertisedImageUnitCount()` (GL_Texture.cpp:6694) now says so by name
|
||||
instead of reading as a missing `NoteImageUnitTouched`.
|
||||
|
||||
### Not changed, and why
|
||||
|
||||
Nothing was disputed on the merits. The reviewers' "checked and cleared" sections (stamp-table
|
||||
equivalence, memo key completeness term by term, single-writer / no new concurrency hazard, the
|
||||
image-unit mark's writers, G1 = 0 in the pull build, the new tests' typing,
|
||||
`ParsePoisonOmissionKnob`'s equivalence) are unaffected by this round: the stamp table, the knob
|
||||
fast path, the image-unit mark and every `#if` guard are byte-identical to the reviewed patch.
|
||||
The only disagreement is the one recorded under the major, and it is a disagreement about the
|
||||
proposed FIX, not about the finding.
|
||||
|
||||
### Risks, restated
|
||||
|
||||
- **Still not built.** This round adds one more `constexpr` function with a loop
|
||||
(`NoP4aFamilyFieldIsWhollySupplied`) beside the stamp table's `MakeAnswerableMaskTable`, in the
|
||||
same standard-conforming shape; no toolchain has seen either.
|
||||
- The new test step depends on `MG_Config::Transport == Monolith` to move the consumer signal and
|
||||
skips its body otherwise, so in a split-transport lane step 4 does not run. That is the right
|
||||
trade today: the assertion it would make there is the same dominated one, and the
|
||||
`static_assert` covers every lane.
|
||||
@@ -0,0 +1,222 @@
|
||||
# P5d round 3, package D - role guards and TLS made nearly free
|
||||
|
||||
Worktree `MobileGL-p5d-D`, branch `p5d-D`, base `85362a85`. Not built, not committed, not pushed.
|
||||
Patch `scratchpad/p5d/patches/D.patch` (618 lines, 11 files). **What changed, per file:**
|
||||
### `MG_Remote/Server/ServerLoop.{h,cpp}` - item 1, the point of the package
|
||||
- New `Server::Detail::g_applyThreadKey` (`inline std::atomic<Uint64>`, header) + `Detail::CurrentThreadKey()`.
|
||||
On aarch64 (`__has_builtin(__builtin_thread_pointer)`) the key is the thread pointer - one
|
||||
`mrs TPIDR_EL0`, no call; elsewhere `std::this_thread::get_id()` memcpy'd into a `Uint64`. x86_64
|
||||
deliberately stays on the fallback: the builtin only grew x86 support in clang 17 / GCC 11, and older
|
||||
toolchains accept `__has_builtin` for it and then fail in codegen - a build break on the desktop
|
||||
ctest lane for a target nobody profiles.
|
||||
- `OnApplyThread()` is now **inline in the header**: `key != 0 && key == CurrentThreadKey()`. Gone:
|
||||
`ServerLoopInstance()`'s function-static guard byte, the acquire loads of `m_running` and of
|
||||
`atomic<std::thread::id>`, `std::this_thread::get_id()`, the out-of-line `pthread_equal`, and the
|
||||
cross-library PLT hop for every caller that includes the header.
|
||||
- `m_applyThreadId` is **deleted**. The key is published as `ApplyThreadMain`'s first statement and
|
||||
cleared **inside the same `m_controlMutex` critical section that clears `m_running`** (the C2 exit
|
||||
block), so the two move together and the "live but exiting apply thread" window is byte-for-byte the
|
||||
old one. `Stop()` no longer clears an identity after the join.
|
||||
- Relaxed on both sides. The one dangerous shape - a new thread reusing the exited apply thread's TLS
|
||||
block (hence its thread pointer) while still reading the pre-clear key - cannot happen: the clear is
|
||||
sequenced before that thread's exit, the exit releases the TLS block under libc's own lock and the
|
||||
receiving thread takes the same lock, a happens-before chain a relaxed load must respect by
|
||||
coherence. A stale *zero* is harmless. The argument is written out at the declaration.
|
||||
### `MG_Impl/Pipe/SlotAllocator.{h,cpp}` and `MG_State/.../MipmapStorage.{h,cpp}` - item 2
|
||||
`g_reverseAnnouncementScopeDepth`, `g_frontendKeyedRegistryScopeDepth` and
|
||||
`g_textureLegacyArmScopeDepth` stop being `thread_local`. The three scopes increment/decrement only
|
||||
when `OnApplyThread()` is true, each keeping that decision in a new `Bool m_counted` so the destructor
|
||||
undoes exactly what the constructor did (re-asking would leak a count across the apply thread's exit).
|
||||
**Single-writer proof, in the files:** each counter's only reader in the tree is its apply-thread
|
||||
guard - `MGPipeRefuseAllocatorFromApplyThread` (`SlotAllocator.cpp:30-31`),
|
||||
`RefuseLegacyTextureArmFromApplyThread` (`MipmapStorage.cpp:44`) - and both return before they look
|
||||
unless the transport is active **and** `OnApplyThread()`; I grepped every `::Active()` call site and
|
||||
those three are all of them. So the apply thread is sole writer and sole reader, and a monolith process
|
||||
never reads them. `Active()` on the apply thread is unchanged; off it, it is now always false, and both
|
||||
headers say a future reader has to account for that.
|
||||
### `MG_State/GLState/BufferState/BufferObject.cpp` - item 3
|
||||
The Fatal body is split out as `FatalLegacyBufferArmFromApplyThread` (`[[noreturn]]`).
|
||||
`SyncPersistentMappedRange` called `RefuseLegacyBufferArmFromApplyThread` and then re-asked
|
||||
`PushIsArmed()`/`OnServerRole()` for an early-out whose condition was *identical* - so its `return` was
|
||||
unreachable, the guard having already aborted. It now asks once; no accessor makes more than one
|
||||
`OnApplyThread()` call. `PushIsArmed()` is a read of `MG_Config::Transport`; left alone.
|
||||
### `MG_Remote/Client/ClientSession.cpp` - item 3
|
||||
`RefusePipeInputsTouchWhileApplierOwnsIt` reordered cheapest-first: `Transport == Monolith`,
|
||||
`Ipc.BatchWaits != 0`, then `OnApplyThread()`, `Started()`, `BarrierArmed()`,
|
||||
`ApplyThreadIsInsideApplier()`, and the `thread_local` `InBarrierWait()` **last**. The chain is a pure
|
||||
conjunction (every row returns "no violation"), so which touches abort is unchanged; only the cost of
|
||||
answering "no" moves. With the shipping default `MOBILEGL_IPC_BATCH_WAITS=1` it is two loads on a path
|
||||
`MGPipeValidateForVerb` reaches per verb (PipeFill.cpp:542, :1951, :2789).
|
||||
### `MG_Remote/Client/WireTables.cpp`, `.../PersistentMapTracker.cpp`
|
||||
Comments only. `RunsAsTheServerRole()` and `PersistentMapTracker::OnServerRole()` stay out of line
|
||||
**on purpose**: inlining them would put a server header inside `WireTables.h` (whose comment says it
|
||||
exists so PipeFill need not include one) and inside `PersistentMapTracker.h` (which MG_State's
|
||||
`BufferObject.cpp` includes). One PLT hop is that boundary's price; the four-part predicate behind it
|
||||
- the measured cost - is gone either way. `IsLivePersistentMap`'s row order is *not* touched: it is
|
||||
contractually the same chain as `SyncPersistentMappedRange`'s, and item 1 already made its one guarded
|
||||
accessor (`IsMapped`) cheaper than the reorder would have.
|
||||
|
||||
## Why (profile at head 56a77348)
|
||||
`OnApplyThread()` 2.66% self / 3.64% incl **on the client thread** for an answer that is "no" every
|
||||
time, plus `@plt` 6.1% self on the apply thread (item 1). `__emutls_get_address` 7.7% and the **top
|
||||
symbol** on the monolith GL thread, 7.4% on the apply thread, 32.7% of it exactly these scopes
|
||||
(FrontendKeyedRegistry ctor 18.75 + dtor 13.95, ReverseAnnouncement 8.4), built at ~20 sites in
|
||||
DirectGLES.cpp including inside `StateBackendObjectRegistry::HandleOf` per draw (item 2). Every cycle
|
||||
cut on the apply thread shortens the client's R-1 wait one-for-one, so both pay on both threads.
|
||||
|
||||
## Red-once
|
||||
- **Item 1**: `MG_Test/Wire/ServerLoopTest.cpp:291-324` - a posted control request records
|
||||
`OnApplyThread()` inside itself (`EXPECT_TRUE`) and the test thread asserts `EXPECT_FALSE` after;
|
||||
publish the key late, clear it early or compare the wrong identity and it goes red. Backed by the 5
|
||||
`MGL_BUFFER_GUARD_TEST` + 9 `MGL_TEXTURE_GUARD_TEST` cases (`RemoteClientTest.cpp:1841-1912`), each
|
||||
needing `OnApplyThread()` true inside `RunOnApplyThread`.
|
||||
- **Item 2**: three cases **added** to `RemoteClientTest.cpp` after the buffer guard block:
|
||||
`AnAllocatorProbeFromTheApplyThreadOutsideEveryScopeIsFatalByName` (the control),
|
||||
`AnAllocatorProbeInsideAnExemptionScopeOnTheApplyThreadIsAllowed` (red if the increment is dropped)
|
||||
and `AnExemptionScopeHeldOnTheGLThreadDoesNotExemptTheApplyThread` - **the one that pins the
|
||||
single-writer argument**: the scope is open on the GL thread across the posted request and the apply
|
||||
thread's probe must still abort; make the counter unconditional again and it goes green while a real
|
||||
violation is silently exempted.
|
||||
- **Item 3, ClientSession**: pure reorder, no new test; `ClientPipeInputsFillWhileTheApplierOwnsIt
|
||||
IsFatalByName` (batch off -> Fatal), `...IsAllowedWhenBatchingIsOn` and
|
||||
`...WithTheApplierIdleIsAllowed` already drive both sides of the row I moved to the front.
|
||||
**BufferObject**: `BufferSyncPersistentMappedRangeFromTheApplyThreadIsFatalByName` still demands
|
||||
`Fatal{RoleViolation, "buffer-legacy-arm"}` from that path.
|
||||
|
||||
## G1 / monolith impact
|
||||
**G1 = 0/0/0/0.** Nothing reaches the pull build: MG_Remote and `MG_Test/Wire` are not compiled there
|
||||
(`MG_Test/CMakeLists.txt:105`), `SlotAllocator.cpp` is a `MOBILEGL_PIPE_PUSH` file (disagg forces push
|
||||
on, `CMakeLists.txt:496-500`), and every hunk in `BufferObject.cpp`, `MipmapStorage.h` and
|
||||
`MipmapStorage.cpp` - the new `m_counted` member included - sits inside `#if MOBILEGL_BUILD_DISAGGREGATED`.
|
||||
**Push build under `Transport=Monolith`**: identical by inspection - no `ServerLoop` runs, so the key
|
||||
is 0 and `OnApplyThread()` is false exactly as the old predicate was; the scopes count nothing and
|
||||
`Active()` is false, which the only reader never reaches (first line returns on Monolith). G14: three
|
||||
names added, none removed. G2: the added names sit in a disagg-only suite beside names already there.
|
||||
|
||||
## Risks
|
||||
1. `__builtin_thread_pointer` is the one new intrinsic; if unavailable the `#if` falls back and only
|
||||
the speedup is lost. **Not compiled** (no-build rule), so this is inspection only.
|
||||
2. The relaxed-load recycling argument leans on libc synchronising TLS-block reuse; it is written out
|
||||
at the declaration so a reviewer can reject it, and an acquire load would cost one `ldar` and keep
|
||||
every other saving. 3. `Active()` is now always false off the apply thread - harmless today (one
|
||||
reader, proven), latent for a future second reader; both headers say so.
|
||||
|
||||
## What I did not do (item 4 inventory)
|
||||
- `g_inBarrierWait` (`ClientSession.cpp:117`) - ~900+ set/clear pairs per frame. **Must stay TLS** (the
|
||||
file's note: a second GL thread would read the first one's wait as its own), so single-writer fails;
|
||||
only mitigated by moving its read to the end of the guard chain.
|
||||
- `t_activeDecoder` (`PipeWireCodec.cpp:359`; read :1289/:1292, saved/restored :1385-1393) - per
|
||||
applied record, so it *is* hot. **Not changed**: its comment says a unit suite drives several
|
||||
decoders from several test threads, so the apply thread is not its only writer.
|
||||
- `g_reply` (`PipeRoute.cpp:61`) - the monolith adapter's one-deep reply mailbox, per reply-slot row
|
||||
rather than per draw, and per-thread by design (R-1 bounds the in-flight depth per thread). **Kept.**
|
||||
- Cold, off the per-draw path: `tl_initializing` (`Init.cpp:31`), `tl_isPoolThread`
|
||||
(`ShaderCompilePool.cpp:30`), `t_current` (GLX/WGL), the `static thread_local Vector<GLuint> names`
|
||||
scratch buffers (`GL_ProgramPipeline.cpp:74,83`, `GL_Sampler.cpp:309,342`), `paddedUbo`
|
||||
(`UniformManager.cpp:2072`, DirectVulkan); `DirectGLES.cpp:12828` already says why the owner stamp is
|
||||
not thread_local.
|
||||
- Also not done: inlining `RunsAsTheServerRole` / `OnServerRole` (layering), reordering
|
||||
`IsLivePersistentMap` (contract) - both argued in the code comments. No build, no ctest, no device
|
||||
run (the brief forbids all three): every claim is by inspection, no number was re-measured.
|
||||
|
||||
## Fix round
|
||||
|
||||
Both reviewers returned `apply-with-minors`, no blockers. Every major and every minor below is
|
||||
addressed in code except the two that are explicitly integration notes. Still not built, not
|
||||
committed, not pushed; no attribution lines. Patch regenerated (770 lines, 12 files).
|
||||
|
||||
### MAJOR (both reviewers) - the key clear at the `ServerLoop.cpp` C2 exit had no red-once. **Fixed.**
|
||||
Both reviewers made the same argument and it is correct: the old predicate was
|
||||
`m_running && id == mine`, so an uncleared identity was harmless by construction; the new one has no
|
||||
second term, and delete-the-store left every test in the tree green. New case in
|
||||
`MG_Test/Wire/ServerLoopTest.cpp:829` (right after the C2 `AForwarderCallAfterTheLoopStopped...`
|
||||
case), `AThreadCreatedAfterTheLoopStoppedIsNotMistakenForTheApplyThread`, in three parts:
|
||||
- `ASSERT_NE(Detail::g_applyThreadKey, 0)` while the loop is parked, so the post-Stop `EXPECT_EQ`
|
||||
cannot pass for the wrong reason (a key that was never published is also 0);
|
||||
- `EXPECT_EQ(Detail::g_applyThreadKey, 0)` after `fixture.Stop()` - reviewer 1's deterministic half;
|
||||
it pins the **presence** of the store on every platform;
|
||||
- reviewer 2's half, which pins **why it matters**: 8 sequentially created-and-joined
|
||||
`std::thread`s, each asserting `ServerLoop::OnApplyThread()` false. With the store removed the
|
||||
first thread to inherit the exited apply thread's TLS block (hence its thread pointer) answers
|
||||
true and the case goes red for its own reason. It is second, not alone, because TLS-block reuse is
|
||||
likely, not guaranteed - the case says so in its own comment.
|
||||
|
||||
**Position** (reviewer 1's second half). I did not add a `Detach()`-time hook: it would put a third D
|
||||
hunk into `ServerLoop.cpp` and deepen the overlap with package T, for a property no unit case can
|
||||
reach anyway. Instead the C2 comment now names the hazard concretely (`ServerLoop.cpp:458-472`):
|
||||
`ServerLoopTest` has no GL context, so `CreateBackend` never runs and the
|
||||
`~BackendObject_DirectGLES` -> `ReleaseEGLResources` self-post does not exist here; move the store
|
||||
above `m_backend.reset()` and that post misses `RunOnApplyThread`'s re-entrancy arm
|
||||
(`ServerLoop.cpp:576`, `if (OnApplyThread()) return work(user);`) while `m_running` is still true,
|
||||
publishes into the one-slot mailbox and waits on `m_controlDone` for an answer only that thread could
|
||||
give. The comment says in as many words: do not move this on the strength of a green ctest.
|
||||
|
||||
### MINOR (both) - the `std::thread::id` fallback copies an object representation with no padding check. **Fixed.**
|
||||
`ServerLoop.h:178`: added `static_assert(std::has_unique_object_representations_v<std::thread::id>)`
|
||||
beside the size assert, plus `#include <type_traits>`. I chose this over
|
||||
`std::hash<std::thread::id>{}(id) | 1u`: the hash is well-defined, but `| 1` is injective only
|
||||
because today's `pthread_t` happens to be pointer-aligned, and on MSVC (an FNV hash of the id) it
|
||||
would fold two distinct threads onto one key with no diagnostic - a guard answering **true off the
|
||||
apply thread**, which is worse than the failure being fixed. A padding implementation is now a build
|
||||
error, which is the loud-not-silent direction CONTRACT-P5C rule E asks for. The comment beside it
|
||||
names the concrete consequence both reviewers derived (lost re-entrancy shortcut -> `m_controlDone`
|
||||
deadlock; exemption scopes stop counting -> `MGPipeRefuseAllocatorFromApplyThread` aborts at the
|
||||
per-draw `HandleOf` probes).
|
||||
|
||||
### MINOR (reviewer 2) - the selector macro is unguarded, never `#undef`'d and not forceable from the build. **Fixed.**
|
||||
`MOBILEGL_SERVER_APPLY_THREAD_POINTER` -> `MOBILEGL_DETAIL_APPLY_THREAD_POINTER`
|
||||
(`ServerLoop.h:150-161`): `#ifndef`-guarded, always **defined** (to 0 when unavailable) and tested
|
||||
with `#if`, not `#ifdef`. `-DMOBILEGL_DETAIL_APPLY_THREAD_POINTER=0` now really disables the builtin
|
||||
path - the escape hatch for risk 1 the package named itself - and no longer collides with the
|
||||
header's own define. The nested `#if defined(__aarch64__) && defined(__has_builtin)` /
|
||||
`#if __has_builtin(...)` shape is kept (GCC < 10 must not evaluate the inner line).
|
||||
|
||||
### MINOR (reviewer 2) - `Active()` is silently thread-conditional in public headers. **Fixed by renaming.**
|
||||
`MGPipeReverseAnnouncementScope::Active()`, `MGPipeFrontendKeyedRegistryScope::Active()` and
|
||||
`MGPipeTextureLegacyArmScope::Active()` are now `ActiveOnApplyThread()` (declarations
|
||||
`SlotAllocator.h:205`, `:229`, `MipmapStorage.h:70`; definitions `SlotAllocator.cpp:83`, `:101`,
|
||||
`MipmapStorage.cpp:94`; the three readers `SlotAllocator.cpp:28-29`, `MipmapStorage.cpp:44`; the
|
||||
comment at `RemoteClientTest.cpp:1861`). I re-grepped the tree myself: those are all of them, so this
|
||||
is a closed rename with no external caller. Chose the rename over an `MGLOG_E_ONCE`/assert inside the
|
||||
query because the guard reads it on the apply thread on the per-draw path, where a diagnostic is
|
||||
either free and useless or costly and hot; a name that carries the precondition costs nothing and
|
||||
cannot be ignored by the future P4b/P7 reader the finding is about. Both headers' comments now say
|
||||
why the name is spelled that way.
|
||||
|
||||
### MINOR (reviewer 1) - `docs/Disaggregated/ROADMAP.md:69`'s G4 citation went stale. **Fixed, but OUTSIDE `D.patch`.**
|
||||
Confirmed: `MobileGLResult ServerLoop::RunOnApplyThread(` is at `:543` at baseline `85362a85` and at
|
||||
`:570` in this worktree (the fix round's C2 comment moved it a further 15 lines), and the old `:598`
|
||||
end maps line-for-line to `:625`. The cell now reads `ServerLoop.cpp:570-625`. **`ROADMAP.md` is not under `MobileGL/`**, so the mandated
|
||||
`git diff HEAD -- MobileGL` cannot carry it; it is delivered as a separate one-line patch beside it,
|
||||
`scratchpad/p5d/patches/D-docs.patch`. Integrator: apply both.
|
||||
`python scripts/check_doc_citations.py docs/Disaggregated/ROADMAP.md` -> 56 citations, 0 problems.
|
||||
Note for the integrator: package T also moves `RunOnApplyThread`, so this cell needs re-checking once
|
||||
T lands - the gate will not catch it (it only resolves the file and counts lines).
|
||||
|
||||
### MINOR (both) - D's `ServerLoop.cpp` hunk overlaps the C2 block package T rewrites. **No code change; integration note, as both reviewers asked.**
|
||||
Accepted as stated, and I deliberately did **not** shrink the hunk: the key clear has to be inside
|
||||
the same `lock_guard(m_controlMutex)` critical section as `m_running.store(false)`, or the window in
|
||||
which a live-but-exiting apply thread answers true moves - which is the property item 1's equivalence
|
||||
argument rests on. Land **D before T**, or rebase T onto D, and at the merge check that T's
|
||||
`m_controlPosted` clear and D's `Detail::g_applyThreadKey.store(0)` both end up inside that same
|
||||
section beside `m_running.store(false)`. The new ServerLoopTest case above is what makes a merge that
|
||||
drops the clear fail loudly instead of landing green. The `ServerLoop.h` overlap (D prepends the key
|
||||
block, T adds park counters) is textual only - different regions of the file.
|
||||
|
||||
### Findings I did not act on
|
||||
None outstanding, and nothing in either review was disputed. The two items left uncoded are the two
|
||||
both reviewers themselves scoped as "flag it to the integrator" (overlap ordering) and the one the
|
||||
patch boundary cannot carry (ROADMAP, delivered separately).
|
||||
|
||||
### Re-verification after the fix round
|
||||
- `python scripts/check_include_closure.py` -> 4 probes, 0 skipped, 0 problems (unchanged; the rename
|
||||
and the `<type_traits>` include add no header->header edge).
|
||||
- `python scripts/check_doc_citations.py docs/Disaggregated/ROADMAP.md` -> 0 problems.
|
||||
- Doc-citation sweep over every file this package touches: the four `ClientSession.cpp` citations in
|
||||
`ROADMAP.md` / `CONTRACT-P5C.md` are all at or below `:880` and both D hunks start at `:996`, so
|
||||
none moved. No document names `Scope::Active()`, so the rename breaks no citation.
|
||||
- G1 unchanged and still 0/0/0/0: the new test is in `MG_Test/Wire` (not in the pull build), the
|
||||
rename and the new comments are all inside `#if MOBILEGL_BUILD_DISAGGREGATED`, and `<type_traits>`
|
||||
enters only `MG_Remote/Server/ServerLoop.h`, which the pull build does not compile.
|
||||
- Still no build, no ctest, no device run (the brief forbids all three). Every claim here is by
|
||||
inspection or by a read-only script.
|
||||
@@ -0,0 +1,288 @@
|
||||
# E - can the draw barrier be deferred by one verb? (research note, no code)
|
||||
|
||||
Head `85362a85`, worktree `MobileGL-p5d-E`, DirectGLES, `MOBILEGL_TRANSPORT=inproc`, push mask
|
||||
`0x1fff` (every subsystem bit on, so every record arm in the backend is the arm that runs).
|
||||
Every `file:line` below is at that head. Paths are under `MobileGL/` unless they start with `docs/`.
|
||||
|
||||
## 0. The lockstep as it is wired today (the thing (c) would move)
|
||||
|
||||
1. `MG_Impl/GLImpl/Drawing/GL_Drawing.cpp:544-550` - `DrawElements_Backend` runs `MGP_FILL(DrawElements)`
|
||||
(= `MGPipeValidateForVerb`, `MG_Impl/Pipe/PipeFill.cpp:2781`) and only then calls the backend table
|
||||
slot, which under a transport is the client emit table.
|
||||
2. `PipeFill.cpp:2789-2790` (`RefusePipeInputsTouchWhileApplierOwnsIt`), `:2814` (withdraw the server
|
||||
stamp), `:3059-3104` step 4: for every field in the kDraw mask not supplied by a record,
|
||||
`MGPipeFillAccess::CopyField` (`:88-330`). The object-class rows copy either a **SharedPtr**
|
||||
(VAO `:113`, dispatch program `:216`, draw program `:219`, XFB program `:262`) or a **raw pointer
|
||||
into the live GLContext arrays** (15 buffer slots `:119-121`, 15 binding-point row bases `:127-129`,
|
||||
2 framebuffer slots `:174-178`, image-unit base `:183`, texture-unit base `:246`). Storage:
|
||||
`MG_Backend/MGPipe/PipeInputs.h:807-815`.
|
||||
3. `MG_Remote/Client/EmitTables.cpp:529-553` `EmitIndexedDraw` -> `ReadDrawBindings` (`:459-482`, reads
|
||||
the VAO element slot + the two indirect slots on the GL thread) -> `EmitDrawRecord` (`:484-525`):
|
||||
`BeforeDrawVerb` (`:222-226`: `PushDrawConsumers`, deferred-destroy drain, `MarkGpuWritesForDraw`)
|
||||
then `EmitAndWaitTails`.
|
||||
4. `MG_Remote/Client/ClientSession.cpp:814-990`: encode, `PublishAndNotify` (`:871`); DrawVbo is
|
||||
`kCtxVerb`, so the BatchWaits early return at `:899-905` does not apply; `WaitForAppliedBudget`
|
||||
(`:916-917`), then `DrainEventRing` (`:951` - "the one instant at which the apply thread is known
|
||||
not to be inside the applier").
|
||||
5. Server: `MG_Remote/Server/PipeApplier.cpp:1043-1080` `ApplyOne` = `ScopedApplierEntry` (`:1057`),
|
||||
`StampVerbBoundary` (`:1061` -> `PipeInputs.cpp:103-130`: every BARRIER_PULLED field's `FilledGen`
|
||||
is set to 0, so each server read of one is counted as `rsp` by `MGPipeInputUnfreshRead`
|
||||
`:137-170`), `DecodeAndApply` -> `OnDrawVbo` (`:430`) -> `gl.DrawElements` ->
|
||||
`MG_Backend/DirectGLES/DirectGLES.cpp:6705-6716` -> `PrepareForDraw` (`:4762-4830`), then
|
||||
`LeaveApplier`. `MG_Remote/Transport/SessionRings.h:378-394`: apply -> `++appliedSeq` ->
|
||||
`PublishApplied` -> ring. The client is runnable again only after step 5 has fully returned.
|
||||
|
||||
What the barrier protects is stated at `PipeInputs.h:904-910`: `m_filled` (`FilledGen[63]`,
|
||||
`CurrentVerbSerial`), `m_serverStampedVerb` and the `rsp` counter are plain, not atomic, because
|
||||
"the verb barrier leaves at most one of {GL thread, apply thread} runnable". `CONTRACT-P5C.md:35-43`
|
||||
(rule E) says "the barrier held" is no longer an accepted mechanism - and then `:157-201` (§3.1) and
|
||||
`:410-437` (§5.4) grant exactly the object-class family and the two scopes a named exemption from it.
|
||||
|
||||
## (a) Every server-side read that reaches client-owned memory through the object rows
|
||||
|
||||
Legend: **live** = a dereference of the frontend array/object the row points at; **id** = a
|
||||
`GetLifetimeId()` read plus a probe of the CLIENT slot allocator (`MGPipeSlots().FindByLifetimeId`,
|
||||
`DirectGLES/SlotTables.h:436-449`, inside `MGPipeFrontendKeyedRegistryScope` or
|
||||
`MGPipeReverseAnnouncementScope`; `MG_Impl/Pipe/SlotAllocator.cpp:22-38` is the guard the scope
|
||||
disarms); **mint** = `GetOrCreate(const StatePtr&)` -> `MGPipeSlots().Acquire` (`SlotTables.h:244-248`),
|
||||
a WRITE into the client allocator; **rec** = the same site has a record arm that reads no frontend
|
||||
memory and is the arm that runs on this workload.
|
||||
|
||||
### a.1 `GetBoundVertexArray` (SharedPtr copy, `PrepareForDraw` `:4770`)
|
||||
| site | what is dereferenced | kind |
|
||||
|---|---|---|
|
||||
| `ResolveVaoTwin` `:1602-1633` (scope `:1609`) | `g_backendVertexArrayObjects.Find(vao.get())` -> `SlotTables.h:411-414` -> `HandleOf` `:436-449`: `vao->GetLifetimeId()` + allocator probe; `GetOrCreate(vao)` on a miss `:1618` | id, mint |
|
||||
| `:4776` | `currentVAO->GetConfigVersion()` | live |
|
||||
| `SyncVaoAttributeBuffersByHandle` `:699-790` | memo hit `:719-737`: per entry `IsBufferDrawCleanByHandle(entry.handle, entry.resource, entry.frontend)` (`Managers.cpp:3109-3153`; under a transport `:3143-3145` does NOT touch `frontend`), dirty repair `:731` `currentVAOObject->GetAttribute(i).Buffer` -> `EnsureBufferResource`; memo miss `:741-780`: `GetAllAttributes()` (32 x `{Enabled, Buffer SharedPtr, ...}`), `attrib.Buffer.get()`, `EnsureBufferResource(bufferObject)` (`Managers.cpp:3448-3455` -> `HandleOfBuffer` `:2922-2936`, scope `:2933`: `GetLifetimeId` + allocator probe), `HandleOfBuffer(bufferKey)` `:767` | live, id |
|
||||
| IBO arm `:823-861` | `currentVAOObject->GetIndexBufferBindingSlot().GetBoundObject()` (SharedPtr in the VAO), `HandleOfBuffer(possibleIBO.get())`, `EnsureBufferResource(possibleIBO)` | live, id |
|
||||
| `SyncCurrentVAO` `:1637-1649` -> `SyncToBackend` (`Managers.cpp:4881-4891`) -> `SyncToBackendFromApplier` (`:5160+`) | nothing: "the frontend VAO is not read at all"; the index arm resolves `st.IndexBuffer.Res` only (`:5320-5331`) | rec |
|
||||
| `SyncCurrentVertexAttributeValues` `:1659-1700` | `GetBoundVertexArray()` again `:1669`, `vao->GetConfigVersion()` `:1682`, memo miss `vao->GetAttribute(location).Enabled` `:1686` | live |
|
||||
| `DrawArrays` `:6718-6738` (scope `:6729`), `MultiDrawArrays` `:6775` | `Find(currentVAO.get())` + `SyncClientSideAttributesForDrawArrays(currentVAO, ...)` | id, live |
|
||||
| `EnsureBufferResourceForHandle` `Managers.cpp:3193-3446` | under a transport the object is HELD, not read: base `:3207-3222` = `resource->hostBytes`, `SyncPersistentMappedRange` skipped `:3329-3341`, content bit from the descriptor `:3376-3381`; it still takes the SharedPtr by reference | held only |
|
||||
|
||||
### a.2 `GetProgramForDraw` (SharedPtr copy, `:4781`; the client-side `GetProgramForDraw` join is `MG_State/GLState/Core.cpp:609-628` and runs at fill, not on the server)
|
||||
| site | what is dereferenced | kind |
|
||||
|---|---|---|
|
||||
| `SyncCurrentProgram` `:4321-4400` (scope `:4329`) | `GetLinkStatus()/GetSpirvStatus()` `:4339`; `g_backendProgramObjects` probes; broadcast count from the framebuffer record when `g_fboRecordsTrusted` `:4350-4372`, else `GetFramebufferBindingSlotChecked(Draw)` + `drawFBO->GetObjectVersion()/GetDrawBuffers()` `:4384-4395` | live, id, rec |
|
||||
| `BindCurrentTextures` `:5245-5305` | `currentProgram.get()` as key, `GetLifetimeId()`, `GetBackendStateVersion()`, `GetLinkStatus()` `:5279-5282`; miss -> `ResolveAndBindUnitTextures` `:4836+`: `GetMaxUniformLocation/GetUniformSamplerOrImageUnitIndex/GetUniformType` `:4845-4855` (only on an alias conflict) | live |
|
||||
| `BindCurrentProgramWithResources` `:5317-5600` (scope `:5328`) | `GetLinkStatus/GetSpirvStatus` `:5320`, registry `Find` fallback `:5336`, `GetUBOSize` `:5347,:5371`, `MapUBO()`/`GetUBOContentVersion()` ONLY when no ShaderCso record `:5364-5387` (record arm `:5364-5368`), `GetUniformBlockBinding(i)` `:5463`, `GetBackendStateVersion` `:5514`, sampler pass miss `GetUniformSamplerOrImageUnitIndex` `:5537-5539` | live, rec (UBO bytes) |
|
||||
| `SyncAtomicCounterBuffers` `:600-660` | `GetBufferBindingPointCount` (sticky forward -> `PipeFill.cpp:1861-1865`, live context) - only for programs with counters | live |
|
||||
|
||||
### a.3 `GetTextureUnitObject` (raw base of `Array<TextureUnit,192>`; a unit is 11 `BindingSlot<ITextureObject>` + one `SharedPtr<SamplerObject>`, `MG_State/GLState/TextureState/TextureUnit.h:15-26`, `MG_Util/Types.h:181-201`)
|
||||
| site | what is dereferenced | kind |
|
||||
|---|---|---|
|
||||
| `CaptureDrawTextureSyncKeys` `:2188-2211` -> `CurrentUnitBindingsEpoch` `:2044-2098` | record arm `:2048-2056` reads two applier serials (`SamplerViewsSerial`, `SamplerStatesSerial`); walk arm `:2073-2097` -> `CaptureUnitBindings/UnitBindingsUnchanged` `:1867-1920`: per unit `GetAllBindingSlots()[i].GetBoundObject()->GetLifetimeId()`, `GetSamplerObject()` | rec; walk = live |
|
||||
| `SyncNeccessaryTextures` `:2333-2480`, memo HIT `:2342-2362` | `PairingsIntact` `:2134-2139`: `entry.slot->get()` where `entry.slot` is `const SharedPtr<ITextureObject>*` INTO the frontend unit's slot (`:2127-2131`, borrowed at `:2374`); then `entry.backend->IsDrawSyncClean(entry.slot->get(), ...)` `:2356` - the texture object's versions. **This runs on every draw even when every memo hits.** | live |
|
||||
| same, memo MISS `:2372-2385` | `GetTextureUnitObject(index)` per unit, every slot's `GetBoundObject()`, `IsUndefinedDefaultTexture`, `SyncTextureObjectToBackend(textureObject)` (`:1725-1760`, scope `:1734`: `Find` id / `GetOrCreate` mint, then params/sampler/mips sync of the object) | live, id, mint |
|
||||
| `ResolveAndBindUnitTextures` `:4861-4880` (memo miss of `BindCurrentTextures`) | per unit `GetAllBindingSlots()`, `GetBoundObject()`, twin `Find` (scope `:4926`) | live, id |
|
||||
| sampler pass `:5562-5582` | `GetTextureUnitObject(unit).GetSamplerObject()`, `GetBindingSlot(Texture2D).GetBoundObject()`, `samplerObject->GetLodBias()`, `texture2D->GetFormat()/GetSamplerObject()` | live |
|
||||
| `BindCurrentUnitSamplers` `:5051-5170` | record arm `:5111-5150` reads `st.BoundSamplerStates[]` only; decline arm `:5157` `GetTextureUnitObject(unit).GetSamplerObject()` | rec |
|
||||
|
||||
### a.4 `GetImageTextureBinding` (raw base of `Array<ImageTextureBinding,192>`, `TextureState.h:18-27`: SharedPtr + 5 scalars)
|
||||
| site | what is dereferenced | kind |
|
||||
|---|---|---|
|
||||
| `SyncImageTextureBindingsForDraw` gate `:2870-2900+` | `g_imageUnitHighWaterMark == 0` early-out (`:2893`): zero reads on a draw with no image unit ever bound | - |
|
||||
| `SyncImageTextureBinding` `:2636-2748` | `imageBinding.Texture` (SharedPtr) `:2650,:2660`, `ResolveShaderImageRecord` (`:2582-2635`, scope `:2588`; identity test `view.Res != HandleOf(boundTexture)` `:2626`), Level/Layer/Layered/Format from the record when it matches `:2670-2676`, **`imageBinding.Access` always frontend** `:2745` (`:2663-2669`: the record's `Access` byte has no written encoding) | live, id, rec (4 of 5 fields) |
|
||||
| `MarkWritableImageBufferTexturesGpuWritten` `:2764-2785` | gated `g_writableImageBufferUnitCount == 0`; else `imageBinding.Texture.get()` -> `TextureObjectBuffer::GetBufferBindingSlot().GetBoundObject()` -> `MarkBufferGpuWritten` (`Managers.cpp:2947`, id) | live, id |
|
||||
|
||||
### a.5 `GetFramebufferBindingSlot` (raw pointers to the two `BindingSlot<FramebufferObject>`)
|
||||
| site | what is dereferenced | kind |
|
||||
|---|---|---|
|
||||
| `SyncCurrentFBO` `:3277-3330` | `SyncCurrentFBOByRecord` first `:3292`; decline (scope `:3303`): slot `:3305`, `GetBoundObject()/GetVersion()`, `currentFBO->GetObjectVersion()`, attachment walk | rec |
|
||||
| `BindCurrentFBO` `:4514-4600` | record arm `:4527-4560` (`FindByHandle(record.Fbo)`); decline `:4568-4590`: slot, `GetBoundObject()`, compare with `pDefaultFramebufferInfo->defaultFBO` `:4577`, `Find(currentFBO.get())` (scope `:4585`) | rec |
|
||||
| `SyncNeccessaryTextures` draw-FBO list `:2405-2470` and `SyncReadFramebufferTextureAttachments` `:2246-2331` | **always** `GetFramebufferBindingSlotChecked(target)`, `GetBoundObject()`, `GetVersion()`, `currentFBO->GetObjectVersion()`, and on a list rebuild `currentFBO->GetAllAttachmentObjects()` / `attachment.GetTexture()` (borrowed SharedPtr slots); the record only replaces the VERSION half of the key, "THE IDENTITY HALF IS STILL A POINTER COMPARE" `:2159-2168` | live |
|
||||
| `Clear` `:6180-6190`, `BlitFramebuffer` bound arm `:8380-8418` | the three functions above, plus `GetFramebufferBindingSlot(Read/Draw).GetBoundObject()` `:8407-8410` for the layered-blit workaround; the named arm `:8330-8378` reads handles and the twin table's `StateForHandle` note | live / rec |
|
||||
|
||||
### a.6 `GetBufferBindingSlot`, `GetBufferBindingPoint`, the rest
|
||||
- `GetBufferBindingSlot` (15 pointers): on the inproc DRAW path there is **no dereference** - the
|
||||
indirect arm reads `MGPipeApplier().VerbIndirectBuffer` (`:941-960`; P5c hd §3.5), the
|
||||
`:962` slot read is the monolith `else`. Readers are readback (`:10232` etc.) and the monolith
|
||||
`ReadIndirectCommandBytes` `:401`. The row is still FILLED every draw (15 x `GetBufferBindingSlot`
|
||||
in `CopyField` `:119-121` - the `GetBindingSlot` 1.73% in the profile).
|
||||
- `GetBufferBindingPoint` (15 row bases into `Array<BindingSlotRange1D,84>`): `SyncBufferBindingPoints
|
||||
(ShaderStorage)` `:477-521` and `MarkShaderStorageBuffersGpuWritten` `:585-598` (count is
|
||||
record-supplied; loop is over 0 entries on Minecraft), the UBO loop `:5463-5480` (1-3 points per
|
||||
program on Minecraft: `point.GetBoundObject()`, `point.GetRange()` - which itself reads
|
||||
`object->GetSize()` when no explicit range, `Types.h:218-224` -, `GetBufferResource`,
|
||||
`IsBufferDrawClean` -> `HandleOfBuffer` id, `EnsureBufferResource`), atomic counters `:628`, XFB
|
||||
`:1407-1420,:569` (only with a pending span). All **live + id**.
|
||||
- `GetProgramForDispatch`: `PrepareForCompute` `:6126` (dispatch only; same shape as a.2).
|
||||
- `GetTransformFeedbackProgram`: `StartPendingTransformFeedback` `:1381-1420` at the tail of every
|
||||
draw (`:4829`) but returns at `:1386` unless a span is pending.
|
||||
- Sticky forwards on the draw path: `GetBufferBindingPointCount` (`:621`, atomic counters only).
|
||||
`GetProgramObject/ValidateProgramName` (`:6152-6157`), `GetTextureObject` (CopyTex),
|
||||
`HasOpenTransformFeedbackSpan`, `RecordError` (`:8804`) are off the steady draw path.
|
||||
|
||||
### a.7 The two exemption scopes
|
||||
`MGPipeFrontendKeyedRegistryScope` (20 sites: `:1609, :1734, :2588, :3303, :4236, :4329, :4585,
|
||||
:4926, :4993, :5328, :5689, :6166, :6729, :6775, :8837, :9161, :9216, :9695, :10029, :11884`) wraps a
|
||||
`Find` / `HandleOf` / `GetOrCreate(const StatePtr&)` on one of the five frontend-keyed twin tables;
|
||||
each is `stateObj->GetLifetimeId()` + `MGPipeSlots().FindByLifetimeId` (`SlotTables.h:436-449`),
|
||||
and a miss on `GetOrCreate` is `MGPipeSlots().Acquire` (`:244-248`). `MGPipeReverseAnnouncementScope`
|
||||
(`Managers.cpp:2933`) wraps the same probe for every buffer (`HandleOfBuffer`), reached from every
|
||||
`EnsureBufferResource` / `IsBufferDrawClean` / `MarkBufferGpuWritten`. Both counters are
|
||||
`thread_local` (`SlotAllocator.cpp:41-70`) - the 7.4% apply-thread emutls and 3.0% `HandleOf<ITextureObject>`
|
||||
in the profile are these probes. On the steady Minecraft draw (memos hitting) the scope sites that
|
||||
still execute per draw are `:1609` (VAO twin), `:4329` (program), `:5328` (program resources),
|
||||
`:2933` for each UBO point, and `:1734`/`:4926` only on memo misses.
|
||||
|
||||
## (b) Which of those reads are UNSAFE if the client runs draw N+1's GL calls before the server applied draw N
|
||||
|
||||
Between `PublishAndNotify` for N and `MGPipeValidateForVerb` for N+1 the GL thread runs arbitrary
|
||||
entry points: `glBindVertexArray`, `glBindBuffer(Base/Range)`, `glActiveTexture`/`glBindTexture`,
|
||||
`glBindSampler`, `glUseProgram`, `glBindFramebuffer`, `glVertexAttribPointer`, `glEnable/DisableVertexAttribArray`,
|
||||
`glBufferData/SubData`, `glTexParameter*`, `glUniform*`, `glLinkProgram`, `glDelete*`, plus the client's
|
||||
own pre-verb work (`PushDrawConsumers` `PersistentMapTracker.cpp:704-745`, `MarkGpuWritesForDraw`
|
||||
`GpuWritePending.cpp:126-138`, `ReadDrawBindings`). Four classes of hazard:
|
||||
|
||||
1. **The raw-pointer rows point INTO live arrays, so every deref reads draw N+1's binding, and the
|
||||
read races the client's write.** `BindingSlot::Bind` is a non-atomic `shared_ptr` move plus
|
||||
`++m_version` (`Types.h:189-195`); `BindingSlotRange1D::SetRange` is a two-word store (`:229-232`).
|
||||
Concretely unsafe: `PairingsIntact` / `IsDrawSyncClean(entry.slot->get())` (`:2136, :2356` - on
|
||||
EVERY draw, memo hit or not), the unit walks (`:1871, :2374, :4861, :5562`), the UBO point reads
|
||||
(`:5464-5466`, incl. `GetRange()` -> `obj->GetSize()` while `glBufferData` may resize), the
|
||||
framebuffer slot reads (`:2405-2410, :2246, :3305, :4568`) and `GetAllAttachmentObjects()` while
|
||||
`glFramebufferTexture` mutates the container, image bindings (`:2650-2660, :2745`). A torn
|
||||
`shared_ptr` read is UB; a texture the client just `glDeleteTextures`'d while the server holds
|
||||
only a borrowed slot pointer is a use-after-free (the borrow rules at `:2103-2126` are written
|
||||
on the barrier). **Safe**: `GetBufferBindingSlot` (no draw-path reader under a transport).
|
||||
2. **The SharedPtr rows pin identity but not internals.** VAO: `GetConfigVersion` (`:4776,:1682`),
|
||||
`GetAllAttributes()/GetAttribute(i).Buffer` (`:731, :743-760, :1686`), the element slot (`:824`)
|
||||
move under `glVertexAttribPointer`/`glBindBuffer(ELEMENT_ARRAY)` on the same VAO - wrong draw AND a
|
||||
`shared_ptr` race. Program: `MapUBO()` (`:5385`, only without a ShaderCso record) copies the
|
||||
scratch `glUniform*` writes into - a torn UBO upload; `GetUniformSamplerOrImageUnitIndex`,
|
||||
`GetUniformBlockBinding`, `GetBackendStateVersion`, link/SPIR-V status move under
|
||||
`glUniform1i`/`glLinkProgram` (`Core.cpp:609-628` joins the pending link on the CLIENT at fill,
|
||||
so a relink issued after publish N is exactly the window the join exists to close). Objects reached
|
||||
through the rows: `GetSize` (`glBufferData`), texture params/shape for `IsDrawSyncClean` and
|
||||
`SyncTextureParamsToBackend` (`glTexParameter`), `GetLodBias` (`glSamplerParameter`), FBO draw
|
||||
buffers/attachments. Only `GetLifetimeId()` is immutable and safe.
|
||||
3. **The client allocator is client memory, mutated by every create/delete.** Every `HandleOf` /
|
||||
`Find` in both scopes (`SlotTables.h:445-446`, `Managers.cpp:2935`) probes `MGPipeSlots()`; a
|
||||
`GetOrCreate` miss WRITES it (`:248`). `glGen*`/`glDelete*` between two draws make this a data
|
||||
race on a hash container whatever draw the probe "means" - and it is unsolved by any snapshot of
|
||||
the pointer rows. This is rule E's own sentence at `SlotAllocator.cpp:30-35`, currently
|
||||
suspended by the two scopes on the strength of the barrier.
|
||||
4. **`gPipeInputs` itself is one block.** N+1's fill rewrites the pointer rows, `CurrentVerbSerial`
|
||||
and `FilledGen[]` (`PipeFill.cpp:2806, :3068-3103`) while N's stamp (`PipeInputs.cpp:110-127`)
|
||||
and N's reads are in flight; `RefusePipeInputsTouchWhileApplierOwnsIt` (`ClientSession.cpp:998-1022`)
|
||||
is disarmed under `BatchWaits=1` on the argument that "the pull-verbs' own wait keeps their
|
||||
pulled reads fenced" (`:1006-1012`, `:889-898`) - deferring the draw wait removes precisely that
|
||||
fence. `DrainEventRing` (`:951`) and the `OnBufferWriteback`/`OnGpuWritten` callbacks that write
|
||||
frontend objects also rest on it.
|
||||
|
||||
Net: on the inproc draw path, every object-class read except `GetBufferBindingSlot` is unsafe under
|
||||
one-verb pipelining; the record arms (a.1 `SyncToBackendFromApplier`, a.3 epoch + sampler bind,
|
||||
a.5 `SyncCurrentFBOByRecord` + `BindCurrentFBO`) are the only parts already safe.
|
||||
|
||||
## (c) The minimal "snapshot at fill" scheme
|
||||
|
||||
Two layers have to be snapshotted, not one:
|
||||
|
||||
**Layer 1 - the block.** `gPipeInputs` becomes a ring of >= 2 `PipeInputs` (63 fields; the value rows
|
||||
are record-supplied and need no copy, but `m_filled`, `m_currentVerb`, `m_serverStampedVerb` are per
|
||||
slot). The three doors (`MGPipeFillAccess` `PipeFill.cpp:69`, `MGPipeStampAccess` `PipeInputs.cpp:95`,
|
||||
`MGPipeApplyAccess`), the generated poison (`PipeFilled.inc`), the verify comparator
|
||||
(`PipeFill.cpp:470-510`) and the stamp all address ONE block today. Cost per draw: 4 SharedPtr copies
|
||||
(8 atomic RMW) + 34 pointers (272 B) - trivial - but the pointers are useless as a snapshot: they
|
||||
name live arrays (b.1).
|
||||
|
||||
**Layer 2 - what the pointers point at.** The closure the server dereferences per draw, if copied
|
||||
by value (upper bound, everything a.1-a.6 reaches on a memo miss):
|
||||
|
||||
| family | per draw | bytes | refcount ops (inc+dec) |
|
||||
|---|---|---|---|
|
||||
| texture units 0..maxTouchedUnit (Minecraft ~4-8) | 12 SharedPtr per unit (11 slots + sampler) + versions | 8 x ~320 B = ~2.5 KB | ~96-192 |
|
||||
| image units 0..highWaterMark | `ImageTextureBinding` x n (0 on Minecraft) | 0-40 B x n | 0-2n |
|
||||
| UBO points per program block (1-3) | SharedPtr + `Range1D` + explicit flag | ~40 B each | 2-6 |
|
||||
| SSBO / atomic / XFB points | 0 on Minecraft | 0 | 0 |
|
||||
| FBO slots x2 + attachment SharedPtrs | 2 SharedPtr + versions (+ n attachments for Iris) | ~64 B (+24n) | 4 (+2n) |
|
||||
| VAO: identity + config version + (memo miss) 32 attributes + IBO | 1 SharedPtr + 4 B; miss: 32 x ~48 B + 1 | ~24 B / ~1.6 KB | 2 / 2+2k |
|
||||
| program: identity + 6-10 scalars, `MapUBO` bytes only without ShaderCso | 1 SharedPtr + ~64 B | ~80 B | 2 |
|
||||
| **total, steady (memos hit)** | | **~3 KB** | **~110-210** |
|
||||
|
||||
At 852 draws/frame: ~2.5 MB/frame of memcpy (same order as the round-1 persistent-map push that
|
||||
was worth cutting) and ~95k-180k lock-prefixed RMW/frame; at 20-40 ns each that is **2-4 ms per
|
||||
15.6 ms frame on the GL thread** - comparable to the wait it would remove - and it does NOT fix
|
||||
(b.2) (object internals still live), (b.3) (allocator probes) or the borrowed-slot lists
|
||||
(`:2127-2131`), which would have to be rebuilt to point at the snapshot rather than at the unit.
|
||||
|
||||
A **handle-only snapshot** (lifetime id / `{slot,gen}` + version per entry, no SharedPtr) is
|
||||
~1.5 KB/draw, zero refcount ops, ~1 us/draw. But the server can only consume it if every twin
|
||||
lookup is BY HANDLE (`FindByHandle`, `SlotTables.h:420-425`) and `IsDrawSyncClean` compares
|
||||
server-owned serials - which is the G6 rekey (ROADMAP.md:71) plus the `set_shader_buffers` emission
|
||||
(CONTRACT-P5C.md:169-176). So the only snapshot that is both minimal and SAFE is not a snapshot:
|
||||
it is the retirement of the frontend-keyed path.
|
||||
|
||||
**Where the client would wait.** Today: `ClientSession.cpp:916-917`, immediately after `:871`.
|
||||
Under one-verb pipelining: `EmitDrawRecord` publishes N and returns; the GL thread runs its
|
||||
inter-draw work; at the top of `MGPipeValidateForVerb(N+1)` (`PipeFill.cpp:2781-2790`, where the
|
||||
`RefusePipeInputsTouch...` guard already stands) it waits `appliedSeq >= seq(N)` BEFORE the residual
|
||||
fill writes the next block (or before it recycles slot N of the ring), then drains events. The
|
||||
reply-slot rows (`:876-879`) and `PushDrawConsumers`' `resource_subdata` records (ordered before
|
||||
N+1 on the ring, `EmitTables.cpp:215-219`) keep their shape. Overlap gained = min(client inter-draw
|
||||
work, server apply of N).
|
||||
|
||||
## (d) What the pushed records already cover, so the server COULD stop reading the live tables
|
||||
|
||||
| record(s), applier state (`MG_Pipe/PipeApply.h`) | draw-path consumer that already uses it | live read that remains and what would replace it |
|
||||
|---|---|---|
|
||||
| `bind/create_vertex_elements` -> `VertexElementsCsos`, `BoundVertexElements` (`:487,:594`); `set_vertex_buffers` -> `VertexBuffers[32]{Res,...}`, `VertexBuffersSerial` (`:598-616`); `set_index_buffer` -> `IndexBuffer.Res` (`:620`) | `SyncToBackendFromApplier` (`Managers.cpp:5160+`, reads nothing frontend); memo KEY of `SyncVaoAttributeBuffersByHandle` (`:703-716`); index resolve `:5320-5331` | the ensure WALK (`:731, :741-780`) and the IBO arm (`:823-861`) still take the VAO's attribute/element SharedPtrs to reach `HandleOfBuffer`; under a transport `EnsureBufferResourceForHandle(nullptr, h)` needs no object (`:3193-3222`, the indirect arm at `:953` already calls it that way) - walk `st.VertexBuffers[Start..+Count].Res` and `st.IndexBuffer.Res` instead. `ResolveVaoTwin`'s twin is keyed by VAO lifetime id, the CSO handle is content-addressed (`SetHashSuppressor`) - the twin needs a rekey (P3b/P4b). `SyncCurrentVertexAttributeValues` can read `rec->Attributes[i].Enabled` |
|
||||
| `set_sampler_views` -> `BoundSamplerViews[unit] = {View, Texture, Unit}` (`MGPipeTypes.h:856-861`), `SamplerViewsSerial`; `bind_sampler_states` -> `BoundSamplerStates[]` (`PipeApply.h:673-681`); tx staged shadow + `set_texture_params` | `CurrentUnitBindingsEpoch` record arm (`:2032-2056`), `BindCurrentUnitSamplers` record arm (`:5111-5150`) | `SyncNeccessaryTextures`' unit list (`:2372-2385`, and its per-draw `PairingsIntact`/`IsDrawSyncClean` on borrowed slots), `ResolveAndBindUnitTextures` (`:4861-4880`), the sampler pass (`:5562-5582`). One view per unit is already the RESOLVED sampled target, so the 11-slot alias resolution (`:4864-4880`) collapses; `IsDrawSyncClean` would key on the applier's texture serials. Caveat A8 (`:2816-2822, :5127-5133`): nothing pins the window to cover `[0, maxTouchedUnit]` |
|
||||
| `set_shader_images` -> `BoundShaderImages[unit] = {Res, Unit, InternalFormat, Layer, Level, Layered, Access}` (`MGPipeTypes.h:877-886`) | `SyncImageTextureBinding` takes Level/Layer/Layered/Format from the record (`:2670-2676`); the window/mark union (`:2837-2842`) is server-owned | `Access` is carried but deliberately unread (`:2663-2669`: encoding undefined) -> define it; the identity test `HandleOf(boundTexture)` (`:2626`) -> `FindByHandle(view.Res)`; `MarkWritableImageBufferTexturesGpuWritten` needs the TexBuffer's backing-buffer handle (not in `MGPImageView`) |
|
||||
| `set_framebuffer_state` -> `FramebufferRecords[h] = {Fbo, Color[8]/Depth/Stencil surfaces, DrawBuffers, IsDefault, ...}` (`MGPipeTypes.h:750-760`), `BoundFramebuffer[2]`, `FramebufferSerial` (`PipeApply.h:639-654`) | `SyncCurrentFBOByRecord` (`:3292`), `BindCurrentFBO` record arm (`:4527-4560`), broadcast count (`:4350-4372`), the ContentHash half of both texture-list keys | the identity half of `g_fboTextureSyncList` / read list (`:2405-2410, :2246-2258`) and their attachment borrows -> `FindByHandle(record.Fbo)` + the record's surface handles; `BlitLayeredDestinationAspects` (`:8407-8410`) -> the twins' `StateForHandle` note as the named arm already does (`:8355-8362`); `pDefaultFramebufferInfo` (G6) |
|
||||
| `set_draw_program` -> `DrawProgram`, `bind_shader_state` -> `BoundShaderCso`, ShaderCso records with `GlobalConstants` bytes + version (`PipeApply.h:503,:689-693`) | UBO bytes and version (`:5364-5368`), program twin caches block indices / sampler bindings at link (`:5449, :5537`) | link/SPIR-V status, `GetLifetimeId`, `GetBackendStateVersion`, `GetUBOSize`, `GetUniformBlockBinding(i)`, `GetUniformSamplerOrImageUnitIndex(loc)` - the reflection/binding table (ROADMAP.md:32 "program registry 重键", ARCHITECTURE 3.2 "explicitly-not-ported") |
|
||||
| `set_context_values` (P5c rv) -> the 8 value rows | `SyncBufferBindingPoints` count, `maxTouchedUnit`, XFB flags | - (RECORD_SUPPLIED) |
|
||||
| `set_shader_buffers` / `set_stream_output_targets` (catalogued, **not emitted until P4b**, CONTRACT-P5C.md:169-176) | nothing | the 84-entry `GetBufferBindingPoint` rows: the UBO loop `:5463-5480` (1-3 per Minecraft draw), SSBO/atomic/XFB. **The one family with no producer at all** |
|
||||
|
||||
Also structurally: `EnsureBufferResourceForHandle` already reads nothing from the object under a
|
||||
transport (a.1 last row) - the frontend argument survives only so `HandleOfBuffer(obj)` can find
|
||||
the handle. Every consumer that has the handle in the applier (`VertexBuffers[].Res`,
|
||||
`IndexBuffer.Res`, `VerbIndirectBuffer`) could pass `(nullptr, h)` today; UBO/SSBO/atomic/XFB
|
||||
cannot until `set_shader_buffers` crosses.
|
||||
|
||||
So the records are ~80% there in SHAPE (vertex / sampler / image / framebuffer / program handles
|
||||
all cross) and ~0% there in the IDENTITY path (every twin table is still frontend-keyed, every
|
||||
buffer ensure still goes through `HandleOfBuffer(obj)`); plus one family (`GetBufferBindingPoint`)
|
||||
has no wire form yet.
|
||||
|
||||
## (e) Cost / benefit and recommendation
|
||||
|
||||
**Benefit bound.** One-verb pipelining overlaps the client's non-wait work with the server's apply.
|
||||
Profile at 56a77348: client thread 39.5% of cycles, `WaitForApplied` 27.9 self / 34.2 incl, so the
|
||||
client's overlappable work is ~5-12% of process cycles; the apply thread's real work (`DrainRing`)
|
||||
is 29%. Best case, with the frame server-bound: ~10-20% of the critical path, ~1.5-3 ms of a
|
||||
15.6 ms frame, i.e. 64 -> ~72-80 fps - and only if the snapshot costs nothing. The gap to monolith
|
||||
(110-118) is the apply thread's own work (49% of it is the idle-poll mutex, package T) and the
|
||||
per-draw guard/TLS/validate overhead (D, C): those cut both sides one-for-one and carry no barrier
|
||||
change. After T/D/C land the client's non-wait share should be re-measured; if it falls under ~10%
|
||||
the pipelining gain is inside the noise of a paired A/B.
|
||||
|
||||
**Cost.** (1) The full-copy snapshot (c) costs 2-4 ms/frame at 852 draws in refcount traffic alone and
|
||||
is still incorrect ((b.2), (b.3)); the handle-only snapshot is cheap but presupposes the G6 rekey.
|
||||
(2) The block itself has to be versioned/double-buffered: every stamp, fill, poison and verify
|
||||
site assumes one `gPipeInputs` - this is exactly what `P5D-INPROC-PERFORMANCE.md:103-106` assigns to
|
||||
P11 ("`gPipeInputs` 版本化/双缓冲"); note ROADMAP.md:37's P11 row (persistent map / adoption) does
|
||||
not yet list that item - a doc gap to close whichever way this goes. (3) The 20 frontend-keyed
|
||||
scope sites + `HandleOfBuffer` + `set_shader_buffers` = ROADMAP.md:32's P3b/P4b row verbatim.
|
||||
(4) R-1's contract text (`ClientSession.cpp:889-898`, `PipeInputs.h:904-910`, ARCHITECTURE.md:528-530),
|
||||
the R-1 negative control, `RefusePipeInputsTouchWhileApplierOwnsIt`, the strict lane's expected-red
|
||||
and `rsp` accounting all have to be re-derived. (5) With the registries still frontend-keyed, the
|
||||
7.4% emutls / 3% `HandleOf` on the apply thread get worse, not better, under any snapshot.
|
||||
|
||||
**Recommendation.** Do not take the deferred draw barrier into P5d round 4. The reads in (a) show
|
||||
that on the inproc draw path every object-class row except `GetBufferBindingSlot` is dereferenced
|
||||
live, and (b) shows that most of those dereferences are data races the moment the client moves on -
|
||||
the racy identity path (twin tables keyed by frontend lifetime id, `HandleOfBuffer` through the
|
||||
client allocator) cannot be snapshotted, only retired, and one family (`GetBufferBindingPoint`) has
|
||||
no wire form at all. The cheap-and-safe subset that round 4 CAN take, because it needs no barrier
|
||||
change and is split-only: (i) under a transport, drive `SyncVaoAttributeBuffersByHandle`'s walk
|
||||
and the IBO arm off `st.VertexBuffers[].Res` / `st.IndexBuffer.Res` with
|
||||
`EnsureBufferResourceForHandle(nullptr, h)`, removing the per-draw VAO attribute reads and their
|
||||
`HandleOfBuffer` probes (also cuts emutls); (ii) skip the 15-slot `GetBufferBindingSlot` copy in the
|
||||
kDraw residual fill when the wire is live, since no draw-path site reads it under a transport
|
||||
(package C's residual-fill territory - coordinate rather than duplicate); (iii) after T/D/C,
|
||||
re-profile the client's non-wait share. Sequence the real thing as P3b/P4b (handle-keyed twins,
|
||||
`set_shader_buffers`) followed by P11's versioned `gPipeInputs`; the deferred barrier is one
|
||||
`WaitForApplied` move (`ClientSession.cpp:917` -> `PipeFill.cpp:2790`) once both are in, and is
|
||||
not worth a package before then.
|
||||
@@ -0,0 +1,286 @@
|
||||
# P5d round 3, package T - transport: apply loop idle poll, clock-free spin, park counters
|
||||
|
||||
Worktree `MobileGL-p5d-T`, branch `p5d-T`, base `85362a85`. Not built, not committed. Patch:
|
||||
`scratchpad/p5d/patches/T.patch` (13 files, +489/-25).
|
||||
|
||||
## What changed, per file
|
||||
|
||||
### `MG_Remote/Server/ServerLoop.{h,cpp}` - item 1, the idle poll's mutex
|
||||
- New `std::atomic<Bool> m_controlPosted` replaces a `std::condition_variable` of the same name
|
||||
that had **no waiter and no notifier anywhere in the tree** (`m_controlDone` carries the whole
|
||||
handshake). `ControlIsPending()` is now one acquire load, no lock, and moved to the public
|
||||
section (a test reads it - see red-once); `PumpControlRequest()` gained a fast negative on the
|
||||
same load before it touches `m_controlMutex`.
|
||||
- Protocol: the poster stores the shadow **last of the four fields, with release**, inside the
|
||||
existing critical section in `RunOnApplyThread`; the pump clears it **under the lock, when it
|
||||
takes the request**; the C2 exit block clears it under the same lock as the `m_running` clear.
|
||||
The C2 critical section, the NOT_INITIALIZED answer at exit, publish-then-ring and the
|
||||
re-entrant inline arm are unchanged in meaning; `ready` now reads only atomics.
|
||||
- Why: simpleperf at `56a77348` put `mutex::lock` 12.4 + `mutex::unlock` 13.6 + the two
|
||||
`pthread_mutex_*` halves at ~49% of `mgl-srv-apply`, with `ApplyThreadMain` the **direct** caller
|
||||
of 58-60% and 99.8% of `pthread_mutex_lock`-via-`mutex::lock`. Those are these two call sites,
|
||||
twice per idle iteration; most of `@plt` 6.1% goes with them.
|
||||
- Item 2 (`DrainRing` on an empty ring): **audited, no code change**, finding written into the
|
||||
function. `ApplyOne` -> `RingConsumer::Pop`, whose first act is one acquire load of `cmdHead`
|
||||
against `m_localTail` (`Ring.cpp:222-232`); `applied == 0` skips the retire block. No mutex, no
|
||||
clock, no allocation.
|
||||
|
||||
### `MG_Remote/Transport/Doorbell.{h,cpp}` - item 3, the clock-free spin
|
||||
- `Detail::g_spinItersPerUs` + `Detail::CalibrateSpinItersPerUs()`, and an inline
|
||||
`SpinItersPerUs()` that is a relaxed load and a predicted branch - deliberately **not** a
|
||||
function-local static, whose guard-variable load on every `Wait` is the cost this removes.
|
||||
`Wait()`'s spin phase is now `SpinItersPerUs() * spinUs` iterations of `ready()` + `CpuRelax()`
|
||||
and reads **no clock at all**; the deadline is computed once, at the top of the park phase, and
|
||||
`kWaitForever` reads no clock even there.
|
||||
- Calibration: two passes of 8192 `load(acquire)` + `CpuRelax()` timed with `steady_clock`, faster
|
||||
pass wins (a preempted pass reads absurdly slow, and under-spinning costs a park), clamped to
|
||||
[4, 2000] iters/us with a 200 fallback for a clock too coarse to see the probe; a race between
|
||||
two calibrators stores two legitimate budgets.
|
||||
- **Overshoot bound** (documented in the file): the old loop overshot by at most one batch of 64
|
||||
yields; this one by the calibration error - a DVFS drop or little-core migration stretches it by
|
||||
the frequency ratio (~3-4x here, so 50 us can reach ~200 us), a preemption adds one slice as
|
||||
before, and the probe omits the caller's `ready()`, so a real iteration is *dearer* than a
|
||||
probed one and the spin ends early by that factor. Not a correctness bound: `ready()` is
|
||||
re-tested every iteration and the deadline is still real-clock. Stated drift: the spin is no
|
||||
longer charged against `timeoutMs`, so a wait is bounded by `timeoutMs + budget` - 50 us against
|
||||
millisecond-scale timeouts, and erring long is the safe direction for a timeout.
|
||||
- `Doorbell::ParkEntries()` + a private relaxed counter incremented at the one place a waiter
|
||||
stops spinning and blocks - unconditional (one relaxed add on a path about to enter a condvar
|
||||
or `poll()`), so a test can read it without `MOBILEGL_PIPE_STATS`.
|
||||
- Why: `steady_clock::now()` 2.34 self / 5.91 incl + `__kernel_clock_gettime` 2.37 + `clock_gettime`
|
||||
1.94 = ~6.6% of the GL thread at 900+ waits/frame, for a duration that is a hint.
|
||||
`WaitForApplied`'s common path paid 2+ reads per wait; it now pays zero unless it parks.
|
||||
|
||||
### `MG_Remote/Transport/SessionRings.h`, `Transport/Ring.cpp` - item 4, client half
|
||||
- `SessionProducer::Waits()` (relaxed atomic, incremented in `Park` *after* the two refusals that
|
||||
never reach the bell) and `Parks()` (delegates to the self bell's `ParkEntries()` - only
|
||||
`Doorbell::Wait` knows whether its budget ran out). Atomic because `ClientSession::Stop`'s drain
|
||||
waits through this producer from the tearing-down thread; it is never copied (checked).
|
||||
|
||||
### `MG_Util/Metrics/PipeStats.{h,cpp}`, `Client/EmitTables.cpp`, `Client/ClientSession.cpp`
|
||||
- Four gauges, **all inside existing `#if MOBILEGL_PIPE_PUSH` blocks**: `ServerWaits`,
|
||||
`ServerParks`, `ClientWaits`, `ClientParks`. New `wait[srv= srvpark= cli= clipark=]` group on
|
||||
the one `MGPipe stats:` line (it closes the `emit[` bracket the ring gauges were riding), same
|
||||
four under long names in the JSON dump, run totals like the gauges above them. Both server
|
||||
numbers, not one: `srv` is `ParkCount()`'s long-standing meaning (an intention, counted on the
|
||||
way toward a park - ServerLoopTest's C10 note), `srvpark` is a real block.
|
||||
- No new include edge into MG_Util: the existing `PublishGauge` pattern already crosses that way
|
||||
(`EmitTables.cpp:1000`). The client pair is published from `EmitPresent`; the **server pair by
|
||||
the apply thread from its own loop**, because CONTRACT-P5C rule E layer 2 forbids a GL-thread
|
||||
read of `ServerLoop`'s memory - PipeStats is the meeting point precisely because it is below
|
||||
both roles. Both publishes sit behind `PipeStats::Enabled()`.
|
||||
- `ClientSession::LogWireLedger` also gained `cliwait=`/`clipark=`: that line is in **every** split
|
||||
private log, and the round-3 profile runs did not set `MOBILEGL_PIPE_STATS`.
|
||||
|
||||
## Red-once
|
||||
|
||||
| change | goes red as |
|
||||
|---|---|
|
||||
| shadow **set** side dropped | `ServerLoopTest.AControlRequestRunsOnTheApplyThreadAndUnparksIt` (existing) - the predicate sees nothing, the poster blocks, the case says so within 5 s |
|
||||
| shadow **clear** side dropped | `ServerLoopTest.TheControlShadowIsClearedSoTheLoopParksAgainAfterARequest` (**new**) - `ready` stays true, the loop spins for ever and never stores `consumerParked`, so the second `WaitUntilTrulyParked()` times out; plus a direct `ControlIsPending()` read |
|
||||
| spin budget zero / calibration broken | `InProcessTransportTest.DoorbellSpinsWithoutParkingWhenReadyFlipsInsideTheBudget` (**new**) - `ParkEntries()` moves although the condition flipped on the 8th probe |
|
||||
| spin never ends / park unreachable | `InProcessTransportTest.DoorbellStillParksWhenTheSpinBudgetRunsOut` (**new**) - `ParkEntries()` does not move |
|
||||
| `wait[]` renamed or dropped | `PipeStatsTest.SummaryLineCarriesEveryClassAndGate` (extended) |
|
||||
|
||||
Kept passing by inspection: `DoorbellWakesAParkedWaiter`, `DoorbellReturnsImmediatelyWhenAlreadyReady`,
|
||||
`DoorbellTimesOutWhenNothingHappens` (the uncharged spin only makes elapsed larger than its `>= 20 ms`),
|
||||
`ShutdownUnparksAWaiterWithNoDeadline` (fence protocol and `Dead()` unchanged), `RingTest`, `SessionTest`.
|
||||
|
||||
## G1 / monolith impact
|
||||
|
||||
**None.** Every touched file is MG_Remote (built only under `MOBILEGL_BUILD_DISAGGREGATED`,
|
||||
`CMakeLists.txt:547-578`), a test, or - for `PipeStats.{h,cpp}`, which *is* in the pull build -
|
||||
edited exclusively inside pre-existing `#if MOBILEGL_PIPE_PUSH` regions (`PipeStats.h:186-236`,
|
||||
`PipeStats.cpp:484-545`, `:600-622`), so `MOBILEGL_PIPE_PUSH=OFF` compiles no added token and the
|
||||
pull build's text is unmoved. No include-closure probe is touched.
|
||||
|
||||
## Risks
|
||||
|
||||
1. **The spin's duration bound is weaker** (see above): it burns CPU before a park, cannot change
|
||||
a result, and the [4, 2000] clamp plus probed-cheaper-than-real bounds both directions. The
|
||||
probe reads the clock twice per pass, once per process (2-30 us) - no steady-state path.
|
||||
2. `m_controlPosted` is a second copy of one mailbox bit. Disagreement is a performance fault in
|
||||
both directions (a spin that never parks; a wakeup the doorbell's own re-test recovers), never
|
||||
a lost request - the pump re-checks `m_controlPending` under the lock and clears a stale shadow.
|
||||
3. `wait[` closes `emit[` earlier than before, but `PipeStatsWindow::CounterOrAbsent` matches
|
||||
`" name="` / `"[name="`, so `fbe`/`sve`/`csom`/`mpr`/`rsp`/... are unaffected and the four new
|
||||
names collide with nothing (`cli=` cannot match `clipark=`).
|
||||
|
||||
## What I did not do
|
||||
|
||||
- Did not build, run, commit or push. No `Co-Authored-By` anywhere.
|
||||
- Did not hoist `ServerSession::Consumer()` / `Applier()` out of `DrainRing` - two out-of-line
|
||||
calls per idle iteration, a small win, but caching them touches `ApplyThreadMain`'s opening
|
||||
lines, which package D owns. Flagged for round 4 / D; nothing of `OnApplyThread`,
|
||||
`m_applyThreadId` or those lines is touched here. Also left: the per-park `steady_clock::now()`
|
||||
under `kWaitForever` (one read per real park), to keep the park phase structurally identical.
|
||||
- Did not update `docs/Disaggregated/ARCHITECTURE.md:623`, which enumerates the summary line's
|
||||
counters and now omits `wait[]` - left to the round's docs pass so four packages do not collide.
|
||||
Same for stale `Doorbell.h:NNN` references nearby: already drifted at `85362a85`.
|
||||
- The brief's "visible to `ControlIsPending` **without the pump**" is not directly constructible:
|
||||
`m_callerMutex` allows one outstanding request and a live apply thread pumps every iteration, so
|
||||
no window exists in which a posted-but-unpumped request is observable by a third thread. The new
|
||||
test pins the clear side (which nothing else could see), the existing unpark test the set side.
|
||||
|
||||
|
||||
---
|
||||
|
||||
# Fix round (after review)
|
||||
|
||||
Two reviews, no blockers, three distinct majors (two of them the same defect seen from two
|
||||
sides) and five minors. Every one is addressed below; nothing was widened. Patch regenerated
|
||||
from `MobileGL-p5d-T`: 16 files, +749/-40 (was 13 / +489/-25). Still not built, not committed.
|
||||
|
||||
## Majors
|
||||
|
||||
### M1 (review 1) - the clear side's red-once did not fire
|
||||
|
||||
**Upheld.** `PumpControlRequest` clears the shadow twice - at the take (`ServerLoop.cpp`, under
|
||||
the lock, when the work is handed out) and defensively on the next pump when it finds the shadow
|
||||
set with `m_controlPending` false. The old case asserted only the END STATE ("the shadow is clear
|
||||
after the request was answered, and the loop parks again"), and the defensive clear supplies that
|
||||
end state on the very next iteration, so deleting the take-clear left the case green. The
|
||||
reviewer's trace is exact.
|
||||
|
||||
**Fix: observe the window, keep both clears.** `ServerLoopTest.TheControlShadowIsClearedSoTheLoop
|
||||
ParksAgainAfterARequest` is replaced by
|
||||
`ServerLoopTest.TheControlShadowIsClearedWhenThePumpTakesTheRequest`. The posted work now blocks
|
||||
on a gate the test thread holds; the test waits until the work is really running and reads
|
||||
`ControlIsPending()` in that window. There `m_controlPending` is still true and no pump is
|
||||
running, so the ONLY thing that can make the accessor answer false is the clear at the take -
|
||||
delete it and the read returns true on every run, deterministically. The end-state assertions
|
||||
(answer returned, `WaitUntilTrulyParked()` again) are kept as the symptom an operator would see,
|
||||
with their comment now saying which clear each one pins.
|
||||
|
||||
The defensive clear is kept: it is a genuine self-heal for a shadow that outlived its request,
|
||||
and with the window observed directly it no longer masks anything.
|
||||
|
||||
### M2 (both reviews) - `Parks()` was not the subset of `Waits()` that everything claimed
|
||||
|
||||
**Upheld, in all three of its parts**, and the fix is one mechanism for all three.
|
||||
|
||||
1. *Counted per `Park()`, not per `Wait()`.* The increment sat inside the park-retry loop, so a
|
||||
wait that parked, woke on a remembered `Notify`, re-tested false and parked again counted
|
||||
twice - a routine path, exactly as review 2 describes.
|
||||
2. *Counted for `Park(0)`.* When the remaining deadline rounds down to 0 ms the loop spins
|
||||
through the counting site without ever blocking.
|
||||
3. *Counted off a shared bell.* `SessionProducer::Parks()` returned `m_selfBell->ParkEntries()`,
|
||||
and `ClientSession::Start` installs that same object as the encoder's stage-retirement bell
|
||||
(`PipeWireCodec::SetStageRetirementDoorbell`). Those waits never touch `m_waits`, and with
|
||||
`spinUs == 0` they now park unconditionally, so `clipark` could print larger than `cli` on
|
||||
any lane with stage retirement (the SmallRing / E3(e) lane asserts `stageReclaimWaits >= 1`).
|
||||
|
||||
**Fix.**
|
||||
- `Doorbell::Wait` gained a fifth, defaulted parameter `std::atomic<std::uint64_t>* parkTally`.
|
||||
It is bumped at the same site as the bell's own counter, and both are now guarded by
|
||||
`chunkMs != 0 && !countedPark`, where `countedPark` is a local hoisted above the park loop. One
|
||||
park per `Wait`, and only when the call really blocks.
|
||||
- `SessionProducer` owns `m_parks` and `SessionProducer::Park` passes `&m_parks`, so the client
|
||||
pair counts the same events on both sides. `Parks()` reads it; `ParkEntries()` is no longer
|
||||
consulted by any ledger.
|
||||
- The server half got the same treatment even though its bell has no second waiter today:
|
||||
`ServerLoop::m_parkBlocks` (+ `ParkBlockCount()`, reset in m-3's per-session block beside
|
||||
`m_parks`) is passed into the loop's one `Wait`, so `srvpark <= srv` cannot be broken later by
|
||||
anything else waiting on the consumer doorbell (`SessionConsumer::WaitForWork` waits on that
|
||||
same bell).
|
||||
- Comments corrected where they asserted the old, false relation: `Doorbell::ParkEntries()`'s
|
||||
block now says it is per BELL and names the two waiters that share the client's,
|
||||
`SessionRings.h`'s `Waits()`/`Parks()` block says why the tally is ours and not the bell's,
|
||||
and `PipeStats.h` says the subset relation holds only because both halves are counted by the
|
||||
same path.
|
||||
|
||||
**Red-once for the fix** (both new, both deterministic):
|
||||
- `InProcessTransportTest.AWaitThatParksTwiceOnARememberedNotifyIsStillOnePark` - `Notify()` with
|
||||
nobody waiting, then one `Wait` whose condition never flips: the first `Park` returns at once on
|
||||
the remembered wakeup and the loop parks again. `ParkEntries()` must move by exactly 1. Revert
|
||||
`countedPark` and it moves by 2.
|
||||
- `SessionTest.TheProducersParkLedgerCountsItsOwnWaitsAndNotTheBellsOtherWaiters` - a foreign
|
||||
`Wait` on `clientTransport->SelfDoorbell()` (spinUs 0, condition never true - the encoder's
|
||||
exact shape) must move neither `producer.Waits()` nor `producer.Parks()`; the producer's own
|
||||
20 ms `WaitForApplied` must move both by exactly one. Revert `Parks()` to `ParkEntries()` and
|
||||
the first `EXPECT_EQ` on `Parks()` goes red.
|
||||
|
||||
## Minors
|
||||
|
||||
### m1 - `spinUs == 0` silently changed meaning (both reviews)
|
||||
|
||||
**Upheld; stated, and the probe no longer runs on that path.** The old batched loop ran 63
|
||||
iterations before its first clock read could end it; the new budget is `itersPerUs * spinUs`, so
|
||||
zero means zero. That is now an explicit `spinUs == 0 ? 0ull : ...` short-circuit with a comment
|
||||
naming the delta, both callers that pass 0 (PipeWireCodec's SEG_STAGE retirement wait, and any
|
||||
lane with `MOBILEGL_IPC_SPIN_US=0`), and the reason it is deliberate rather than accidental. The
|
||||
short-circuit also keeps a zero-spin caller from paying the one-time calibration probe - review
|
||||
1's second half of the finding. `PipeWireCodec.cpp`'s `0` is left as it is: the retirement wait is
|
||||
rare, already slow, and wants the park; changing it is a codec decision, not this package's.
|
||||
|
||||
### m2 - `ControlIsPending()` no longer means what its name says
|
||||
|
||||
**Upheld; documented rather than renamed.** The brief names the accessor, and three call sites'
|
||||
worth of churn buys nothing, so the header block now states the predicate literally - "posted and
|
||||
not yet taken" - says that it answers false for the whole duration of `work(user)` while
|
||||
`m_controlPending` is still true, says it is a PARK PREDICATE and not a liveness query, and notes
|
||||
that the only in-tree reader besides the test is the apply thread's own `ready` lambda, which by
|
||||
construction cannot be inside that window. The new test in M1 is built on exactly that window, so
|
||||
the documented meaning is now the tested one.
|
||||
|
||||
### m3 - the calibration clamp's two opposite readings (review 2)
|
||||
|
||||
**Upheld.** `perUs` truncates to 0 for any pass longer than 8.192 ms, i.e. the heavily preempted
|
||||
probe, and that landed on the same `best == 0` arm as "no usable clock" and installed 200 iters/us
|
||||
- 50x the floor - on the machine least able to afford it. `CalibrateSpinItersPerUs` now carries
|
||||
`sawUsableClock`, set whenever `elapsedNs > 0`: measured-but-slow clamps UP to
|
||||
`kMinSpinItersPerUs`, and `kFallbackSpinItersPerUs` is reserved for the case where no pass was
|
||||
measurable at all. The `Doorbell.h` paragraph already described this rule ("a probe that was
|
||||
preempted cannot produce a budget smaller than kMinSpinItersPerUs"); it is now true.
|
||||
|
||||
### m4 - `Doorbell.h:NNN` citations invalidated by this package's insert (review 2)
|
||||
|
||||
**Upheld - and the first report's "already drifted at 85362a85" was wrong.** Checked at baseline:
|
||||
`139-178` really was the spin loop through the `Dead()` arm, `186-193` really was the `};` plus
|
||||
NotifyIfParked's PRECONDITION, `211-221` really was CondVarDoorbell/`Kill`. The +60 lines moved all
|
||||
three, and `check_doc_citations.py` cannot see it because the file only grew. Every in-tree
|
||||
citation is re-pointed to SYMBOL NAMES rather than to new line numbers, so the next insert cannot
|
||||
break them again: `ServerLoop.cpp` (x3), `ServerLoop.h`, `SessionRings.h` (x3), `EventRing.h`,
|
||||
`ClientSession.cpp`, `ClientSession.h` (x2) and `CONTRACT-P5.md:355`. The checker only resolves
|
||||
`path:line` pairs, so dropping the numbers leaves every gate green.
|
||||
|
||||
### m5 (review 2's note, not filed as a finding) - `srv=0` under `spawn`
|
||||
|
||||
Not a defect of this patch, but the new `PipeStats.h` comment asserted "PipeStats is the meeting
|
||||
point precisely because it is below both", which is true in one transport mode. The comment now
|
||||
says so: under `spawn` the server pair is published into the server process's PipeStats and the
|
||||
client's line prints `srv=0 srvpark=0` for the whole run - the same property `rsp` already has -
|
||||
and the round this pair was added for is `inproc`.
|
||||
|
||||
## Red-once table, updated
|
||||
|
||||
| change | goes red as |
|
||||
|---|---|
|
||||
| shadow **set** side dropped | `ServerLoopTest.AControlRequestRunsOnTheApplyThreadAndUnparksIt` (existing) |
|
||||
| shadow **take-clear** dropped | `ServerLoopTest.TheControlShadowIsClearedWhenThePumpTakesTheRequest` (**new, rewritten**) - reads `ControlIsPending()` while the work is still running, where the defensive clear cannot cover for it |
|
||||
| park counted per `Park` instead of per `Wait` | `InProcessTransportTest.AWaitThatParksTwiceOnARememberedNotifyIsStillOnePark` (**new**) |
|
||||
| client parks read off the shared bell again | `SessionTest.TheProducersParkLedgerCountsItsOwnWaitsAndNotTheBellsOtherWaiters` (**new**) |
|
||||
| spin budget zero / calibration broken | `InProcessTransportTest.DoorbellSpinsWithoutParkingWhenReadyFlipsInsideTheBudget` |
|
||||
| spin never ends / park unreachable | `InProcessTransportTest.DoorbellStillParksWhenTheSpinBudgetRunsOut` |
|
||||
| `wait[]` renamed or dropped | `PipeStatsTest.SummaryLineCarriesEveryClassAndGate` (extended) |
|
||||
|
||||
## G1 / monolith impact
|
||||
|
||||
**Still none.** The fix round touched `Doorbell.{h,cpp}`, `Ring.cpp`, `SessionRings.h`,
|
||||
`EventRing.h`, `ServerLoop.{h,cpp}`, `ClientSession.{h,cpp}`, `CONTRACT-P5.md` (all MG_Remote or
|
||||
a doc), three test files, and `PipeStats.h` - the last one inside the same pre-existing
|
||||
`#if MOBILEGL_PIPE_PUSH` region as before, comment text only. No new include edge; the new `Wait`
|
||||
parameter is defaulted, so the two call sites that do not pass a tally (`PipeWireCodec.cpp`,
|
||||
`SessionConsumer::Park` in `Ring.cpp`) are unchanged.
|
||||
|
||||
## Not changed, with reasons
|
||||
|
||||
- **The defensive clear in `PumpControlRequest` stays.** Review 1 offered "drop it so the
|
||||
take-clear is load-bearing" as an alternative; dropping a self-heal to make a test work is the
|
||||
wrong trade when the test can observe the real window instead.
|
||||
- **`ControlIsPending` is not renamed** (see m2).
|
||||
- **`PipeWireCodec.cpp`'s `spinUs == 0` is not changed** (see m1).
|
||||
- **`ARCHITECTURE.md:623`'s summary-line enumeration** still omits `wait[]` - left to the round's
|
||||
docs pass, as before, so four packages do not collide on one file. The `Doorbell.h` citations
|
||||
were different: those were made wrong BY this patch, so this patch fixes them.
|
||||
- Did not build, run, commit or push. No attribution lines anywhere.
|
||||
Reference in New Issue
Block a user