Files
MobileGL/docs/archive/notes/p5/p5-results/v1-v2.md
T

252 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# v1-v2 — server role round 2: the confirmed set, the five codex findings, and the first 21/21 reduced-path lane
Branch `p5/v1`, round-2 range **`3fb97cc5..c9d84e33`** (base = round 1 `571cbabb` then `git merge feat/disaggregated`
→ `3fb97cc5`, head `7110cdb6` as the charter's step 0 named; nine single-line commits follow). Identity
`Swung0x48 <swung0x48@outlook.com>` on every one; no attribution trailer. Submodule links toggled per
PACKAGE-PREAMBLE around every git/build.
Commits, oldest first:
| sha | what |
|---|---|
| `e7fe3b97` | ServerLoop: C2 (m_running under m_controlMutex + re-check), M-7 (Fatal not app-thread fallback), M-4/codex-11 (effective affinity mask), C7/ID-54 (once-per-tuple make-current, release recorded not forwarded), m-3, m-4 |
| `0244fc4a` | Init.cpp: M-6 (ShutdownSplitRoles stops the apply thread on the early-Start-failure path), m-6 (re-assert consumer mask after step 4) |
| `0fde9535` | Managers.cpp: M-2 (server staged copy is the only base under split), M-3 (whole-store readers through RequireCoverage), m-5 (first cut) |
| `7a925b32` | DirectGLES C6: the seven format-capability reads use the server-private cache under split; #if-guarded so pull is byte-identical |
| `61d6cd8e` | PipeApplier: ID-49 (neutral-pack tight read), m-1 (present 1:1 is a convention), m-7 (reply ordering documented) |
| `05bcb734` | PipeApply.cpp: item-12 (resource_flush_range carries no bytes under split; consumer reads the staged shadow) |
| `a85aa954` | ServerLoopTest: M-1 production gate, C2/M-6/M-7/C6/C7 controls, C10 actual-park arming, M-4 explicit mask, m-2 SIGABRT death |
| `455b475c` | m-5 corrected: null a stale hostBytes only when its shadow is really gone (StagedShadowStore::HasShadow), not on a twin's first-ensure generation sync |
| `c9d84e33` | R-11 completion: a resource_subdata that finds no twin under split MINTS one, so its bytes reach the server shadow (the twin is otherwise lazy until draw, D-A2) |
Files: `MG_Remote/Server/{ServerLoop.{h,cpp},PipeApplier.cpp,StagedShadow.h}`, `MG_Backend/Init.cpp`,
`MG_Backend/DirectGLES/{Managers.cpp,Utils.{h,cpp},BackendObject_DirectGLES.cpp}`, `MG_Pipe/PipeApply.cpp`,
`MG_Test/Wire/ServerLoopTest.cpp`. PipeApply.cpp is p1's / Utils.cpp+BackendObject_DirectGLES.cpp are
DirectGLES: edited under the integrator's explicit C6/item-12 assignment (flagged §"ownership" below).
---
## The confirmed set (each item: what changed, the red-once line)
**ID-49 (codex 1, ReadPixels blocker), server half.** `ServerVerbSink::OnReadPixels` now reads with NEUTRAL
pack state (ROW_LENGTH 0, SKIP_ROWS/PIXELS/IMAGES 0, ALIGNMENT 1, restored after via
`MGPipeApplySetPixelPackState`) into a **tight `w*h*bytesPerPixel`** scratch that IS the reply payload;
`DstSize` is that tight size. **The DstSize formula both sides agree on: `w * h * bytesPerPixel`** (bpp from
`GetInputBytesPerPixel(ConvertGLEnumToTextureInputFormat(Format), …DataType(Type))`, the same helper c1's
`PackedReadbackBytes` uses). A non-default server-visible pack state can no longer change the reply's size or
bytes. **Red once:** the two `DepthReadbackHonoursThePackPixelStoreParameters` SEGFAULTs — round 1's census
omitted them — are the pre-fix state; the round-2 joint inproc census has **0 SEGFAULT** and that case now
**passes** (probe `p5-v1-probe-depthpack_ok`). End-to-end control is the joint, per the charter.
**M-1 / codex 8 — R-11 production gate.** New `StagedShadowProductionTest.SubDataThroughTheRealOpsTable…`
installs the real ops table (`BufferImpl::RegisterBufferBackendOps`), drives
`MGPipeGetResourceOps()->SubData` — the exact `Managers.cpp:2118` call site — mints the twin, then poisons the
source and reads `FindBufferResourceForHandle(res)->hostBytes`. **Red once (redcheck `m1-subdata-site-raw-offset`):**
restore `hostBytes = raw - offset` → `kept a pointer into SEG_STAGE`. `StagedShadowTest` stays the container
test; this is the gate.
**M-2 / codex 3 (item 3) — the audit becomes real.** `liveHostBase()` no longer falls back to the client
object's `MappedData()` under an active transport — the server's staged copy is the only base (R-2 / table 3).
This exposed and drove two further fixes (see §"the M-2 cascade" below). **Red once:** under
`MOBILEGL_IPC_AUDIT=1` the joint reduced-path lane is **21/21**; reverting M-2 makes the draw read `MappedData`
again (proven — reverting it turned the blue triangle green), so a corrupt staged upload could no longer be
seen. With M-2 the draw reads the shadow, and audit-green means no 0xDD reached a draw.
**M-3 / codex 4 — whole-store readers.** The pool-reuse upload (`Managers.cpp:~2993`) and the whole-store
respecify (`~3068`/`~3086`, `initialData != nullptr`) now call `MGL_SERVER_STAGED_REQUIRE(... "pool_reuse_whole_store"` /
`"respecify_whole_store")` before uploading. No-op for the legacy `MappedData()` arm and for a non-copying
store; a sparse shadow forced through storage recreation dies by name. **Red once:** the unit
`StagedShadowTest.ADrainOutsideTheStagedCoverageIsFatalByName` (now an `EXPECT_EXIT(…, KilledBySignal(SIGABRT), …)`
+ the `Fatal{StageSnapshotTooNarrow}` log grep) is RequireCoverage's own control; the production wiring is
exercised end-to-end on the joint. ID-41: no edit inside `FlushPendingRangesFrom`; both G5 scripts stay as
b1 re-pinned them (I touched no pinned body).
**m-5 / codex 5 — DropAll and the freed base.** `DropAll` frees shadows; a twin surviving context loss kept a
`hostBytes` into freed memory. Nulled in the generation-reset block — **but only when the shadow is really
gone**, keyed on the new `StagedShadowStore::HasShadow(key)`, because that block also runs on a twin's FIRST
ensure and an unguarded null dropped a base a subdata had just staged (this is what first broke the reduced
path — §"the M-2 cascade"). **Red once (unit `StagedShadowTest.HasShadowIsTrueAfterAdoptAndFalseAfterTheShadowIsDropped`):**
Adopt → HasShadow true; DropAll → false, both asserted.
**M-4 + codex 11 — affinity gate.** `ApplyAffinity` now returns the EFFECTIVE mask read back with
`sched_getaffinity`, and the log/`ResolvedAffinityMask()` report it, not the requested one. New
`TheExplicitAffinityMaskResolvesToItselfAndIsLogged` (`"0x3"` → `0x3`, log `RESOLVED mask 0x3`), and the `off`
case now also asserts `off` was a RECOGNISED string. **Red once (redcheck `resolved-mask-not-logged`):** rename
the log token → `did not log its resolved affinity mask`. Deleting `off`'s branch makes off unrecognised → the
`off` case's new assertion goes red.
**M-6 — teardown on early Start failure.** `ShutdownSplitRoles()` now calls `Server::ServerLoopInstance().Stop()`
itself after `ClientSession::Stop()`; Stop is idempotent (its `!joinable` arm drops a backend that never ran a
thread). **Red once (unit `StopWithoutAStartedThreadStillDropsThePrivateBackend`):** without Stop's reset a
second `CreateBackend` returns `INVALID_ARGUMENT`.
**M-7 + C2 (codex 2) — the mailbox race and the no-thread arm.** `m_running` is now cleared INSIDE the
`m_controlMutex` critical section in `ApplyThreadMain`'s exit, and `RunOnApplyThread` re-checks it under that
same lock before publishing (C2 — the verifier's latch harness proved the pre-fix hang, `CALLER_HUNG 30000 ms`
while `Stop()` returned in 0 ms). The `!m_running` arm returns `NOT_INITIALIZED` when the backend is gone and is
a named `Fatal{ApplyThreadNotRunning}` when a backend exists but no thread does (M-7 — never a silent
EGL-on-the-app-thread fallback). **Red once:** unit
`AForwarderCallAfterTheLoopStoppedReturnsNotInitializedAndDoesNotRunInline` (C2; redcheck `c2-inline-after-stop`
→ `did not return NOT_INITIALIZED`) and the `EXPECT_EXIT` death test
`AForwarderWithABackendButNoThreadIsFatalNotAnAppThreadFallback` (M-7).
**The minors.** m-1: the present↔swap 1:1 comment now says it is a convention c1 upholds (Present is reached
only from a present record; ServerSwapEGLBuffers never calls Present), not structural. m-2: the coverage death
test names the mode — `EXPECT_EXIT(…, KilledBySignal(SIGABRT), …)` + the exact Fatal grep (the log flush is
pinned by `Log.cpp`'s per-write fflush before abort). m-3: `ServerLoop::Start` resets m_drained/m_parks/the C7
counters. m-4: the doorbell claim in the header and `Fatal{ApplyThreadJoinTimeout}` corrected — a Notify
suffices for the stop case (m_stopRequested is in the predicate and published before the ring); Kill is
load-bearing only for a predicate with nothing to see. m-6: `AssertConsumerMaskIsHonest` re-runs after step 4.
m-7: OnReadPixels' reply is written mid-apply and that is safe because the client reaches a slot only through
`appliedSeq` (published after LeaveApplier + the bracket drop), never the slot stamp — documented against
ReplySlot.h rather than reordered across the decoder boundary.
**item 12 (integrator, mid-round) — the last blocker between 14/21 and 21/21.** `MGPipeApplyResourceFlushRange`
(`PipeApply.cpp:1873`) faulted `record.Size != 0 && bytes == nullptr` as corruption; under an active transport
that is the NORMAL case (R-13.2: the flush carries no bytes, the covering subdata already staged them into the
server shadow). Guarded the fault with `&& Transport == Monolith` (a constexpr `true` in a non-disaggregated
build, so monolith codegen is unchanged); a genuinely-missing subdata is now caught more precisely downstream
by R-11's `Fatal{StageSnapshotTooNarrow}`. **Red once (executed on the joint):** dropping the `&& Monolith`
guard turned `integration-split` 21→14 and `DirectGLES.Split.PersistentCoherentMapScenario.AWrite…` aborts
`Subprocess aborted` (the exact c1-v2 §6 Fatal); restored, back to 21/21.
---
## The five codex verifier findings (all CONFIRMED; `v1-codex-verify.md`) — all fixed this round
- **C2** — fixed with M-7 (above). Control: the deterministic post-Stop case; the verifier's latch harness is
cited as the race-closure evidence (the fix works by mutual exclusion, so a shipped latch cannot straddle
the pre/post-fix line — I ship the deterministic half and cite the harness).
- **C6** — `ActiveBackendFormatCaps()` (new, `#if MOBILEGL_BUILD_DISAGGREGATED`) returns the server's own
backend cache under split, the process global otherwise; the seven reads (Utils.cpp ×5,
ClampSamplesToBackendSupport ×2) route through it, each `#if`-guarded so the pull build is byte-identical
(G1 confirms). Control `TheServerFormatCapsAccessorReadsTheServersOwnBackendNotTheGlobal` (pointer identity:
the accessor returns `&server->GetFormatCapabilities()`, not the global's); redcheck `c6-accessor-reads-global`
→ `returned the process global's cache`. c1 owes nothing beyond routing through the forwarders it already
owes. The wrong *value* is joint-only (needs c1's BackendObject_Remote linked), as the verifier said.
- **C7 / ID-54** — `ServerMakeEGLCurrent` binds natively once per (dpy,draw,read,ctx) tuple, treats an
identical repeat as a no-op (and skips the R-12 republish), and RECORDS a client release-current without
forwarding it (the context is held for life). `NativeBindCount`/`ClientReleaseCount` diagnostics added. The
comment was NOT struck (charter §2 stands). Control: the pure decision
`MakeCurrentClassifiesRepeatAndReleaseWithoutRebinding`; the instrumented native-bind COUNT is a joint-lane
control (needs a real EGL context).
- **C9** — the redcheck runner now carries an `expected` regex per perturbation and accepts a control ONLY when
it went red AND that regex matched; a **C9 META-CONTROL** asserts the runner rejects a stubbed
`(8, "unrelated fixture failure")`. Run: META-CONTROL **PASS**, all **14** perturbations red for their own
reason, `P5_V1_REDCHECK_ALL_WENT_RED`.
- **C10** — the park tests arm on `RingControl::consumerParked` (set only inside the blocking Park), not
`ParkCount()`. New fixture `WaitUntilTrulyParked()`; the three park cases use it. Redcheck
`c10-wait-never-blocks` (replace the Wait with `true`) → `never entered the blocking park`.
---
## The M-2 cascade (why the milestone briefly regressed, and the two fixes it forced)
M-2 is correct and required, but it removed a crutch: with `liveHostBase()` reading the server shadow instead of
the client's `MappedData()`, the reduced-path VBO rendered a **shifted/empty** store (TriangleScenario's interior
came back blue, not green — `aColor` read one float early). Two real gaps that inproc-reading-`MappedData` had
masked, both now fixed:
1. **m-5's null was too broad** (`455b475c`): the generation-reset block runs on a twin's FIRST ensure
(generation starts mismatched), and my first m-5 cut nulled a `hostBytes` a subdata had *just* staged in the
same generation. `HasShadow` discriminates freed-on-context-loss from live-just-staged.
2. **the twin is lazy** (`c9d84e33`): `Ops_H_Create` is a no-op (D-A2), so a buffer's twin is minted at the
first DRAW — *after* its `resource_subdata`. Under monolith that is fine (`MappedData` is the source); under
split the subdata found no twin, returned early, and the bytes were lost. So under an active transport a
subdata that finds no twin now MINTS one (`GetOrCreateBufferResourceForHandle`) to stage into. Monolith keeps
the lazy twin.
Diagnosed by instrumentation, not assertion (`p5-v1-joint2-tri.log`: `Ops_H_SubData bytes=0x… (non-null)` then
`ensure hostBase=(nil) hasShadow=0`).
---
## M-5 triage — the joint inproc census, by family, one probe each
Census on the round-2 joint (`MOBILEGL_TRANSPORT=inproc ctest -L integration-gpu`, `p5-v1-joint2-inproc.log`):
**575 passed / 183 skipped / 551 subprocess-aborted / 23 failed / 0 SEGFAULT** (round 1 was
428/185/506/28/**2**). The 551 aborts are the class-C `Fatal{UnmigratedVerb}` slots the reduced path does not
serve (BRIEF §4). The 23 wrong-answer failures — none is v1's; the reduced-path lane is 21/21:
| family | count | probe (one line) | owner / class |
|---|---|---|---|
| `LayeredAttachmentShapeScenario` (3D / cubemap-array attachments) | 12 (6 GLES + 6 VK) | `…:507 Failure`, no Fatal — wrong pixels, ran the split path | reduced-path-by-design: layered framebuffer attachment is P4b/P7, not the six verbs. c1 emit + shared backend |
| `DepthStencilReadbackMatrixScenario.GetTexImageReadsAPackedDepthStencilTexture` | 3 | `…:530 Failure`, no Fatal — wrong equality | get_texture_image of a PACKED depth/stencil format; not the E2 tight-RGBA8 read. Shared backend readback conversion / c1 emit |
| `PrimitivesGeneratedNoXfbScenario` (+PrimGenReroute) | 3 | **passes standalone** (`p5-v1-probe-primgen` rc=0) | flaky under `-j 6` — an XFB/query scenario contended between concurrent inproc processes, not a deterministic split defect |
| `HandleRecycleScenario.AFramebufferAtARecycledAddress…` | 3 | **passes standalone** (`p5-v1-probe-handlerecyc` rc=0) | flaky under `-j 6` (same shape); the framebuffer handle-recycle family is c1/shared, not v1 |
| `TextureParamsWithoutASamplerViewScenario` (attachment-only swizzle) | 1 | `…:415 Failure`, no Fatal | set_texture_params swizzle; c1 emit + shared backend |
| `P4aFinalFixScenario` (renderbuffer+framebuffer deleted) | 1 | `…:420` draw-after-delete reads black, no Fatal | P4a framebuffer/renderbuffer deletion debt; c1/shared |
None aborts (no Fatal), so all ran the split path; the six-verb reduced path is not among them. §8's round-1
"what passes" list is corrected below.
---
## Report corrections (codex "claims not supported")
- The round-1 inproc census is **428 / 185 / 506 / 28 +2 SEGFAULT = 1149**; the two SEGFAULTs were
`DepthReadbackHonoursThePackPixelStoreParameters` (both backends) — ID-49's shape, now fixed (round-2 census
0 SEGFAULT).
- The 506 (round-1) aborts are attributed to class-C **by one probed example**, not per-abort; stated as such.
- The `VERB_BARRIER=0` / `STRICT_ERRORS=1` failure NAMES come from the MobileGL log FILES (`MGLOG_F` →
stdout + file, not ctest's gtest capture), cited as such; ctest logs hold no `Fatal{` lines.
- "No 0xDD reached a draw" is now ESTABLISHED (not inferred): M-2 makes the server shadow the draw's only base,
and the joint reduced-path lane is 21/21 under `MOBILEGL_IPC_AUDIT=1`.
- "The thread really PARKED first" now rests on `consumerParked` (set inside the blocking Park), not on
`ParkCount()` which increments before the wait (C10).
- §8's "what passes … HandleRecycle.*" was wrong; corrected in the M-5 table (HandleRecycle passes standalone,
flakes under -j).
---
## The four self-tests (final head `c9d84e33`, `p5-v1-gate.sh`, log `p5-v1-r2-gaterun2.log`)
| | pull | push | verify | split |
|---|---|---|---|---|
| build | rc=0 | rc=0 | rc=0 | rc=0 |
| `ctest -L unit` | **1814/1814** | **1814/1814** | **1814/1814** | **1994, 1 known red** (`PipeWireCodecTest.NoReplyIsWrittenForARowTheCatalogueGivesNoReplySlot` = c1's ID-31) |
- **G1** (pull vs `p5-before-libMobileGL.so`, marker `4595163c`): `.text 10806611 → 10806611 (+0)`,
`27814 → 27814: 0 added, 0 removed, 0 resized, 0 renamed`. `nm pull MG_Remote = 0`, split = 431. C6's
`#if MOBILEGL_BUILD_DISAGGREGATED` guards are what kept the pull build byte-identical.
- **Integration:** push `integration-gpu` monolith **1128/1128**; split `integration-gpu` monolith
**1149/1149** (the 7 ID-42 PersistentCoherentMap reds retired by the merged b1/s1 round 2).
- **G2** pull-vs-push 0 diff lines; **G14** 0 removed (40 added pull/push = wave 1's; 201 split-only = wave-1 +
v1's two suites). **Flake sweep** v1's suite ×100 under `-j 8`: 100% (18 cases).
- **Red-check** (`p5-v1-redcheck.py`, C9-hardened, log `p5-v1-redcheck.log`): META-CONTROL PASS,
`P5_V1_REDCHECK_ALL_WENT_RED`, 14 perturbations each red for its own asserted string.
## The joint (`p5/v1-joint2` = `c9d84e33` + `p5/c1@41992649` merged, + the two merge-time hunks)
- `git merge p5/c1` was CONFLICT-FREE. **`integration-split` under `inproc`: 21/21** (19 pass + 2 design skips —
the persistent-map counting-lane gate). **The first IPC frame renders**, and so do the whole VBO-triangle and
persistent-map families — a strict superset of c1's 14/21. Under `MOBILEGL_IPC_AUDIT=1`: also **21/21**.
- The two hunks the integrator applies at the real merge (they need c1's headers, so they are NOT on `p5/v1`):
the direct `MakeUnique<BackendObject_Remote>()` in `Init.cpp` and the `ScopedApplierEntry` bracket at the top
of `PipeApplier::ApplyOne`. Applied and verified on the joint scratch (`p5-v1-joint2-hunks.py`). The weak
`CreateRemoteBackendObject` placeholder is deleted on the scratch (static-archive link); on `p5/v1` it stays.
## What c1 must provide
1. The two merge hunks above are the integrator's, not c1's — c1's round 2 supplied the headers they need.
2. Route `BackendObject_Remote`'s `MakeEGLCurrent`/`InitCapabilities` through v1's `ServerMakeEGLCurrent` /
`ServerInitCapabilities` forwarders (the R-12 republish); c1 already owes this.
3. Nothing else for the reduced path — it is complete at 21/21.
## Ownership note
`PipeApply.cpp` (p1's) and `Utils.cpp`+`BackendObject_DirectGLES.cpp` (DirectGLES) were edited under the
integrator's explicit assignment: item-12 (the flush-range consumer half is v1's, coordinator message) and C6
(ID-52 ruling, "you own the conversion of those seven reads"). Both are `#if`-guarded to keep the pull build
byte-identical (G1 confirms) and monolith codegen unchanged.
## Debts
- The 23 non-reduced-path inproc wrong-answers (M-5 table) are P4b/P7 + c1/shared-backend, and two families flake
under `-j`; owners named, none v1's.
- ReadPixels' client-side scatter is c1's (ID-49); v1 reads tight and neutral.
- `OnDrawVbo` declines instancing/multi-draw/indirect (P8); `OnBlit` uses the bound framebuffers; `OnReadPixels`
has no PBO destination (b1's).
- The C7 native-bind COUNT and the C6 wrong-value are joint-lane controls (need a real EGL context / c1's object).
- `rsp` is a floor for non-verb records (needs p1's "enter without stamping" entry point).