Files
MobileGL/docs/Disaggregated/notes/p5/p5-results/w1-v1.md
T
swung0x48 aba329cc53 [Docs] (Disaggregated): P6's plan and contract draft; the working notes move from a local directory into the repository
Two things land together: the plan and contract draft P6 needs before it starts, and the
working notes that until now existed on exactly one machine.

1. P6's plan and contract draft (neither started; not one line of code)

     docs/Disaggregated/P6-SPAWN-PLAN.md      the package plan
     docs/Disaggregated/P6-CONTRACT-DRAFT.md  the contract draft (moves to
                                              MG_Remote/CONTRACT-P6.md when c6 lands)

   The three findings that mattered, all written into the documents:

   a. P6 is SMALLER than its roadmap cell suggests. The transport primitives are already in
      the tree and already tested: SocketDoorbell including hangup detection and the two
      lost-wakeup fences (Doorbell.cpp:211-360, four cases in FdPassingTest), SCM_RIGHTS fd
      passing, ShmSegment::Adopt, and the SurfaceOp / WindowKind schema already defined in
      protocol.fbs. inproc uses a real second shm mapping rather than an alias, and
      SessionRings.h:22-28 says that was done so today is not the first time it runs. P6 is
      assembly plus a control plane plus a process, not "write a socket transport".

   b. The evidence for "P6 is only a transport swap" has EXPIRED. It was bought by P5c, whose
      audit found 59 direct memory accesses in a tree that already looked clean. P5e changed
      the tree again. So the first package, a6, is a read-only audit that writes no code, and
      every contract row that depends on it is marked PENDING a6 rather than decided in
      advance.

   c. P6 does not get a game running on the phone. ANativeWindow* is meaningless in the child,
      and protocol.fbs:186-196's own comment says Android transfers the window out of band -
      that band is P12. P6 lands pbuffer / surfaceless / offscreen, which is what HeadlessGL,
      the split-equivalent lane and trace replay need, and which is exactly what P6's own exit
      gate measures. A real window reaching the server is a named refusal. This gets its own
      section because "P6 is done" would otherwise read as "the game runs now".

   The contract adds rule G: a role may not name a process-local handle. It extends P5c's
   rule E (a role may not name the other role's memory) from memory to identity - EGLDisplay /
   EGLSurface / EGLContext / ANativeWindow* / fd number / pid / mapped address either cross as
   a token this contract defines or they do not cross, and a descriptor crosses only through
   SCM_RIGHTS, where the kernel renumbers it.

   What P5e left for P6 gets its own section. The latch semantics are already designed
   (ARCHITECTURE.md:504) and are copied unchanged. What is new is that after run-ahead the
   client no longer waits while the peer process can DIE: the client holds records that were
   published and will never be applied, and Android's freezer suspends the child without
   closing anything. So slow and dead must be distinguishable WITHOUT a timeout threshold.
   That is P6's only new design; everything else is carriage. It shares a predicate with E1's
   outstanding re-specification (ID-122), and the two should be designed together.

   Two gates carried over from P5e's lessons: G2 requires integration-spawn's test-name set to
   match integration-split's name for name, because a differing set means the lane is quietly
   running less; and the arm-proof gate requires every entry to record the child pid and
   transport=spawn in its OWN private log - ID-124 established that this kind of false green
   really happens, and a spawn lane that actually ran monolith would pass every other gate.
   S2's shape must be settled before it is written, and the document says why: E1 is the
   cautionary case.

2. The working notes move into the repository (docs/Disaggregated/notes/, 415 .md files)

   ROADMAP / CURRENT_STAGE_PROGRESS / MEASUREMENTS cite ruling numbers, audit line numbers and
   the provenance of gate figures from ~/w7/notes/, a directory that exists on one machine and
   is under no version control. A record cited hundreds of times that lives only locally is not
   a record, so it moves in whole, with its directory structure intact.

   Only .md moves. Scripts, run logs, census and gate result JSON, patches, APKs and nm dumps
   stay where they are: they are evidence DATA rather than documents - large, regenerated per
   run, and mostly carrying absolute paths and device serials that mean something only on the
   machine that produced them. The mapping and this split are written in
   docs/Disaggregated/notes/README.md.

   Two exceptions are recorded there too: P6's BRIEF has been promoted to a first-class
   document and keeps no copy under notes/, so the two cannot drift apart; and recovered/ holds
   material salvaged from failed workflow runs, which may duplicate the like-named phase
   directories and is kept because a few of those were the only complete version at the time.

   CI's citation lint takes docs/Disaggregated/*.md only (non-recursive), and notes/ is
   deliberately outside it: those notes' file:line citations were written against their own
   code heads, and resolving them against today's HEAD would fail wholesale - which is not an
   error, it is their correct state as a historical record. To check one, use the revision it
   declares (--rev).

3. Inconsistencies fixed in passing

   - README.md's status line still said P5e was in progress while the stage table already said
     closed.
   - CURRENT_STAGE_PROGRESS's next-step item 1 still listed the exit gate as pending, though
     (a) and (b) were run in the previous commit; rewritten to closed plus the one item that
     actually remains (E1).
   - ROADMAP's P6 row still had the pre-phase plan in three of its cells; it now carries the
     package order, the scope boundary and the two new gates.

Checked with check_doc_citations.py: 125 citations across docs/Disaggregated/*.md, 0 problems.
2026-09-18 23:31:39 -04:00

513 lines
36 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# P5 w1 — the G3 wire codec
Branch `p5/w1`, worktree `~/w7/p5-w1`, base `feat/disaggregated @ 5175f9cc`.
Commits **`bc969f9a..4f96e810`** (9, the last one the review's fix round - see §10), not pushed. 7 files, +3622 / −132;
**`MG_Pipe/PipeApply.{h,cpp}` are untouched** — `git diff 5175f9cc..HEAD` over them is empty,
which is the decoder's "no semantics" rule stated as a number.
| commit | what |
|---|---|
| `bc969f9a` | the codec: `SegmentTable`, the R-2 honesty arms, `PipeWireEncoder`, `PipeWireDecoder` and its 71 arms |
| `996bb591` | `gen_pipe.py` + `PipeWire.inc`: `MGPipeApplyWireRecord` hands off to the installed decoder |
| `8f9f0e99` | `CapsCodec.cpp`: the two blob serializers and `CapsAbiFingerprint` |
| `1ea5db54` | `MG_Test/Wire/PipeWireCodecTest.cpp` (new) + its CMake registration |
| `f6a84ae4` | an empty var-tail needs no placeholder entry |
| `5dc8fe1c` | a comment's arithmetic corrected (~16 KiB of dense caps table, not half a megabyte) |
| `c1b5c2a0` | **the decoder stops writing `RingControl`** — s1's correction, §8 |
| `52bf9acc` | **declared padding crosses byte for byte** — b1's correction, §8 |
| `4f96e810` | **the review's fix round** — §10 |
---
## 1. What landed, against BRIEF §5 w1's seven items
1. **The encoder.** `StageBytes` / `EncodeRecord` / `Publish` / `EmitSeq` / `MaxRecordBytesSeen`
as c0 declared them, plus three additive members the work needed — a two-tail `EncodeRecord`
overload, `ReclaimStagedBytes()` and `StagedBytesInFlight()`. **No signature c0 shipped
moved**, so the other six packages compile against exactly what they compiled against on
day one.
2. **The decoder.** `MGPipeApplyWireRecord`'s skeleton is gone: every arm now bounds-checks,
cross-checks its tail, resolves its segments and calls an existing `MGPipeApply*` free
function, the verb sink, or `false`. **`MG_Pipe/PipeApply.cpp` is unchanged** — `git diff`
over it is empty.
3. **The tail cross-check.** `MGPipeWireRecordLayout(op, payload, out)` computes the record's
total from the payload's own count fields; the **encoder** refuses a caller whose tails
disagree and the **decoder** refuses a record whose `MGPWireRecHeader::Size` disagrees. One
arithmetic, two readers, so the two sides cannot drift.
4. **`SEG_STAGE`.** A linear allocator over the segment, encoder-local, reclaimed behind
`RingControl::retiredSeq` (which this package only READS), with the `0xDD` fill under
`MOBILEGL_IPC_AUDIT=1` (the decoder reads `MG_Config::Ipc.Audit` at construction).
**`RingControl`'s stage cursor triple is written by nobody in P5** — §10/M2.
5. **R-2's four Fatal arms**, all through `WireProtocolFatal*`, each with a forked death case.
6. **`MGPCaps`'s two blob serializers** + `CapsAbiFingerprint()`.
7. **R-10's proof obligation**: `MaxRecordBytesSeen()` on the encoder.
---
## 2. Acceptance numbers
Numbers below are from `52bf9acc`. **They were all re-run at `4f96e810` after the fix
round and are in §10** — G1 identical, split unit 1902/1902, 60 cases; nothing regressed,
the counts grew.
**The four self-tests.**
| # | gate | result |
|---|---|---|
| 1 | **G1** (`symbol_report.py --threshold 0`, pull build vs `~/w7/p5-before-libMobileGL.so`) | **`.text` 10 806 323 → 10 806 323 (+0, +0.000%)`; 27 811 → 27 811 defined symbols, 0 added / 0 removed / 0 resized / 0 renamed.** `nm --defined-only build-linux/libMobileGL.so \| grep -ic MG_Remote` = **0**; the split build = **215**. |
| 2 | `ctest -L unit`, four directories | pull **1785/1785**, push **1785/1785**, verify **1785/1785**, split **1893/1893** (1842 + my 51) |
| 3 | `ctest -L integration-gpu` under `MOBILEGL_TRANSPORT=monolith` | build-push **1117/1117**, build-split **1117/1117**. G14: integration-gpu names present in build-push but not at the P5 baseline = **0**. |
| 4 | `gen_pipe.py --check` / `--self-test` | up to date; **9 negative-control trips, positive control OK** (unchanged from c0). `gen_pipe_dirty_surface.py --check` rc=0, `check_include_closure.py` 4 probes / 0 problems. |
Builds: `build-linux`, `build-push`, `build-verify`, `build-split` all rc=0, each with a
`CTestTestfile.cmake` present before any verdict was believed (C-10).
**The new suite.** `MG_Test/Wire/PipeWireCodecTest` — **51 cases**, all green. Coverage:
one round trip per flag class (`kNone`, `kHasBlob`, `kVarTail`, `kHostSpan`, `kReplySlot`,
`kNeedsAck`, `kOptional`), both double-tailed rows, `DrawVbo` with and without its conditional
tail, `CreateShaderState`'s seven blob members, `SetResidualValueState`, `CreateSamplerState`'s
`borderColorForm`, both caps blob codecs with every truncation, and **twelve forked death
cases** — the four R-2 arms plus the half-declared blob, the two tail-cross-check shapes, the
mask/count disagreement, the host-span-count rule, the pad record, the declared per-stage
SPIR-V run and the second process resolver. Death cases fork and read the Fatal line out of a
per-pid log file; **no `EXPECT_DEATH`** (PipeInputsTest's rule).
**One flake, named as a flake.** The gate's first pull-lane run reported 1 failure out of 1785
and did not name it; **four subsequent runs of the same lane were 1785/1785**. It cannot be
mine: G1 says the pull binary has `.text` +0 and 0 symbols added, i.e. this package contributes
literally no code to a pull build. Not chased further.
**And one that was my harness, not the tree.** An earlier `integration-gpu` run on build-push
reported 8 failures, all `SIGHUP`, all in `AdvertisedLimitsScenario` /
`XfbAfterClipDistanceScenario`. They died because I had launched ctest from a detached shell
whose parent exited; run in the foreground the same lane is **1117/1117**. Recording it beside
C-10's lesson: a P5 script that reports a test result from a backgrounded `wsl.exe -e bash -c
'... &'` can manufacture failures as easily as the earlier one manufactured passes.
---
## 3. The finding I would most like the other packages to read
**`MGPWireRecHeader::Flags` and `Transport::RingRecordHeader::flags` are the same 16-bit
field, and the two flag spaces collide.**
- `MG_Pipe::kVarTail` is `1<<2`. `Transport::kRecPad` is `1<<2`.
- `MG_Pipe::kHostSpan` is `1<<3`. `Transport::kRecBorrowSlot` is `1<<3`.
- `MG_Pipe::kReplySlot` is `1<<4`. `Transport::kRecVarTail` is `1<<4`. **(Found by the
reviewer, not by me — §10.)** Five of the six call-flag bits alias a ring bit; only
`kOptional` is free, and only because `RingRecordFlags` has not reached `1<<5`.
`PipeWire.inc:37` documents the field as *"MGPipeCallFlags of the call, for asserts and
tracing"*, which invites an encoder to stamp `MGPipeCallFlagsFor(op)` straight into it. Doing
that makes `RingConsumer::Pop` **skip every one of the nine `kVarTail` records as a wrap
filler** — silently, with no checksum anywhere on this ring — and makes both `kHostSpan` rows
look like slots borrowed into the GPU timeline, so they would retire on
`completedFrameSerial` instead of on apply.
**The encoder translates**: `kNeedsAck -> kRecNeedsAck`, `kHasBlob -> kRecHasBlob`,
`kVarTail -> kRecVarTail`; `kReplySlot`, `kOptional` and `kHostSpan` have no ring bit and get
none. The call's own flags are read from the **opcode**, through `MGPipeCallFlagsFor` — which
is what c0 generated that table for. The `static_assert`s in `PipeWireCodec.cpp` now pin
**both enums exhaustively** — each full mask, the exact overlap, every colliding pair by name,
and the completeness of the translation — and
`PipeWireCodecTest.AVarTailRecordIsNotSkippedAsAWrapFiller` is the trip wire.
**Ask for c0:** `PipeWire.inc`'s comment on `MGPWireRecHeader::Flags` should say *ring framing
flags* rather than *MGPipeCallFlags of the call*. It is a one-line generator edit and it is the
comment that would have caused the bug.
---
## 4. Rulings I made, each with what would overturn it
**W-1 — `CreateShaderState` crosses as ONE blob, not seven.** `EncodeProgramArtifacts` already
serialises `SpirvArtifacts::generatedSpirv`, i.e. every stage module, so `Reflection` names the
whole archive and `Spirv[0..5]` stay all-zero ("absent"). A declared `Spirv[i]` is
`Fatal{ProtocolCorruption}` rather than ignored, for `resource_flush_range`'s reason (contract
§6.3): a second, forgeable way to say the same thing is worse than no way at all. Evidence that
this costs nothing: `PipeApply.cpp` reads **neither** `desc.Spirv[]` nor `desc.Reflection` —
`MGPipeApplyCreateShaderState` takes `(desc, link*, spirv*)` and reads the modules out of
`spirv->generatedSpirv`. Shipping them twice would double the largest record in the catalogue
for a reader that does not exist.
*Overturn:* a measurement showing per-stage runs beat one archive — and then the archive has to
stop carrying `generatedSpirv` **in the same change**, or the two disagree.
**W-2 — every var-tail starts 8-byte aligned inside the record; the encoder zero-fills the
gap.** Eight of the nine `kVarTail` rows are already aligned by construction. `DrawVbo` is not:
`MGPDrawRange` is **twelve** bytes, so an odd `NumDraws` leaves the conditional `MGHostSpan`
(a pointer and two `Uint64`s) on a 4-byte boundary. P5 emits no host span at all, so the rule
costs nothing now; it is stated now because the phase that arms `kDrawHasUserIndices` would
otherwise discover it as a misaligned load on a device. Pinned by
`DrawVboConditionalSpanTailIsEightAlignedAndSurvives`.
*Overturn:* nothing — it is free. (Contract table 1 row 16 already rules that the span **stays
in the tail** in P5, which this implements unchanged.)
**W-3 — `SEG_STAGE` is reclaimed by the ENCODER, behind `retiredSeq`, from marks it holds
itself.** `SEG_STAGE` is client-owned memory (table 1: *"client stages, server copies"*) and the
server only reads it, so the client is both producer and reclaimer; what it may not do is free
ahead of `retiredSeq`, which is R-11's whole content on this side. The record carries no field
saying where its staged bytes end, so the encoder remembers `{seq, stage cursor}` per record
and releases up to the newest mark the server has retired. **This does not depend on the verb
barrier**, so it keeps working when R-1's barrier retires family by family.
*Overturn:* a phase that lets the server borrow a staged run into the GPU timeline — then the
mark has to move to `completedFrameSerial` and the two stage tails stop being published
together.
**W-4 — the `0xDD` audit fill is the DECODER's, and it is exact.** R-2.5 says "the server fills
a retired record's SEG_STAGE bytes"; the thing that knows which bytes those are is the decoder,
which resolved them. It records up to eight runs per record (`CreateShaderState`'s seven blob
members plus a tail) and memsets exactly those after the applier **returns**, leaving the stage
ring's own record headers intact. `PoisonedStageBytes()` is the counter a t1 lane asserts is
non-zero with the knob on — an instrumentation that cannot be observed to have run is
decoration.
**W-5 — `CheckBlobIsHonest` could not carry R-2 arm 2, so arm 2 is its own function.** Whether
a record *carries content* is a property of its OTHER fields (`ChunkMask`, the destination
range, the stage mask), which `(op, blob)` does not see. So `CheckBlobIsHonest` implements arms
3 and 4, `RequireDeclaredBlob` is arm 2, and the decoder's per-op arm calls the second one
exactly where the payload says content is implied.
**And it needed a third shape the brief does not name.** A blobref is honest when it is either
fully declared or **all three fields zero**. `Seg` or `Offset` non-zero with `Size == 0` is
neither — and it is precisely what a MONOLITH emitter writes today (`Seg = None`,
`Offset = (Uint64)&link`, `Size = 0`, e.g. `ProgramEmit.h:180-195`). Reading that as "absent"
would **silently drop the bytes of every record an unconverted emitter sent**, so it is Fatal.
Pinned by `AHalfDeclaredBlobIsFatalRatherThanReadAsAbsent`.
**W-6 — `map_persistent` declines WITHOUT calling the applier.** Two reasons and the second is
the finding: the answer is a host pointer, which cannot cross; and
`MGPWireRec_MapPersistent`'s payload is a bare `MGPHandleOnly`, so **the record carries neither
the `size` nor the `seedBytes` that `MGPipeApplyMapPersistent(handle, size, seedBytes)`
takes**. R-6's constant decline is what makes that not matter in P5. See §6.
**W-7 — the decoder writes NO `RingControl` field.** It keeps its own `AppliedSeq()` tally
instead. See §8.1: this started life the other way round and s1's landing corrected it.
**W-8 — the three blob length cross-checks that were inert are now computed, not trusted.**
`CreateRenderState` against `MGPipePipelineChunkBlobBytes(desc.ChunkMask)`, `SetDynamicState`
against `MGPipeDynamicChunkBlobBytes(dyn.ChunkMask)`, `CreateSamplerState` against
`sizeof(SamplerParameters)`, `SetResidualValueState` against `sizeof(ResidualValueBlock)`. The
expected length is derived from a field the record itself carries, so it is a check and not a
restatement. `CreateVertexElements`' and `ResourceSubData`'s remain the applier's
(`PipeApply.cpp:1990-1999`, `:702`) — arming `Size` is what makes those live for the first
time, which is the whole point of rule A.
---
## 5. What the other packages have to know
**v1 — three things.**
1. `PipeWireDecoder` **writes no `RingControl` field at all** — `appliedSeq`'s single writer is
s1's `SessionConsumer::ApplyOne`. `PipeWireDecoder::AppliedSeq()` is the decoder's own
tally, kept *beside* the session's on purpose: the two must agree after every record, and
that comparison is what makes R-9's batching ban checkable rather than merely stated. Whoever
publishes `retiredSeq` should publish it together with `appliedSeq` in P5, because nothing in
this phase borrows a ring slot into the GPU timeline — the day something does, that is the
line that splits. **`retiredSeq` is the one watermark this package reads**: the encoder's
SEG_STAGE reclaim is gated on it, so a `retiredSeq` left at 0 ends the first
`MOBILEGL_IPC_STAGE_MB` of staging in `Fatal{RingOverrun, "SEG_STAGE"}`.
**And do not attach a `RingConsumer` to `RingCursorSet::Stage`** — §10/M2.
2. **`Wire::WireVerbSink` is yours to implement.** Contract §7's five class-B verbs — `Clear`,
`Blit`, `ReadPixels`, `DrawVbo`, `Present` — have **no `MGPipeApply*` entry point at all**
(the 37 that exist are the object and state families), so the codec decodes, validates and
hands over a checked argument list, and the backend call is the server's. With no sink
installed those five arms return `false`. `OnReadPixels` is handed the record's seq and the
`ReplySink` because P5 blocks on `read_pixels` and the pixels come back in the reply slot.
3. The server role calls `SegmentTable::InstallProcessResolver()` **before the apply thread
starts** and `UninstallProcessResolver()` **after the join**. A second install is Fatal.
**s1 — two things.**
1. The codec owns no `Doorbell` and must not: `PipeWireEncoder::Publish()` release-stores the
two ring heads and `submittedSeq` and stops there, so a unit case can drive
encoder → ring → decoder with no transport. **Ringing the consumer doorbell after Publish is
the session's line**, and it is publish-then-notify, never the other way round.
2. `CapsAbiFingerprint()` is ready for `Hello`/`Welcome`. It mixes
`sizeof(DynamicBackendParameters)`, `sizeof(MGPCaps)`, `sizeof(GLFunctionsTable)`, both
table dimensions, both codec versions, `MGPWireOp::kOpCount` and the git stamp — the stamp
because two builds with the same sizes can still disagree about a field order, which no
`sizeof` can see.
**c1 / b1 — three things.**
1. `EncodeRecord` cross-checks the tails you hand it **against the counts in your own
payload**, so a `Count` that does not match the bytes is Fatal on the producing side. A tail
whose count is zero may be omitted entirely.
2. `StageBytes(bytes, 0)` is **Fatal**, not an empty ref: "declared no blob" and "declared an
empty blob" must not be spelled the same way on a wire. Stage first, then fill the payload's
`MGPBlobRef` from what it returns — the encoder's honesty pass refuses a payload whose
blobrefs were not filled that way.
3. **`MaxRecordBytesSeen()` is on the encoder; publishing it into `PipeStats` is b1's line**
(`MG_Util/Metrics/PipeStats.*` is b1's file, so w1 did not touch it). R-10's number goes
into MEASUREMENTS from there.
4. **The codec preserves declared padding byte for byte.** One line, because the next package
to put a field in a pad will want it without reading the code: `EncodeRecord` `memcpy`s the
whole payload struct and the decoder hands it to the applier by reference; nothing is built
field by field, no pad is normalised, and there is no assertion anywhere that a pad is zero.
The only `memset`s are strictly OUTSIDE `sizeof(Payload)` — the record's own alignment
padding and the gaps between tails, which are zeroed so a record is a function of what it
declares rather than of whatever the ring last held. Pinned by
`EveryPayloadByteCrossesIncludingTheOnesSpelledPad`, which stamps the whole struct with a
recognisable byte, writes only the fields the decoder validates, and `memcmp`s the lot — it
**names no pad member**, so it will not stop compiling the day someone claims one.
*The one place a pad is minted rather than preserved:* `StageBytes` returns a brand-new
`MGPBlobRef` with `Pad0 = 0`. A phase that puts a field in `MGPBlobRef::Pad0` must set it
**after** `desc.Blob = StageBytes(...)`, not before.
**t1 — one thing.** `MOBILEGL_IPC_AUDIT=1` arms the `0xDD` fill and
`PipeWireDecoder::PoisonedStageBytes()` is what proves it ran. A lane that sets the knob and
asserts nothing has not exercised R-2.5.
---
## 6. Debt this phase leaves, named
1. **`map_persistent` has no carrier for `size` or `seedBytes`** (W-6). The phase that lands a
real one needs an `MGPipeTypes.h` payload change — c0's file — before it can even call the
applier. R-6 hides this completely in P5.
2. **`SetShaderBuffers` (38) and `SetStreamOutputTargets` (39) still have no consumer.** Both
are on BRIEF §4's exclusion list, so inventing one would be a semantics nobody can test this
phase. What they *do* have now is their tail arithmetic — including the
`HostSpanCount is 0 or Count` rule that keeps the two arrays index-aligned — so the phase
that implements them inherits a validated record rather than a validation problem.
3. **`GetCaps` (1) decodes to `false`.** The snapshot is built from the server's live backend
and posted by `ServerSession::PublishCapsSnapshot`; the codec validates the record's two
blobrefs and declines rather than inventing a caps source.
4. **FOUR counts have no GL constant bounding them**, not two (review m6):
`ResourceSubData.RegionCount`, `DrawVbo.NumDraws`, `SetShaderBuffers.Count` and
`SetStreamOutputTargets.Count`. All four are bounded only by the record fitting the ring —
the first two through `Fatal{RingOverrun}` naming R-10 and the integrator, the other two
through `MGPipeWireRecordLayout`'s 32-bit `record.Size` bound and then `MaxRecordBytes()`.
Every other count is held to `kMGPipeMaxVertexAttribs` / `kMGPipeMaxTextureUnits` /
`kMGPipeMaxImageUnits` / `popcount(Mask)`.
5. **`MGPipeApplyWireRecord`'s `remaining` parameter is now inert on the split path**: the
decoder passes `size` for it, because `RingConsumer::Pop` has already proved the record fits
what the producer published. It is kept because the signature is the generator's.
6. **`MG_Pipe/MGPipeHostSpan.h` was not touched.** The resolver install point named in the
ownership table is exercised from `SegmentTable::InstallProcessResolver()`; adding anything
to that header risks a pull-build symbol for no gain (G1).
7. **No chunking** (R-10), and no compression in `CapsCodec` beyond the sparse tables.
8. **`SEG_STAGE` bytes staged for a record that is never encoded are never reclaimed.** The
mark is taken in `EncodeRecord`, so a caller that stages and then abandons leaks staging
until the ring is rebuilt. Nothing does that today (staging is always immediately followed
by the record that names it), and making the allocator transactional would need a
scope object the c0 signatures do not have — so it is written down rather than guarded.
---
## 7. Where I think the contract or the brief is wrong
1. **`PipeWire.inc:37`'s description of `MGPWireRecHeader::Flags`** — see §3. This is the one
I want changed.
2. **BRIEF §5 w1 says "71 条的 encoder".** The encoder is *generic by opcode*: one
`EncodeRecord` plus `StageBytes`, held to a per-opcode layout and a per-opcode honesty pass.
A 71-way encoder switch would be 71 places for a caller to be wrong instead of one, and c0's
signature (`EncodeRecord(op, payload, payloadBytes, tail, tailBytes)`) already says so. What
*is* 71-way is the decoder, because each arm names a different applier.
3. **Contract table 1 row 3 says `CreateShaderState` is "seven independent runs".** It is one —
see W-1. The row's own evidence (the archive codec, and the applier taking two typed
pointers) is what makes it one.
4. **The brief's R-2.3 lists four Fatal arms; there are five.** The half-declared blobref
(W-5) is a real shape, it is what every monolith emitter writes today, and reading it as
"absent" loses bytes silently.
5. **Contract §0 rule A says a content record "must set `Seg` … `Offset` … and `Size`".** True,
but the predicate *"is this a content record"* is per-op and lives in the payload, which is
why `CheckBlobIsHonest(op, blob, segments)`'s signature cannot express it. Not an error in
the contract, but the split into two functions is worth recording so nobody looks for arm 2
inside the first one.
---
## 8. The three corrections that arrived from other packages mid-flight
Recorded in full, with which of them needed a code change, because the integrator asked and
because two of the three are the kind that go green while being wrong.
### 8.1 s1 — the single-record cap, and `appliedSeq` has one writer
> **Superseded in part.** s1's own fix round reversed the cap back to **4 MiB** — `MOBILEGL_IPC_RING_MB` now names the ring and the segment adds a page on top, so the contract is true as written. The reasoning below stands and the numbers are corrected; §10 records the reversal. Part (b), `appliedSeq`, is unaffected.
**(a) The 2 MiB cap needed NO change, and that is the finding.** `CONTRACT-P5.md §5` and
`Config.h` both say 8 MiB of `SEG_CMD` caps a record at 4 MiB; s1 found the 4096-byte control
page at the head of the segment, which makes the power-of-two ring 4 MiB and
`RingProducer::MaxRecordBytes()` **2 MiB**. Nothing in the codec had to move, because **every
comparison already went through `RingProducer::MaxRecordBytes()` and `Capacity()` at runtime**
— there is no constant anywhere in `PipeWireCodec.{h,cpp}` derived from `MOBILEGL_IPC_RING_MB`,
and the two Fatal messages print the live numbers. The unit case now says so in its header.
**And R-10's headroom is much larger than the 2 MiB figure suggests, for a structural reason
worth stating.** `CreateShaderState` is **not** the record most likely to approach the cap — it
is one of the *smallest*. Its record is `8 + sizeof(MGPProgramDesc)` = **200 bytes** whether
the archive is one kilobyte or one megabyte, because R-10's own rule sends every blob through
`SEG_STAGE` and leaves `{Seg, Offset, Size}` in the record. Pinned by
`ABigProgramArchiveDoesNotGrowItsRecordAtAll`: a 32 KiB module archive produces a 200-byte
record and 32 KiB of staging. The archive's bound is `MOBILEGL_IPC_STAGE_MB` (32 MiB) with its
own named `Fatal{RingOverrun, "SEG_STAGE"}`, not the record cap.
What *can* approach 2 MiB is a **var-tail**, and only two of them are unbounded by a GL
constant:
| row | worst case | vs the 4 MiB cap |
|---|---|---|
| `SetSamplerViews` / `SetShaderImages` | 192 units × 24 B + 24 = **4 632 B** | **0.11 %** |
| `BindSamplerStates` | 192 × 8 + 24 = 1 560 B | 0.04 % |
| `SetVertexBuffers` | 32 × 32 + 32 = 1 056 B | 0.03 % |
| `SetVertexAttribDefaults` | 32 × 24 + 16 = 784 B | 0.02 % |
| `SetFramebufferState` (largest fixed payload) | 312 B | 0.007 % |
| **`DrawVbo`** | `NumDraws` × 12 B — **no GL constant bounds it** | 4 MiB ⇒ ~349 000 draws in one `glMultiDraw*` |
| **`ResourceSubData`** | `RegionCount` × 40 B — likewise | 4 MiB ⇒ ~104 800 regions in one record |
| **`SetShaderBuffers`** / **`SetStreamOutputTargets`** | `Count` × 24 B (+ 32 B or 4 B) — likewise | caught by the 32-bit `record.Size` bound and then by `MaxRecordBytes()` |
So **nothing in the bounded part of the catalogue comes within 900× of the cap**, and I do
not think P5 has to chunk. **Four** rows are unbounded by any GL constant, not two (review m6):
`DrawVbo.NumDraws`, `ResourceSubData.RegionCount`, `SetShaderBuffers.Count` and
`SetStreamOutputTargets.Count`. All four are caught — the first two by
`Fatal{RingOverrun, "<call>"}` naming R-10 and the integrator, the last two by the 32-bit
`record.Size` bound in `MGPipeWireRecordLayout` and then by `MaxRecordBytes()` — so all four
are loud rather than silent, but only the first two say R-10 in the message. **A real measurement on OpenRA is still owed** — `MaxRecordBytesSeen()` is the
counter, and publishing it into `PipeStats` is b1's line, so the number reaches MEASUREMENTS
from there, not from here.
**(b) `appliedSeq` needed a real change** (`c1b5c2a0`). The decoder used to publish both
`appliedSeq` and `retiredSeq` after every record, on the reasoning that the decoder is the half
that knows when a record's staged runs stop being read. s1 owns that watermark and has exactly
one writer, so the decoder now writes **no** `RingControl` field and keeps a private tally.
That is the better shape anyway: two counts that must agree make R-9's batching ban
*checkable*, which one count could not, and `TheDecoderWritesNoRingControlFieldOfItsOwn` plus
`AppliedSeqAdvancesByExactlyOnePerRecordAndIsNeverBatched` are the two cases. The test fixture
now plays the session's role, which is also more honest about where the line is.
### 8.2 b1 — `MGPSubData::Pad0` is no longer padding
**No code change was needed; a test was.** The codec `memcpy`s whole payload structs and never
constructs one field by field, never normalises a pad and never asserts one is zero, so b1's
`HasLiveHostWrites` bit crosses intact — see §5's item 4 for the one-line statement, which is
what the next package to claim a pad should read.
What was missing was a **test that could see it**: the round trips compared named fields, and
a codec that zeroed a pad would have gone green through every one of them. `52bf9acc` adds
`EveryPayloadByteCrossesIncludingTheOnesSpelledPad` and `ResourceDescPadsCrossToo`, which stamp
the whole struct and `memcmp` the payload as it lands in `SEG_CMD`. Both deliberately **name no
pad member**, so b1's rename will not break them — a case that said `Pad0` would stop compiling
on exactly the day it became load-bearing.
This is also the second time in one phase that a "pad" turned out to be a field: P5 itself put
`HasRespecifiedLevel` / `RespecifiedUploadTarget` / `RespecifiedLevel` into `MGPResourceDesc`'s
two pads (contract table 1 row 19b). **Two for two is a pattern, not a coincidence**, and a
codec is the wrong layer to have an opinion about which bytes are meaningful.
---
## 9. Retained files
Everything cited above, and nothing else. Re-runnable scripts: `~/w7/p5-w1-gate.sh` (four
directories, generators, purity, G1, the four unit lanes), `~/w7/p5-w1-units.sh`,
`~/w7/p5-w1-final2.sh` (G1 + integration-gpu + the name lists), `~/w7/p5-w1-submodlinks.sh`,
and **`~/w7/p5-w1-redcheck.py`** — the one that breaks each fix in turn and requires its
control to go red (§10); its output is `~/w7/p5-w1-redcheck.log`.
Logs: `~/w7/p5-w1-gaterun.log`, `~/w7/p5-w1-symbols.md`,
`~/w7/p5-w1-unit-{linux,push,verify,split}.log`, `~/w7/p5-w1-itest-{push,split}.log`, and the
two name lists `~/w7/p5-w1-{itest-names-push,split-unit-names}.txt`. Everything else under
`~/w7/p5-w1-*` is deleted.
**Run any of them with `~/w7/p5-w1-submodlinks.sh on` first** (the worktree's submodules are
symlinks into `~/w7/pipe`), and `off` before any git command — git refuses to run at all while
they are symlinks.
**Not run here, and not w1's:** `ctest -L integration-verify`, the retrace sweeps, the device
lane, and any `MOBILEGL_TRANSPORT=inproc` run — there is no session yet to start one, which is
s1's and v1's.
---
## 10. Fix round 1 — the adversarial review, and s1's reversal
Review: `~/w7/notes/p5/p5-results/w1-review-v1.md`, 0 BLOCKER / 5 MAJOR / 8 MINOR / 3 QUESTION.
One commit, **`4f96e810`**, so the range is now **`bc969f9a..4f96e810`** (9 commits).
**How each control was shown to work.** `~/w7/p5-w1-redcheck.py` breaks the FIX — never the
test — rebuilds, runs the case that should notice, requires it RED, restores, and requires
green again. Eight for eight, and the output is in `~/w7/p5-w1-redcheck.log`. Two of them only
worked after the *test* was fixed, which is the part worth recording:
- **M3's first control stayed green.** It encoded and applied one record per iteration, which
drains the mark queue at every reclaim — the single regime the buggy "clear only when front
reaches `size()`" code handled correctly. Priming with one extra record is what makes
`front < size()` permanent.
- **And then it stayed green again**, because `StageMarksHeld()` returned `size() - front`.
That is the number *in flight*, which stays at one or two while the vector behind it grows
for ever. The accessor now reports the storage. A control that measures the wrong number is
indistinguishable from no control.
### The five MAJOR
| # | disposition |
|---|---|
| **M1** | **Fixed.** Added `CheckHostSpanIsHonest(span, const SegmentTable&)` — the old signature is c0's and still compiles for other packages, but it cannot do arm 4 and now says so. Both host-span rows (`SetShaderBuffers`, `DrawVbo`) and the encoder's own pass use the new one. Two forged-input death cases, one per row. |
| **M2** | **Fixed, by deleting the shared cursors from the design rather than arguing about them.** `SEG_STAGE` is now a genuine linear allocator (which is what BRIEF §5 w1 asked for) with encoder-local monotonic head/tail; `RingControl`'s stage triple is written by **nobody** in P5 and stays at its `InitRingControl` values. The reviewer's scenario is closed at the root: there is no cursor to walk backwards. **s1 must not attach a `RingConsumer` to `RingCursorSet::Stage`** — that is now stated in `ReclaimStagedBytes`' header, and the case asserts all three cursors are still zero after a full stage-encode-apply-reclaim cycle. |
| **M3** | **Fixed.** The consumed prefix is erased once it is half the queue — amortised O(1), bounded at twice the records in flight. |
| **M4** | **Fixed, and the contract conflict raised rather than papered over.** `PostReply` now Fatals for any op the catalogue gives no `kReplySlot`, so the pool can never be written outside its own reservation. The four Bool acceptance answers no longer ride a slot; they are recorded and read through `LastAcceptance()` / `AcceptedRecords()` / `DeclinedRecords()`. **This needs an integrator ruling** (see below) — the contract says those four use DECLINED and the catalogue gives them no slot, and w1 cannot move `PipeCalls.def`. |
| **M5** | **Fixed, my side was wrong.** The catalogue row is right: `BufferSubDataResident` is the buffer half only and `MGPipeApplyBufferSubDataResident` takes no regions. The layout no longer shares `ResourceSubData`'s arm, and a non-zero `RegionCount` on the resident op is now Fatal. No integrator ruling needed. |
**The integrator ruling M4 still needs.** `CONTRACT-P5.md` table 0's reply-slot-header row says
DECLINED "is how the four `Bool` acceptance entry points say `false` (R-5)"; `PipeCalls.def`
gives `ResourceCreate`, `ResourceRespecify`, `ResourceSubData` and `SetTextureParams` no
`kReplySlot`. Both cannot be true. Either those four rows gain the flag (and `ReplyPool` sizes
for `ResourceSubData`, the highest-frequency record class in the catalogue) or R-5's answer
rides `kNeedsAck`'s ack path and table 0's sentence is corrected. **The codec is safe either
way today** — it writes no slot it was not given — but c1/b1 need the answer before they can
gate a dirty-flag clear on acceptance.
### The third collision, and the MINORs
**Pinned exhaustively rather than one at a time.** `kReplySlot == kRecVarTail` is real and was
unpinned; it bites in the *reverse* direction of the headline, because the encoder stamps
`kRecVarTail` on nine opcodes. The asserts now cover **the whole of both enums**: each enum's
full mask, the exact overlap (five of six call-flag bits alias a ring bit — only `kOptional` is
free, and only because `RingRecordFlags` has not reached `1<<5`), every colliding pair by name,
and the completeness of the translation, so a seventh enumerator that needs a ring bit is a
build break instead of a silent drop.
| # | disposition |
|---|---|
| **m1** | Fixed — above. |
| **m2** | Fixed: three `offsetof` asserts tie `MGPWireRecHeader` to `RingRecordHeader` field by field. The decoder walks backwards from `payload` to that header, so "both are 8 bytes" was not enough. |
| **m3** | Fixed: `PoisonResolvedRuns` uses the subtraction form, like `Resolve`. |
| **m4** | Fixed: the hook is installed once, from the constructor, beside the resolver it is modelled on; `UninstallApplyHook()` added for teardown. |
| **m5** | **Evaporated.** Staged runs no longer carry a ring header at all (M2's allocator), so there is no kind to collide with `kRingPadRecordKind`. |
| **m6** | Fixed in §6.4 below — **four** unbounded counts, not two. |
| **m7** | Fixed: the header now says what the audit actually covers (at most one run per record today, because tails live in SEG_CMD and the six per-stage runs are Fatal), and overflowing `m_resolved` is Fatal rather than a silent drop. |
| **m8** | Fixed: the encoder rejects a declared `Spirv[i]` too, so it no longer accepts a record the decoder Fatals on. |
### The three QUESTIONs
- **q1 — will the `0xDD` fill survive P6's spawn?** Not fixed, and it should not be fixed here:
it is a contract line, not code. **Ask for the integrator:** `CONTRACT-P5.md` should say that
`SEG_STAGE` may not be mapped `PROT_READ` on the server while `MOBILEGL_IPC_AUDIT` can be
set, or R-2.5's only mechanical control becomes a SIGSEGV the day someone passes
`readOnly = true` to `ShmSegmentPosix`.
- **q2 — is `W && H && D` the right content predicate?** **Fixed.** A box is now either EMPTY
(every extent zero) or WHOLE; a partially-zero extent is `Fatal{ProtocolCorruption}`, and a
non-zero `RegionCount` also counts as content. The old predicate let a record name a real
destination and declare no bytes, with rule A's arm 2 sitting out.
- **q3 — the reclaim had no trigger in this package.** **Fixed.** `Publish()` now calls
`ReclaimStagedBytes()`, so SEG_STAGE's liveness no longer depends on a c1 barrier that does
not exist yet. It still depends on somebody advancing `retiredSeq` — that half is v1's and is
in §5.
### s1's cap reversal: 4 MiB again
The earlier "2 MiB, not 4" correction was itself reversed: `MOBILEGL_IPC_RING_MB` now names the
**ring**, and the segment adds one control page on top, so `MaxRecordBytes()` is **4 MiB** at
the default and `CONTRACT-P5.md §5` and `Config.h` are true as originally written. **No code
changed either time**, because every comparison goes through `RingProducer::MaxRecordBytes()`
at runtime — which is the argument for never deriving a cap from a config constant, made twice
in one afternoon. §8.1's table and headroom figure are corrected below; the largest bounded
record is unchanged at **4 632 B**, so the headroom is **~900×**, not ~400×.
### Acceptance, re-run at `4f96e810`
G1 unmoved (`.text` 10 806 323 → 10 806 323, +0; 27 811 → 27 811 symbols, 0/0/0/0; pull
`MG_Remote` 0, split 225). `ctest -L unit`: pull/push/verify **1785** each, split
**1902/1902** (1842 + 60). `integration-gpu` **1117/1117** on build-push and build-split, G14
name drift **0**. `gen_pipe.py --check` up to date, `--self-test` 9 negative controls,
`check_include_closure` 4 probes / 0 problems. `PipeWireCodecTest` is **60 cases**, 19 of them
forked death controls.