mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-19 07:51:57 +09:00
Two things land together: the plan and contract draft P6 needs before it starts, and the
working notes that until now existed on exactly one machine.
1. P6's plan and contract draft (neither started; not one line of code)
docs/Disaggregated/P6-SPAWN-PLAN.md the package plan
docs/Disaggregated/P6-CONTRACT-DRAFT.md the contract draft (moves to
MG_Remote/CONTRACT-P6.md when c6 lands)
The three findings that mattered, all written into the documents:
a. P6 is SMALLER than its roadmap cell suggests. The transport primitives are already in
the tree and already tested: SocketDoorbell including hangup detection and the two
lost-wakeup fences (Doorbell.cpp:211-360, four cases in FdPassingTest), SCM_RIGHTS fd
passing, ShmSegment::Adopt, and the SurfaceOp / WindowKind schema already defined in
protocol.fbs. inproc uses a real second shm mapping rather than an alias, and
SessionRings.h:22-28 says that was done so today is not the first time it runs. P6 is
assembly plus a control plane plus a process, not "write a socket transport".
b. The evidence for "P6 is only a transport swap" has EXPIRED. It was bought by P5c, whose
audit found 59 direct memory accesses in a tree that already looked clean. P5e changed
the tree again. So the first package, a6, is a read-only audit that writes no code, and
every contract row that depends on it is marked PENDING a6 rather than decided in
advance.
c. P6 does not get a game running on the phone. ANativeWindow* is meaningless in the child,
and protocol.fbs:186-196's own comment says Android transfers the window out of band -
that band is P12. P6 lands pbuffer / surfaceless / offscreen, which is what HeadlessGL,
the split-equivalent lane and trace replay need, and which is exactly what P6's own exit
gate measures. A real window reaching the server is a named refusal. This gets its own
section because "P6 is done" would otherwise read as "the game runs now".
The contract adds rule G: a role may not name a process-local handle. It extends P5c's
rule E (a role may not name the other role's memory) from memory to identity - EGLDisplay /
EGLSurface / EGLContext / ANativeWindow* / fd number / pid / mapped address either cross as
a token this contract defines or they do not cross, and a descriptor crosses only through
SCM_RIGHTS, where the kernel renumbers it.
What P5e left for P6 gets its own section. The latch semantics are already designed
(ARCHITECTURE.md:504) and are copied unchanged. What is new is that after run-ahead the
client no longer waits while the peer process can DIE: the client holds records that were
published and will never be applied, and Android's freezer suspends the child without
closing anything. So slow and dead must be distinguishable WITHOUT a timeout threshold.
That is P6's only new design; everything else is carriage. It shares a predicate with E1's
outstanding re-specification (ID-122), and the two should be designed together.
Two gates carried over from P5e's lessons: G2 requires integration-spawn's test-name set to
match integration-split's name for name, because a differing set means the lane is quietly
running less; and the arm-proof gate requires every entry to record the child pid and
transport=spawn in its OWN private log - ID-124 established that this kind of false green
really happens, and a spawn lane that actually ran monolith would pass every other gate.
S2's shape must be settled before it is written, and the document says why: E1 is the
cautionary case.
2. The working notes move into the repository (docs/Disaggregated/notes/, 415 .md files)
ROADMAP / CURRENT_STAGE_PROGRESS / MEASUREMENTS cite ruling numbers, audit line numbers and
the provenance of gate figures from ~/w7/notes/, a directory that exists on one machine and
is under no version control. A record cited hundreds of times that lives only locally is not
a record, so it moves in whole, with its directory structure intact.
Only .md moves. Scripts, run logs, census and gate result JSON, patches, APKs and nm dumps
stay where they are: they are evidence DATA rather than documents - large, regenerated per
run, and mostly carrying absolute paths and device serials that mean something only on the
machine that produced them. The mapping and this split are written in
docs/Disaggregated/notes/README.md.
Two exceptions are recorded there too: P6's BRIEF has been promoted to a first-class
document and keeps no copy under notes/, so the two cannot drift apart; and recovered/ holds
material salvaged from failed workflow runs, which may duplicate the like-named phase
directories and is kept because a few of those were the only complete version at the time.
CI's citation lint takes docs/Disaggregated/*.md only (non-recursive), and notes/ is
deliberately outside it: those notes' file:line citations were written against their own
code heads, and resolving them against today's HEAD would fail wholesale - which is not an
error, it is their correct state as a historical record. To check one, use the revision it
declares (--rev).
3. Inconsistencies fixed in passing
- README.md's status line still said P5e was in progress while the stage table already said
closed.
- CURRENT_STAGE_PROGRESS's next-step item 1 still listed the exit gate as pending, though
(a) and (b) were run in the previous commit; rewritten to closed plus the one item that
actually remains (E1).
- ROADMAP's P6 row still had the pre-phase plan in three of its cells; it now carries the
package order, the scope boundary and the two new gates.
Checked with check_doc_citations.py: 125 citations across docs/Disaggregated/*.md, 0 problems.
513 lines
36 KiB
Markdown
513 lines
36 KiB
Markdown
# P5 w1 — the G3 wire codec
|
||
|
||
Branch `p5/w1`, worktree `~/w7/p5-w1`, base `feat/disaggregated @ 5175f9cc`.
|
||
Commits **`bc969f9a..4f96e810`** (9, the last one the review's fix round - see §10), not pushed. 7 files, +3622 / −132;
|
||
**`MG_Pipe/PipeApply.{h,cpp}` are untouched** — `git diff 5175f9cc..HEAD` over them is empty,
|
||
which is the decoder's "no semantics" rule stated as a number.
|
||
|
||
| commit | what |
|
||
|---|---|
|
||
| `bc969f9a` | the codec: `SegmentTable`, the R-2 honesty arms, `PipeWireEncoder`, `PipeWireDecoder` and its 71 arms |
|
||
| `996bb591` | `gen_pipe.py` + `PipeWire.inc`: `MGPipeApplyWireRecord` hands off to the installed decoder |
|
||
| `8f9f0e99` | `CapsCodec.cpp`: the two blob serializers and `CapsAbiFingerprint` |
|
||
| `1ea5db54` | `MG_Test/Wire/PipeWireCodecTest.cpp` (new) + its CMake registration |
|
||
| `f6a84ae4` | an empty var-tail needs no placeholder entry |
|
||
| `5dc8fe1c` | a comment's arithmetic corrected (~16 KiB of dense caps table, not half a megabyte) |
|
||
| `c1b5c2a0` | **the decoder stops writing `RingControl`** — s1's correction, §8 |
|
||
| `52bf9acc` | **declared padding crosses byte for byte** — b1's correction, §8 |
|
||
| `4f96e810` | **the review's fix round** — §10 |
|
||
|
||
---
|
||
|
||
## 1. What landed, against BRIEF §5 w1's seven items
|
||
|
||
1. **The encoder.** `StageBytes` / `EncodeRecord` / `Publish` / `EmitSeq` / `MaxRecordBytesSeen`
|
||
as c0 declared them, plus three additive members the work needed — a two-tail `EncodeRecord`
|
||
overload, `ReclaimStagedBytes()` and `StagedBytesInFlight()`. **No signature c0 shipped
|
||
moved**, so the other six packages compile against exactly what they compiled against on
|
||
day one.
|
||
2. **The decoder.** `MGPipeApplyWireRecord`'s skeleton is gone: every arm now bounds-checks,
|
||
cross-checks its tail, resolves its segments and calls an existing `MGPipeApply*` free
|
||
function, the verb sink, or `false`. **`MG_Pipe/PipeApply.cpp` is unchanged** — `git diff`
|
||
over it is empty.
|
||
3. **The tail cross-check.** `MGPipeWireRecordLayout(op, payload, out)` computes the record's
|
||
total from the payload's own count fields; the **encoder** refuses a caller whose tails
|
||
disagree and the **decoder** refuses a record whose `MGPWireRecHeader::Size` disagrees. One
|
||
arithmetic, two readers, so the two sides cannot drift.
|
||
4. **`SEG_STAGE`.** A linear allocator over the segment, encoder-local, reclaimed behind
|
||
`RingControl::retiredSeq` (which this package only READS), with the `0xDD` fill under
|
||
`MOBILEGL_IPC_AUDIT=1` (the decoder reads `MG_Config::Ipc.Audit` at construction).
|
||
**`RingControl`'s stage cursor triple is written by nobody in P5** — §10/M2.
|
||
5. **R-2's four Fatal arms**, all through `WireProtocolFatal*`, each with a forked death case.
|
||
6. **`MGPCaps`'s two blob serializers** + `CapsAbiFingerprint()`.
|
||
7. **R-10's proof obligation**: `MaxRecordBytesSeen()` on the encoder.
|
||
|
||
---
|
||
|
||
## 2. Acceptance numbers
|
||
|
||
Numbers below are from `52bf9acc`. **They were all re-run at `4f96e810` after the fix
|
||
round and are in §10** — G1 identical, split unit 1902/1902, 60 cases; nothing regressed,
|
||
the counts grew.
|
||
|
||
**The four self-tests.**
|
||
|
||
| # | gate | result |
|
||
|---|---|---|
|
||
| 1 | **G1** (`symbol_report.py --threshold 0`, pull build vs `~/w7/p5-before-libMobileGL.so`) | **`.text` 10 806 323 → 10 806 323 (+0, +0.000%)`; 27 811 → 27 811 defined symbols, 0 added / 0 removed / 0 resized / 0 renamed.** `nm --defined-only build-linux/libMobileGL.so \| grep -ic MG_Remote` = **0**; the split build = **215**. |
|
||
| 2 | `ctest -L unit`, four directories | pull **1785/1785**, push **1785/1785**, verify **1785/1785**, split **1893/1893** (1842 + my 51) |
|
||
| 3 | `ctest -L integration-gpu` under `MOBILEGL_TRANSPORT=monolith` | build-push **1117/1117**, build-split **1117/1117**. G14: integration-gpu names present in build-push but not at the P5 baseline = **0**. |
|
||
| 4 | `gen_pipe.py --check` / `--self-test` | up to date; **9 negative-control trips, positive control OK** (unchanged from c0). `gen_pipe_dirty_surface.py --check` rc=0, `check_include_closure.py` 4 probes / 0 problems. |
|
||
|
||
Builds: `build-linux`, `build-push`, `build-verify`, `build-split` all rc=0, each with a
|
||
`CTestTestfile.cmake` present before any verdict was believed (C-10).
|
||
|
||
**The new suite.** `MG_Test/Wire/PipeWireCodecTest` — **51 cases**, all green. Coverage:
|
||
one round trip per flag class (`kNone`, `kHasBlob`, `kVarTail`, `kHostSpan`, `kReplySlot`,
|
||
`kNeedsAck`, `kOptional`), both double-tailed rows, `DrawVbo` with and without its conditional
|
||
tail, `CreateShaderState`'s seven blob members, `SetResidualValueState`, `CreateSamplerState`'s
|
||
`borderColorForm`, both caps blob codecs with every truncation, and **twelve forked death
|
||
cases** — the four R-2 arms plus the half-declared blob, the two tail-cross-check shapes, the
|
||
mask/count disagreement, the host-span-count rule, the pad record, the declared per-stage
|
||
SPIR-V run and the second process resolver. Death cases fork and read the Fatal line out of a
|
||
per-pid log file; **no `EXPECT_DEATH`** (PipeInputsTest's rule).
|
||
|
||
**One flake, named as a flake.** The gate's first pull-lane run reported 1 failure out of 1785
|
||
and did not name it; **four subsequent runs of the same lane were 1785/1785**. It cannot be
|
||
mine: G1 says the pull binary has `.text` +0 and 0 symbols added, i.e. this package contributes
|
||
literally no code to a pull build. Not chased further.
|
||
|
||
**And one that was my harness, not the tree.** An earlier `integration-gpu` run on build-push
|
||
reported 8 failures, all `SIGHUP`, all in `AdvertisedLimitsScenario` /
|
||
`XfbAfterClipDistanceScenario`. They died because I had launched ctest from a detached shell
|
||
whose parent exited; run in the foreground the same lane is **1117/1117**. Recording it beside
|
||
C-10's lesson: a P5 script that reports a test result from a backgrounded `wsl.exe -e bash -c
|
||
'... &'` can manufacture failures as easily as the earlier one manufactured passes.
|
||
|
||
---
|
||
|
||
## 3. The finding I would most like the other packages to read
|
||
|
||
**`MGPWireRecHeader::Flags` and `Transport::RingRecordHeader::flags` are the same 16-bit
|
||
field, and the two flag spaces collide.**
|
||
|
||
- `MG_Pipe::kVarTail` is `1<<2`. `Transport::kRecPad` is `1<<2`.
|
||
- `MG_Pipe::kHostSpan` is `1<<3`. `Transport::kRecBorrowSlot` is `1<<3`.
|
||
- `MG_Pipe::kReplySlot` is `1<<4`. `Transport::kRecVarTail` is `1<<4`. **(Found by the
|
||
reviewer, not by me — §10.)** Five of the six call-flag bits alias a ring bit; only
|
||
`kOptional` is free, and only because `RingRecordFlags` has not reached `1<<5`.
|
||
|
||
`PipeWire.inc:37` documents the field as *"MGPipeCallFlags of the call, for asserts and
|
||
tracing"*, which invites an encoder to stamp `MGPipeCallFlagsFor(op)` straight into it. Doing
|
||
that makes `RingConsumer::Pop` **skip every one of the nine `kVarTail` records as a wrap
|
||
filler** — silently, with no checksum anywhere on this ring — and makes both `kHostSpan` rows
|
||
look like slots borrowed into the GPU timeline, so they would retire on
|
||
`completedFrameSerial` instead of on apply.
|
||
|
||
**The encoder translates**: `kNeedsAck -> kRecNeedsAck`, `kHasBlob -> kRecHasBlob`,
|
||
`kVarTail -> kRecVarTail`; `kReplySlot`, `kOptional` and `kHostSpan` have no ring bit and get
|
||
none. The call's own flags are read from the **opcode**, through `MGPipeCallFlagsFor` — which
|
||
is what c0 generated that table for. The `static_assert`s in `PipeWireCodec.cpp` now pin
|
||
**both enums exhaustively** — each full mask, the exact overlap, every colliding pair by name,
|
||
and the completeness of the translation — and
|
||
`PipeWireCodecTest.AVarTailRecordIsNotSkippedAsAWrapFiller` is the trip wire.
|
||
|
||
**Ask for c0:** `PipeWire.inc`'s comment on `MGPWireRecHeader::Flags` should say *ring framing
|
||
flags* rather than *MGPipeCallFlags of the call*. It is a one-line generator edit and it is the
|
||
comment that would have caused the bug.
|
||
|
||
---
|
||
|
||
## 4. Rulings I made, each with what would overturn it
|
||
|
||
**W-1 — `CreateShaderState` crosses as ONE blob, not seven.** `EncodeProgramArtifacts` already
|
||
serialises `SpirvArtifacts::generatedSpirv`, i.e. every stage module, so `Reflection` names the
|
||
whole archive and `Spirv[0..5]` stay all-zero ("absent"). A declared `Spirv[i]` is
|
||
`Fatal{ProtocolCorruption}` rather than ignored, for `resource_flush_range`'s reason (contract
|
||
§6.3): a second, forgeable way to say the same thing is worse than no way at all. Evidence that
|
||
this costs nothing: `PipeApply.cpp` reads **neither** `desc.Spirv[]` nor `desc.Reflection` —
|
||
`MGPipeApplyCreateShaderState` takes `(desc, link*, spirv*)` and reads the modules out of
|
||
`spirv->generatedSpirv`. Shipping them twice would double the largest record in the catalogue
|
||
for a reader that does not exist.
|
||
*Overturn:* a measurement showing per-stage runs beat one archive — and then the archive has to
|
||
stop carrying `generatedSpirv` **in the same change**, or the two disagree.
|
||
|
||
**W-2 — every var-tail starts 8-byte aligned inside the record; the encoder zero-fills the
|
||
gap.** Eight of the nine `kVarTail` rows are already aligned by construction. `DrawVbo` is not:
|
||
`MGPDrawRange` is **twelve** bytes, so an odd `NumDraws` leaves the conditional `MGHostSpan`
|
||
(a pointer and two `Uint64`s) on a 4-byte boundary. P5 emits no host span at all, so the rule
|
||
costs nothing now; it is stated now because the phase that arms `kDrawHasUserIndices` would
|
||
otherwise discover it as a misaligned load on a device. Pinned by
|
||
`DrawVboConditionalSpanTailIsEightAlignedAndSurvives`.
|
||
*Overturn:* nothing — it is free. (Contract table 1 row 16 already rules that the span **stays
|
||
in the tail** in P5, which this implements unchanged.)
|
||
|
||
**W-3 — `SEG_STAGE` is reclaimed by the ENCODER, behind `retiredSeq`, from marks it holds
|
||
itself.** `SEG_STAGE` is client-owned memory (table 1: *"client stages, server copies"*) and the
|
||
server only reads it, so the client is both producer and reclaimer; what it may not do is free
|
||
ahead of `retiredSeq`, which is R-11's whole content on this side. The record carries no field
|
||
saying where its staged bytes end, so the encoder remembers `{seq, stage cursor}` per record
|
||
and releases up to the newest mark the server has retired. **This does not depend on the verb
|
||
barrier**, so it keeps working when R-1's barrier retires family by family.
|
||
*Overturn:* a phase that lets the server borrow a staged run into the GPU timeline — then the
|
||
mark has to move to `completedFrameSerial` and the two stage tails stop being published
|
||
together.
|
||
|
||
**W-4 — the `0xDD` audit fill is the DECODER's, and it is exact.** R-2.5 says "the server fills
|
||
a retired record's SEG_STAGE bytes"; the thing that knows which bytes those are is the decoder,
|
||
which resolved them. It records up to eight runs per record (`CreateShaderState`'s seven blob
|
||
members plus a tail) and memsets exactly those after the applier **returns**, leaving the stage
|
||
ring's own record headers intact. `PoisonedStageBytes()` is the counter a t1 lane asserts is
|
||
non-zero with the knob on — an instrumentation that cannot be observed to have run is
|
||
decoration.
|
||
|
||
**W-5 — `CheckBlobIsHonest` could not carry R-2 arm 2, so arm 2 is its own function.** Whether
|
||
a record *carries content* is a property of its OTHER fields (`ChunkMask`, the destination
|
||
range, the stage mask), which `(op, blob)` does not see. So `CheckBlobIsHonest` implements arms
|
||
3 and 4, `RequireDeclaredBlob` is arm 2, and the decoder's per-op arm calls the second one
|
||
exactly where the payload says content is implied.
|
||
**And it needed a third shape the brief does not name.** A blobref is honest when it is either
|
||
fully declared or **all three fields zero**. `Seg` or `Offset` non-zero with `Size == 0` is
|
||
neither — and it is precisely what a MONOLITH emitter writes today (`Seg = None`,
|
||
`Offset = (Uint64)&link`, `Size = 0`, e.g. `ProgramEmit.h:180-195`). Reading that as "absent"
|
||
would **silently drop the bytes of every record an unconverted emitter sent**, so it is Fatal.
|
||
Pinned by `AHalfDeclaredBlobIsFatalRatherThanReadAsAbsent`.
|
||
|
||
**W-6 — `map_persistent` declines WITHOUT calling the applier.** Two reasons and the second is
|
||
the finding: the answer is a host pointer, which cannot cross; and
|
||
`MGPWireRec_MapPersistent`'s payload is a bare `MGPHandleOnly`, so **the record carries neither
|
||
the `size` nor the `seedBytes` that `MGPipeApplyMapPersistent(handle, size, seedBytes)`
|
||
takes**. R-6's constant decline is what makes that not matter in P5. See §6.
|
||
|
||
**W-7 — the decoder writes NO `RingControl` field.** It keeps its own `AppliedSeq()` tally
|
||
instead. See §8.1: this started life the other way round and s1's landing corrected it.
|
||
|
||
**W-8 — the three blob length cross-checks that were inert are now computed, not trusted.**
|
||
`CreateRenderState` against `MGPipePipelineChunkBlobBytes(desc.ChunkMask)`, `SetDynamicState`
|
||
against `MGPipeDynamicChunkBlobBytes(dyn.ChunkMask)`, `CreateSamplerState` against
|
||
`sizeof(SamplerParameters)`, `SetResidualValueState` against `sizeof(ResidualValueBlock)`. The
|
||
expected length is derived from a field the record itself carries, so it is a check and not a
|
||
restatement. `CreateVertexElements`' and `ResourceSubData`'s remain the applier's
|
||
(`PipeApply.cpp:1990-1999`, `:702`) — arming `Size` is what makes those live for the first
|
||
time, which is the whole point of rule A.
|
||
|
||
---
|
||
|
||
## 5. What the other packages have to know
|
||
|
||
**v1 — three things.**
|
||
1. `PipeWireDecoder` **writes no `RingControl` field at all** — `appliedSeq`'s single writer is
|
||
s1's `SessionConsumer::ApplyOne`. `PipeWireDecoder::AppliedSeq()` is the decoder's own
|
||
tally, kept *beside* the session's on purpose: the two must agree after every record, and
|
||
that comparison is what makes R-9's batching ban checkable rather than merely stated. Whoever
|
||
publishes `retiredSeq` should publish it together with `appliedSeq` in P5, because nothing in
|
||
this phase borrows a ring slot into the GPU timeline — the day something does, that is the
|
||
line that splits. **`retiredSeq` is the one watermark this package reads**: the encoder's
|
||
SEG_STAGE reclaim is gated on it, so a `retiredSeq` left at 0 ends the first
|
||
`MOBILEGL_IPC_STAGE_MB` of staging in `Fatal{RingOverrun, "SEG_STAGE"}`.
|
||
**And do not attach a `RingConsumer` to `RingCursorSet::Stage`** — §10/M2.
|
||
2. **`Wire::WireVerbSink` is yours to implement.** Contract §7's five class-B verbs — `Clear`,
|
||
`Blit`, `ReadPixels`, `DrawVbo`, `Present` — have **no `MGPipeApply*` entry point at all**
|
||
(the 37 that exist are the object and state families), so the codec decodes, validates and
|
||
hands over a checked argument list, and the backend call is the server's. With no sink
|
||
installed those five arms return `false`. `OnReadPixels` is handed the record's seq and the
|
||
`ReplySink` because P5 blocks on `read_pixels` and the pixels come back in the reply slot.
|
||
3. The server role calls `SegmentTable::InstallProcessResolver()` **before the apply thread
|
||
starts** and `UninstallProcessResolver()` **after the join**. A second install is Fatal.
|
||
|
||
**s1 — two things.**
|
||
1. The codec owns no `Doorbell` and must not: `PipeWireEncoder::Publish()` release-stores the
|
||
two ring heads and `submittedSeq` and stops there, so a unit case can drive
|
||
encoder → ring → decoder with no transport. **Ringing the consumer doorbell after Publish is
|
||
the session's line**, and it is publish-then-notify, never the other way round.
|
||
2. `CapsAbiFingerprint()` is ready for `Hello`/`Welcome`. It mixes
|
||
`sizeof(DynamicBackendParameters)`, `sizeof(MGPCaps)`, `sizeof(GLFunctionsTable)`, both
|
||
table dimensions, both codec versions, `MGPWireOp::kOpCount` and the git stamp — the stamp
|
||
because two builds with the same sizes can still disagree about a field order, which no
|
||
`sizeof` can see.
|
||
|
||
**c1 / b1 — three things.**
|
||
1. `EncodeRecord` cross-checks the tails you hand it **against the counts in your own
|
||
payload**, so a `Count` that does not match the bytes is Fatal on the producing side. A tail
|
||
whose count is zero may be omitted entirely.
|
||
2. `StageBytes(bytes, 0)` is **Fatal**, not an empty ref: "declared no blob" and "declared an
|
||
empty blob" must not be spelled the same way on a wire. Stage first, then fill the payload's
|
||
`MGPBlobRef` from what it returns — the encoder's honesty pass refuses a payload whose
|
||
blobrefs were not filled that way.
|
||
3. **`MaxRecordBytesSeen()` is on the encoder; publishing it into `PipeStats` is b1's line**
|
||
(`MG_Util/Metrics/PipeStats.*` is b1's file, so w1 did not touch it). R-10's number goes
|
||
into MEASUREMENTS from there.
|
||
4. **The codec preserves declared padding byte for byte.** One line, because the next package
|
||
to put a field in a pad will want it without reading the code: `EncodeRecord` `memcpy`s the
|
||
whole payload struct and the decoder hands it to the applier by reference; nothing is built
|
||
field by field, no pad is normalised, and there is no assertion anywhere that a pad is zero.
|
||
The only `memset`s are strictly OUTSIDE `sizeof(Payload)` — the record's own alignment
|
||
padding and the gaps between tails, which are zeroed so a record is a function of what it
|
||
declares rather than of whatever the ring last held. Pinned by
|
||
`EveryPayloadByteCrossesIncludingTheOnesSpelledPad`, which stamps the whole struct with a
|
||
recognisable byte, writes only the fields the decoder validates, and `memcmp`s the lot — it
|
||
**names no pad member**, so it will not stop compiling the day someone claims one.
|
||
*The one place a pad is minted rather than preserved:* `StageBytes` returns a brand-new
|
||
`MGPBlobRef` with `Pad0 = 0`. A phase that puts a field in `MGPBlobRef::Pad0` must set it
|
||
**after** `desc.Blob = StageBytes(...)`, not before.
|
||
|
||
**t1 — one thing.** `MOBILEGL_IPC_AUDIT=1` arms the `0xDD` fill and
|
||
`PipeWireDecoder::PoisonedStageBytes()` is what proves it ran. A lane that sets the knob and
|
||
asserts nothing has not exercised R-2.5.
|
||
|
||
---
|
||
|
||
## 6. Debt this phase leaves, named
|
||
|
||
1. **`map_persistent` has no carrier for `size` or `seedBytes`** (W-6). The phase that lands a
|
||
real one needs an `MGPipeTypes.h` payload change — c0's file — before it can even call the
|
||
applier. R-6 hides this completely in P5.
|
||
2. **`SetShaderBuffers` (38) and `SetStreamOutputTargets` (39) still have no consumer.** Both
|
||
are on BRIEF §4's exclusion list, so inventing one would be a semantics nobody can test this
|
||
phase. What they *do* have now is their tail arithmetic — including the
|
||
`HostSpanCount is 0 or Count` rule that keeps the two arrays index-aligned — so the phase
|
||
that implements them inherits a validated record rather than a validation problem.
|
||
3. **`GetCaps` (1) decodes to `false`.** The snapshot is built from the server's live backend
|
||
and posted by `ServerSession::PublishCapsSnapshot`; the codec validates the record's two
|
||
blobrefs and declines rather than inventing a caps source.
|
||
4. **FOUR counts have no GL constant bounding them**, not two (review m6):
|
||
`ResourceSubData.RegionCount`, `DrawVbo.NumDraws`, `SetShaderBuffers.Count` and
|
||
`SetStreamOutputTargets.Count`. All four are bounded only by the record fitting the ring —
|
||
the first two through `Fatal{RingOverrun}` naming R-10 and the integrator, the other two
|
||
through `MGPipeWireRecordLayout`'s 32-bit `record.Size` bound and then `MaxRecordBytes()`.
|
||
Every other count is held to `kMGPipeMaxVertexAttribs` / `kMGPipeMaxTextureUnits` /
|
||
`kMGPipeMaxImageUnits` / `popcount(Mask)`.
|
||
5. **`MGPipeApplyWireRecord`'s `remaining` parameter is now inert on the split path**: the
|
||
decoder passes `size` for it, because `RingConsumer::Pop` has already proved the record fits
|
||
what the producer published. It is kept because the signature is the generator's.
|
||
6. **`MG_Pipe/MGPipeHostSpan.h` was not touched.** The resolver install point named in the
|
||
ownership table is exercised from `SegmentTable::InstallProcessResolver()`; adding anything
|
||
to that header risks a pull-build symbol for no gain (G1).
|
||
7. **No chunking** (R-10), and no compression in `CapsCodec` beyond the sparse tables.
|
||
8. **`SEG_STAGE` bytes staged for a record that is never encoded are never reclaimed.** The
|
||
mark is taken in `EncodeRecord`, so a caller that stages and then abandons leaks staging
|
||
until the ring is rebuilt. Nothing does that today (staging is always immediately followed
|
||
by the record that names it), and making the allocator transactional would need a
|
||
scope object the c0 signatures do not have — so it is written down rather than guarded.
|
||
|
||
---
|
||
|
||
## 7. Where I think the contract or the brief is wrong
|
||
|
||
1. **`PipeWire.inc:37`'s description of `MGPWireRecHeader::Flags`** — see §3. This is the one
|
||
I want changed.
|
||
2. **BRIEF §5 w1 says "71 条的 encoder".** The encoder is *generic by opcode*: one
|
||
`EncodeRecord` plus `StageBytes`, held to a per-opcode layout and a per-opcode honesty pass.
|
||
A 71-way encoder switch would be 71 places for a caller to be wrong instead of one, and c0's
|
||
signature (`EncodeRecord(op, payload, payloadBytes, tail, tailBytes)`) already says so. What
|
||
*is* 71-way is the decoder, because each arm names a different applier.
|
||
3. **Contract table 1 row 3 says `CreateShaderState` is "seven independent runs".** It is one —
|
||
see W-1. The row's own evidence (the archive codec, and the applier taking two typed
|
||
pointers) is what makes it one.
|
||
4. **The brief's R-2.3 lists four Fatal arms; there are five.** The half-declared blobref
|
||
(W-5) is a real shape, it is what every monolith emitter writes today, and reading it as
|
||
"absent" loses bytes silently.
|
||
5. **Contract §0 rule A says a content record "must set `Seg` … `Offset` … and `Size`".** True,
|
||
but the predicate *"is this a content record"* is per-op and lives in the payload, which is
|
||
why `CheckBlobIsHonest(op, blob, segments)`'s signature cannot express it. Not an error in
|
||
the contract, but the split into two functions is worth recording so nobody looks for arm 2
|
||
inside the first one.
|
||
|
||
---
|
||
|
||
## 8. The three corrections that arrived from other packages mid-flight
|
||
|
||
Recorded in full, with which of them needed a code change, because the integrator asked and
|
||
because two of the three are the kind that go green while being wrong.
|
||
|
||
### 8.1 s1 — the single-record cap, and `appliedSeq` has one writer
|
||
|
||
> **Superseded in part.** s1's own fix round reversed the cap back to **4 MiB** — `MOBILEGL_IPC_RING_MB` now names the ring and the segment adds a page on top, so the contract is true as written. The reasoning below stands and the numbers are corrected; §10 records the reversal. Part (b), `appliedSeq`, is unaffected.
|
||
|
||
**(a) The 2 MiB cap needed NO change, and that is the finding.** `CONTRACT-P5.md §5` and
|
||
`Config.h` both say 8 MiB of `SEG_CMD` caps a record at 4 MiB; s1 found the 4096-byte control
|
||
page at the head of the segment, which makes the power-of-two ring 4 MiB and
|
||
`RingProducer::MaxRecordBytes()` **2 MiB**. Nothing in the codec had to move, because **every
|
||
comparison already went through `RingProducer::MaxRecordBytes()` and `Capacity()` at runtime**
|
||
— there is no constant anywhere in `PipeWireCodec.{h,cpp}` derived from `MOBILEGL_IPC_RING_MB`,
|
||
and the two Fatal messages print the live numbers. The unit case now says so in its header.
|
||
|
||
**And R-10's headroom is much larger than the 2 MiB figure suggests, for a structural reason
|
||
worth stating.** `CreateShaderState` is **not** the record most likely to approach the cap — it
|
||
is one of the *smallest*. Its record is `8 + sizeof(MGPProgramDesc)` = **200 bytes** whether
|
||
the archive is one kilobyte or one megabyte, because R-10's own rule sends every blob through
|
||
`SEG_STAGE` and leaves `{Seg, Offset, Size}` in the record. Pinned by
|
||
`ABigProgramArchiveDoesNotGrowItsRecordAtAll`: a 32 KiB module archive produces a 200-byte
|
||
record and 32 KiB of staging. The archive's bound is `MOBILEGL_IPC_STAGE_MB` (32 MiB) with its
|
||
own named `Fatal{RingOverrun, "SEG_STAGE"}`, not the record cap.
|
||
|
||
What *can* approach 2 MiB is a **var-tail**, and only two of them are unbounded by a GL
|
||
constant:
|
||
|
||
| row | worst case | vs the 4 MiB cap |
|
||
|---|---|---|
|
||
| `SetSamplerViews` / `SetShaderImages` | 192 units × 24 B + 24 = **4 632 B** | **0.11 %** |
|
||
| `BindSamplerStates` | 192 × 8 + 24 = 1 560 B | 0.04 % |
|
||
| `SetVertexBuffers` | 32 × 32 + 32 = 1 056 B | 0.03 % |
|
||
| `SetVertexAttribDefaults` | 32 × 24 + 16 = 784 B | 0.02 % |
|
||
| `SetFramebufferState` (largest fixed payload) | 312 B | 0.007 % |
|
||
| **`DrawVbo`** | `NumDraws` × 12 B — **no GL constant bounds it** | 4 MiB ⇒ ~349 000 draws in one `glMultiDraw*` |
|
||
| **`ResourceSubData`** | `RegionCount` × 40 B — likewise | 4 MiB ⇒ ~104 800 regions in one record |
|
||
| **`SetShaderBuffers`** / **`SetStreamOutputTargets`** | `Count` × 24 B (+ 32 B or 4 B) — likewise | caught by the 32-bit `record.Size` bound and then by `MaxRecordBytes()` |
|
||
|
||
So **nothing in the bounded part of the catalogue comes within 900× of the cap**, and I do
|
||
not think P5 has to chunk. **Four** rows are unbounded by any GL constant, not two (review m6):
|
||
`DrawVbo.NumDraws`, `ResourceSubData.RegionCount`, `SetShaderBuffers.Count` and
|
||
`SetStreamOutputTargets.Count`. All four are caught — the first two by
|
||
`Fatal{RingOverrun, "<call>"}` naming R-10 and the integrator, the last two by the 32-bit
|
||
`record.Size` bound in `MGPipeWireRecordLayout` and then by `MaxRecordBytes()` — so all four
|
||
are loud rather than silent, but only the first two say R-10 in the message. **A real measurement on OpenRA is still owed** — `MaxRecordBytesSeen()` is the
|
||
counter, and publishing it into `PipeStats` is b1's line, so the number reaches MEASUREMENTS
|
||
from there, not from here.
|
||
|
||
**(b) `appliedSeq` needed a real change** (`c1b5c2a0`). The decoder used to publish both
|
||
`appliedSeq` and `retiredSeq` after every record, on the reasoning that the decoder is the half
|
||
that knows when a record's staged runs stop being read. s1 owns that watermark and has exactly
|
||
one writer, so the decoder now writes **no** `RingControl` field and keeps a private tally.
|
||
That is the better shape anyway: two counts that must agree make R-9's batching ban
|
||
*checkable*, which one count could not, and `TheDecoderWritesNoRingControlFieldOfItsOwn` plus
|
||
`AppliedSeqAdvancesByExactlyOnePerRecordAndIsNeverBatched` are the two cases. The test fixture
|
||
now plays the session's role, which is also more honest about where the line is.
|
||
|
||
### 8.2 b1 — `MGPSubData::Pad0` is no longer padding
|
||
|
||
**No code change was needed; a test was.** The codec `memcpy`s whole payload structs and never
|
||
constructs one field by field, never normalises a pad and never asserts one is zero, so b1's
|
||
`HasLiveHostWrites` bit crosses intact — see §5's item 4 for the one-line statement, which is
|
||
what the next package to claim a pad should read.
|
||
|
||
What was missing was a **test that could see it**: the round trips compared named fields, and
|
||
a codec that zeroed a pad would have gone green through every one of them. `52bf9acc` adds
|
||
`EveryPayloadByteCrossesIncludingTheOnesSpelledPad` and `ResourceDescPadsCrossToo`, which stamp
|
||
the whole struct and `memcmp` the payload as it lands in `SEG_CMD`. Both deliberately **name no
|
||
pad member**, so b1's rename will not break them — a case that said `Pad0` would stop compiling
|
||
on exactly the day it became load-bearing.
|
||
|
||
This is also the second time in one phase that a "pad" turned out to be a field: P5 itself put
|
||
`HasRespecifiedLevel` / `RespecifiedUploadTarget` / `RespecifiedLevel` into `MGPResourceDesc`'s
|
||
two pads (contract table 1 row 19b). **Two for two is a pattern, not a coincidence**, and a
|
||
codec is the wrong layer to have an opinion about which bytes are meaningful.
|
||
|
||
---
|
||
|
||
## 9. Retained files
|
||
|
||
Everything cited above, and nothing else. Re-runnable scripts: `~/w7/p5-w1-gate.sh` (four
|
||
directories, generators, purity, G1, the four unit lanes), `~/w7/p5-w1-units.sh`,
|
||
`~/w7/p5-w1-final2.sh` (G1 + integration-gpu + the name lists), `~/w7/p5-w1-submodlinks.sh`,
|
||
and **`~/w7/p5-w1-redcheck.py`** — the one that breaks each fix in turn and requires its
|
||
control to go red (§10); its output is `~/w7/p5-w1-redcheck.log`.
|
||
Logs: `~/w7/p5-w1-gaterun.log`, `~/w7/p5-w1-symbols.md`,
|
||
`~/w7/p5-w1-unit-{linux,push,verify,split}.log`, `~/w7/p5-w1-itest-{push,split}.log`, and the
|
||
two name lists `~/w7/p5-w1-{itest-names-push,split-unit-names}.txt`. Everything else under
|
||
`~/w7/p5-w1-*` is deleted.
|
||
|
||
**Run any of them with `~/w7/p5-w1-submodlinks.sh on` first** (the worktree's submodules are
|
||
symlinks into `~/w7/pipe`), and `off` before any git command — git refuses to run at all while
|
||
they are symlinks.
|
||
|
||
**Not run here, and not w1's:** `ctest -L integration-verify`, the retrace sweeps, the device
|
||
lane, and any `MOBILEGL_TRANSPORT=inproc` run — there is no session yet to start one, which is
|
||
s1's and v1's.
|
||
|
||
---
|
||
|
||
## 10. Fix round 1 — the adversarial review, and s1's reversal
|
||
|
||
Review: `~/w7/notes/p5/p5-results/w1-review-v1.md`, 0 BLOCKER / 5 MAJOR / 8 MINOR / 3 QUESTION.
|
||
One commit, **`4f96e810`**, so the range is now **`bc969f9a..4f96e810`** (9 commits).
|
||
|
||
**How each control was shown to work.** `~/w7/p5-w1-redcheck.py` breaks the FIX — never the
|
||
test — rebuilds, runs the case that should notice, requires it RED, restores, and requires
|
||
green again. Eight for eight, and the output is in `~/w7/p5-w1-redcheck.log`. Two of them only
|
||
worked after the *test* was fixed, which is the part worth recording:
|
||
|
||
- **M3's first control stayed green.** It encoded and applied one record per iteration, which
|
||
drains the mark queue at every reclaim — the single regime the buggy "clear only when front
|
||
reaches `size()`" code handled correctly. Priming with one extra record is what makes
|
||
`front < size()` permanent.
|
||
- **And then it stayed green again**, because `StageMarksHeld()` returned `size() - front`.
|
||
That is the number *in flight*, which stays at one or two while the vector behind it grows
|
||
for ever. The accessor now reports the storage. A control that measures the wrong number is
|
||
indistinguishable from no control.
|
||
|
||
### The five MAJOR
|
||
|
||
| # | disposition |
|
||
|---|---|
|
||
| **M1** | **Fixed.** Added `CheckHostSpanIsHonest(span, const SegmentTable&)` — the old signature is c0's and still compiles for other packages, but it cannot do arm 4 and now says so. Both host-span rows (`SetShaderBuffers`, `DrawVbo`) and the encoder's own pass use the new one. Two forged-input death cases, one per row. |
|
||
| **M2** | **Fixed, by deleting the shared cursors from the design rather than arguing about them.** `SEG_STAGE` is now a genuine linear allocator (which is what BRIEF §5 w1 asked for) with encoder-local monotonic head/tail; `RingControl`'s stage triple is written by **nobody** in P5 and stays at its `InitRingControl` values. The reviewer's scenario is closed at the root: there is no cursor to walk backwards. **s1 must not attach a `RingConsumer` to `RingCursorSet::Stage`** — that is now stated in `ReclaimStagedBytes`' header, and the case asserts all three cursors are still zero after a full stage-encode-apply-reclaim cycle. |
|
||
| **M3** | **Fixed.** The consumed prefix is erased once it is half the queue — amortised O(1), bounded at twice the records in flight. |
|
||
| **M4** | **Fixed, and the contract conflict raised rather than papered over.** `PostReply` now Fatals for any op the catalogue gives no `kReplySlot`, so the pool can never be written outside its own reservation. The four Bool acceptance answers no longer ride a slot; they are recorded and read through `LastAcceptance()` / `AcceptedRecords()` / `DeclinedRecords()`. **This needs an integrator ruling** (see below) — the contract says those four use DECLINED and the catalogue gives them no slot, and w1 cannot move `PipeCalls.def`. |
|
||
| **M5** | **Fixed, my side was wrong.** The catalogue row is right: `BufferSubDataResident` is the buffer half only and `MGPipeApplyBufferSubDataResident` takes no regions. The layout no longer shares `ResourceSubData`'s arm, and a non-zero `RegionCount` on the resident op is now Fatal. No integrator ruling needed. |
|
||
|
||
**The integrator ruling M4 still needs.** `CONTRACT-P5.md` table 0's reply-slot-header row says
|
||
DECLINED "is how the four `Bool` acceptance entry points say `false` (R-5)"; `PipeCalls.def`
|
||
gives `ResourceCreate`, `ResourceRespecify`, `ResourceSubData` and `SetTextureParams` no
|
||
`kReplySlot`. Both cannot be true. Either those four rows gain the flag (and `ReplyPool` sizes
|
||
for `ResourceSubData`, the highest-frequency record class in the catalogue) or R-5's answer
|
||
rides `kNeedsAck`'s ack path and table 0's sentence is corrected. **The codec is safe either
|
||
way today** — it writes no slot it was not given — but c1/b1 need the answer before they can
|
||
gate a dirty-flag clear on acceptance.
|
||
|
||
### The third collision, and the MINORs
|
||
|
||
**Pinned exhaustively rather than one at a time.** `kReplySlot == kRecVarTail` is real and was
|
||
unpinned; it bites in the *reverse* direction of the headline, because the encoder stamps
|
||
`kRecVarTail` on nine opcodes. The asserts now cover **the whole of both enums**: each enum's
|
||
full mask, the exact overlap (five of six call-flag bits alias a ring bit — only `kOptional` is
|
||
free, and only because `RingRecordFlags` has not reached `1<<5`), every colliding pair by name,
|
||
and the completeness of the translation, so a seventh enumerator that needs a ring bit is a
|
||
build break instead of a silent drop.
|
||
|
||
| # | disposition |
|
||
|---|---|
|
||
| **m1** | Fixed — above. |
|
||
| **m2** | Fixed: three `offsetof` asserts tie `MGPWireRecHeader` to `RingRecordHeader` field by field. The decoder walks backwards from `payload` to that header, so "both are 8 bytes" was not enough. |
|
||
| **m3** | Fixed: `PoisonResolvedRuns` uses the subtraction form, like `Resolve`. |
|
||
| **m4** | Fixed: the hook is installed once, from the constructor, beside the resolver it is modelled on; `UninstallApplyHook()` added for teardown. |
|
||
| **m5** | **Evaporated.** Staged runs no longer carry a ring header at all (M2's allocator), so there is no kind to collide with `kRingPadRecordKind`. |
|
||
| **m6** | Fixed in §6.4 below — **four** unbounded counts, not two. |
|
||
| **m7** | Fixed: the header now says what the audit actually covers (at most one run per record today, because tails live in SEG_CMD and the six per-stage runs are Fatal), and overflowing `m_resolved` is Fatal rather than a silent drop. |
|
||
| **m8** | Fixed: the encoder rejects a declared `Spirv[i]` too, so it no longer accepts a record the decoder Fatals on. |
|
||
|
||
### The three QUESTIONs
|
||
|
||
- **q1 — will the `0xDD` fill survive P6's spawn?** Not fixed, and it should not be fixed here:
|
||
it is a contract line, not code. **Ask for the integrator:** `CONTRACT-P5.md` should say that
|
||
`SEG_STAGE` may not be mapped `PROT_READ` on the server while `MOBILEGL_IPC_AUDIT` can be
|
||
set, or R-2.5's only mechanical control becomes a SIGSEGV the day someone passes
|
||
`readOnly = true` to `ShmSegmentPosix`.
|
||
- **q2 — is `W && H && D` the right content predicate?** **Fixed.** A box is now either EMPTY
|
||
(every extent zero) or WHOLE; a partially-zero extent is `Fatal{ProtocolCorruption}`, and a
|
||
non-zero `RegionCount` also counts as content. The old predicate let a record name a real
|
||
destination and declare no bytes, with rule A's arm 2 sitting out.
|
||
- **q3 — the reclaim had no trigger in this package.** **Fixed.** `Publish()` now calls
|
||
`ReclaimStagedBytes()`, so SEG_STAGE's liveness no longer depends on a c1 barrier that does
|
||
not exist yet. It still depends on somebody advancing `retiredSeq` — that half is v1's and is
|
||
in §5.
|
||
|
||
### s1's cap reversal: 4 MiB again
|
||
|
||
The earlier "2 MiB, not 4" correction was itself reversed: `MOBILEGL_IPC_RING_MB` now names the
|
||
**ring**, and the segment adds one control page on top, so `MaxRecordBytes()` is **4 MiB** at
|
||
the default and `CONTRACT-P5.md §5` and `Config.h` are true as originally written. **No code
|
||
changed either time**, because every comparison goes through `RingProducer::MaxRecordBytes()`
|
||
at runtime — which is the argument for never deriving a cap from a config constant, made twice
|
||
in one afternoon. §8.1's table and headroom figure are corrected below; the largest bounded
|
||
record is unchanged at **4 632 B**, so the headroom is **~900×**, not ~400×.
|
||
|
||
### Acceptance, re-run at `4f96e810`
|
||
|
||
G1 unmoved (`.text` 10 806 323 → 10 806 323, +0; 27 811 → 27 811 symbols, 0/0/0/0; pull
|
||
`MG_Remote` 0, split 225). `ctest -L unit`: pull/push/verify **1785** each, split
|
||
**1902/1902** (1842 + 60). `integration-gpu` **1117/1117** on build-push and build-split, G14
|
||
name drift **0**. `gen_pipe.py --check` up to date, `--self-test` 9 negative controls,
|
||
`check_include_closure` 4 probes / 0 problems. `PipeWireCodecTest` is **60 cases**, 19 of them
|
||
forked death controls.
|