6bccf98d7 put TakeSendStats/TakeReadStats on the concrete StreamLink class, so
ClientSession.cpp and ServerLoop.cpp had to include <MG_Remote/Transport/StreamLink.h> -
which the P6.5 seam gate (scripts/ci/link_seam_purity.py, "concrete link included above
Transport") rejects, and the MGPipe generators and hygiene gates job has been red since.
The counters stay in StreamLink.cpp beside the recv/sendmsg loops that feed them; only the
struct and the two Take functions move, to LinkMetrics.h as LinkIoStats +
LinkMetricsTakeReadStats/TakeSendStats. The readers drop the concrete include; the hot
loops are untouched.
transport_wait_ns measures wall time in Ring::Wait's flush/spin/sleep, so
the client's parked time is read directly instead of inferred by
subtracting CPU clocks that run concurrently with the server.
frame_stats.py matches client and server present serials into per-frame
avg/p50/p99 splits; the README documents the MOBILEGL_PIPE_STATS=1
workflow.
The three things that made the deferral inert are all fixed, in the order they were
found: the reply ring could not hold an answer across the ~1,000 records between two
creates; a residency budget could not help because what displaces an answer is what
the SERVER sends, not what the client emits; and so the mechanism had to be retention.
WHAT LANDED
* ILink::DeclareReplyRead + LinkCapabilities::RetainsReplies. The wire declares a
seq will be read BEFORE it publishes, so the declaration cannot race its own
answer; a link with the capability false ignores it, and the window does not
defer on one (CreateWindowEffective() is gated on it).
* StreamLink's ring of the last 8 answers becomes a declared-answer store: a
monotone whitelist plus a cursor, answers kept until their reader TAKES them
(take-and-delete under one lock, bytes swapped into the scratch), and a capacity
of 24 that is FATAL to exceed rather than evicting. An evicted wanted answer is
unrecoverable - re-latching by sending another create wipes the server's record
(PipeApply.cpp:2033) and its PendingUploads (PipeApply.h:332) while the client
cleared its dirty flags at emission (PipeApply.h:298-304) - so the capacity is
sized so it cannot be reached and reaching it says its own name.
* An answer nobody declared is DROPPED, which is what keeps the store bounded by the
client's deferral depth instead of by the server's answer rate. Retain-on-arrival
grows ~11.3 entries per create against a server that ignores the no-reply bit.
* A read that fails after WaitForApplied has returned Reached is now Fatal{ReplyMissing}
naming the seq: on the stream arm the reply frame strictly precedes the progress
frame that publishes appliedSeq, so that combination is a broken invariant, and
retrying it (what the row used to do) wedges the window for the rest of the
process while leaving every deferred object latched on a provisional accept.
* The residency budget is deleted (WireTables.cpp keeps its history), the window
retirement branch drains once before falling back - it could otherwise strand the
answers it was holding - and Detach clears the store.
MEASURED ON THE DEVICE, load frame, both ends rebuilt, frame-sent identical per op
(create 4,536, records 103,097) throughout:
window 1 (R-5's shape) 4,536 takes 28.66 s
window 2 explicit 2,500 17.27 s
window 2 AS THE SHIPPED DEFAULT 2,563 16.52 s
window 4 1,644 11.02 s
window 2 against an OLD SERVER 2,663 16.38 s (zero Fatals)
The last row is the version-skew case the design review named as a must: a server
built before the no-reply bit answers ~55,903 records a frame, the whitelist drops
every answer nobody declared, and the deferral still reads its own back.
GATES (172 + 15 green), and the important one is proved to go red:
* ADeclaredAnswerSurvivesFarMoreInterveningAnswersThanAnyFixedBuffer - reverting the
store to "keep the last 8 arrivals" fails it immediately with "the declared answer
did not survive 64 intervening ones". That is the line between the ring-1024
stand-in and a bounded mechanism.
* AnAnswerNobodyDeclaredIsNotKeptAtAll - red if everything is retained.
* TheWindowRefusesToDeferOnALinkThatCannotRetainAnswers - on the in-process (ShmLink)
arm the knob at 4 must still block every create, red if the capability gate goes.
* The refused-create case now reads the repair that exists on THAT arm (the latch),
and asserts the window's own books are empty there.
NOT DONE, and named: the window has no unit gate on the STREAM arm yet - only the
device runs. Three rounds of this feature were green in the lanes while inert on the
device because the lanes and the device run different transports; the transport-level
retention gates close that for the store, and the window-level one should be built on
StreamClientPublicationTest's real stream session next.
The create window could not collect its deferred answers. The previous commit
diagnosed that as the reply pool being 8 slots deep against 55,903 replies in one
frame, and left a no-reply bit as the way out. This lands that bit, measures it on
the device, and reports that the diagnosis was half wrong.
WHAT LANDED. ClientSession::EmitAndWaitTails takes a second question beside
wantReply: willReadReply. They are not the same question and the window is the case
that separates them - it emits with wantReply=false and READS the answer later, so
its records must still be answered. kRecNoReply (RingRecordFlags bit 5, which
aliases kOptional in the call-flag space; the flag-space asserts were extended and
the alias named) is stamped when willReadReply is false, and the decoder skips
PostReply while keeping every acceptance counter - "nobody reads the answer" is not
"nothing happened". The two illegal combinations are Fatal by name.
Three call sites say it: SetTextureParams for a confirmed object, the respecify
row's fire-and-forget half, and sub-data where MGPipeSubDataWantsItsReply decides.
MEASURED ON THE DEVICE (aikjs4s8cys8ba7l, rebuilt APK + client, both halves new):
server: skipped=50,001 answered=4,521 (a frame's replies: 55,903 -> 4,543)
client: Reply frames received seen=4,501, latest seq=97,181
frame-sent: identical per op, records untouched
create row: by_op 2 = 9,070 (was 4,536) - because the drain's blocking take is
now counted, so the row's number stays honest instead of reading as
"create stopped waiting" while the waiting merely moved
AND THE WINDOW STILL READ BACK NOTHING: readfail 8,951 -> 8,872, i.e. -0.9%.
THE REAL CAUSE, from the miss probe:
P12miss n=1 want=310 replySeq=1504 ring=[1175..1504]
P12miss n=9001 want=310 replySeq=102915 ring=[102866..102915]
want=310 every time. The drain is STRICTLY FIFO and a failed read has no way out -
it retries the same head for ever. Any reply buffer is bounded by design (8 pool
slots, or the 8-frame ring the stream link now keeps), so with 4,536 deferred
answers at least one is overwritten before it is read, and that one wedges the
window permanently. Cutting the churn by two orders of magnitude does not help: it
turns a certainty into a certainty. That is the constraint, not the depth.
THE FIX IS NAMED AND NOT TAKEN (the goal this work ran under stops here on purpose).
On a failed read of the head, abandon it: MGPipeNoteHandleUnpublished +
NoteCreateSuspect + pop - the DECLINE path reused for "unknown". The window then
self-heals through the blocking path and the object merely gets slower. It needs its
own gate: a failed read must let the window ADVANCE.
Gates, all green (172 + 13 + 22 + 103):
PipeWireCodecTest.TheNoReplyBitSkipsTheAnswerAndKeepsTheAcceptanceRecord - the skip,
the kept tally, the answered/skipped split, and the negative control that a record
WITHOUT the bit is still answered (a server that stopped answering everything
would otherwise pass)
RemoteClientControls.ACreateWindowDefersTheAnswersAndTheWindowOffDoesNot - now also
asserts the answers CAME BACK (answers_taken=11, outstanding=1, fallbacks=0 against
12 records). RED-ONCE PROVED: giving the window's creates the literal
"fire-and-forget => skip" predicate makes this case fail.
RingTest - the sixth and last flag alias, and the hand-written exhaustiveness list
extended (with a note that it is only as exhaustive as that list).
Server side: AnsweredRecords()/SkippedAnswers() - the number that made this
debuggable, since a wire change whose effect is invisible is undebuggable.
Client-first is safe: an old server ignores bit 5 and replies as before, so the two
cross-version directions degrade to today's behaviour - this is "both ends must
upgrade to take EFFECT", not "to avoid breaking". Hit that in practice first.
ILink::ReadReply takes a SEQ (ILink.h:285) and ShmLink implements it against a
ReplySlotPool, so any answer the pool still holds is readable. StreamLink kept ONE
reply - replySeq/replyStatus/reply - and then asked the wrong question of it:
x.replyReady.wait(lock, [&]{ return closed || x.replySeq >= seq || appliedSeq >= seq; });
if (x.replySeq != seq) return ... MOBILEGL_ERR_PROTOCOL_MISMATCH;
The wait is written as "at least this seq"; the check is "exactly this seq". Every
answer older than the newest was therefore unreadable, which nothing noticed because
every caller read the answer it had just waited for - i.e. always the newest one. It
is the transport half of the P12 create window's failure: the window defers up to
four answers and collects them in order, and this link could not hand one back.
THE RING, AT THE POOL'S OWN DEPTH. kReplyRing = kDefaultReplySlotCount (8), because
that is the pool this link stands in for: an answer a reader has not taken by the time
eight newer ones have arrived is one the shm arm has already overwritten too, so the
two transports now lose the same answers and no more. Vectors are sized per reply, so
an idle link pays eight empty ones.
This does NOT make the create window work - on the device it changed nothing, because
the pool itself is 8 slots against 59,673 replies in one frame (see the next commit).
It is kept because the two transports disagreeing about ILink's contract is a bug in
its own right, and because the single slot silently failed any read that raced a newer
reply.
Gate: StreamLinkTest.AnOlderReplyIsStillReadableByItsSeq posts three replies and reads
them oldest-first - the order the window's drain takes them in.
- A wait count alone cannot say whether a frame is slow because it sends too much or because it
waits too often. The device run made that the next question: one atlas frame paid 43,232 waits
while the upload row paid none.
- LinkMetricsNoteRecord is called at the encoder's one commit point (PipeWireCodec.cpp, beside the
existing CallClass::WireRecords count, for the comment's reason: after chunking no emitter knows
how many records its call produced). LinkMetricsRecordsFor reads it back, and every frame that
emitted something now logs `P65LinkMetrics kind=frame-sent frame=N records=M by_op=...`.
- The host measurement now prints and asserts both halves for the same workload. An atlas-shaped
load (64 sub-uploads into one level) sends 128 ResourceSubData records and waits for ZERO of
them, while ResourceCreate 76/76, ResourceRespecify 130/130 and SetTextureParams 130/130 are
one wait per record. That is item B stated exactly: the records still go out; the round trips do
not. It also says what the next thing to fix is - the record COUNT of those three rows, not their
waits, which the contract rules they must keep.
- The device run that motivated this made the point: 43,232 of a session's 43,629 reply waits were
in ONE atlas frame, and the exit summary never appeared at all - a SIGTERM does not take the
session's clean stop path, so LinkMetricsEnd is never reached. The split added one commit ago
was therefore invisible on exactly the run that needed it.
- Emit now writes a second line for any frame that bought a wait:
`P65LinkMetrics kind=frame-op frame=N wait_replies=M by_op=<id>=<count>,...`. The formatting is
factored into FormatOpSplit and shared with the exit line. Frames that bought nothing stay
quiet, so a steady-state session does not double its log.
- Measured on the device (MC 26.2 into a world over TCP, MOBILEGL_PIPE_STATS=1): the steady state
is one wait per frame and it is op 9 (Present); the upload-heavy startup frames are
`2=50,3=68,47=62` - ResourceCreate / ResourceRespecify / SetTextureParams, with ResourceSubData
(48) absent, which is item B working on the wire it was written for.
- The tally added beside this is only useful if somebody can read it where the problem is. The
handoff says why that was hard: under spawn the client does not advance the frame window, so the
per-frame P65LinkMetrics lines never print, and a SIGKILL takes the exit-time dump with it -
"only the dump at exit, and SIGKILL does not get it".
- LinkMetricsEnd writes the split on its own line from the one place a client that ends the
ordinary way always reaches: P65LinkMetrics kind=per-op wait_replies=N by_op=<id>=<count>,... .
Ids stay raw (this file includes only <cstdint> and friends, not the op table) and the reader
names them: 2 ResourceCreate, 3 ResourceRespecify, 47 SetTextureParams, 48 ResourceSubData. The
mapping is written down in mgl-device/README.md, which is the page the device run reads.
- The handoff measured the total (55,428 reply waits in one Minecraft frame) and left "which rows"
open: a total is the same number whether the waits are uploads, creates or parameter sets, and
those three have different fixes - so knowing the split is what says whether P12 item B is the
whole story or a fraction of one.
- LinkMetricsBeginReply now takes the op and Window carries a per-op tally, so
LinkMetricsReplyWaitsFor(op) reads back what the P65LinkMetrics line prints as `wait_replies`
split by row. Same activation as everything else in that file (MOBILEGL_PIPE_STATS=1), one array
in the window, and a value past the table lands on the last slot so the per-op numbers still SUM
to the total.
- RemoteClientControls.ReplyWaitsByOpForAnAtlasShapedLoad drives the two shapes a stitcher
produces and prints the table: an atlas stitch (64 glTexSubImage2D into one level) goes from 64
reply waits to ZERO, and a texture per sprite (64 textures, 256 glTexParameteri) from 384 to 320
with the remainder all ResourceCreate 76 / ResourceRespecify 130 / SetTextureParams 130.
- It prints before it asserts: the table is the point of the case, and a gate that fired first
would hide the numbers on exactly the run that needs them - the red one. The assertions are the
structural ones that can be wrong about the mechanism (upload rows zero, the other rows not, the
per-op tally summing to the total); the numbers themselves are recorded, not gated.
- The A/B is a real run: with the wire-arm arm test removed the case goes red with the full table,
which is where the 64 and 384 above come from.
- MGPipeSubDataWantsItsReply is the ONE place that decides whether a resource_subdata record waits
for its answer (WireTables.cpp:370 and PipeRoute.h:328 both read it, and the comment beside it
says that a second spelling is how the client comes to wait for an answer the emitter told the
server not to bother with). The buffer half has answered "no" since P5e; the texture half now
does too, ON THE WIRE ARM ONLY.
- THE ANSWER IS DERIVABLE THERE, which is the argument - not that nobody reads it:
* The emission is already gated on the answer's content. A texture record is produced only where
FamilyIsLive(kMGPipeSubsystemTextureResources, ...) holds, and under split that gate IS
CapsMirrorInstance().ServerConsumes(bit 10) (PipeFill.cpp:1719-1740) - the same published fact
the applier's own acceptance belt reads (PipeApply.cpp:1408-1419, NoP4aConsumer). So the one
refusal this acceptance exists to carry, "no consumer" (ID-39 / D-D5 / ID-18 M3, which is why
the texture half ever asked), cannot be the answer to a record this client emitted.
* What remains is a bug class, and it is loud already: a record that names no texture, declares
no texels or carries no bytes is an MGP_TRIP_WIRE_REPORT (PipeApply.cpp:1201-1218, :2245-2252),
and a handle the applier does not hold is counted and named (ResolveResourceIn, :721-728) with
the client's own tracker preventing it from arising (PipeFill.cpp:1045-1051).
* Every answer the client can still get for free, it still gets: the emitter reads this route's
Bool exactly as before, so a record the CLIENT cancelled (the device-lost latch, an exhausted
stage) still answers "not accepted" and the level stays dirty and is retried - D-D5's safe
direction, unchanged.
- THE WAIT MOVES TO THE STAGE. A run that does not fit SEG_STAGE reclaims retired bytes and parks
on the stage doorbell (PipeWireCodec.cpp StageAllocate / ReclaimStagedBytes) - the one wait a
run-ahead client owes, and it is proportional to the WINDOW rather than to the call count. On
Minecraft 26.2's atlas frame that is the handoff's 55,428 reply waits at ~4.4 ms each: 96 MiB of
uploads through a 32 MiB window.
- THE GATE IS THE NUMBER ITSELF. LinkMetricsReplyWaits is added so a case can read back the same
`wait_replies` the per-frame P65LinkMetrics line prints - LinkMetrics.cpp:76-80 counts exactly
the records that own a reply slot. TextureUploadsStopOwningAReplyOnTheWireArm asserts that a pure
glTexSubImage2D crosses the wire and moves it by ZERO, while a resource_create row - kWaitReply,
and it always was - still moves it, so the zero cannot be "nothing was counted". Red-once run:
with the arm test removed the child exits 142.
- Monolith and every non-wire arm are untouched, and that is pinned twice: the existing
RemoteRunAhead.OnlyTheTextureHalfOfResourceSubDataWantsItsReply still reads true with no arm
installed, and the new test sits inside MOBILEGL_BUILD_DISAGGREGATED with the original expression
as its fall-through, so a pull build preprocesses to the line it had (G1).
- Verification debt, stated rather than implied: the host gate is the COUNT, not a wall clock.
In-process control sessions run lockstep (the caps peer publishes no kCapRunAheadApply), so the
verb barrier still waits there; the end-to-end wall-clock win is the device measurement the
handoff describes and it has not been taken yet.
- AuthToken.h: ConstantTimeTokenMatch loops kAuthTokenCompareBytes (64) iterations with both operands read as zero past their ends - its trip count no longer tells a peer the configured token's length; a token longer than 64 is still compared to its last byte (SocketTransportTest pins the 70-byte case)
- ServerSession.cpp: an ask in Hello.linkTerms above the granted terms logs one MGLOG_W naming both sides ("the ask is ignored and the session is server-sized"); the production client asks for nothing, so a session that was never mis-configured never logs it; SessionHandshakeTest's 64 GiB case requires the line (red before: absent)
- ServerSession.cpp: a MintNonce failure sends Refuse{LinkTerms} "stream data plane unavailable: the server could not mint a data nonce" before returning the fault it is (still exit 67, still counted) instead of a bare close
- ServerMain.cpp: a first frame with the wrong magic or over 1 MiB while a session is live is Refuse{MalformedHello} "first frame is not a control frame", drained before the close so the refusal is not RST away; Busy stays the answer for a well-formed frame that is not a DataBind
- tcp_supervisor_smoke.py: garbage_first_frame_while_busy (magic XXXX on a live session -> code 8; red before: code 7)
- fatal census unchanged: 79 abort sites, 3 refusal words (the new refusals go through the enum, not a literal)
- ServerMain.cpp: RunSession takes the descriptor its data-connection source reads from (the child's hand-off end under --serve, the listener itself without it) and closes it the moment Accept returns; the source object goes with it
- the supervisor's hand-off send is MSG_DONTWAIT (FdPassing::SendFd dontWait), so a peer that stopped reading answers at once; "peer gone" on a datagram pair (ECONNREFUSED, ENOTCONN) is an answer the caller names, not a logged fault
- before: every DataBind after the bind sat in the child's receive queue with no reply, its TCP connection held open until the session ended, and past the queue limit (net.unix.max_dgram_qlen 10 on the phone; ~288 datagrams of sender buffer on this host) the supervisor blocked in sendmsg for the rest of the session - no accepts, no Busy, no reaping
- before, without --serve: the listener stayed open after the data connection bound, so a second client or a retry completed a TCP handshake nobody would accept and waited out its 20 s connect budget instead of the ECONNREFUSED it got before PH-7 (4)
- tcp_supervisor_smoke.py: data_bind_after_bound (12 extra DataBinds on a bound session each get Refuse{Authentication} "data connection could not be handed to the live session", and a fresh control connection is still Busy) and single_session_listener (a connect after `ready` is ECONNREFUSED); connect_control retries ECONNREFUSED at startup for pair() and held_session() alike
- red-once at da313010: smoke TimeoutError on the first extra's receive; probe with qlen+2 fire-and-forget DataBinds: 288 queued silently, then the supervisor stopped accepting (fresh control: connect timed out) while still alive; single-session connect after ready succeeded
`scripts/ci/fatal_census.py` scanned `MG_Remote/` and saw four abort sites. CONTRACT-P6
§12.1 already says what the server image is - "P6's server image is the whole
libMobileGL.so" - and the server process was executing seventy-nine more deaths the gate
could not see: 15 in DirectGLES.cpp, 12 in Managers.cpp, 11 in VkBufferManager.cpp, 7 in
PipeRoute.cpp. SCAN_ROOTS is now MG_Remote / MG_Pipe / MG_Backend / MG_Impl/Pipe /
MG_State, `.inc` joins `.cpp`/`.h` (the Magma wire arms and the generated wire records are
.inc files, and 20 sites are reachable only through one), and MG_Benchmark /
MG_IntegrationTest join MG_Test in the exclusion.
Four rules now, where there were two:
1. an abort names its family within twelve lines, OR it is argued into FUNNEL_FILES /
the new FUNNEL_SITES table. FUNNEL_SITES exempts an individual funnel inside a file
that has ordinary named deaths around it, and only when one of the file's anchors
appears in the same window - so the exemption follows the funnel if it moves, expires
if it is deleted, and does not cover an unrelated abort added to the same file. Two
entries, one line of reason each: PipeApply.cpp's MGP_TRIP_WIRE_REPORT and
PipeWire.inc's MGPipeWireProtocolFatal.
2. unchanged: a WireLogFatal / SessionFail call carries a `Fatal{` in its string.
3. NEW: every family word found in the perimeter has a row in FatalFamilies.def. Eleven
words did not, and now do (see below).
4. NEW: every `Refuse{Word}` names a RefuseCode enumerator. This is what catches
`Refuse{AuthenticationRequired}` (SocketTransport.cpp:161), a word protocol.fbs has
never had - PH-7 (2). Renamed to the enum's own `Authentication`, and
SocketTransportTest pins the spelling to `EnumNameRefuseCode` so the two cannot
drift again.
`abort_sites` was recorded and never compared. It is now a DOWNWARD ratchet with a
per-file map beside it, so "83 became 84" reports as "VkBufferManager.cpp went 11 -> 12,
here are its lines"; fewer sites - the direction `@P7` retirement moves the number - asks
for a re-baseline rather than blocking one.
Eleven new `.def` rows for the eleven rowless words. Ten are ProtocolCorruption for the
reason that bucket already states. ResourceUnavailable is ServerCrashed, not SegmentMismatch:
SegmentMismatch means a shared SEGMENT was the wrong size, and telling a peer that when a
VkBuffer allocation failed would be a lie about whose geometry was wrong. A twelfth word was
collapsed rather than given a row - `InitialBytesNotCarried` (PipeFill.cpp) said the same
thing as `UncarriedInitialBytes` (WireTables.cpp) about the same record, and two words for
one family is the drift a6 censused.
Red-once, four ways: an unnamed `std::abort()` appended under MG_State reds naming the file
and the line; deleting the `Fatal{` from PipeRoute.cpp's ResidualBlockSize reds; putting
`Refuse{AuthenticationRequired}` back reds rule 4; a typo'd family word reds rule 3 and the
vocabulary ratchet together.
The end state is re-ruled: client and server may sit on different machines,
OSes and architectures, joined over TCP. The AVF pVM / AF_VSOCK end state the
P6 end-state review assumed is withdrawn (retraction at its head, form D added
to its table, the vsock probe and both external unknowns struck).
The transport stack becomes two independently selectable axes, negotiated in
the handshake and mixable: control plane (ITransport: fork / unix:<path> /
tcp://host:port) and data plane (ILink: ShmLink / StreamLink). On one machine
control over TCP with data over shared segments is a legal pair; ShmLink brings
its own AF_UNIX aux rendezvous for SCM_RIGHTS, so ITransport narrows to control
frames. `auto` is decided by a successful segment delivery, never by inspecting
addresses, and a fall-back is named. Nothing above Transport/ may branch on the
link kind. ARCHITECTURE.md gains 11.9 for this; CONTRACT-P6.md gains 10.3.
ROADMAP: P6.5 and Ph get rows (both were referenced by the contract and absent
from the stage table); the IPC track becomes P6 -> P6.5 -> Ph -> P12 -> P9 ->
P10 -> P11(same-machine arm). Every later row is reconciled against the tree:
items already landed are struck (client vertex arrays, client-minted query
handles, credit default 1, SEG_EVENT overflow mechanism), the unreachable P7
gate (`nm | grep glslang` on a server image that IS libMobileGL.so) is replaced
by the 184-symbol ratchet share, fixed-width wire form moves from P7 to P6.5,
P12 is cut down per the review (server-owned display, no Surface passing), and
the debt table's stale claims are corrected: class C is down to two slots, not
15 and not 511; PACK-PBO and the 2 MiB reply cap are P9's, not P6's.
Verified against the tree and recorded honestly: the contract's three hs rows
(LinkTerms, Refuse, the fingerprint split) never landed - protocol.fbs has
neither - so CURRENT_STAGE_PROGRESS.md's claim that S6 ran red is withdrawn and
the rows move to P6.5; the 184-symbol ratchet gate does not exist in CI; the
server process never initialises PipeStats, so gate 8's `srv=0 srvpark=0` is a
structural zero (fix in progress in the tree, not in this commit).
check_doc_citations.py: 159 citations, one pre-existing basename ambiguity.
CONTRACT-P6 4.2 and 4.3. Three defects, all of them things the handshake
believed it was checking and was not.
1. THE BUILD STAMP WAS "" AND NOTHING SAID SO. CMakeLists took the git hash
through an execute_process with NO RESULT_VARIABLE, so a failed
`git rev-parse` yielded the empty string. Verified live rather than
inferred: this worktree's `.git` is a file holding a WINDOWS path, WSL's
git answers `fatal: not a git repository`, result 128 - and every WSL build
mixed "" into its ABI fingerprint. Two builds from different commits then
agreed exactly, which is the one thing the stamp exists to prevent. The
already-failing SessionHandshakeTest was reporting this the whole time.
The fix may NOT touch GIT_COMMIT_HASH_SHORT: it is baked into
glGetString(GL_VERSION), so redefining it changes an application-visible GL
string and the pull build's .rodata - a behaviour leak and a G1 break at
once. MGGitHash.h.in gains a SECOND pair instead, consumed only by the
fingerprint, plus -DMOBILEGL_BUILD_STAMP for trees git cannot read (CI has
${GITHUB_SHA} in hand). A stampless build is now a configure-time WARNING
naming the consequence, not silence.
MOBILEGL_BUILD_STAMP_PRESENT IS MIXED AS ITS OWN FIELD. "this build cannot
name its commit" and "this build's commit is the empty string" are different
claims, and collapsing them is how every stampless build in the world agreed
with every other. The test asserts the flag moves the fingerprint in EITHER
build state, and the `BuildStamp = ""` perturbation is now guarded - in a
stampless build production's value already IS "", so that assertion was
passing while proving nothing.
2. THE CLIENT NEVER READ abiMajor/abiMinor. Hello has carried them since P5
and the SERVER checks them; the client compared only the fingerprint. The
fingerprint mixes AbiVersion so a mismatch would fire anyway - but as
"struct shapes", which sends the reader looking for a layout change that is
not there. Under spawn the peer is a SEPARATE IMAGE the operator chose with
MOBILEGL_IPC_SERVER_PATH, so this is exactly the failure that needs its own
sentence. Both handshake sites call one helper; there are two sites, and
only one of them was reachable before `sm`, which is how the gap survived.
3. Hello::pid WAS 0 AND Welcome::serverPid ECHOED IT. So both were always 0 and
neither named anything, while §9.5's arm-proof gate wants exactly the child
pid. Both now carry real ids, and the pair is the cheapest arm proof in the
protocol: under spawn they MUST differ, and a same-process session cannot
produce two different values. The client keeps what the server STATED -
local launcher knowledge is a different fact from one that crossed the wire,
and only the second can be asserted from the server's own mouth.
R-16 both ways: revert serverPid to echoing hello->pid() and the new test reds
on both halves, naming 4.3. G1 re-measured after the CMake and header changes:
0 symbol diff, .text unchanged at 0xa52203.
Unit suite goes from two long-standing failures to one - the remaining
PipeCatalogue.FrontendNeverTakesAnApplierAddress is an environmental path
problem in this tree, unrelated. All six OpenRA arms still green.
Before this, CONTRACT-P6 5.3's own sentence held: "a dead server leaves the
client silently DECLINE-ing every verb forever with glGetError clean".
FatalCode::DeviceLost had zero producers and zero consumers and
glGetGraphicsResetStatus returned GL_NO_ERROR unconditionally, so an
application had no way to learn that the process holding its GPU objects was
gone. Measured cost of that: a server killed on its first Clear left the
client waiting out the full 120 s barrier and reporting Fatal{BarrierTimeout}
- the wrong diagnosis, two minutes late. It now notices in 0.01 s.
THE MACHINERY WAS ALREADY THERE. Doorbell::Wait breaks out on Dead(),
SessionProducer::Park turns that into SessionWait::ShutDown, and ClientSession
already had the named DECLINED path. What did not exist was the death FACT:
D5c's finding is that the client parks on clientBell[0] and rings ITSELF
through clientBell[1], holding both ends, so EOF can never arrive on the one
descriptor it watches. A bell that can wake itself cannot hear a death.
SetDeathWitness(fd) fixes that with no new descriptor: the control socket's
far end is held only by the server. The subtlety is that the bell must watch a
socket it must NEVER read - the transport's reassembler owns those bytes and a
control reply is not a death. So the witness enters the poll set with
`events = POLLRDHUP` and nothing else: POLLHUP/POLLERR/POLLNVAL arrive in
revents unrequested, so the hangup is seen while POLLIN never is. Nothing is
recv'd from it and its ownership stays with SocketTransport.
Dead() AND PeerHungUp() ARE TWO FACTS. Park sets m_dead for POLLERR, POLLNVAL
and unrecognised revents too - each a fault in THIS process's descriptor - and
CondVarDoorbell::Kill() sets it on every orderly Stop(). A latch taken from
Dead() would report a lost GPU for a local fd bug and a context reset for
every clean exit, so only PeerHungUp() arms it.
GL_UNKNOWN_CONTEXT_RESET, not GUILTY or INNOCENT: those assign blame, and the
server died for a reason that never crossed the wire.
D5b, in the control-reply wait, which had the right comment and no code: EOF
on the control stream is device loss; a TIMEOUT with the socket still open is
"ALIVE BUT SILENT" and says so by name. 5.4 forbids calling a frozen or merely
slow peer dead, and under P5e run-ahead a server one frame behind is the
INTENDED steady state - a latch armed by a deadline would fire on a healthy
session under load.
MOBILEGL_IPC_RESPAWN is now a NAMED REFUSAL rather than a variable nothing
reads. The latch is one-way on purpose: every handle this client minted names
an object inside the server process, so recovery means re-pushing the world.
THREE TEST NAMES ADDED, none renamed (G14), and R-16 run both ways:
* remove the witness -> AKilledServerLatchesDeviceLostFromTheHangupAndNot-
ADeadline goes red, takes 2 s instead of 0.01 s, and names D5c.
* collapse PeerHungUp() into Dead() -> ADeadBellIsNotAlwaysAHungUpPeer goes
red naming the local-fd-bug consequence.
* AnOrderlyStopIsNotADeviceLoss stays GREEN under that collapse, and its
comment now says so. The first draft claimed it was the control for it;
running the falsification proved otherwise - under spawn the self bell is a
SocketDoorbell and Stop() never kills one. The comment was corrected rather
than the claim quietly kept.
G1 RE-MEASURED, NOT ASSUMED: 0 symbol diff against the c6 baseline and .text
unchanged at 0xa52203. That measurement earned its keep - it caught a 16-byte
drift that had nothing to do with dl: P6's O_APPEND log-sink fix was compiling
into the PULL build, where a second process can never exist. Now guarded, and
.text is back to the baseline exactly.
Full unit suite: 2350 tests, the same two pre-existing environmental failures
and no others. All six OpenRA arms (2 backends x monolith/inproc/spawn) green.
OpenRA.DirectGLES, transport=spawn, on a Redmi: ssim=1.000000,
mismatchPixels=0, zero Fatal{ lines, three runs for three. The server is a
process the client launched out of the APK's lib/arm64/, and the client's own
log names its pid.
Four things stood between the desktop green and the device green, and every
one of them was invisible on desktop for the same reason: a build tree is not
a deployment, and Linux is not Android.
1. THE SERVER COULD NOT LINK. It links libMobileGL.so and is EXEC'd, not
dlopen'd, so it gets a fresh linker namespace with the DEFAULT search path
- and an APK's lib/<abi>/ is not on it. `CANNOT LINK EXECUTABLE ...:
library "libMobileGL.so" not found: needed by main executable`, before a
single line of MobileGL ran. On desktop CMake had been giving the target an
absolute build-tree RPATH, so the WSL server resolved its dependency
through a path no phone has. $ORIGIN is asked for via BUILD_RPATH (a raw
-Wl,-rpath,$ORIGIN reaches the generator unquoted and the SHELL eats it -
the device linker said `normalize_path - invalid input: "$"`), and because
the NDK drops RPATH from the link anyway, the LAUNCHER states
LD_LIBRARY_PATH: it is the one component that has already resolved where
the image lives, and a pair that is not co-located is not a pair.
2. THE RENDEZVOUS NEEDED A WRITABLE DIRECTORY. An Android app has no /tmp, so
bind() in the child had nowhere to put the node and the client refused by
name 20 s later. AF_UNIX endpoints now accept the ABSTRACT namespace with a
leading '@' - the systemd/D-Bus convention - which has no filesystem
presence at all: no directory, no mode bits, and no stale node to unlink
after a crash, which is the bind() failure Listen() has been working
around. Its reach is the network namespace, which for an app is the app.
StartSpawned mints '@' names; an explicit path still works everywhere, and
that is what P12's "connect to something somebody else is listening on"
will use.
3. PBUFFER WAS A DirectGLES-ONLY KNOB. Twice over: trace-replay-ci.sh gated
`--ez use_pbuffer true` on the backend name, and the JNI demanded a native
window for DirectVulkan unconditionally. A surface SHAPE is not a backend
property, and the coupling mattered because the Android GLWS makes a window
surface whenever gNativeWindow is non-null - so handing Magma a window was
the same as asking for one. Until P12 a spawned server cannot be given a
window at all: an ANativeWindow* is a pointer into the CLIENT's process and
SetWindowHandle is refused on the way across with Fatal{UnmigratedSurface,
"AndroidNativeWindow@P12"} (Rule H). Which is exactly what happened, one op
after the server had come up green.
4. THE RUNNER LOOKED IN ONE OF THE TWO OUTPUT DIRECTORIES. assembleTraceRelease
writes apk/trace/release; CI has staged the same artifact under
apk/trace/debug. A tree that has built locally has both, and picking the
wrong one retraces last month's APK and says nothing about it. Newest of
either, now.
MEASURED AND NOT FIXED: DirectVulkan on this device diverges under BOTH split
transports and does so nondeterministically - inproc scored ssim=0.988998 /
9836 px, spawn scored 0.976494 / 14658 px and then 0.988998 / 9836 px on a
repeat, while monolith with the same pbuffer surface scores 1.000000 / 0 px.
The two transports draw from the same set of outcomes, so this is the split
path on Magma against a real GPU, not the process boundary: P6 neither caused
it nor can fix it. DirectVulkan spawn IS green on desktop (lavapipe, 1.000000).
G14: one test name added, AnAbstractEndpointNeedsNoWritableDirectoryAndLeaves-
NoFile, which pins both halves - the handshake crosses an abstract name, and
nothing is left on the filesystem. A '@' mistaken for a path would have bound
a file of that literal name, passed here, and failed on the one platform this
exists for.