`scripts/ci/fatal_census.py` scanned `MG_Remote/` and saw four abort sites. CONTRACT-P6
§12.1 already says what the server image is - "P6's server image is the whole
libMobileGL.so" - and the server process was executing seventy-nine more deaths the gate
could not see: 15 in DirectGLES.cpp, 12 in Managers.cpp, 11 in VkBufferManager.cpp, 7 in
PipeRoute.cpp. SCAN_ROOTS is now MG_Remote / MG_Pipe / MG_Backend / MG_Impl/Pipe /
MG_State, `.inc` joins `.cpp`/`.h` (the Magma wire arms and the generated wire records are
.inc files, and 20 sites are reachable only through one), and MG_Benchmark /
MG_IntegrationTest join MG_Test in the exclusion.
Four rules now, where there were two:
1. an abort names its family within twelve lines, OR it is argued into FUNNEL_FILES /
the new FUNNEL_SITES table. FUNNEL_SITES exempts an individual funnel inside a file
that has ordinary named deaths around it, and only when one of the file's anchors
appears in the same window - so the exemption follows the funnel if it moves, expires
if it is deleted, and does not cover an unrelated abort added to the same file. Two
entries, one line of reason each: PipeApply.cpp's MGP_TRIP_WIRE_REPORT and
PipeWire.inc's MGPipeWireProtocolFatal.
2. unchanged: a WireLogFatal / SessionFail call carries a `Fatal{` in its string.
3. NEW: every family word found in the perimeter has a row in FatalFamilies.def. Eleven
words did not, and now do (see below).
4. NEW: every `Refuse{Word}` names a RefuseCode enumerator. This is what catches
`Refuse{AuthenticationRequired}` (SocketTransport.cpp:161), a word protocol.fbs has
never had - PH-7 (2). Renamed to the enum's own `Authentication`, and
SocketTransportTest pins the spelling to `EnumNameRefuseCode` so the two cannot
drift again.
`abort_sites` was recorded and never compared. It is now a DOWNWARD ratchet with a
per-file map beside it, so "83 became 84" reports as "VkBufferManager.cpp went 11 -> 12,
here are its lines"; fewer sites - the direction `@P7` retirement moves the number - asks
for a re-baseline rather than blocking one.
Eleven new `.def` rows for the eleven rowless words. Ten are ProtocolCorruption for the
reason that bucket already states. ResourceUnavailable is ServerCrashed, not SegmentMismatch:
SegmentMismatch means a shared SEGMENT was the wrong size, and telling a peer that when a
VkBuffer allocation failed would be a lie about whose geometry was wrong. A twelfth word was
collapsed rather than given a row - `InitialBytesNotCarried` (PipeFill.cpp) said the same
thing as `UncarriedInitialBytes` (WireTables.cpp) about the same record, and two words for
one family is the drift a6 censused.
Red-once, four ways: an unnamed `std::abort()` appended under MG_State reds naming the file
and the line; deleting the `Fatal{` from PipeRoute.cpp's ResidualBlockSize reds; putting
`Refuse{AuthenticationRequired}` back reds rule 4; a typo'd family word reds rule 3 and the
vocabulary ratchet together.
OpenRA.DirectGLES, transport=spawn, on a Redmi: ssim=1.000000,
mismatchPixels=0, zero Fatal{ lines, three runs for three. The server is a
process the client launched out of the APK's lib/arm64/, and the client's own
log names its pid.
Four things stood between the desktop green and the device green, and every
one of them was invisible on desktop for the same reason: a build tree is not
a deployment, and Linux is not Android.
1. THE SERVER COULD NOT LINK. It links libMobileGL.so and is EXEC'd, not
dlopen'd, so it gets a fresh linker namespace with the DEFAULT search path
- and an APK's lib/<abi>/ is not on it. `CANNOT LINK EXECUTABLE ...:
library "libMobileGL.so" not found: needed by main executable`, before a
single line of MobileGL ran. On desktop CMake had been giving the target an
absolute build-tree RPATH, so the WSL server resolved its dependency
through a path no phone has. $ORIGIN is asked for via BUILD_RPATH (a raw
-Wl,-rpath,$ORIGIN reaches the generator unquoted and the SHELL eats it -
the device linker said `normalize_path - invalid input: "$"`), and because
the NDK drops RPATH from the link anyway, the LAUNCHER states
LD_LIBRARY_PATH: it is the one component that has already resolved where
the image lives, and a pair that is not co-located is not a pair.
2. THE RENDEZVOUS NEEDED A WRITABLE DIRECTORY. An Android app has no /tmp, so
bind() in the child had nowhere to put the node and the client refused by
name 20 s later. AF_UNIX endpoints now accept the ABSTRACT namespace with a
leading '@' - the systemd/D-Bus convention - which has no filesystem
presence at all: no directory, no mode bits, and no stale node to unlink
after a crash, which is the bind() failure Listen() has been working
around. Its reach is the network namespace, which for an app is the app.
StartSpawned mints '@' names; an explicit path still works everywhere, and
that is what P12's "connect to something somebody else is listening on"
will use.
3. PBUFFER WAS A DirectGLES-ONLY KNOB. Twice over: trace-replay-ci.sh gated
`--ez use_pbuffer true` on the backend name, and the JNI demanded a native
window for DirectVulkan unconditionally. A surface SHAPE is not a backend
property, and the coupling mattered because the Android GLWS makes a window
surface whenever gNativeWindow is non-null - so handing Magma a window was
the same as asking for one. Until P12 a spawned server cannot be given a
window at all: an ANativeWindow* is a pointer into the CLIENT's process and
SetWindowHandle is refused on the way across with Fatal{UnmigratedSurface,
"AndroidNativeWindow@P12"} (Rule H). Which is exactly what happened, one op
after the server had come up green.
4. THE RUNNER LOOKED IN ONE OF THE TWO OUTPUT DIRECTORIES. assembleTraceRelease
writes apk/trace/release; CI has staged the same artifact under
apk/trace/debug. A tree that has built locally has both, and picking the
wrong one retraces last month's APK and says nothing about it. Newest of
either, now.
MEASURED AND NOT FIXED: DirectVulkan on this device diverges under BOTH split
transports and does so nondeterministically - inproc scored ssim=0.988998 /
9836 px, spawn scored 0.976494 / 14658 px and then 0.988998 / 9836 px on a
repeat, while monolith with the same pbuffer surface scores 1.000000 / 0 px.
The two transports draw from the same set of outcomes, so this is the split
path on Magma against a real GPU, not the process boundary: P6 neither caused
it nor can fix it. DirectVulkan spawn IS green on desktop (lavapipe, 1.000000).
G14: one test name added, AnAbstractEndpointNeedsNoWritableDirectoryAndLeaves-
NoFile, which pins both halves - the handshake crosses an abstract name, and
nothing is left on the filesystem. A '@' mistaken for a path would have bound
a file of that literal name, passed here, and failed on the one platform this
exists for.
The cross-process session is up. A server process is started on its own with an
endpoint and nothing else, it listens; a client started separately connects to
that name; Hello crosses, Welcome comes back with four SegmentRefs, six
descriptors cross by SCM_RIGHTS, and the client maps memory another process
created. ClientSession::Active() is non-null at the end of it, which is the flag
that arms the thirty-seven wire emitters.
INDEPENDENT, NOT FORKED - and that is a correction to this package's first
shape. The version committed an hour ago forked the server and handed it fds
3..6. It worked, and it was less code, but inheritance can only ever produce a
server that is a CHILD OF ITS CLIENT, which is the one arrangement the end state
cannot use: there the server is a standing application and the client is
somewhere else entirely, possibly on another kernel. CONTRACT-P6 §3.1 called
this Dial == Connect and filed it as a later shape. It is the primary one, and
fork/execve survives only as a way to START a process - every descriptor it used
to carry is gone.
What that costs and buys, concretely: the rendezvous becomes a NAME, so
SocketTransport grows Listen/AcceptPair/ConnectTo, with a bounded connect retry
that is the CLIENT's (a launcher that waited for the server to be ready would be
coupling the two again by the back door). The bells stop being inherited fds and
are created by the server and handed over with the segments - slots 4 and 5 of
six SCM_RIGHTS offers. And the client connects TWICE to the same path: an fd
offer is a sendmsg whose ancillary data rides with specific bytes, so it may not
share a socket with the framed control stream, or the frame reassembler and the
descriptor receiver race for the same bytes.
The pieces behind it:
- SessionSegments::AdoptFromDescriptors, which is the same Adopt+Map loop
AttachInProcess has run on every inproc session since P5. That was the point
of writing the inproc attach as dup+Adopt+Map rather than as an alias: only
the origin of the descriptor is new.
- ServerSession::SetExternalDoorbells, the narrow fix for a6 row A7-1 - the
§3.9 accessors abort for every role but InProcess, so a spawn apply thread
died before popping its first record. lk deletes them; until then an injected
pair is honest about where the bells come from.
- MG_Backend::InitServerRoleForSpawn: steps 1 and 2 of the split bring-up
extracted so the server process runs the SAME code the inproc server role
runs. The capability bits are a statement about the server's own backend and
a second copy would diverge the first time a backend registers a slot.
- ClientSession::FinishStartup: everything below the attach - producer, reply
pool, event ring, segment table, encoder, first CapsSnapshot, emitter tables -
shared verbatim between inproc and spawn. Its one parameter is not a
transport question: under inproc the client starts the apply thread because
there is no server-side main; under spawn there is one, and it started the
thread before it sent the first descriptor.
G1 CAUGHT A REAL LEAK on the way: InitServerRoleForSpawn was guarded on the
inside, so the pull build still DEFINED the symbol - 2 added, 1 removed, .text
+16 bytes. The whole function and its declaration are now inside the guard.
"It returns false there" is not the same as "it is not there", and the gate is
what said so.
Gates: ServerSpawnTest 4/4; unit suite 2344 with the same two pre-existing
environmental failures; G1 pull 0 added / 0 removed, .text 10822147 unchanged.
The first P6 code that runs. ITransport over a connected AF_UNIX SOCK_STREAM
socket, with SCM_RIGHTS descriptor passing on a dedicated SOCK_DGRAM aux pair.
No shared memory: this is the CONTROL plane, and the hot path bypasses
ITransport entirely by its own header's account. What the socket carries is the
handshake, surface ops, resync, aux requests and fatals - and, once sm exists,
the four segment descriptors that let the peer map the data plane. It moves the
keys, not the bytes.
CONTRACT-P6 §8 and the spawn plan both require this green over a plain
socketpair with two threads BEFORE a child is ever forked, for the reason
SessionRings.h gives about the attach half: a transport first exercised on the
day the process appears is a transport whose bugs all arrive at once, on the
hardest day to debug them. That requirement is now met.
It reinvents nothing. Framing.h already had AppendFrame and a real byte-stream
reassembler with exactly the BUFFER_TOO_SMALL keep-the-message semantics
ITransport specifies - written for this and never before driven by a socket.
FdPassing.h already had SCM_RIGHTS over a dedicated datagram pair. What is new
is ~250 lines joining them, plus the platform refusal: Windows has no
SCM_RIGHTS and ShmSegment::Adopt is POSIX-only, so CreatePair refuses BY NAME
at construction rather than handing back a transport that fails every later
call.
SOCK_STREAM rather than SOCK_SEQPACKET, deliberately: Doorbell.h:390-398
already chose a stream socket so a closed peer surfaces as POLLIN|POLLHUP with
recv()==0 rather than as a hang, and the control plane may not disagree with
the bell about what death looks like. Framing.h is what makes a stream safe -
the message boundary is in the payload, not in the socket.
TESTS: the same nine ITransport-contract cases InProcessTransportTest runs,
copied rather than shared. Folding both suites into one parameterised fixture
would rename the sixteen existing cases and G14 says names only grow; and the
two suites do not share a handle anyway, since InProcessTransport's tests reach
into condvar doorbells this class does not have. Nine names added, zero
removed. The seven Doorbell* cases are deliberately absent: they exercise
CondVarDoorbell, not ITransport.
Plus three a stream has and a message queue cannot: a 256 KiB payload that
provably crosses in several reads; a garbage stream that must latch failed and
must NOT resync onto a boundary it invented; and adopting an already-connected
fd with no aux channel, which answers UNSUPPORTED to ShareFd rather than
pretending - rule G, a transport declares what it supports.
Red-once, both run: drop the message on BUFFER_TOO_SMALL and six cases go red;
make the framing latch recoverable and AGarbageStreamIsLatchedFailed fails by
name.
Gates: 12/12 in 677 ms. Unit suite 2340 tests, up 12, with the same two
pre-existing environmental failures. G1 pull 0 added / 0 removed, .text size
10822147 unchanged, and zero SocketTransport symbols in the pull build.
One note for the VM track this eventually serves: swapping AF_UNIX for
AF_VSOCK is one line here and nothing at all in Framing. It is the shared-memory
half that does not cross a kernel.
swung0x48