Breaks the one process = one server = one session = one context equation
that the whole stack's correctness used to rest on. Design and per-slice
shape: docs/Disaggregated/design/11-state-ownership.md.
S1 wire: bind_context ring record (opcode 84) announces the current
context at the make-current edge; CreateContext/DestroyContext join the
SurfaceOp control frames; control revision 5 -> 6; the server keeps a
per-session token table and refuses unknown tokens by name.
S2 server: SessionRuntime per session with thread-scoped resolution
(singletons become fallbacks), per-thread gPipeInputs, per-session latch
domains, per-thread segment resolvers; the in-process unix supervisor
serves up to MOBILEGL_IPC_INPROC_MAX_SESSIONS (16) concurrent sessions;
surface mode is declared per session and the display lease refuses a
second on-screen session by name.
S3 frontend: ShareGroupState owns the shared object stores and name
generators (buffer/texture/renderbuffer/sampler/program), GLContext keeps
per-context bindings/errors/container objects; pGLContext becomes
thread-local with a process default context preserving the single-context
world verbatim; eglCreateContext builds a real GLContext and joins the
sharing context's group.
S4 backend: native EGL tuples keyed by (session, context token);
InitDisplayAndContext's unconditional teardown is gone - creating a
second context or surface no longer destroys the first; eglCreateContext
sharing is wired to the native driver; bind_context is the native switch
point.
S5 applier: MGPipeApplierState splits into MGPipeObjectRecords (per
share group) and MGPipeWorkingState (per context), resolved through a
{session, token} registry with a thread-local cached fast path.
Tests: MultiSessionTest (incl. two sessions x two contexts interleaved
with real clear/readback witnesses), ApplierOwnershipTest (with a
null-resolver negative control), ContextStateOwnershipTest,
ServerLoopEglTest native-context liveness; 99/99 of the acceptance set
green on the host build.
Known debts (documented in the design doc): twin registries still keyed
by handle alone, no per-record ContextSlot, process-global backend
generations, MakeCurrent dedup per current binding.
A Wayland client's window is its own wl_surface, and what the compositor shows is the wl_buffer
attached to it - but the pixels of a MobileGL frame are drawn where the backend is, on a split
build in another process on another OS. So far a wl_egl_window was taken for an X11 window and
its client-side address sent to the Android server (EGL_BAD_NATIVE_WINDOW at best), and under
glvnd a Wayland display never reached this vendor at all.
A window on a Wayland display - asked for as EGL_PLATFORM_WAYLAND, or a wl_display recognised by
its first word being wl_display_interface - is now a window surface in the EGL state at the
wl_egl_window's size with a pbuffer of that size behind it in the backend, whatever
MOBILEGL_IPC_SURFACE says. At every swap the frame is read back out of the default framebuffer
through this library's own glReadPixels (the application's read framebuffer, pack buffer and pack
state put back as they were), its rows flipped and swizzled into a wl_shm buffer, and attached,
damaged and committed on the application's wl_surface. Three buffers rotate on their release
events, dispatched on an event queue of the surface's own. libwayland-client is loaded at run
time, so neither the build nor the library depends on it. EGL_KHR_platform_wayland and
EGL_EXT_platform_wayland are announced on Linux.
Measured: glmark2-wayland runs its build scene at 13 FPS (400x300) and the window shows the frame
under kwin. A window resize is not followed yet; the surface keeps its creation size.
On a system with libglvnd the libEGL.so.1 an application links is a dispatcher, not an
implementation, and the supported way to be the implementation it talks to is to be a vendor
library it loads and routes to. On this machine it is also the ONLY way that reaches a caller which
does not link EGL directly: kwin goes through libepoxy, which dlopen()s libEGL.so.1 and dlsyms its
entry points at startup, so a library injected with LD_PRELOAD is invisible to it however loudly it
is named - measured, with the client in kwin's maps and not one call into it.
So this is the vendor half of that ABI: __egl_Main answers glvnd with the entry points by name, the
platforms this implementation serves, and the device it is associated with - the render node of the
machine, because that is what lets a compositor match its own node to the display it is about to
draw into, and because glvnd drops a vendor that does not answer at all.
Two details are the difference between a vendor that is loaded and a vendor that is used. glvnd
requires setDispatchIndex to exist even when a vendor has nothing to record, which this one has -
its entry points are the implementation, they never re-enter glvnd. And a vendor must hand over ITS
OWN entry points: with default visibility a reference inside this library resolves against the whole
process, where another libEGL that loaded first wins. That is what -Bsymbolic-functions is for;
declaring the entry points protected says the same thing but does not compile against the platform
headers, which declare them default.
MOBILEGL_BUILD_TRACE_REPLAY now also produces mobilegl_trace_replay on
ANDROID: the same CLI as desktop, linked against the APK runner's
glws/glproc glue. On the device it dials a render server with no
Activity at all - MOBILEGL_IPC_CONTROL=unix:@name with shm data to the
offscreen supervisor (a cross-uid abstract connect needs root or an
SELinux waiver; loopback TCP works from plain adb shell), or
tcp:// with MOBILEGL_IPC_SURFACE=server to the on-screen display server.
glws_android grows the desktop glws_egl's server-owned null-window
createWindowSurface, gated on MOBILEGL_TRACE_SURFACE=window so the APK's
pbuffer mode keeps its old answer, and the config carries EGL_WINDOW_BIT
for it. The control screen refuses unix endpoints in on-screen mode:
the in-process display server serves tcp:// only.
Verified on-device with an OpenRA retrace, ssim=1.0 both ways:
offscreen service on @mgl-cli with the CLI as root, and the display
Activity on tcp://127.0.0.1:40613 with the replay's frames on the
phone's screen.
Step two of CONTRACT-P6 5.2. The ~90 scattered `MGLOG_F("...Fatal{...}"); std::abort();`
pairs under MG_Remote/ become one call each to SessionFail(MGFatalFamily, fmt, ...),
which logs, echoes to stderr, counts, and aborts in one place. a6's census
found no single chokepoint existed; now there is one, and step three hangs the
SessionFault-to-peer publish off it.
THE MESSAGE STRING IS PASSED VERBATIM. The contract requires the log lines, the
CI greps and every red-once byte-identical, so SessionFail does not compose the
line from the family - it takes the SAME string the site handed MGLOG_F and
forwards it unchanged. The family enum rides alongside, redundantly, so the
funnel has the coarse wire FatalCode without parsing the string. The census now
checks that redundancy the way it checks WireLogFatal: every SessionFail call's
string must carry a Fatal{ word, so the enum and the string cannot silently
disagree.
The conversion was mechanical - 89 sites by a transform that injects the family
as MGLOG_F's new first argument and drops the trailing abort - with three sites
fixed by hand that the transform would have broken: a stub macro, and two
if/else-then-shared-abort blocks (PersistentMapTracker's adopt-tier and
SurfaceOpCodec's three surface-error branches) where a naive rewrite stole the
abort from the earlier branches and would have let them fall through. The
death tests are the safety net for byte-identity, and every one stayed green.
Four std::abort() remain under MG_Remote/, all inside the two funnels
(WireLog.cpp, FatalFunnel.cpp); the census exempts both files because their
callers carry the family word, and its baseline drops from 93 aborts to 4.
SessionFaultCount() is the one telemetry point S8 will assert zero over a good
run.
G1 re-measured: 0 symbol diff, .text unchanged at 0xa52203 (FatalFunnel and
FatalFamily live under MG_Remote/, compiled only in the disaggregated build).
Unit 2358 (one pre-existing environmental failure), integration-split 186/186,
integration-spawn 102/102, retrace 6/6, census 0 unmarked.
CONTRACT-P6 4.2 and 4.3. Three defects, all of them things the handshake
believed it was checking and was not.
1. THE BUILD STAMP WAS "" AND NOTHING SAID SO. CMakeLists took the git hash
through an execute_process with NO RESULT_VARIABLE, so a failed
`git rev-parse` yielded the empty string. Verified live rather than
inferred: this worktree's `.git` is a file holding a WINDOWS path, WSL's
git answers `fatal: not a git repository`, result 128 - and every WSL build
mixed "" into its ABI fingerprint. Two builds from different commits then
agreed exactly, which is the one thing the stamp exists to prevent. The
already-failing SessionHandshakeTest was reporting this the whole time.
The fix may NOT touch GIT_COMMIT_HASH_SHORT: it is baked into
glGetString(GL_VERSION), so redefining it changes an application-visible GL
string and the pull build's .rodata - a behaviour leak and a G1 break at
once. MGGitHash.h.in gains a SECOND pair instead, consumed only by the
fingerprint, plus -DMOBILEGL_BUILD_STAMP for trees git cannot read (CI has
${GITHUB_SHA} in hand). A stampless build is now a configure-time WARNING
naming the consequence, not silence.
MOBILEGL_BUILD_STAMP_PRESENT IS MIXED AS ITS OWN FIELD. "this build cannot
name its commit" and "this build's commit is the empty string" are different
claims, and collapsing them is how every stampless build in the world agreed
with every other. The test asserts the flag moves the fingerprint in EITHER
build state, and the `BuildStamp = ""` perturbation is now guarded - in a
stampless build production's value already IS "", so that assertion was
passing while proving nothing.
2. THE CLIENT NEVER READ abiMajor/abiMinor. Hello has carried them since P5
and the SERVER checks them; the client compared only the fingerprint. The
fingerprint mixes AbiVersion so a mismatch would fire anyway - but as
"struct shapes", which sends the reader looking for a layout change that is
not there. Under spawn the peer is a SEPARATE IMAGE the operator chose with
MOBILEGL_IPC_SERVER_PATH, so this is exactly the failure that needs its own
sentence. Both handshake sites call one helper; there are two sites, and
only one of them was reachable before `sm`, which is how the gap survived.
3. Hello::pid WAS 0 AND Welcome::serverPid ECHOED IT. So both were always 0 and
neither named anything, while §9.5's arm-proof gate wants exactly the child
pid. Both now carry real ids, and the pair is the cheapest arm proof in the
protocol: under spawn they MUST differ, and a same-process session cannot
produce two different values. The client keeps what the server STATED -
local launcher knowledge is a different fact from one that crossed the wire,
and only the second can be asserted from the server's own mouth.
R-16 both ways: revert serverPid to echoing hello->pid() and the new test reds
on both halves, naming 4.3. G1 re-measured after the CMake and header changes:
0 symbol diff, .text unchanged at 0xa52203.
Unit suite goes from two long-standing failures to one - the remaining
PipeCatalogue.FrontendNeverTakesAnApplierAddress is an environmental path
problem in this tree, unrelated. All six OpenRA arms still green.
OpenRA.DirectGLES, transport=spawn, on a Redmi: ssim=1.000000,
mismatchPixels=0, zero Fatal{ lines, three runs for three. The server is a
process the client launched out of the APK's lib/arm64/, and the client's own
log names its pid.
Four things stood between the desktop green and the device green, and every
one of them was invisible on desktop for the same reason: a build tree is not
a deployment, and Linux is not Android.
1. THE SERVER COULD NOT LINK. It links libMobileGL.so and is EXEC'd, not
dlopen'd, so it gets a fresh linker namespace with the DEFAULT search path
- and an APK's lib/<abi>/ is not on it. `CANNOT LINK EXECUTABLE ...:
library "libMobileGL.so" not found: needed by main executable`, before a
single line of MobileGL ran. On desktop CMake had been giving the target an
absolute build-tree RPATH, so the WSL server resolved its dependency
through a path no phone has. $ORIGIN is asked for via BUILD_RPATH (a raw
-Wl,-rpath,$ORIGIN reaches the generator unquoted and the SHELL eats it -
the device linker said `normalize_path - invalid input: "$"`), and because
the NDK drops RPATH from the link anyway, the LAUNCHER states
LD_LIBRARY_PATH: it is the one component that has already resolved where
the image lives, and a pair that is not co-located is not a pair.
2. THE RENDEZVOUS NEEDED A WRITABLE DIRECTORY. An Android app has no /tmp, so
bind() in the child had nowhere to put the node and the client refused by
name 20 s later. AF_UNIX endpoints now accept the ABSTRACT namespace with a
leading '@' - the systemd/D-Bus convention - which has no filesystem
presence at all: no directory, no mode bits, and no stale node to unlink
after a crash, which is the bind() failure Listen() has been working
around. Its reach is the network namespace, which for an app is the app.
StartSpawned mints '@' names; an explicit path still works everywhere, and
that is what P12's "connect to something somebody else is listening on"
will use.
3. PBUFFER WAS A DirectGLES-ONLY KNOB. Twice over: trace-replay-ci.sh gated
`--ez use_pbuffer true` on the backend name, and the JNI demanded a native
window for DirectVulkan unconditionally. A surface SHAPE is not a backend
property, and the coupling mattered because the Android GLWS makes a window
surface whenever gNativeWindow is non-null - so handing Magma a window was
the same as asking for one. Until P12 a spawned server cannot be given a
window at all: an ANativeWindow* is a pointer into the CLIENT's process and
SetWindowHandle is refused on the way across with Fatal{UnmigratedSurface,
"AndroidNativeWindow@P12"} (Rule H). Which is exactly what happened, one op
after the server had come up green.
4. THE RUNNER LOOKED IN ONE OF THE TWO OUTPUT DIRECTORIES. assembleTraceRelease
writes apk/trace/release; CI has staged the same artifact under
apk/trace/debug. A tree that has built locally has both, and picking the
wrong one retraces last month's APK and says nothing about it. Newest of
either, now.
MEASURED AND NOT FIXED: DirectVulkan on this device diverges under BOTH split
transports and does so nondeterministically - inproc scored ssim=0.988998 /
9836 px, spawn scored 0.976494 / 14658 px and then 0.988998 / 9836 px on a
repeat, while monolith with the same pbuffer surface scores 1.000000 / 0 px.
The two transports draw from the same set of outcomes, so this is the split
path on Magma against a real GPU, not the process boundary: P6 neither caused
it nor can fix it. DirectVulkan spawn IS green on desktop (lavapipe, 1.000000).
G14: one test name added, AnAbstractEndpointNeedsNoWritableDirectoryAndLeaves-
NoFile, which pins both halves - the handshake crosses an abstract name, and
nothing is left on the filesystem. A '@' mistaken for a path would have bound
a file of that literal name, passed here, and failed on the one platform this
exists for.
The first time this project has had a second process. fork + execve of a real
server image, the control socket surviving the exec, bytes crossing a process
boundary, and the child exiting on EOF with nothing left behind.
mobilegl_server_main EXISTS NOW. a6 found ARCHITECTURE.md:488-496 had described
it for three phases while the symbol was nowhere in the tree, and the only
MobileGLServer target was the P0 spike stub, which links no MobileGL library at
all. The spike keeps its evidence and its name becomes MobileGLServerSpike; the
real image links the library, so an undefined symbol is a build error rather
than a dlsym failure in a child whose stderr is /dev/null.
WHAT sm OWNS AND WHAT IT LEAVES: the process, the inherited descriptors, the
transport and the exit discipline. Not the session, not the applier, not the
four segments - wiring those from ServerMain before lk has a seam to wire them
THROUGH is how call sites get written twice. The loop is a read/answer/exit-on-
EOF stand-in whose SHAPE is the part that matters: one reader, EOF ends the
process, and no timeout is ever allowed to stand in for the death fact (§5.4).
ANTI-RECURSION IS TWO CATCHES, and neither is ARCHITECTURE.md:488's. That rule
says the child hard-sets Transport to Monolith; a6 measured that 226 live lines
test `Transport != Monolith` to select the SERVER arm, 148 in MG_Backend, so
obeying it flips the whole backend to frontend glue in the one process with no
frontend. Instead: (a) MOBILEGL_IPC_DIAL=no, structural and separate from role,
and (b) MOBILEGL_TRANSPORT and every MOBILEGL_IPC_* scrubbed from the child's
envp. The child refuses with 64 for (a) and 65 for (b) so the two safeties are
distinguishable - two that cannot be told apart are one.
TWO BUGS WORTH RECORDING, both found by the test rather than by review:
dup2(n, n) IS A NO-OP THAT DOES NOT CLEAR CLOSE-ON-EXEC. FdPassing sets
SOCK_CLOEXEC on the aux pair, so the child depends on dup2 to clear it - but in
a gtest process a fresh socketpair lands on exactly fds 3..8, so a source fd
often IS its own target. The no-op left CLOEXEC set and the following close()
then shut the descriptor we had just "installed". The child came up with no
socket and the parent saw TRANSPORT_CLOSED on its first send. Now spelled out
rather than written cleverly.
THE CHILD'S DIAGNOSTICS WERE INVISIBLE. WireLogError reaches the file sink
only - Defines.h builds the logger with the console sink off - so a child that
refused to start said so into a file the parent does not know the path of.
ARCHITECTURE.md §15.2 warns about exactly this on Android. Both refusal paths
now also write stderr directly, and both carry a distinct exit code the parent
can name without reading anything.
TESTS: six cases, 6/6. Each exists because something would otherwise pass
silently - the reply carries the child's own getpid() so a test that talked to
itself cannot pass (ID-124's lesson at the smallest scale); the process tree is
COUNTED before, during and after, not promised; the child is proven not to exit
while merely idle before it is proven to exit on EOF; an unresolvable image is a
named refusal that spawns nothing (S1); a surviving client knob is caught by
name (S3); and a descriptor crosses execve by SCM_RIGHTS, which is the mechanism
the four segments will ride on and is now exercised before it is needed.
Also records the asio decision in the contract: declined for P6 because spawn
needs five platform mechanisms and asio covers one - its own socketpair returns
operation_not_supported on Windows, it has no descriptor passing, segment
delivery is POSIX-only by design, and there is no fork. Revisit at P6.5, where
asio::generic::stream_protocol carries a raw family/type/protocol triple and
therefore gives AF_VSOCK from one implementation.
Gates: unit suite 2346, up 6, same two pre-existing environmental failures. G1
pull 0 added / 0 removed, .text size 10822147 unchanged, and zero spawn, server
or SocketTransport symbols in the pull build.
The first P6 code that runs. ITransport over a connected AF_UNIX SOCK_STREAM
socket, with SCM_RIGHTS descriptor passing on a dedicated SOCK_DGRAM aux pair.
No shared memory: this is the CONTROL plane, and the hot path bypasses
ITransport entirely by its own header's account. What the socket carries is the
handshake, surface ops, resync, aux requests and fatals - and, once sm exists,
the four segment descriptors that let the peer map the data plane. It moves the
keys, not the bytes.
CONTRACT-P6 §8 and the spawn plan both require this green over a plain
socketpair with two threads BEFORE a child is ever forked, for the reason
SessionRings.h gives about the attach half: a transport first exercised on the
day the process appears is a transport whose bugs all arrive at once, on the
hardest day to debug them. That requirement is now met.
It reinvents nothing. Framing.h already had AppendFrame and a real byte-stream
reassembler with exactly the BUFFER_TOO_SMALL keep-the-message semantics
ITransport specifies - written for this and never before driven by a socket.
FdPassing.h already had SCM_RIGHTS over a dedicated datagram pair. What is new
is ~250 lines joining them, plus the platform refusal: Windows has no
SCM_RIGHTS and ShmSegment::Adopt is POSIX-only, so CreatePair refuses BY NAME
at construction rather than handing back a transport that fails every later
call.
SOCK_STREAM rather than SOCK_SEQPACKET, deliberately: Doorbell.h:390-398
already chose a stream socket so a closed peer surfaces as POLLIN|POLLHUP with
recv()==0 rather than as a hang, and the control plane may not disagree with
the bell about what death looks like. Framing.h is what makes a stream safe -
the message boundary is in the payload, not in the socket.
TESTS: the same nine ITransport-contract cases InProcessTransportTest runs,
copied rather than shared. Folding both suites into one parameterised fixture
would rename the sixteen existing cases and G14 says names only grow; and the
two suites do not share a handle anyway, since InProcessTransport's tests reach
into condvar doorbells this class does not have. Nine names added, zero
removed. The seven Doorbell* cases are deliberately absent: they exercise
CondVarDoorbell, not ITransport.
Plus three a stream has and a message queue cannot: a 256 KiB payload that
provably crosses in several reads; a garbage stream that must latch failed and
must NOT resync onto a boundary it invented; and adopting an already-connected
fd with no aux channel, which answers UNSUPPORTED to ShareFd rather than
pretending - rule G, a transport declares what it supports.
Red-once, both run: drop the message on BUFFER_TOO_SMALL and six cases go red;
make the framing latch recoverable and AGarbageStreamIsLatchedFailed fails by
name.
Gates: 12/12 in 677 ms. Unit suite 2340 tests, up 12, with the same two
pre-existing environmental failures. G1 pull 0 added / 0 removed, .text size
10822147 unchanged, and zero SocketTransport symbols in the pull build.
One note for the VM track this eventually serves: swapping AF_UNIX for
AF_VSOCK is one line here and nothing at all in Framing. It is the shared-memory
half that does not cross a kernel.
The twelve Server* forwarders' one-slot mailbox carried a raw function
pointer plus a void* to a stack-local Args struct (P5c audit row G4);
neither means anything across a process boundary. The slot now carries
one SurfaceControlFrame by value: kind is the op, scalars are the
arguments, and ok/eglMajor/eglMinor come back in the same value. The
blocking handshake, the shadow, the doorbell order and the C2 exit
block are unchanged.
- protocol.fbs: append-only - SurfaceOpKind gains SetSwapInterval=8,
ReleaseResources=9, SetWindowHandle=10; WindowKind gains MetalLayer=6;
SurfaceOp gains readSurface and context (the MakeCurrent four-tuple).
protocol_generated.h regenerated with the pinned flatc.
- Server/SurfaceControlFrame.{h,cpp}: the frame (trivially copyable by
static_assert) and the op enum - ten wire kinds plus four inproc-only
kinds (InitCapabilities is answered by CapsSnapshot on the wire,
SwapBuffers/InitWindowSurface have no production caller - f0-egl F8,
kept for the inproc test lane).
- Protocol/SurfaceOpCodec.{h,cpp}: the frame <-> SurfaceOp/SurfaceReply
codec, the explicit WindowBackend <-> WindowKind table, and the wire
entry ServerApplyWireSurfaceOp whose refusals are named:
Fatal{UnmigratedSurface, "AndroidNativeWindow@P12"} and
Fatal{ProtocolCorruption, "SurfaceOp"}.
- ServerLoop: RunOnApplyThread(fn, void*) -> RunSurfaceControlFrame(frame)
plus the dispatch ApplySurfaceControlFrame; the old per-forwarder Args
bodies moved into the dispatch unchanged (C7/ID-54, N-3, R-12 included).
RunProbeOnApplyThreadForTesting is the test seam through the same
channel (the hook lives in a member, never in the slot).
- Tests: RemoteClientTest/ServerLoopTest migrated to the probe seam; new
SurfaceControlFrameTest suite (mapping tables, wire round-trips, the
two named refusals) plus ServerLoopTest's AVoidForwarderCrossesAsOne-
DispatchedFrame (the frame channel's red-once handle) and
ServerLoopEglTest's TheInitializeDisplayReplyArrivesThroughTheFrame.
swung0x48
BZLZHH