MOBILEGL_BUILD_TRACE_REPLAY now also produces mobilegl_trace_replay on
ANDROID: the same CLI as desktop, linked against the APK runner's
glws/glproc glue. On the device it dials a render server with no
Activity at all - MOBILEGL_IPC_CONTROL=unix:@name with shm data to the
offscreen supervisor (a cross-uid abstract connect needs root or an
SELinux waiver; loopback TCP works from plain adb shell), or
tcp:// with MOBILEGL_IPC_SURFACE=server to the on-screen display server.
glws_android grows the desktop glws_egl's server-owned null-window
createWindowSurface, gated on MOBILEGL_TRACE_SURFACE=window so the APK's
pbuffer mode keeps its old answer, and the config carries EGL_WINDOW_BIT
for it. The control screen refuses unix endpoints in on-screen mode:
the in-process display server serves tcp:// only.
Verified on-device with an OpenRA retrace, ssim=1.0 both ways:
offscreen service on @mgl-cli with the CLI as root, and the display
Activity on tcp://127.0.0.1:40613 with the replay's frames on the
phone's screen.
It was one case on one backend under one transport: OpenRA, DirectGLES,
inproc. It is now every CI case, both backends, inproc AND spawn - 154 CI
entries, 156 local ctest entries - because every defect the spawn work turned
up was a question about the SERVER's process and not about any one trace or
either backend.
THE DirectVulkan ARM. tools/trace_replay/CMakeLists.txt carried "a
DirectVulkan arm would be measuring a server nobody has written yet". True
when P5 wrote it; false since P5f/P7 landed Magma's server consumers and its
run-ahead readiness gate. Leaving it would have shipped a split shape whose
second backend nothing ever ran - the same class of hole as a lane that runs
monolith under a name that says split. Magma now brings up a server process
of its own and OpenRA retraces across it at ssim=1.000000, which is only
possible because this branch also taught the child to load its config: before
that, a DirectVulkan session brought up a DirectGLES server and the two
halves disagreed about which backend the capability mask describes.
`split` BECOMES AN OPT-OUT. It began as an opt-in naming one case, because
one case was the work. Marking 39 cases `"split": true` would carry no
information and would reinstate the hole the key was written to close, only
mirrored: a case added later would get the monolith arms and be silently
absent from the split ones, and absent is the one state nothing reds on. A CI
case is now in the subset unless it says `"split": false`, which is a
reviewable claim that a trace cannot cross a role boundary.
THE CI MATRIX GAINS A THIRD DIMENSION. The job took {backend, case} and
hard-coded MOBILEGL_TRANSPORT=inproc in its own run block, so `spawn` had no
way into CI at all; the .SPAWN ctest entries existed only for a local build.
It is {backend, case, transport} now, the backends are the case's own
ci_backends rather than a hard-coded DirectGLES, and the artifact names carry
the transport or the two arms of one {backend, case} would overwrite each
other's uploads.
inproc STAYS AS THE CONTROL. Same library, same codec, same applier, same
picture; the only difference is which process the server role runs in.
Running only spawn would leave "did the transport break it?" and "did the
phase break it?" answerable only together - which is the question every
failure in this phase has turned on, including the SIGSEGV that D1 explained.
BOTH NEGATIVE CONTROLS FOLLOW THE ARM. retrace_pull_library_control.sh and
retrace_drop_draw_control.sh each exported MOBILEGL_TRANSPORT=inproc
themselves, so on a spawn job they would have gone on proving something about
the other arm while spawn ran unguarded. They read SPLIT_TRANSPORT now, and
default to inproc so a caller that sets nothing behaves exactly as before.
scripts/ci/control_smoke_test.sh: 22 passed, 0 failed against the stubbed
ctest, including the two modes that matter most - a red whose SSIM never fell
and a knob that armed and dropped nothing.
The spawn arm needs a second binary and the inproc arm needs none, so the job
places libMobileGLServer.so only on the spawn arm and a missing one is a
named failure there rather than a LaunchServer refusal 78 jobs deep.
Measured locally as this lands: 39 of the 78 SPAWN entries complete, all
green. The rest are still running.
OpenRA.DirectGLES.SPAWN is green: ssim=1.000000, mismatchPixels=0, zero
Fatal{ lines, against a server in a process the client started by name. The
inproc SPLIT arm stays beside it as the control, so "did the transport break
it" and "did the phase break it" remain separately answerable.
Five defects stood between the handshake and that picture. Each one was
silent in its own way, and none of them looked like what it was.
1. THE DOORBELL COULD NOT WAKE ITSELF. SocketDoorbell's one-fd form sends on
the end it polls, so a write is delivered to the PEER - correct for waking
the other process, useless for waking a thread in this one. The server's
control pump posts into the apply thread's mailbox and must ring the bell
that thread parks on. Symptom: every eglInitialize crossed, landed in the
mailbox, and timed out at 5 s. The two-fd form (park on [0], notify on [1])
is restored here, and the bell handover goes from two descriptors to THREE
- the client needs the same split for the same reason. Seven offers now.
2. InitCapabilities HAD NO WIRE KIND. CONTRACT-P6 §2 predicted this exactly
("the request has a live client caller and the codec refuses to encode
it"). SurfaceOpKind gains 11, append-only; the decode range's upper bound
moves with it, or the new tag reads as out-of-range and surfaces as
Fatal{ProtocolCorruption} rather than the missing row it is. The op's name
loses its InprocOnly suffix, which stopped being true here.
3. THE CHILD LOADED NO CONFIG AT ALL. mobilegl_server_main calls
InitServerRoleForSpawn directly, and MG_ConfigLoader::Init is a step of
MobileGL::Initialize - so every MG_Config value in the server was its
static default. MOBILEGL_BACKEND_TYPE survives the launcher's scrub on
purpose and was read by nobody: a DirectVulkan session would have brought
up a DirectGLES server and then disagreed with its client about which
backend the capability mask describes.
4. AND THEREFORE Transport READ AS Monolith, which CONTRACT-P6 §3.1 (D1)
forbids in as many words. a6 measured the cost: 226 live lines test
`Transport != Monolith` to select the SERVER arm, 148 in MG_Backend.
Monolith flips all of them to frontend glue in the one process with no
frontend. Observed precisely as predicted - UnitTexturesByHandle() answered
false, Clear took the legacy arm that dereferences borrowed FRONTEND slot
pointers, and the apply thread took a SIGSEGV inside SyncNeccessaryTextures
on the first Clear record of the trace. It is pinned in-process rather than
through the environment because the environment is where it must NOT be:
LooksLikeAClientEnvironment exits 65 on a surviving MOBILEGL_TRANSPORT, and
that check is anti-recursion catch (b). Dial, not Transport, is that axis.
5. THE LOG WAS CORRUPT, NOT MERELY INTERLEAVED. Two processes opened the same
MOBILEGL_LOG_FILE_PATH with "w" and each carried its OWN file offset, so
they overwrote each other's bytes - half-lines like `ote client: CapsMirror
generation 1 adopted`, and the server's whole startup block vanishing under
the client's later writes. This cost real diagnosis time twice, because a
missing line read as a code path not taken. O_APPEND now; the client still
truncates once so a fresh run is not a pile of old ones, and the server
never does - it would delete the very startup lines a failed session needs.
run_trace_case.cmake's Fatal{ census reads exactly this file, so it was
also reading a lie.
The retrace runner's split assertions become transport-aware: one row per
transport it admits, and a transport with no row is a FATAL rather than a
fall-through, because the fall-through is the accident the block exists to
prevent. The spawn row carries a SECOND sentence and it is the stronger half
- ConfigLoader's marker proves the value RESOLVED, while `spawn ARMED - the
server role runs in pid N` is written only after the launch, the connect and
the handshake all succeeded, and carries a pid that is not this process's.
inproc resolves its own marker while running the server role on a thread
here; nothing on that path can write this one.
G14: one test name added (TheEleventhFramedOpIsInitCapabilitiesAndItCrosses-
TheWire), none removed. The ten-op table keeps its name and its scope rather
than being renamed to eleven.
Two unit failures predate this change and are environmental in this tree:
PipeCatalogue.FrontendNeverTakesAnApplierAddress (the frontend source root
resolves empty) and SessionHandshakeTest.TheAbiFingerprintChangesWhenAnyOf-
ItsInputsDoes (GIT_COMMIT_HASH_SHORT is empty here, which a6 already recorded
as red in this configuration).
- A retrace that exports MOBILEGL_PIPE_VERIFY=1 at a library configured without
-DMOBILEGL_PIPE_VERIFY=ON is a no-op: the frames still match their goldens and the
case reports green having compared nothing. run_trace_case.cmake now demands the
evidence whenever the variable is set to anything but 0/false - mobilegl.log must
exist, must carry "MGPipe: verify armed", and must carry neither
Fatal{PipeVerifyDiffer nor Fatal{UnmigratedPipeInput. With the variable unset the
script is byte-for-byte the old one.
- The Fatal scan is not redundant with the replay's exit status:
MOBILEGL_PIPE_VERIFY_FATAL=0 is the supported triage configuration, and there a
divergence is logged and counted rather than aborted, so the run would finish 0
with its own report sitting unread in the log.
- LABELS retrace on both registrations: the lane was selectable only by regex, so
`ctest -L retrace --no-tests=error` - the spelling that reds a lane which
registered nothing - could not be written at all.
- "verify": true on eight cases (the five 180s cases plus minecraft-1.21.4-in-world,
minecraft-1.21.4-fabric-sodium-in-world and improved-transparency-minecraft-26.3),
and --format github-verify-matrix over that subset: 16 entries, against the full
lane's 77. The verify build compares at every verb boundary and again at every
accessor read, which the design budgets at 5-10x, so the per-push job runs the
subset and the full sweep is a phase-exit / workflow_dispatch run.
- The flag is validated in the manifest loader, not at the matrix, so a non-boolean
or a verify case excluded from CI is a loud error in every consumer instead of a
subset that is quietly one case short.
- PLAN-B.md §8.2 and appendix B add a batch of new runtime switches
(MOBILEGL_PIPE_PUSH / _VERIFY / _STATS / _LEGACY_MEMOS / _TEXEL_RETAIN_MB /
_INDEX_MIRROR_MB, plus MOBILEGL_IPC_* later), and §11 P0 wants them parsed beside
the existing ones. Today every knob that has to reach an Android replay costs an
edit in five files - run_android_retrace_local.py, trace-replay-ci.sh,
TraceReplayActivity's request record, the JNI marshalling, and the setenv block in
trace_replay_core.cpp. That per-knob tax is what this replaces: one extra,
`--es mobilegl_env "K=V;K=V"`, carries all of them.
- Applied last, immediately before dlopen(libMobileGL.so), so it can also override
the dedicated fields above it - MobileGL's config is read during the load, and an
escape hatch that cannot beat the defaults is not one. An entry with no '=' unsets
the variable, which is the only way to clear a default the marshalling sets.
- The existing per-knob flags stay: they carry semantics beyond a setenv (use_angle
also selects a variant, the dump lists are joined, DirectVulkan forces the
R11G11B10F fallback), and rewriting them as env strings would move that logic into
the callers.
- Surface: --env / MOBILEGL_TRACE_ENV in trace-replay-ci.sh, repeatable --env
KEY=VALUE in run_android_retrace_local.py, `mobilegl_env` intent extra,
Request::envOverrides.
- The two-level parse now lives in trace_env_overrides.hpp, beside the semicolon
splitter it shares with the texture and FBO dump lists, and
tools/trace_replay/trace_env_overrides_test.cpp pins it: the empty entries a
trailing ';' leaves behind must not become unsetenv(""), `K=` must stay a Set of
the empty string rather than an Unset (a knob read with getenv() != nullptr sees
those as opposite answers), and only the FIRST '=' may separate, or a value
carrying '=' is truncated without a word of warning. The whole MOBILEGL_PIPE_*
batch rides on this parse, and the only lane that exercised it end to end was an
on-device retrace, which would have reported a splitting bug as "the knob had no
effect".
- The check is built and RUN at build time and mobilegl_trace_replay depends on it,
so `cmake --build ... --target mobilegl_trace_replay` - the exact command of
test.yml's "Build trace replay" job, which never invokes ctest - runs it. It is
assert-free on purpose: that lane configures Release, and <cassert> under NDEBUG
would compile every check into a green run that checked nothing. Negative control:
swapping find('=') for rfind('=') fails 2 checks, and keeping the splitter's empty
entries fails 2 more.
- compare actual output against primary and alternate golden images
- record the matched golden path in trace replay results
- allow APK and Linux retrace fixtures to pass alternate golden paths
- keep nostalgia validation accepting both Mesa and PC goldens