[MG_Remote, CI] (P6): the split retrace lane becomes the whole CI matrix, both backends, both transports

It was one case on one backend under one transport: OpenRA, DirectGLES,
inproc. It is now every CI case, both backends, inproc AND spawn - 154 CI
entries, 156 local ctest entries - because every defect the spawn work turned
up was a question about the SERVER's process and not about any one trace or
either backend.

THE DirectVulkan ARM. tools/trace_replay/CMakeLists.txt carried "a
DirectVulkan arm would be measuring a server nobody has written yet". True
when P5 wrote it; false since P5f/P7 landed Magma's server consumers and its
run-ahead readiness gate. Leaving it would have shipped a split shape whose
second backend nothing ever ran - the same class of hole as a lane that runs
monolith under a name that says split. Magma now brings up a server process
of its own and OpenRA retraces across it at ssim=1.000000, which is only
possible because this branch also taught the child to load its config: before
that, a DirectVulkan session brought up a DirectGLES server and the two
halves disagreed about which backend the capability mask describes.

`split` BECOMES AN OPT-OUT. It began as an opt-in naming one case, because
one case was the work. Marking 39 cases `"split": true` would carry no
information and would reinstate the hole the key was written to close, only
mirrored: a case added later would get the monolith arms and be silently
absent from the split ones, and absent is the one state nothing reds on. A CI
case is now in the subset unless it says `"split": false`, which is a
reviewable claim that a trace cannot cross a role boundary.

THE CI MATRIX GAINS A THIRD DIMENSION. The job took {backend, case} and
hard-coded MOBILEGL_TRANSPORT=inproc in its own run block, so `spawn` had no
way into CI at all; the .SPAWN ctest entries existed only for a local build.
It is {backend, case, transport} now, the backends are the case's own
ci_backends rather than a hard-coded DirectGLES, and the artifact names carry
the transport or the two arms of one {backend, case} would overwrite each
other's uploads.

inproc STAYS AS THE CONTROL. Same library, same codec, same applier, same
picture; the only difference is which process the server role runs in.
Running only spawn would leave "did the transport break it?" and "did the
phase break it?" answerable only together - which is the question every
failure in this phase has turned on, including the SIGSEGV that D1 explained.

BOTH NEGATIVE CONTROLS FOLLOW THE ARM. retrace_pull_library_control.sh and
retrace_drop_draw_control.sh each exported MOBILEGL_TRANSPORT=inproc
themselves, so on a spawn job they would have gone on proving something about
the other arm while spawn ran unguarded. They read SPLIT_TRANSPORT now, and
default to inproc so a caller that sets nothing behaves exactly as before.
scripts/ci/control_smoke_test.sh: 22 passed, 0 failed against the stubbed
ctest, including the two modes that matter most - a red whose SSIM never fell
and a knob that armed and dropped nothing.

The spawn arm needs a second binary and the inproc arm needs none, so the job
places libMobileGLServer.so only on the spawn arm and a missing one is a
named failure there rather than a LaunchServer refusal 78 jobs deep.

Measured locally as this lands: 39 of the 78 SPAWN entries complete, all
green. The rest are still running.
This commit is contained in:
2026-09-21 12:31:49 -04:00
parent 7e244b3775
commit dc5bd935de
5 changed files with 146 additions and 59 deletions
+31 -20
View File
@@ -449,10 +449,14 @@ endfunction()
# --- P5: the SPLIT arm of the retrace matrix -------------------------------------------------
#
# One variant entry per case that trace_cases.json marks `split: true` - today that is OpenRA
# alone, which is the only target the phase gate names and (measurably) the only fixture hydrated
# locally. DirectGLES only: the split shape's first backend is Espryt, and a DirectVulkan arm
# would be measuring a server nobody has written yet.
# One variant entry per case that trace_cases.json marks `split: true`.
#
# BOTH BACKENDS, as of P6. The DirectGLES-only note this comment used to carry -
# "a DirectVulkan arm would be measuring a server nobody has written yet" - was
# true when P5 wrote it and stopped being true at P5f/P7, which landed Magma's
# server consumers and its run-ahead readiness gate. Leaving it would have meant
# shipping a split shape whose second backend nothing ever ran, which is the
# same class of hole as a lane that runs monolith under a name that says split.
#
# REGISTERED ONLY UNDER MOBILEGL_BUILD_DISAGGREGATED, because the arm is a statement about a
# library that compiled MG_Remote. A build without the option accepts MOBILEGL_TRANSPORT=inproc
@@ -463,24 +467,31 @@ function(add_trace_replay_split_test CASE_NAME)
if(NOT MOBILEGL_BUILD_DISAGGREGATED)
return()
endif()
add_trace_replay_test(${CASE_NAME} DirectGLES VARIANT SPLIT TRANSPORT inproc
IPC_SERVER_PATH "${CMAKE_BINARY_DIR}/libMobileGLServer.so" ${ARGN})
# --- P6: the SPAWN arm, beside the inproc one rather than instead of it ------------------
# --- P6: TWO TRANSPORTS x TWO BACKENDS, four entries per case ---------------------------
#
# TWO ARMS, NOT ONE RENAMED. The inproc arm is the CONTROL: it is the same
# library, the same codec, the same applier and the same picture, with the
# only difference being which process the server role runs in. A spawn arm
# that had replaced it would have left "did the transport break it?" and
# "did the phase break it?" answerable only together.
# TWO TRANSPORTS, NOT ONE RENAMED. The inproc arm is the CONTROL: same
# library, same codec, same applier, same picture, with the only difference
# being which PROCESS the server role runs in. A spawn arm that had replaced
# it would leave "did the transport break it?" and "did the phase break it?"
# answerable only together.
#
# G14 is satisfied by construction - a name is added, none is removed - and
# the VARIANT gives it its own output and artifact directories, which is the
# half the comment above says is easy to forget: both arms write
# output/mobilegl.log, and that log is the ONLY valid refusal census because
# the console sink is compiled out of these configurations.
add_trace_replay_test(${CASE_NAME} DirectGLES VARIANT SPAWN TRANSPORT spawn
IPC_SERVER_PATH "${CMAKE_BINARY_DIR}/libMobileGLServer.so" ${ARGN})
# TWO BACKENDS, because every bug this phase found so far was a question
# about the SERVER's process rather than about Espryt - the child loading no
# config, Transport reading as Monolith there, a doorbell that could not wake
# its own thread. None of those is backend-shaped, and all of them would have
# been just as invisible on a Magma server nobody ran.
#
# G14 is satisfied by construction - names are added, none removed - and the
# VARIANT gives each entry its own output and artifact directories, which is
# the half that is easy to forget: all four write output/mobilegl.log, and
# that log is the ONLY valid refusal census because the console sink is
# compiled out of these configurations.
foreach(backend DirectGLES DirectVulkan)
add_trace_replay_test(${CASE_NAME} ${backend} VARIANT SPLIT TRANSPORT inproc
IPC_SERVER_PATH "${CMAKE_BINARY_DIR}/libMobileGLServer.so" ${ARGN})
add_trace_replay_test(${CASE_NAME} ${backend} VARIANT SPAWN TRANSPORT spawn
IPC_SERVER_PATH "${CMAKE_BINARY_DIR}/libMobileGLServer.so" ${ARGN})
endforeach()
endfunction()
set(MOBILEGL_TRACE_CASES_CMAKE "${CMAKE_CURRENT_BINARY_DIR}/trace_cases.cmake")
+47 -19
View File
@@ -12,8 +12,10 @@ CI_BACKENDS = ("DirectGLES", "DirectVulkan")
# silent no-op, which is review finding N-4: `"split": true` mistyped as `"splitt": true` loaded
# clean, the split subset became [], the GitHub matrix became {"include":[]}, and `retrace-split`
# was skipped with no red anywhere. Every other way of getting `split` wrong already raised
# (`"ci": false`, a backend list without DirectGLES, a non-bool value) - the typo was the one hole,
# and it is the shape that makes a whole CI job quietly stop existing.
# (`"ci": false`, a non-bool value) - the typo was the one hole, and it is the shape that makes a
# whole CI job quietly stop existing. P6 inverted the key's default (see load_trace_case_manifest),
# which closes the mirror-image hole: a case added later is IN the split subset unless it says
# otherwise, so forgetting the key can no longer drop a case out of the lane.
#
# Adding a key means adding it here, deliberately, in the same commit. That is the point.
KNOWN_CASE_KEYS = frozenset({
@@ -80,23 +82,32 @@ def load_trace_case_manifest(path=TRACE_CASES_JSON):
raise ValueError(
f"{name} is marked verify but excluded from CI, so the verify matrix would drop it"
)
# "split" opts a case into the MOBILEGL_TRANSPORT=inproc retrace subset, the same shape
# and the same reason as "verify" above: a typo has to be a loud manifest error in every
# consumer, not a subset that is quietly one case short. The split arm additionally runs
# DirectGLES ONLY - the server the arm exercises is Espryt's - so a case that excluded
# DirectGLES from CI would leave the split matrix with nothing to run.
split = merged.get("split", False)
# "split" IS AN OPT-OUT, NOT AN OPT-IN, and P6 inverted it deliberately.
#
# It began as an opt-in listing one case, because the split arm was one
# case's worth of work. Now that the whole CI matrix runs split, 40
# identical `"split": true` lines would carry no information AND would
# reinstate the exact hole the key was written to close: a case added
# later gets the monolith arms by default and is silently absent from
# the split ones. Absent is the one state nothing reds on.
#
# So: a CI case is in the split subset unless it says `"split": false`,
# and saying so is a reviewable claim that this trace cannot cross a
# role boundary. `ci: false` removes a case from CI entirely and takes
# its split arms with it - the subset is a SUBSET of the CI matrix, and
# a case CI never runs cannot be one CI runs split.
#
# The arm runs the case's OWN ci_backends, both of them by default. The
# DirectGLES-only requirement that used to live here is gone with the
# DirectGLES-only arm (tools/trace_replay/CMakeLists.txt says why).
split = merged.get("split", merged.get("ci", True))
if not isinstance(split, bool):
raise ValueError(f"split must be true or false for {name}")
if split and not merged.get("ci", True):
raise ValueError(
f"{name} is marked split but excluded from CI, so the split matrix would drop it"
)
if split and "DirectGLES" not in ci_backends(merged):
raise ValueError(
f"{name} is marked split but does not run DirectGLES in CI; the split arm is "
f"DirectGLES-only, so the entry would be registered with no backend"
)
merged["split"] = split
cases.append(merged)
return {"defaults": defaults, "cases": cases}
@@ -154,11 +165,12 @@ def verify_trace_cases(cases):
def split_trace_cases(cases):
"""The subset the split (MOBILEGL_TRANSPORT=inproc) CI mode retraces.
"""The subset the split CI modes retrace - inproc AND spawn, both backends.
P5's phase gate names exactly one: OpenRA, at SSIM >= 0.99. It is also the only fixture that
is hydrated locally, so keeping the subset explicit in the manifest is what stops a later
phase from widening the arm into an LFS fetch by accident.
P5's phase gate named exactly one case, OpenRA at SSIM >= 0.99, because one case was the
work. P6 runs the whole CI matrix across a process boundary, so the subset is now every CI
case that has not explicitly opted out. load_cases() has already resolved the default, so
"split" is present and boolean on every case by the time this reads it.
"""
return [case for case in cases if case.get("split", False)]
@@ -193,12 +205,28 @@ def github_verify_matrix(cases):
return github_test_matrix(verify_trace_cases(cases))
# The two transports the split retrace lane runs. inproc is the CONTROL - same library, same
# codec, same picture, server role on a thread HERE - and spawn is the claim. Both, always: a
# spawn failure that inproc also shows is a phase bug, and one inproc does not show is a
# transport bug, and collapsing the two makes that distinction unaskable.
SPLIT_TRANSPORTS = ("inproc", "spawn")
def github_split_matrix(cases):
"""{backend, case} for the split arm. DirectGLES only - see split_trace_cases."""
"""{backend, case, transport} for the split arm.
THREE DIMENSIONS, NOT TWO. The job used to take {backend, case} and hard-code
MOBILEGL_TRANSPORT=inproc in its own run block, so `spawn` had no way into CI at all. The
backend list is the case's own ci_backends - the same list the monolith matrix uses - so a
case restricted to one backend stays restricted here instead of being registered with a
backend it does not run.
"""
return {
"include": [
{"backend": "DirectGLES", "case": case["name"]}
{"backend": backend, "case": case["name"], "transport": transport}
for case in split_trace_cases(cases)
for backend in ci_backends(case)
for transport in SPLIT_TRANSPORTS
]
}