run_matrix.sh drives interleaved monolith/inproc/spawn+shm/spawn+tcp
benchmark rounds on an adb device with per-run arm-marker verification;
profile_matrix.sh layers simpleperf, ftrace wakeup capture and a /proc
sampler on top; make_report.py aggregates runs into REPORT.md. The wifi
kit carries the cross-host control-plane hang repro and phone-side
strace/fd-reconcile helpers. .trace-work/ (run output) is now ignored.
MOBILEGL_BUILD_TRACE_REPLAY now also produces mobilegl_trace_replay on
ANDROID: the same CLI as desktop, linked against the APK runner's
glws/glproc glue. On the device it dials a render server with no
Activity at all - MOBILEGL_IPC_CONTROL=unix:@name with shm data to the
offscreen supervisor (a cross-uid abstract connect needs root or an
SELinux waiver; loopback TCP works from plain adb shell), or
tcp:// with MOBILEGL_IPC_SURFACE=server to the on-screen display server.
glws_android grows the desktop glws_egl's server-owned null-window
createWindowSurface, gated on MOBILEGL_TRACE_SURFACE=window so the APK's
pbuffer mode keeps its old answer, and the config carries EGL_WINDOW_BIT
for it. The control screen refuses unix endpoints in on-screen mode:
the in-process display server serves tcp:// only.
Verified on-device with an OpenRA retrace, ssim=1.0 both ways:
offscreen service on @mgl-cli with the CLI as root, and the display
Activity on tcp://127.0.0.1:40613 with the replay's frames on the
phone's screen.
transport_wait_ns measures wall time in Ring::Wait's flush/spin/sleep, so
the client's parked time is read directly instead of inferred by
subtracting CPU clocks that run concurrently with the server.
frame_stats.py matches client and server present serials into per-frame
avg/p50/p99 splits; the README documents the MOBILEGL_PIPE_STATS=1
workflow.
- README, ARCHITECTURE, ROADMAP, MEASUREMENTS and CURRENT_STAGE_PROGRESS are rewritten as indexes
(84 KB -> 15 KB): what the project is, where it stands, and where each detail lives. ROADMAP is one
plain sentence per phase; MEASUREMENTS is the five conclusions that matter plus one line per phase;
CURRENT_STAGE_PROGRESS is the P12 goal, a checklist and the next steps.
- ARCHITECTURE's body moves unchanged into ten design/ files, keeping §1-§17 and appendices A/B, so
`ARCHITECTURE.md §N` references in code resolve through the table in ARCHITECTURE.md.
- guide/ holds build-and-run, code map, glossary and the engineering discipline (from README and
ROADMAP), and the device pinning record that used to sit in devices/.
- The open-debt table moves to notes/DEBTS.md; the detailed current-stage tables (with decision IDs,
the open line items and evidence locations) move into notes/p12/README.md. notes/README.md maps
every moved path and section.
- The documentation citation lint in test.yml also covers design/ and guide/, which describe the
current tree as the top-level files did.
- ROADMAP, MEASUREMENTS and ARCHITECTURE are rewritten as summaries (333 KB of index text -> 84 KB).
ROADMAP's phase table is one line per phase plus a link; MEASUREMENTS keeps headline numbers under
its old §1-§13 numbering (plus §14-§16 for P6.5, P7 and P12); ARCHITECTURE keeps every decided rule
under the same §1-§17 / appendix numbering, so the `ARCHITECTURE.md §N` references in code still
land, and §17 becomes an index of the phase landing shapes.
- CURRENT_STAGE_PROGRESS now describes the current stage (the P12 on-screen-server subset) and absorbs
CURRENT_STAGE_STATUS, which is removed. The P7 status tables, the P6 closing snapshot and the P6.5
first-wave snapshot move to notes/p7, notes/p6 and notes/p65.
- Each phase gets notes/<phase>/README.md: a short summary, then the verbatim material that left
the index documents (its ROADMAP row, its MEASUREMENTS section with subsection numbers kept, its
ARCHITECTURE §17 part, milestones) and a file index. P8-P11 and P13 get their planned scope;
the full open-question list moves to notes/OPEN-QUESTIONS.md.
- The phase reports that sat at the top level move into their phase folders with unchanged names:
P5D-INPROC-PERFORMANCE, P5E-RUNAHEAD, P5F-WIRE-COMPLETENESS, P6-SPAWN-PLAN, P6-CONTRACT-DRAFT,
P6-ENDSTATE-REVIEW. Their relative links, CONTRACT-P6.md's links to them and one tool docstring
are updated; notes/README.md maps every old path and section number to its new place.
- 71 files under notes/recovered/ that were byte-identical copies of phase-folder files are removed.
- Corrections made while condensing: the server process reads Transport as Spawn (CONTRACT-P6 §3.1
amended the old "child forces monolith" rule); both backends now publish kCapRunAheadApply;
MOBILEGL_IPC_RESPAWN is a named refusal and MOBILEGL_IPC_IDLE_EXIT_S is not parsed; the Redmi
pin row is in tools/device_bench/pin_device.sh.
check_doc_citations over docs/Disaggregated/*.md: 0 problems. No relative link is newly broken.
- the round-2 merge folded "TCP session has no forwarded server log" into the one server-log
check for every transport, and its message named only MOBILEGL_TRANSPORT - which is `spawn`
for the tcp arm too (tcp = spawn + MOBILEGL_IPC_CONTROL=tcp://), so the tcp arm's missing
forwarded log read as a plain spawn red
- the message carries MOBILEGL_IPC_CONTROL='...' after the transport (empty for inproc and
plain spawn); message only, no check moved
- the spawn server-log fatal from the previous round sat BEFORE the marker check and the
`spawn ARMED ... pid` check, so a launch or handshake failure - no server process, no ARMED
line - reported "the run wrote no server log" instead of the more specific "resolved, but
carries no spawn ARMED", and sent the reader to the wrong process. It now sits just before
the census; the tcp arm's "no forwarded server log" check, which had the same placement,
moves with it
- the inproc row demands the server log too. Under inproc every server-role line is routed
by thread role to the `.server.log` sink, opened lazily on the first such line, and at the
INFO level this lane demands ServerLoop writes one when mgl-srv-apply starts - so a clean
inproc run always has the file, and "no file" read as "0 lines" was a green Fatal{ and
MGWIRE-FLOOR census of the role that emits both. The two census reads are unconditional
now: the fatal above them is what makes the file exist
- fail-closed demonstrations, OpenRA DirectVulkan, the replay wrapped to damage one log after
a real run: inproc with the server log deleted went rc 0 on the previous runner (both
censuses printed 0) and is now a fatal at the census; spawn with the server log deleted is
the same fatal; spawn with the server log deleted AND the ARMED line removed reports the
ARMED message, where the previous runner reported the server-log one; spawn with only the
ARMED line removed reports the ARMED message. Undamaged, OpenRA DirectVulkan and DirectGLES
SPLIT and SPAWN stay green, 0 unsound lines
Fable review of the B3 rework, should-fix: run_trace_case.cmake read the
server log only `if(EXISTS ...)`. Under spawn the server process writes its
own file, and the lines the runner counts out of it - the applier Fatal{
lines and the MGWIRE-FLOOR unsound-serial-complete line, which only the
server's VulkanRenderer::OnSubmitsCompletedUpTo emits - are nowhere else,
so a server that died before its first line read as "0 lines" and went
green. The client log was already fail-closed, and so was the tcp arm's
forwarded server log; plain spawn was not.
The plain-spawn branch of the transport block now FATALs when
mobilegl.server.log does not exist, beside the tcp arm's identical check.
Demonstration: the SPAWN ctest command line with TRACE_REPLAY_EXE replaced
by a wrapper that runs the real replay and then deletes
output/mobilegl.server.log (541 lines):
runner at aab01275 -> rc 0, "Fatal{ lines across ... : 0",
"MGWIRE-FLOOR unsound-serial-complete lines: 0" (green, half unseen)
this runner -> rc 1:
CMake Error at run_trace_case.cmake:321 (message):
OpenRA DirectVulkan: MOBILEGL_TRANSPORT=spawn but the run wrote no
.../output/mobilegl.server.log, so the server role's Fatal{ and
MGWIRE-FLOOR lines cannot be counted. A spawn retrace with no
server log cannot be counted as a clean one.
MobileGLTraceReplay.OpenRA.DirectVulkan.SPLIT / .SPAWN unmodified: both
pass, ssim 1.000000, 0 px, 0 MGWIRE-FLOOR lines.
Review ID-P7-34, item 2. The primary B3 fix - the clamp in
VulkanRenderer::OnSubmitsCompletedUpTo that raises the completed-frame-
serial floor only to a serial no in-flight submission still carries - had
nothing behind it: deleting the clamp loop left every lane green and the
OpenRA picture gold on lavapipe (the host's barriers hide the tear). Its
only observable was the per-event MGLOG_W "MGWIRE-FLOOR
unsound-serial-complete".
run_trace_case.cmake now reads that line out of both roles' logs on every
split run, exactly as it already reads Fatal{: the count is printed pass
or fail and any nonzero count is a FATAL_ERROR. The R-16 counter (26 -> 0
on the OpenRA retrace) is thereby a gate.
Red/green on MobileGLTraceReplay.OpenRA.DirectVulkan.SPLIT / .SPAWN:
clamp loop deleted -> both FAIL at ssim 1.000000 / 0 px:
MGPipe split: OpenRA DirectVulkan transport=inproc, MGWIRE-FLOOR
unsound-serial-complete lines: 26
[mgl-srv-apply/WARN]: MGWIRE-FLOOR unsound-serial-complete #1: advancing
the completed frame-serial floor to 3, which submission 3 (serial 3)
still carries
(spawn: identical, 26)
clamp restored -> both PASS, 0 lines.
run_cts.py silently tolerates a missing --skip-file, so the known device-killers from
the skill (clip_distance.functional, the multisampled_to_singlesampled blit family,
the rgb565 repeat-mode case) would each have cost a reboot in the gate-5 window.
KHR-GL46 has no uniform_buffer_object group; the 90 UBO cases are
GTF-GL46.gtf31.GL3Tests.uniform_buffer_object*, present only when the deployed glcts
was built with GTF - the device probe decides. KHR-GL46.shader_storage_buffer_object
(124) is the block that exercises what cluster A actually refuses today: unaligned
and over-range SSBO bindings, not only UBO byte tails.
Cut from VK-GL-CTS 0b04c470ed78569b0b778ece1262d88455ab2b00 (main; the deployed glcts build's source) mustpass
gl46-main.txt and gl33-main.txt: texture_*, shader_image*, packed_pixels,
direct_state_access, uniform_buffer*. The gl33 variants are the fallback the runbook
names for an Android runner that cannot open a 4.6 core context; whichever version the
probe selects, BASE and AFTER read the same list. A 0.5 pp gate needs a committed
denominator, not a glob over someone's checkout.
wave 0 item 5. Three instruments P7's device window needs, and the CI steps that
can make each of them red.
run_tcp_matrix.py --backend DirectVulkan end to end. --backend already existed and
already filtered by each case's ci_backends; what did not exist is a per-backend
GOLDEN. The two backends shared `golden` plus an optional `alternate_golden`, and
alternate_golden is an OR across backends, not a per-backend key - so a
DirectVulkan run that matched the DirectGLES picture passed, which is the "went
green on the wrong arm" shape the exit gate exists to catch. trace_cases.json
gains `backend_overrides.<backend>` ({golden, alternate_golden, ssim_threshold}
only: restating target_call or a crop would stop the two backends retracing the
same frame through the same window). A per-backend golden REPLACES the shared
pair, matching another backend's declared image is refused BY NAME, and a --case
named explicitly that does not run a requested backend is an error instead of one
plan fewer and no message. The CMake emitter and the APK matrix refuse a case that
declares the key rather than silently emitting one golden for both arms.
DirectGLES is unchanged and that is asserted, not asserted-in-prose: a case with no
override resolves to itself, and the key is dropped from the resolved case even for
the backend that overrides nothing - so giving DirectVulkan its own golden does not
move the DirectGLES identity fingerprint and does not invalidate its checkpoints.
Each results.json row now records the golden, alternate golden and threshold it was
scored against.
run_android_retrace_local.py gets --matrix (the CI split subset, 39 cases, each
running only its own ci_backends - not --all, which adds the rd12 workload ID-P7-4
excludes), --repeat N, and --archive-dir. The archive is the point: the result root
is keyed by case and backend and by NOTHING else, so repeat 2 overwrites repeat 1
and the only artefact that survives a three-repeat run is the last one, which makes
"were the three repeats the same picture?" unanswerable after the fact - in the one
place where re-running to find out is expensive. It also refuses --benchmark with a
non-monolith --transport, which has always produced a monolith run under a spawn
label (run_benchmark_case passes args.env alone; p6/README.md records the pitfall).
compare_actuals.py computes the pair result.json cannot: SSIM between two actual
PNGs, plus a case x repeat x ssim-vs-golden x ssim-vs-first-repeat table over an
archive. It is the retrace gate's OWN SSIM - a transcription of ComputeChannelSsim
/ ComputeRgbSsim (trace_replay_core.cpp:630/:671), the only SSIM in the tree - and
the crop comes from each repeat's own result.json, so the two columns describe the
same rectangle. PNG decode is stdlib zlib, because an unavailable Pillow in the
device window would turn "the repeats diverged" into "the tool did not run".
TraceReplayActivity: resolveSpawnServerPath's two decisions were String.contains()
on the whole joined K=V;K=V list and neither was entry-aware. The one that bit:
`--env MOBILEGL_IPC_SERVER_PATH` with no '=' is the documented UNSET spelling
(trace_env_overrides_test.cpp:92) and the only way to ask for "spawn with no image"
and watch LaunchServer refuse by name - it did not match "MOBILEGL_IPC_SERVER_PATH="
so a resolved path was appended AFTER it and, because ApplyEnvOverrides applies in
order with setenv(...,1), the deliberate unset was silently undone. That is exactly
the "silently replacing it would make that knob untestable" the method's own comment
forbids. Two more: the key inside another entry's VALUE flipped both decisions the
wrong way, and the transport match was case-sensitive while ConfigLoader lower-cases
the value before matching (ConfigLoader.cpp:322). The parse moves to
SpawnServerPath, a class with no android.* import, so test_android_lifecycle.py
compiles and runs it with plain javac - the parse had no host-side coverage at all,
and its on-device failure mode is "the knob had no effect".
Only spawn gets a path, deliberately: it is the only transport that execs an image.
Red-once, all four: a stub case whose result names the DirectGLES golden fails a
--backend DirectVulkan run with "matched a golden another backend declares"; a
synthetic PNG pair with a closed-form SSIM (identical = 1.0 exactly, constant
black vs white = C1/(255^2+C1), inverted halves = (C2-2v)/(2v+C2)); the pre-fix
resolveSpawnServerPath body fails eight of SpawnServerPathTest's checks; and the
DirectVulkan TCP proof battery is the DirectGLES one under the other prefix,
because an empty prefix selection raises nothing and reads as a clean pass.
CI: test_run_tcp_matrix.py and test_compare_actuals.py join the R-16 step. The
matrix driver's stub regressions had NO CI step - they were run by hand and
recorded in a note - and they guard the tool that decides whether a device golden
run passed.
Counts in WSL: test_run_tcp_matrix 12 -> 18, tcp_lane_tools 4 -> 6,
test_compare_actuals 12 new, test_android_lifecycle green with the new Java unit.
swung0x48