Commit Graph
51 Commits
Author SHA1 Message Date
swung0x48 ee9919ba39 [Fix] (CI): merge Arch WSL runner and prefer it over hosted runners 2026-10-05 09:16:06 -04:00
swung0x48 e3e890ed5d [Feat] (Trace): the replay runner builds as an adb-shell executable for Android, and unix endpoints carry a headless window
MOBILEGL_BUILD_TRACE_REPLAY now also produces mobilegl_trace_replay on
ANDROID: the same CLI as desktop, linked against the APK runner's
glws/glproc glue. On the device it dials a render server with no
Activity at all - MOBILEGL_IPC_CONTROL=unix:@name with shm data to the
offscreen supervisor (a cross-uid abstract connect needs root or an
SELinux waiver; loopback TCP works from plain adb shell), or
tcp:// with MOBILEGL_IPC_SURFACE=server to the on-screen display server.

glws_android grows the desktop glws_egl's server-owned null-window
createWindowSurface, gated on MOBILEGL_TRACE_SURFACE=window so the APK's
pbuffer mode keeps its old answer, and the config carries EGL_WINDOW_BIT
for it. The control screen refuses unix endpoints in on-screen mode:
the in-process display server serves tcp:// only.

Verified on-device with an OpenRA retrace, ssim=1.0 both ways:
offscreen service on @mgl-cli with the CLI as root, and the display
Activity on tcp://127.0.0.1:40613 with the replay's frames on the
phone's screen.
2026-09-27 06:28:00 -04:00
swung0x48 018fab0f5b [WIP] (MG_Remote): implement P6.5 TCP and stream transport 2026-09-22 07:19:18 -04:00
swung0x48 dc5bd935de [MG_Remote, CI] (P6): the split retrace lane becomes the whole CI matrix, both backends, both transports
It was one case on one backend under one transport: OpenRA, DirectGLES,
inproc. It is now every CI case, both backends, inproc AND spawn - 154 CI
entries, 156 local ctest entries - because every defect the spawn work turned
up was a question about the SERVER's process and not about any one trace or
either backend.

THE DirectVulkan ARM. tools/trace_replay/CMakeLists.txt carried "a
DirectVulkan arm would be measuring a server nobody has written yet". True
when P5 wrote it; false since P5f/P7 landed Magma's server consumers and its
run-ahead readiness gate. Leaving it would have shipped a split shape whose
second backend nothing ever ran - the same class of hole as a lane that runs
monolith under a name that says split. Magma now brings up a server process
of its own and OpenRA retraces across it at ssim=1.000000, which is only
possible because this branch also taught the child to load its config: before
that, a DirectVulkan session brought up a DirectGLES server and the two
halves disagreed about which backend the capability mask describes.

`split` BECOMES AN OPT-OUT. It began as an opt-in naming one case, because
one case was the work. Marking 39 cases `"split": true` would carry no
information and would reinstate the hole the key was written to close, only
mirrored: a case added later would get the monolith arms and be silently
absent from the split ones, and absent is the one state nothing reds on. A CI
case is now in the subset unless it says `"split": false`, which is a
reviewable claim that a trace cannot cross a role boundary.

THE CI MATRIX GAINS A THIRD DIMENSION. The job took {backend, case} and
hard-coded MOBILEGL_TRANSPORT=inproc in its own run block, so `spawn` had no
way into CI at all; the .SPAWN ctest entries existed only for a local build.
It is {backend, case, transport} now, the backends are the case's own
ci_backends rather than a hard-coded DirectGLES, and the artifact names carry
the transport or the two arms of one {backend, case} would overwrite each
other's uploads.

inproc STAYS AS THE CONTROL. Same library, same codec, same applier, same
picture; the only difference is which process the server role runs in.
Running only spawn would leave "did the transport break it?" and "did the
phase break it?" answerable only together - which is the question every
failure in this phase has turned on, including the SIGSEGV that D1 explained.

BOTH NEGATIVE CONTROLS FOLLOW THE ARM. retrace_pull_library_control.sh and
retrace_drop_draw_control.sh each exported MOBILEGL_TRANSPORT=inproc
themselves, so on a spawn job they would have gone on proving something about
the other arm while spawn ran unguarded. They read SPLIT_TRANSPORT now, and
default to inproc so a caller that sets nothing behaves exactly as before.
scripts/ci/control_smoke_test.sh: 22 passed, 0 failed against the stubbed
ctest, including the two modes that matter most - a red whose SSIM never fell
and a knob that armed and dropped nothing.

The spawn arm needs a second binary and the inproc arm needs none, so the job
places libMobileGLServer.so only on the spawn arm and a missing one is a
named failure there rather than a LaunchServer refusal 78 jobs deep.

Measured locally as this lands: 39 of the 78 SPAWN entries complete, all
green. The rest are still running.
2026-09-21 12:31:49 -04:00
swung0x48 7e244b3775 [MG_Remote] (P6/cp): a real trace retraces across two processes, pixel-for-pixel
OpenRA.DirectGLES.SPAWN is green: ssim=1.000000, mismatchPixels=0, zero
Fatal{ lines, against a server in a process the client started by name. The
inproc SPLIT arm stays beside it as the control, so "did the transport break
it" and "did the phase break it" remain separately answerable.

Five defects stood between the handshake and that picture. Each one was
silent in its own way, and none of them looked like what it was.

1. THE DOORBELL COULD NOT WAKE ITSELF. SocketDoorbell's one-fd form sends on
   the end it polls, so a write is delivered to the PEER - correct for waking
   the other process, useless for waking a thread in this one. The server's
   control pump posts into the apply thread's mailbox and must ring the bell
   that thread parks on. Symptom: every eglInitialize crossed, landed in the
   mailbox, and timed out at 5 s. The two-fd form (park on [0], notify on [1])
   is restored here, and the bell handover goes from two descriptors to THREE
   - the client needs the same split for the same reason. Seven offers now.

2. InitCapabilities HAD NO WIRE KIND. CONTRACT-P6 §2 predicted this exactly
   ("the request has a live client caller and the codec refuses to encode
   it"). SurfaceOpKind gains 11, append-only; the decode range's upper bound
   moves with it, or the new tag reads as out-of-range and surfaces as
   Fatal{ProtocolCorruption} rather than the missing row it is. The op's name
   loses its InprocOnly suffix, which stopped being true here.

3. THE CHILD LOADED NO CONFIG AT ALL. mobilegl_server_main calls
   InitServerRoleForSpawn directly, and MG_ConfigLoader::Init is a step of
   MobileGL::Initialize - so every MG_Config value in the server was its
   static default. MOBILEGL_BACKEND_TYPE survives the launcher's scrub on
   purpose and was read by nobody: a DirectVulkan session would have brought
   up a DirectGLES server and then disagreed with its client about which
   backend the capability mask describes.

4. AND THEREFORE Transport READ AS Monolith, which CONTRACT-P6 §3.1 (D1)
   forbids in as many words. a6 measured the cost: 226 live lines test
   `Transport != Monolith` to select the SERVER arm, 148 in MG_Backend.
   Monolith flips all of them to frontend glue in the one process with no
   frontend. Observed precisely as predicted - UnitTexturesByHandle() answered
   false, Clear took the legacy arm that dereferences borrowed FRONTEND slot
   pointers, and the apply thread took a SIGSEGV inside SyncNeccessaryTextures
   on the first Clear record of the trace. It is pinned in-process rather than
   through the environment because the environment is where it must NOT be:
   LooksLikeAClientEnvironment exits 65 on a surviving MOBILEGL_TRANSPORT, and
   that check is anti-recursion catch (b). Dial, not Transport, is that axis.

5. THE LOG WAS CORRUPT, NOT MERELY INTERLEAVED. Two processes opened the same
   MOBILEGL_LOG_FILE_PATH with "w" and each carried its OWN file offset, so
   they overwrote each other's bytes - half-lines like `ote client: CapsMirror
   generation 1 adopted`, and the server's whole startup block vanishing under
   the client's later writes. This cost real diagnosis time twice, because a
   missing line read as a code path not taken. O_APPEND now; the client still
   truncates once so a fresh run is not a pile of old ones, and the server
   never does - it would delete the very startup lines a failed session needs.
   run_trace_case.cmake's Fatal{ census reads exactly this file, so it was
   also reading a lie.

The retrace runner's split assertions become transport-aware: one row per
transport it admits, and a transport with no row is a FATAL rather than a
fall-through, because the fall-through is the accident the block exists to
prevent. The spawn row carries a SECOND sentence and it is the stronger half
- ConfigLoader's marker proves the value RESOLVED, while `spawn ARMED - the
server role runs in pid N` is written only after the launch, the connect and
the handshake all succeeded, and carries a pid that is not this process's.
inproc resolves its own marker while running the server role on a thread
here; nothing on that path can write this one.

G14: one test name added (TheEleventhFramedOpIsInitCapabilitiesAndItCrosses-
TheWire), none removed. The ten-op table keeps its name and its scope rather
than being renamed to eleven.

Two unit failures predate this change and are environmental in this tree:
PipeCatalogue.FrontendNeverTakesAnApplierAddress (the frontend source root
resolves empty) and SessionHandshakeTest.TheAbiFingerprintChangesWhenAnyOf-
ItsInputsDoes (GIT_COMMIT_HASH_SHORT is empty here, which a6 already recorded
as red in this configuration).
2026-09-21 12:23:41 -04:00
swung0x48 7fd83133c0 [Android] retain trace replay ownership across Activity recreation 2026-09-20 10:32:18 -04:00
swung0x48 55ed1c8784 [Test, Tools] (tools/trace_replay): give the retrace harness a SPLIT variant whose output and artifact directories carry the variant too, and assert the transport resolved - a pull library under MOBILEGL_TRANSPORT=inproc replays OpenRA at ssim 1.000, so the picture cannot be the gate 2026-09-11 14:00:14 -04:00
swung0x48 5f8e8db1b9 [Test] (Retrace): make a retrace under MOBILEGL_PIPE_VERIFY prove it armed, and give the lane a label
- A retrace that exports MOBILEGL_PIPE_VERIFY=1 at a library configured without
  -DMOBILEGL_PIPE_VERIFY=ON is a no-op: the frames still match their goldens and the
  case reports green having compared nothing. run_trace_case.cmake now demands the
  evidence whenever the variable is set to anything but 0/false - mobilegl.log must
  exist, must carry "MGPipe: verify armed", and must carry neither
  Fatal{PipeVerifyDiffer nor Fatal{UnmigratedPipeInput. With the variable unset the
  script is byte-for-byte the old one.
- The Fatal scan is not redundant with the replay's exit status:
  MOBILEGL_PIPE_VERIFY_FATAL=0 is the supported triage configuration, and there a
  divergence is logged and counted rather than aborted, so the run would finish 0
  with its own report sitting unread in the log.
- LABELS retrace on both registrations: the lane was selectable only by regex, so
  `ctest -L retrace --no-tests=error` - the spelling that reds a lane which
  registered nothing - could not be written at all.
- "verify": true on eight cases (the five 180s cases plus minecraft-1.21.4-in-world,
  minecraft-1.21.4-fabric-sodium-in-world and improved-transparency-minecraft-26.3),
  and --format github-verify-matrix over that subset: 16 entries, against the full
  lane's 77. The verify build compares at every verb boundary and again at every
  accessor read, which the design budgets at 5-10x, so the per-push job runs the
  subset and the full sweep is a phase-exit / workflow_dispatch run.
- The flag is validated in the manifest loader, not at the matrix, so a non-boolean
  or a verify case excluded from CI is a loud error in every consumer instead of a
  subset that is quietly one case short.
2026-09-06 04:41:28 -04:00
swung0x48 38d4c2372c [Feat] (TraceApp, CI): pass arbitrary env vars through the retrace lane
- PLAN-B.md §8.2 and appendix B add a batch of new runtime switches
  (MOBILEGL_PIPE_PUSH / _VERIFY / _STATS / _LEGACY_MEMOS / _TEXEL_RETAIN_MB /
  _INDEX_MIRROR_MB, plus MOBILEGL_IPC_* later), and §11 P0 wants them parsed beside
  the existing ones. Today every knob that has to reach an Android replay costs an
  edit in five files - run_android_retrace_local.py, trace-replay-ci.sh,
  TraceReplayActivity's request record, the JNI marshalling, and the setenv block in
  trace_replay_core.cpp. That per-knob tax is what this replaces: one extra,
  `--es mobilegl_env "K=V;K=V"`, carries all of them.
- Applied last, immediately before dlopen(libMobileGL.so), so it can also override
  the dedicated fields above it - MobileGL's config is read during the load, and an
  escape hatch that cannot beat the defaults is not one. An entry with no '=' unsets
  the variable, which is the only way to clear a default the marshalling sets.
- The existing per-knob flags stay: they carry semantics beyond a setenv (use_angle
  also selects a variant, the dump lists are joined, DirectVulkan forces the
  R11G11B10F fallback), and rewriting them as env strings would move that logic into
  the callers.
- Surface: --env / MOBILEGL_TRACE_ENV in trace-replay-ci.sh, repeatable --env
  KEY=VALUE in run_android_retrace_local.py, `mobilegl_env` intent extra,
  Request::envOverrides.
- The two-level parse now lives in trace_env_overrides.hpp, beside the semicolon
  splitter it shares with the texture and FBO dump lists, and
  tools/trace_replay/trace_env_overrides_test.cpp pins it: the empty entries a
  trailing ';' leaves behind must not become unsetenv(""), `K=` must stay a Set of
  the empty string rather than an Unset (a knob read with getenv() != nullptr sees
  those as opposite answers), and only the FIRST '=' may separate, or a value
  carrying '=' is truncated without a word of warning. The whole MOBILEGL_PIPE_*
  batch rides on this parse, and the only lane that exercised it end to end was an
  on-device retrace, which would have reported a splitting bug as "the knob had no
  effect".
- The check is built and RUN at build time and mobilegl_trace_replay depends on it,
  so `cmake --build ... --target mobilegl_trace_replay` - the exact command of
  test.yml's "Build trace replay" job, which never invokes ctest - runs it. It is
  assert-free on purpose: that lane configures Release, and <cassert> under NDEBUG
  would compile every check into a green run that checked nothing. Negative control:
  swapping find('=') for rfind('=') fails 2 checks, and keeping the splitter's empty
  entries fails 2 more.
2026-09-05 20:49:57 -04:00
swung0x48 c1d6a3c908 [Feat] (TraceReplay): benchmark mode with per-frame timing for device fixtures 2026-08-26 02:17:35 -04:00
swung0x48 38eb9589f9 [Test] (tools/trace_replay): dump every live framebuffer attachment at a call boundary 2026-08-11 22:25:57 -04:00
swung0x48 3e4ce5caa7 [Refactor] (MG_Impl/GLImpl): replace Flywheel dispatch sync hack with MOBILEGL_COHERENT_AS_FLUSH 2026-07-15 21:51:20 -04:00
swung0x48 f10f7df389 [Chore] (trace-replay): use manifest file to organize retrace fixtures 2026-07-02 20:51:14 +08:00
swung0x48 9b06475811 [Fix] (replay-trace) refresh Minecraft 26.2 main menu fixture 2026-07-02 11:02:46 +08:00
swung0x48 233277d94b [Fix] (replay-trace) add Minecraft 26.2 vanilla fixtures 2026-07-02 09:07:20 +08:00
swung0x48 d2cbd2f596 [Fix] (replay-trace) recapture normal-world Fabric mod fixtures 2026-07-01 12:49:56 +08:00
swung0x48 a762346a3b [Fix] (TraceReplay): support local Android retrace reports 2026-06-29 13:53:30 +08:00
swung0x48 d78892cad0 [Fix] (replay-trace): enable GLES mod retraces and crop minimap 2026-06-29 10:17:30 +08:00
swung0x48 0ef9c76224 [Feat] (MG_Backend/DirectVulkan): support macOS Metal surfaces 2026-06-28 14:19:51 +08:00
swung0x48 46f3192c67 [CI] add normal-world per-mod retrace fixtures 2026-06-28 13:00:12 +08:00
swung0x48 6a11f96a5b [CI] add per-mod common mod retrace fixtures 2026-06-28 10:40:52 +08:00
swung0x48 56be5318ab [CI] add common mod retrace fixtures 2026-06-28 07:40:31 +08:00
swung0x48 59c9b94d76 [CI] add Mali main menu golden 2026-06-27 19:46:24 +08:00
swung0x48 d23e08f564 [CI] add remaining retrace fixtures 2026-06-27 19:20:23 +08:00
swung0x48 d2f2a0039f [Fix] (trace-replay): shorten iterationT fixtures 2026-06-26 15:58:53 +08:00
swung0x48 2f52264f01 [Fix] (trace-replay): use SSIM for golden validation 2026-06-23 21:26:55 +08:00
swung0x48 a1e2007b82 [Fix] (trace-replay): support alternate golden images
- compare actual output against primary and alternate golden images

- record the matched golden path in trace replay results

- allow APK and Linux retrace fixtures to pass alternate golden paths

- keep nostalgia validation accepting both Mesa and PC goldens
2026-06-20 19:04:52 +08:00
swung0x48 be6818effe [Fix] (trace-replay): relax fixture tolerance 2026-06-20 17:15:24 +08:00
swung0x48 0329f40df5 [Feat] (trace-replay): add Minecraft Sundial Lite shader fixture 2026-06-20 07:10:26 +08:00
swung0x48 2f9ccb84d9 [Feat] (trace-replay): add Minecraft Super Duper Vanilla shader fixture 2026-06-20 06:32:25 +08:00
swung0x48 f4cf398651 [Feat] (trace-replay): validate remaining Minecraft shader fixtures
- register Chocapic with a Linux Mesa golden

- register Nostalgia with a Linux Mesa golden

- register Photon shader fixture
2026-06-20 05:44:43 +08:00
swung0x48 84dba77275 [Fix]: fix Bliss
- enable glslang NaN min/max/clamp semantics

- register Bliss retrace fixture

- track remaining fixture validation queue
2026-06-19 21:20:02 +08:00
swung0x48 19e4ba386d [Fix]: fix Chocapic V6 Lite
- prune unused SPIR-V interface variables before GLES transpilation

- remap shader varyings through glslang IO resolver bindings

- initialize opaque uniforms from explicit sampler bindings only

- avoid side effects in texture binding assertions

- register Chocapic V6 Lite retrace fixture
2026-06-19 18:42:32 +08:00
swung0x48 7d101182cd [Feat] (trace-replay): validate Minecraft Complementary Unbound shader fixture 2026-06-19 10:50:40 +08:00
swung0x48 e60b044ff7 [Fix] (trace-replay): validate Complementary Reimagined shader fixture
- clear stale temporary framebuffer attachments before DirectGLES color/depth blits

- add Complementary Reimagined in-world trace to retrace validation
2026-06-19 10:28:47 +08:00
swung0x48 4366909cf0 [Feat] (trace-replay): validate Minecraft iterationT no-DSA shader fixture 2026-06-19 09:25:20 +08:00
swung0x48 7b1d8ce9dd [Feat] (trace-replay): validate Minecraft iterationT shader fixture 2026-06-19 09:14:33 +08:00
swung0x48 94fdeb633f [Fix] (trace-replay): relax Minecraft BSL tolerance 2026-06-19 08:21:50 +08:00
swung0x48 78dcf43c72 [Feat] (trace-replay): add Minecraft Mellow shader fixture 2026-06-17 23:51:17 +08:00
swung0x48 70951f46e5 [Feat] (trace-replay): add Minecraft MakeUP shader fixture 2026-06-17 20:09:21 +08:00
swung0x48 a6b4c4b049 [Feat] (trace-replay): add Minecraft BSL shader fixture 2026-06-17 17:52:30 +08:00
swung0x48 db3d569ed0 [Feat] (trace-replay): add Minecraft Sodium in-world fixture 2026-06-17 16:38:12 +08:00
swung0x48 61349ac0c2 [Fix] (trace-replay): shorten Minecraft in-world fixture 2026-06-17 12:34:32 +08:00
swung0x48 44c3ea7844 [Feat] (trace-replay): add Minecraft in-world fixture 2026-06-17 00:05:09 +08:00
swung0x48 722bddf916 [Feat] (trace-replay): test retrace APK on Android 2026-06-16 14:07:33 +08:00
swung0x48 b33ae1c481 [Feat] (trace-replay): add Minecraft main menu trace 2026-06-16 13:00:01 +08:00
swung0x48 0dc3a0916e [Feat] (trace-replay): add Minecraft startup trace 2026-06-16 12:22:27 +08:00
swung0x48 69a3985f9d [Fix] (ci): reuse MobileGL artifact for retrace 2026-06-16 09:32:53 +08:00
swung0x48 a88ca75c14 [Fix] (trace-replay): run Vulkan retrace headlessly 2026-06-15 20:37:40 +08:00
swung0x48 e453a75ea9 [Fix] (trace-replay): validate OpenRA on both backends 2026-06-15 19:59:21 +08:00