swung0x48/MobileGL
1
0
Fork 0
mirror of https://github.com/MobileGL-Dev/MobileGL synced 2026-11-07 01:53:24 +09:00
Code Issues Packages Projects Releases Wiki Activity

[CI] (.github/workflows, docs/ci, MG_IntegrationTest/Harness, scripts/data): reuse build-linux for split

Browse Source 
- build-linux and build-linux-split each probe the other's configure line with build_fingerprint.py on their own runner (disaggregation branches only); when it matches, build-linux-split skips its compile and the split consumers take build-linux's runtime and library
- the consumers red when build-linux-split skipped but build-linux's guard did not agree, or when the link ratchet, now also run in build-linux, did not pass; a difference falls back to the old two builds, and dev is unchanged apart from the consumers also waiting for build-linux
- the transport resolution check moves after 'Split lanes' and proves the IPC defaults from each passed entry's client log (split_log_paths.py ipc-config, canned tests); with both proven, Transport A/B and the non-strict split unit run are skipped as repeats
- the test job carries REQUIRE_GPU and libegl-mesa0 on the three branches; skip census: arm-monolith, arm-inproc and unit-split become optional rows
This commit is contained in:
swung0x48
2026-10-17 15:59:57 -04:00
parent 72e7d64c2b
commit eebbb1a213
5 changed files with 459 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test.yml
+293 -110
View File
@@ -39,6 +39,9 @@ jobs:
permissions:
actions: write
contents: read
outputs:
split-identical: ${{ steps.split-guard.outputs.identical }}
link-ratchet: ${{ steps.ratchet.outcome }}
env:
BUILD_DIR: build-linux
MOBILEGL_CI_DISAGGREGATED: ${{ (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && 'ON' || 'OFF' }}
@@ -121,6 +124,25 @@ jobs:
CI_BUILD_TYPE: ${{ secrets.ACTIONS_STEP_DEBUG == 'true' && 'Debug' || 'Release' }}
run: bash scripts/ci/configure_build.sh primary "${BUILD_DIR}"
# On the disaggregation branches build-linux-split configures the same sources with only the
# benchmarks left out. This configures its line here as a probe, on this runner, and compares
# the two trees (scripts/ci/build_fingerprint.py). Equal means the split consumers take this
# job's artifacts; a difference, or a guard that cannot decide, means false, and
# build-linux-split builds its own as before. It never reds this job.
- name: Split configuration guard
id: split-guard
if: env.MOBILEGL_CI_DISAGGREGATED == 'ON'
run: |
if python3 scripts/ci/build_fingerprint.py --self-test > "${RUNNER_TEMP}/fingerprint-self-test.txt"; then
python3 scripts/ci/build_fingerprint.py guard --built "${BUILD_DIR}" --probe split \
--probe-dir build-guard-split --github-output identical || echo "identical=false" >> "${GITHUB_OUTPUT}"
else
cat "${RUNNER_TEMP}/fingerprint-self-test.txt"
echo "::warning::build_fingerprint.py's self-test failed, so the split build is not reused"
echo "identical=false" >> "${GITHUB_OUTPUT}"
fi
rm -rf build-guard-split
- name: Build
run: cmake --build "${BUILD_DIR}" --parallel "$(nproc)"
@@ -132,6 +154,32 @@ jobs:
python3 scripts/ci/runtime_mode_proof.py build --build-dir "${BUILD_DIR}" \
--mode "${mode}" --output "${BUILD_DIR}/runtime-mode.json"
# build-linux-split's server link-closure ratchet, run on this build when that job reuses it.
# continue-on-error keeps a ratchet red from skipping this job's consumers; integration-split
# turns it into a red of its own instead (Split runtime agreement).
- name: Server link-closure ratchet
id: ratchet
if: steps.split-guard.outputs.identical == 'true'
continue-on-error: true
run: |
mkdir -p ci-artifacts
python3 scripts/link_ratchet.py --self-test
python3 scripts/link_ratchet.py \
--build-dir "${BUILD_DIR}" \
--baseline scripts/data/link_ratchet_baseline.txt \
--assert-monotone \
--bucket \
--json ci-artifacts/link-ratchet.json
- name: Upload link ratchet
if: ${{ always() && steps.split-guard.outputs.identical == 'true' }}
uses: actions/upload-artifact@v7
with:
name: link-ratchet
path: ci-artifacts/link-ratchet.json
if-no-files-found: warn
overwrite: true
- name: Show ccache stats
if: ${{ always() }}
run: ccache --show-stats
@@ -302,6 +350,7 @@ jobs:
# Initialize()'s glslang table prewarm. The retrace legs keep it: they replay real apps.
env:
MOBILEGL_DISABLE_SHADER_PREWARM: "1"
MOBILEGL_CI_DISAGGREGATED: ${{ (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && 'ON' || 'OFF' }}
steps:
- name: Checkout repo
@@ -321,7 +370,7 @@ jobs:
if: env.MOBILEGL_CI_NATIVE_GPU != '1'
run: |
sudo apt-get update
sudo apt-get install -y libvulkan1 libegl1 libgles2 libgl1-mesa-dri mesa-vulkan-drivers
sudo apt-get install -y libvulkan1 libegl1 libgles2 libgl1-mesa-dri mesa-vulkan-drivers ${{ env.MOBILEGL_CI_DISAGGREGATED == 'ON' && 'libegl-mesa0' || '' }}
- name: Download Linux runtime
uses: actions/download-artifact@v8
@@ -338,8 +387,13 @@ jobs:
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
done
# On the three branches this is also the split runtime's unit run (integration-split skips
# its copy when build-linux-split reuses this build), so it keeps that run's hardening:
# REQUIRE_GPU makes the server loop's EGL cases fail rather than skip without a driver.
- name: Test
working-directory: build-linux
env:
MOBILEGL_ITEST_REQUIRE_GPU: ${{ env.MOBILEGL_CI_DISAGGREGATED == 'ON' && '1' || '' }}
run: |
ulimit -c unlimited
if [ "${{ secrets.ACTIONS_STEP_DEBUG }}" = "true" ]; then
@@ -1053,7 +1107,6 @@ jobs:
cp -p "${RUNNER_TEMP}/verify-split.xml" verify-split-logs-lane/ || true
echo "kept $(ls verify-split-logs-lane | wc -l) file(s) from the green lane run"
# NEGATIVE CONTROL A ON THE SPLIT ARM - integration-verify's step A, same knob, same shape
# (this step passes when ctest FAILS), aimed at the ambient VerifySplit. entries of BOTH
# backends. Measured at landing: 10 of 10 red, each with
@@ -1114,28 +1167,28 @@ jobs:
path: /tmp/core.*
if-no-files-found: ignore
# P5's split build. The verify pair cloned, because the problem is the same one: a third
# configuration of the same sources whose whole value depends on the option having taken.
# The split build: the same sources with the transport and its in-process arm always on, which the
# split integration and retrace lanes consume. On the disaggregation branches build-linux has the
# same configuration apart from the benchmarks; the guard below proves that on this runner, and
# when it holds this job skips its compile and the consumers take build-linux's artifacts
# (build-linux runs the same guard, and the consumers require both). On dev it builds as before.
#
# This dedicated build keeps the protocol/strict controls independent of the branch's
# primary runtime mode. MOBILEGL_BUILD_DISAGGREGATED implies MOBILEGL_PIPE_PUSH (the split path decodes
# into the MGPipe applier and PIPE_PUSH is what compiles the applier), so the split arm needs a
# build of its own exactly as the verify arm does.
#
# THE `nm` STEP IS THE POINT OF THIS JOB. In a build without the option, MG_Remote is not
# compiled at all and MOBILEGL_TRANSPORT's parser does not exist - so the variable is accepted by
# the environment and silently ignored (CONTRACT-P5 5). Every downstream lane would then run
# monolith and go green under a name that says split. CMake will not complain about a typo'd -D,
# so the build-level assertion is the only guard, and its absence is precisely what would make
# the whole split arm meaningless.
# The `nm` step is the point of a build of its own. In a build without the option, MG_Remote is
# not compiled and MOBILEGL_TRANSPORT's parser does not exist, so the variable is accepted and
# ignored and every downstream lane would run monolith under a name that says split. CMake does
# not complain about a typo'd -D, so the build-level assertion is the guard; in reuse mode
# build-linux's runtime_mode_proof (--mode disaggregated) is the same assertion and more.
build-linux-split:
runs-on: ubuntu-26.04
timeout-minutes: 120
permissions:
actions: write
contents: read
outputs:
reuse-primary: ${{ steps.guard.outputs.identical == 'true' && 'true' || 'false' }}
env:
BUILD_DIR: build-split
MOBILEGL_CI_DISAGGREGATED: ${{ (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && 'ON' || 'OFF' }}
CCACHE_BASEDIR: ${{ github.workspace }}
CCACHE_COMPRESS: "true"
CCACHE_DIR: ${{ github.workspace }}/.ccache
@@ -1143,19 +1196,13 @@ jobs:
CCACHE_NOHASHDIR: "true"
steps:
- name: Set swap space
uses: pierotofy/set-swap-space@v1.0
with:
swap-size-gb: 32
- name: Checkout repo
uses: actions/checkout@v6
with:
# recursive, and load-bearing here beyond the usual: MOBILEGL_BUILD_DISAGGREGATED
# SHADOWS ITSELF BACK TO OFF when 3rdparty/flatbuffers/include is missing
# (CMakeLists.txt:471-486, a normal variable rather than a cache force, deliberately).
# A shallow checkout would therefore configure cleanly, build a monolith library, and
# be caught only by the nm step below.
# shadows itself back to OFF when 3rdparty/flatbuffers/include is missing (a normal
# variable rather than a cache force, deliberately). A shallow checkout would therefore
# configure cleanly, build a monolith library, and be caught only by the nm step below.
submodules: recursive
- name: Configure runner environment
@@ -1167,14 +1214,6 @@ jobs:
with:
cmakeVersion: 3.31.10
- name: Restore ccache
uses: actions/cache/restore@v5
with:
path: .ccache
key: ${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-v1
restore-keys: |
${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-
- name: Prepare Vulkan SDK
id: netsetup1274
continue-on-error: true
@@ -1218,7 +1257,42 @@ jobs:
- name: Configure CMake
run: bash scripts/ci/configure_build.sh split "${BUILD_DIR}"
# build-linux's configuration, probed on this runner and compared with this tree. Equal
# means this job builds nothing and ends green; the consumers then take build-linux's
# artifacts, and red if build-linux's own guard disagrees. It never reds this job.
- name: Primary configuration guard
id: guard
if: env.MOBILEGL_CI_DISAGGREGATED == 'ON'
env:
CI_BUILD_TYPE: ${{ secrets.ACTIONS_STEP_DEBUG == 'true' && 'Debug' || 'Release' }}
run: |
if python3 scripts/ci/build_fingerprint.py --self-test > "${RUNNER_TEMP}/fingerprint-self-test.txt"; then
python3 scripts/ci/build_fingerprint.py guard --built "${BUILD_DIR}" --probe primary \
--probe-dir build-guard-primary --github-output identical || echo "identical=false" >> "${GITHUB_OUTPUT}"
else
cat "${RUNNER_TEMP}/fingerprint-self-test.txt"
echo "::warning::build_fingerprint.py's self-test failed, so this job builds its own library"
echo "identical=false" >> "${GITHUB_OUTPUT}"
fi
rm -rf build-guard-primary
- name: Set swap space
if: steps.guard.outputs.identical != 'true'
uses: pierotofy/set-swap-space@v1.0
with:
swap-size-gb: 32
- name: Restore ccache
if: steps.guard.outputs.identical != 'true'
uses: actions/cache/restore@v5
with:
path: .ccache
key: ${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-v1
restore-keys: |
${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-
- name: Build
if: steps.guard.outputs.identical != 'true'
run: cmake --build "${BUILD_DIR}" --parallel "$(nproc)"
# `nm` and NOT `nm -D`, for the reason build-linux-verify spells out: everything under
@@ -1227,6 +1301,7 @@ jobs:
# stripped library would make the grep fail for a third, silent reason.
#
- name: MG_Remote symbols in split library
if: steps.guard.outputs.identical != 'true'
run: |
test -f "${BUILD_DIR}/libMobileGL.so"
defined=$(nm --defined-only "${BUILD_DIR}/libMobileGL.so" | wc -l)
@@ -1250,10 +1325,11 @@ jobs:
# from the per-object partition the frontend symbols a server-only image still has to link
# and compares the list with the stored baseline. Only new symbols fail; a vanished one is
# printed with a request to re-baseline in the same commit. The baseline is a list, not a
# count, so swapping one symbol for another fails. It runs here, not in build-linux, because
# this job always builds MG_Remote (the script re-checks CMakeCache.txt), and before anything
# deletes the *.o files it reads.
# count, so swapping one symbol for another fails. It runs where MG_Remote is built (the
# script re-checks CMakeCache.txt) and before anything deletes the *.o files it reads: here,
# or in build-linux when this job reuses that build.
- name: Server link-closure ratchet
if: steps.guard.outputs.identical != 'true'
run: |
mkdir -p ci-artifacts
python3 scripts/link_ratchet.py --self-test
@@ -1265,27 +1341,30 @@ jobs:
--json ci-artifacts/link-ratchet.json
# Uploaded even when the gate fired: a red ratchet is exactly the moment someone needs the
# per-symbol definer/referrer attribution the JSON carries, and a job that failed before
# uploading its evidence is a job nobody can act on.
# per-symbol definer/referrer attribution the JSON carries.
- name: Upload link ratchet
if: ${{ always() }}
if: ${{ always() && steps.guard.outputs.identical != 'true' }}
uses: actions/upload-artifact@v7
with:
name: link-ratchet
path: ci-artifacts/link-ratchet.json
if-no-files-found: warn
overwrite: true
- name: Show ccache stats
if: ${{ always() }}
if: ${{ always() && steps.guard.outputs.identical != 'true' }}
run: ccache --show-stats
# Only after a build: with nothing built, deleting would drop this branch's entry for good.
- name: Release superseded ccache entry
if: steps.guard.outputs.identical != 'true'
env:
GH_TOKEN: ${{ github.token }}
CACHE_KEY: ${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-v1
run: gh cache delete "${CACHE_KEY}" --ref "${GITHUB_REF}" || true
- name: Save ccache
if: steps.guard.outputs.identical != 'true'
continue-on-error: true
uses: actions/cache/save@v5
with:
@@ -1293,9 +1372,11 @@ jobs:
key: ${{ runner.os }}-ubuntu26-cpu-test-${{ github.job }}-ccache-v1
- name: Prepare packaged test metadata
if: steps.guard.outputs.identical != 'true'
run: python3 scripts/ci/prepare_test_runtime.py "${BUILD_DIR}"
- name: Package Linux split runtime
if: steps.guard.outputs.identical != 'true'
run: |
mkdir -p ci-artifacts
mapfile -t SHARED_LIBS < <(find "${BUILD_DIR}" -type f \( -name '*.so' -o -name '*.so.*' \) -print | sort)
@@ -1313,6 +1394,7 @@ jobs:
"${SHARED_LIBS[@]}"
- name: Upload Linux split runtime
if: steps.guard.outputs.identical != 'true'
uses: actions/upload-artifact@v7
with:
name: mobilegl-linux-runtime-split
@@ -1322,12 +1404,14 @@ jobs:
# The retrace legs load only the library, and the server image on the spawn arm, so they
# take this instead of the whole test runtime.
- name: Package retrace library
if: steps.guard.outputs.identical != 'true'
run: |
files=("${BUILD_DIR}/.ci-source-root" "${BUILD_DIR}/libMobileGL.so")
if [ -f "${BUILD_DIR}/libMobileGLServer.so" ]; then files+=("${BUILD_DIR}/libMobileGLServer.so"); fi
tar -czf ci-artifacts/mobilegl-linux-lib-split.tgz "${files[@]}"
- name: Upload retrace library
if: steps.guard.outputs.identical != 'true'
uses: actions/upload-artifact@v7
with:
name: mobilegl-linux-lib-split
@@ -1339,16 +1423,39 @@ jobs:
integration-split:
runs-on: ubuntu-26.04
timeout-minutes: 120
needs: [build-linux-split]
needs: [build-linux-split, build-linux]
if: ${{ !cancelled() && needs.build-linux-split.result == 'success' && (needs.build-linux-split.outputs.reuse-primary != 'true' || needs.build-linux.result == 'success') }}
# Short-lived test processes skip the glslang prewarm, as in the test job.
env:
MOBILEGL_DISABLE_SHADER_PREWARM: "1"
SPLIT_REUSE: ${{ needs.build-linux-split.outputs.reuse-primary }}
SPLIT_BUILD_DIR: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'build-linux' || 'build-split' }}
SPLIT_RUNTIME: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'mobilegl-linux-runtime' || 'mobilegl-linux-runtime-split' }}
MOBILEGL_CI_DISAGGREGATED: ${{ (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && 'ON' || 'OFF' }}
steps:
- name: Checkout repo
uses: actions/checkout@v6
# The two build guards must agree. build-linux-split skips its compile on its own guard's
# word; the artifacts these lanes then take are build-linux's, so build-linux's guard must
# have proved the same equality on the runner that built them.
- name: Split runtime agreement
env:
PRIMARY_IDENTICAL: ${{ needs.build-linux.outputs.split-identical }}
PRIMARY_RATCHET: ${{ needs.build-linux.outputs.link-ratchet }}
run: |
if [ "${SPLIT_REUSE}" = "true" ] && [ "${PRIMARY_IDENTICAL}" != "true" ]; then
echo "::error::build-linux-split skipped its build, but build-linux's guard did not prove the two configurations equal (split-identical='${PRIMARY_IDENTICAL}'): the runners disagree, so these lanes have no split runtime they can trust"
exit 1
fi
if [ "${SPLIT_REUSE}" = "true" ] && [ "${PRIMARY_RATCHET}" != "success" ]; then
echo "::error::the server link-closure ratchet ran in build-linux for this reused build and did not pass (outcome '${PRIMARY_RATCHET}'); its link-ratchet artifact names the symbols"
exit 1
fi
echo "split runtime: ${SPLIT_BUILD_DIR} (reuse of build-linux: ${SPLIT_REUSE})"
- name: Configure runner environment
uses: ./.github/actions/setup-test-runner
with:
@@ -1370,13 +1477,17 @@ jobs:
- name: Download Linux split runtime
uses: actions/download-artifact@v8
with:
name: mobilegl-linux-runtime-split
name: ${{ env.SPLIT_RUNTIME }}
path: .
- name: Unpack Linux split runtime
run: |
tar --zstd -xf mobilegl-linux-runtime-split.tar.zst
test -f build-split/libMobileGL.so
tar --zstd -xf "${SPLIT_RUNTIME}.tar.zst"
test -f "${SPLIT_BUILD_DIR}/libMobileGL.so"
if [ "${SPLIT_REUSE}" = "true" ]; then
python3 -c 'import json, sys; mode = json.load(open(sys.argv[1]))["mode"]; sys.exit(0 if mode == "disaggregated" else f"build-linux runtime mode is {mode}, not disaggregated")' \
"${SPLIT_BUILD_DIR}/runtime-mode.json"
fi
- name: Prepare downloaded test metadata
run: |
@@ -1385,7 +1496,7 @@ jobs:
done
- name: Spawn/split lane parity (9.2)
run: python3 scripts/ci/spawn_lane_parity.py build-split
run: python3 scripts/ci/spawn_lane_parity.py "${SPLIT_BUILD_DIR}"
# P8-A (ID-P8-2): every monolith case, tails normalised, is on a gated split arm or is named
# with a reason class in scripts/data/split_coverage_exemptions.txt. The same `ctest -N`
@@ -1394,7 +1505,7 @@ jobs:
- name: Split coverage against monolith (P8-A)
run: |
python3 scripts/ci/split_coverage.py --self-test
python3 scripts/ci/split_coverage.py build-split
python3 scripts/ci/split_coverage.py "${SPLIT_BUILD_DIR}"
# P8-SE: PipeCatalogueTest's unmigrated-emulation list equals the MGPipeUnmigratedEmulation
# call sites in the tree, one site per name. A source lint: no build needed.
@@ -1403,8 +1514,11 @@ jobs:
python3 scripts/ci/unmigrated_emulation_sites.py --self-test
python3 scripts/ci/unmigrated_emulation_sites.py
# The test job runs the same unit lane, with the same REQUIRE_GPU, on this same build when
# build-linux-split was reused; the strict unit step in integration-split-strict always runs.
- name: Unit tests on the split runtime
working-directory: build-split
if: env.SPLIT_REUSE != 'true'
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
run: ctest --output-on-failure -L unit --no-tests=error -j "$(nproc)" --output-junit "${RUNNER_TEMP}/unit-split.xml"
@@ -1429,7 +1543,7 @@ jobs:
# the reason and never goes green - MG_IntegrationTest/CMakeLists.txt probes MG_Remote for
# c0's signature stubs and disarms the lane while any remain.
- name: Split lanes (inproc, spawn, TCP)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
run: |
@@ -1453,14 +1567,63 @@ jobs:
python3 ../scripts/ci/junit_tally.py "${RUNNER_TEMP}/tcp-baseline.xml" --require-tcp-ran --require-run-ahead --discovery "${RUNNER_TEMP}/tcp-discovery.json"
python3 ../scripts/ci/junit_tally.py "${RUNNER_TEMP}/tcp-baseline.xml" --require-entry-passed TcpLane.SupervisorProtocolControls --require-entry-passed TcpLane.EventForfeitPeer --require-entry-passed TcpLane.ControlFrameFuzz
# THE RUNTIME HALF OF "THIS IS REALLY A SPLIT BUILD". The build-level nm check in
# build-linux-split proves the library CARRIES MG_Remote; this proves the transport
# RESOLVED in a process of this lane. ConfigLoader::InitTransport logs one INFO line when it
# selects InProcess, and the DirectGLES.Split.PersistentMapArm. entry retains its private
# counting log (all Split entries now have private paths). It runs right after the lane, so
# it reads the lane's own run rather than a later step's rerun of the same entry.
# The line is written during bring-up, before any scenario decides to skip, so this
# check is live from the day the lanes land rather than from the day they stop skipping.
- name: Transport and default-knob resolution check
id: resolution
working-directory: ${{ env.SPLIT_BUILD_DIR }}
run: |
# P6: the sink writes one file per ROLE from the base MOBILEGL_LOG_FILE_PATH; the
# transport-resolution line is ConfigLoader's, on the CLIENT thread, so it is in the
# .client half. The base name itself names no file now.
log=MobileGL/MG_IntegrationTest/persistent-map-arm-split-DirectGLES.client.log
if [ ! -f "${log}" ]; then
echo "::error::${log} does not exist: the DirectGLES.Split.PersistentMapArm. entry never ran (or the per-role split renamed it), so nothing in this job establishes that MOBILEGL_TRANSPORT ever resolved to inproc in a live process"
exit 1
fi
# THE DISTINCTIVE PART OF THE INFO LINE, not the bare KEY=VALUE (review finding M-5):
# ConfigLoader logs `Config: Accepted env variable: MOBILEGL_TRANSPORT=inproc` for the
# env dump too, unconditionally and in a PULL build, and at DEBUG that line is live.
if ! grep -q "MOBILEGL_TRANSPORT=inproc - the MGPipe record stream" "${log}"; then
echo "::error::${log} carries no transport-resolution line. ConfigLoader::InitTransport logs it at INFO when it selects InProcess, and that code exists only in a MOBILEGL_BUILD_DISAGGREGATED build - so this lane ran a monolith library while claiming to be the split lane. (A bare MOBILEGL_TRANSPORT=inproc substring is NOT accepted: the env dump prints one in every build.)"
exit 1
fi
echo "the split lane resolved MOBILEGL_TRANSPORT=inproc"
# The IPC knobs every passed entry resolved, from its own client log: the defaults the
# lane relies on (run-ahead and the role-split state on, strict off) unless the entry
# pins its own. Proven defaults make Transport A/B below a repeat of the integration
# job's two legs; exit 3 says a default changed, and A/B then runs.
ctest -L integration-split --show-only=json-v1 > "${RUNNER_TEMP}/split-lane.json"
rc=0
python3 "${GITHUB_WORKSPACE}/MobileGL/MG_IntegrationTest/Harness/split_log_paths.py" ipc-config \
"${RUNNER_TEMP}/split-lane.json" "${RUNNER_TEMP}/split-baseline.xml" "${RUNNER_TEMP}/split-ipc.json" \
strict=0 role-split-state=1 run-ahead=1 --assumed role-split-state,run-ahead || rc=$?
if [ "${rc}" -eq 0 ]; then
echo "defaults-on=true" >> "${GITHUB_OUTPUT}"
elif [ "${rc}" -eq 3 ]; then
echo "::warning::an IPC default differs from what the lanes assume (split-ipc.json names the entries), so Transport A/B runs"
echo "defaults-on=false" >> "${GITHUB_OUTPUT}"
else
exit "${rc}"
fi
# Both full transport runs are hard gates. The per-case comparison below
# remains diagnostic because explicit mechanism controls retain their own mode.
# With build-linux reused and the defaults proven above, the integration job's two legs
# already ran these entries on this library in these configurations, so this is skipped.
#
# The entries that name MOBILEGL_TRANSPORT in their OWN property (the Split. lanes) keep
# their value in both passes, which is correct: they are the split family in both arms and
# the comparison is about the other 1100.
- name: Transport A/B (monolith vs inproc)
working-directory: build-split
if: env.SPLIT_REUSE != 'true' || steps.resolution.outputs.defaults-on != 'true'
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_ITRP_FIX: "ForceOn"
@@ -1497,33 +1660,6 @@ jobs:
print(f"the two arms agree on all {len(a)} entries, name and status")
PY
# THE RUNTIME HALF OF "THIS IS REALLY A SPLIT BUILD". The build-level nm check in
# build-linux-split proves the library CARRIES MG_Remote; this proves the transport
# RESOLVED in a process of this lane. ConfigLoader::InitTransport logs one INFO line when it
# selects InProcess, and the DirectGLES.Split.PersistentMapArm. entry retains its private
# counting log (all Split entries now have private paths; nothing else writes this file).
# The line is written during bring-up, before any scenario decides to skip, so this
# check is live from the day the lanes land rather than from the day they stop skipping.
- name: Transport resolution check
working-directory: build-split
run: |
# P6: the sink writes one file per ROLE from the base MOBILEGL_LOG_FILE_PATH; the
# transport-resolution line is ConfigLoader's, on the CLIENT thread, so it is in the
# .client half. The base name itself names no file now.
log=MobileGL/MG_IntegrationTest/persistent-map-arm-split-DirectGLES.client.log
if [ ! -f "${log}" ]; then
echo "::error::${log} does not exist: the DirectGLES.Split.PersistentMapArm. entry never ran (or the per-role split renamed it), so nothing in this job establishes that MOBILEGL_TRANSPORT ever resolved to inproc in a live process"
exit 1
fi
# THE DISTINCTIVE PART OF THE INFO LINE, not the bare KEY=VALUE (review finding M-5):
# ConfigLoader logs `Config: Accepted env variable: MOBILEGL_TRANSPORT=inproc` for the
# env dump too, unconditionally and in a PULL build, and at DEBUG that line is live.
if ! grep -q "MOBILEGL_TRANSPORT=inproc - the MGPipe record stream" "${log}"; then
echo "::error::${log} carries no transport-resolution line. ConfigLoader::InitTransport logs it at INFO when it selects InProcess, and that code exists only in a MOBILEGL_BUILD_DISAGGREGATED build - so this lane ran a monolith library while claiming to be the split lane. (A bare MOBILEGL_TRANSPORT=inproc substring is NOT accepted: the env dump prints one in every build.)"
exit 1
fi
echo "the split lane resolved MOBILEGL_TRANSPORT=inproc"
# Negative controls for the verb barrier (MOBILEGL_IPC_VERB_BARRIER=0) and the persistent-map
# push (MOBILEGL_IPC_PERSISTENT_BLOCK_KB=0): each knob must turn its selected entries red.
# The body is scripts/ci/split_negative_controls.sh, so scripts/ci/control_smoke_test.sh can
@@ -1532,7 +1668,7 @@ jobs:
# and accepts a red only when it carries the control's own diagnostic.
- name: Negative controls (barrier, map push)
if: ${{ !cancelled() }}
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
CONTROL_TMPDIR: ${{ runner.temp }}
@@ -1551,8 +1687,8 @@ jobs:
with:
name: integration-split-logs
path: |
build-split/MobileGL/MG_IntegrationTest/*.log*
build-split/MobileGL/MG_IntegrationTest/split-logs/*.log
${{ env.SPLIT_BUILD_DIR }}/MobileGL/MG_IntegrationTest/*.log*
${{ env.SPLIT_BUILD_DIR }}/MobileGL/MG_IntegrationTest/split-logs/*.log
${{ runner.temp }}/arm-inproc.xml
${{ runner.temp }}/arm-monolith.xml
if-no-files-found: warn
@@ -1582,16 +1718,35 @@ jobs:
integration-split-strict:
runs-on: ubuntu-26.04
timeout-minutes: 60
needs: [build-linux-split]
needs: [build-linux-split, build-linux]
if: ${{ !cancelled() && needs.build-linux-split.result == 'success' && (needs.build-linux-split.outputs.reuse-primary != 'true' || needs.build-linux.result == 'success') }}
# Short-lived test processes skip the glslang prewarm, as in the test job.
env:
MOBILEGL_DISABLE_SHADER_PREWARM: "1"
SPLIT_REUSE: ${{ needs.build-linux-split.outputs.reuse-primary }}
SPLIT_BUILD_DIR: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'build-linux' || 'build-split' }}
SPLIT_RUNTIME: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'mobilegl-linux-runtime' || 'mobilegl-linux-runtime-split' }}
MOBILEGL_CI_DISAGGREGATED: ${{ (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && 'ON' || 'OFF' }}
steps:
- name: Checkout repo
uses: actions/checkout@v6
# The two build guards must agree. build-linux-split skips its compile on its own guard's
# word; the artifacts these lanes then take are build-linux's, so build-linux's guard must
# have proved the same equality on the runner that built them.
- name: Split runtime agreement
env:
PRIMARY_IDENTICAL: ${{ needs.build-linux.outputs.split-identical }}
PRIMARY_RATCHET: ${{ needs.build-linux.outputs.link-ratchet }}
run: |
if [ "${SPLIT_REUSE}" = "true" ] && [ "${PRIMARY_IDENTICAL}" != "true" ]; then
echo "::error::build-linux-split skipped its build, but build-linux's guard did not prove the two configurations equal (split-identical='${PRIMARY_IDENTICAL}'): the runners disagree, so these lanes have no split runtime they can trust"
exit 1
fi
echo "split runtime: ${SPLIT_BUILD_DIR} (reuse of build-linux: ${SPLIT_REUSE})"
- name: Configure runner environment
uses: ./.github/actions/setup-test-runner
with:
@@ -1613,13 +1768,17 @@ jobs:
- name: Download Linux split runtime
uses: actions/download-artifact@v8
with:
name: mobilegl-linux-runtime-split
name: ${{ env.SPLIT_RUNTIME }}
path: .
- name: Unpack Linux split runtime
run: |
tar --zstd -xf mobilegl-linux-runtime-split.tar.zst
test -f build-split/libMobileGL.so
tar --zstd -xf "${SPLIT_RUNTIME}.tar.zst"
test -f "${SPLIT_BUILD_DIR}/libMobileGL.so"
if [ "${SPLIT_REUSE}" = "true" ]; then
python3 -c 'import json, sys; mode = json.load(open(sys.argv[1]))["mode"]; sys.exit(0 if mode == "disaggregated" else f"build-linux runtime mode is {mode}, not disaggregated")' \
"${SPLIT_BUILD_DIR}/runtime-mode.json"
fi
- name: Prepare downloaded test metadata
run: |
@@ -1628,7 +1787,7 @@ jobs:
done
- name: Unit tests on the split runtime (strict)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_STRICT_ERRORS: "1"
@@ -1640,7 +1799,7 @@ jobs:
# admitted (ARCHITECTURE.md §9.2.1). The lane must be green, and its markers must match the
# admitted list and strict-expected-markers.txt in both directions.
- name: Strict split lane
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_STRICT_ERRORS: "1"
@@ -1723,7 +1882,7 @@ jobs:
# These entries own assertions removed from generic transport registrations.
# Discovery, execution and zero-skip accounting must agree on the exact set.
- name: Specialized configuration controls
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
run: |
@@ -1744,7 +1903,7 @@ jobs:
# debt somebody now owes, a vanished pair must be removed in the commit that retired it.
# With the census empty, this lane is a hard green gate.
- name: Dual-block census lane
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_TRANSPORT: "inproc"
@@ -1789,7 +1948,7 @@ jobs:
# The per-frame zero-residual-read case must execute and pass on both backends.
- name: P5f per-frame zero-residual check
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_ROLE_SPLIT_STATE: "1"
@@ -1806,7 +1965,7 @@ jobs:
# The mechanism's own negative control (P5F §6): the same named case must be green with
# the knob on and red - on its own assertion text - with it off.
- name: Dual-block negative control (P5f f1)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
run: bash "${GITHUB_WORKSPACE}/scripts/ci/dualblock_negative_control.sh"
# ---- MAGMA'S OWN WIRE LANE: FRAMEBUFFER, TEXTURE AND APPLICATION BUFFERS ------------
@@ -1814,7 +1973,7 @@ jobs:
# P5f fm: Magma's framebuffer, copy and mipmap verbs consume server records.
# The label remains separate so both backends' coverage stays explicit.
- name: Magma split lane (strict, dual-block)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_STRICT_ERRORS: "1"
@@ -1830,7 +1989,7 @@ jobs:
# integration-magma-all-{split,spawn,tcp}, is not gated: it may go red while CI stays green,
# and a case moves into a gating step once its blocker is fixed.
- name: Magma spawn and TCP lanes
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_STRICT_ERRORS: "1"
@@ -1842,7 +2001,7 @@ jobs:
# Every named Magma application-buffer wire case must execute and pass.
- name: Magma buffer wire cases (inproc)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
run: |
ctest -L integration-magma-buffers --show-only=json-v1 > "${RUNNER_TEMP}/magma-buffer-lane.json"
ctest -L integration-magma-buffers --no-tests=error -j 2 --output-on-failure \
@@ -1902,7 +2061,7 @@ jobs:
# fixture entries under the label as well - and they must be green too: a lane whose
# supervisor failed to start has nothing to say about the twenty-one.
- name: Magma buffer wire cases (spawn, TCP)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
run: |
@@ -1959,7 +2118,7 @@ jobs:
# Run-ahead must queue real GPU work and enforce credits 1 and 3.
- name: Magma run-ahead cases (credits 1, 3)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
run: |
ctest -L integration-magma-runahead --show-only=json-v1 > "${RUNNER_TEMP}/magma-runahead-lane.json"
ctest -L integration-magma-runahead --no-tests=error -j 2 --output-on-failure \
@@ -1986,17 +2145,17 @@ jobs:
# Run-ahead off must go red on the lost queue lead; run-ahead back on must be green.
- name: Magma run-ahead negative control
run: |
python3 scripts/ci/magma_runahead_checks.py negative --build-dir build-split \
python3 scripts/ci/magma_runahead_checks.py negative --build-dir "${SPLIT_BUILD_DIR}" \
--out "${RUNNER_TEMP}/magma-runahead-negative" --execute
- name: Magma run-ahead sync validation
run: |
python3 scripts/ci/magma_runahead_checks.py validation --build-dir build-split \
python3 scripts/ci/magma_runahead_checks.py validation --build-dir "${SPLIT_BUILD_DIR}" \
--out "${RUNNER_TEMP}/magma-runahead-syncval" --execute
- name: Magma view/framebuffer cache sync validation
run: |
python3 scripts/ci/magma_cache_checks.py --build-dir build-split \
python3 scripts/ci/magma_cache_checks.py --build-dir "${SPLIT_BUILD_DIR}" \
--out "${RUNNER_TEMP}/magma-cache-syncval" --execute
- name: Upload Magma run-ahead and cache evidence
@@ -2016,7 +2175,7 @@ jobs:
# evidence behind it: the case must EXECUTE, PASS, and never skip - a lane-level exit code
# alone would not separate "generated" from "quietly absent".
- name: Magma packed-float mipmap case (P7)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_IPC_STRICT_ERRORS: "1"
@@ -2039,7 +2198,7 @@ jobs:
# Client vertex/index inputs are copied into owned wire buffer resources.
# The original dedicated entries now have to execute successfully with run-ahead.
- name: Client-array split lane (run-ahead)
working-directory: build-split
working-directory: ${{ env.SPLIT_BUILD_DIR }}
env:
MOBILEGL_ITEST_REQUIRE_GPU: "1"
MOBILEGL_TRANSPORT: "inproc"
@@ -2083,8 +2242,8 @@ jobs:
with:
name: integration-split-strict-logs
path: |
build-split/MobileGL/MG_IntegrationTest/*.log*
build-split/MobileGL/MG_IntegrationTest/split-logs/*.log
${{ env.SPLIT_BUILD_DIR }}/MobileGL/MG_IntegrationTest/*.log*
${{ env.SPLIT_BUILD_DIR }}/MobileGL/MG_IntegrationTest/split-logs/*.log
${{ runner.temp }}/specialized*.json
${{ runner.temp }}/specialized-tests.txt
${{ runner.temp }}/specialized.xml
@@ -2916,10 +3075,16 @@ jobs:
# The setup action selects lavapipe on GitHub-hosted runners.
needs:
- build-linux-split
- build-linux
- build-retrace
- trace-cases
- trace-fixtures
if: ${{ !cancelled() && needs.build-linux-split.result == 'success' && needs.build-retrace.result == 'success' && needs.trace-cases.result == 'success' }}
if: ${{ !cancelled() && needs.build-linux-split.result == 'success' && needs.build-retrace.result == 'success' && needs.trace-cases.result == 'success' && (needs.build-linux-split.outputs.reuse-primary != 'true' || needs.build-linux.result == 'success') }}
env:
SPLIT_REUSE: ${{ needs.build-linux-split.outputs.reuse-primary }}
SPLIT_LIB: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'mobilegl-linux-lib' || 'mobilegl-linux-lib-split' }}
SPLIT_LIB_DIR: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'build-linux' || 'build-split' }}
SPLIT_BUILD_DIR: ${{ needs.build-linux-split.outputs.reuse-primary == 'true' && 'build-linux' || 'build-split' }}
strategy:
fail-fast: false
max-parallel: 8
@@ -2935,6 +3100,20 @@ jobs:
- name: Checkout repo
uses: actions/checkout@v6
# The two build guards must agree. build-linux-split skips its compile on its own guard's
# word; the artifacts these lanes then take are build-linux's, so build-linux's guard must
# have proved the same equality on the runner that built them.
- name: Split runtime agreement
env:
PRIMARY_IDENTICAL: ${{ needs.build-linux.outputs.split-identical }}
PRIMARY_RATCHET: ${{ needs.build-linux.outputs.link-ratchet }}
run: |
if [ "${SPLIT_REUSE}" = "true" ] && [ "${PRIMARY_IDENTICAL}" != "true" ]; then
echo "::error::build-linux-split skipped its build, but build-linux's guard did not prove the two configurations equal (split-identical='${PRIMARY_IDENTICAL}'): the runners disagree, so these lanes have no split runtime they can trust"
exit 1
fi
echo "split runtime: ${SPLIT_BUILD_DIR} (reuse of build-linux: ${SPLIT_REUSE})"
- name: Configure runner environment
uses: ./.github/actions/setup-test-runner
with:
@@ -2969,7 +3148,7 @@ jobs:
- name: Download Linux split library
uses: actions/download-artifact@v8
with:
name: mobilegl-linux-lib-split
name: ${{ env.SPLIT_LIB }}
path: .
- name: Download trace replay
@@ -2983,12 +3162,14 @@ jobs:
# <workspace>/build-linux/libMobileGL.so frozen into every case, so the swap happens here
# rather than through a variable, exactly as in retrace-verify. `nm`, not `nm -D`.
run: |
tar -xzf mobilegl-linux-lib-split.tgz
tar -xzf "${SPLIT_LIB}.tgz"
tar -xzf mobilegl-trace-replay.tgz
test -f build-split/libMobileGL.so
test -f "${SPLIT_LIB_DIR}/libMobileGL.so"
test -f build-retrace/tools/trace_replay/mobilegl_trace_replay
mkdir -p build-linux
cp build-split/libMobileGL.so build-linux/libMobileGL.so
if [ "${SPLIT_LIB_DIR}" != build-linux ]; then
mkdir -p build-linux
cp "${SPLIT_LIB_DIR}/libMobileGL.so" build-linux/libMobileGL.so
fi
if ! nm --defined-only build-linux/libMobileGL.so | grep -q -i MG_Remote; then
echo "::error::the library unpacked at build-linux/libMobileGL.so defines no MG_Remote symbol, so this retrace would replay against a monolith build, ignore MOBILEGL_TRANSPORT entirely and match its golden having split nothing"
exit 1
@@ -3004,17 +3185,19 @@ jobs:
# THE SPAWN ARM NEEDS A SECOND BINARY and the inproc arm needs none, which is the whole
# difference between the two at this level: inproc's server role is a thread in the replay
# process, spawn's is a PROCESS the client launches from this image. It ships in
# build-linux-split's retrace library - libMobileGLServer.so is an executable that wears a
# the split retrace library - libMobileGLServer.so is an executable that wears a
# .so name so an APK will install it somewhere exec-able - and a missing one must be a NAMED
# failure here rather than a LaunchServer refusal 78 jobs deep.
- name: Place the server image (spawn arm only)
if: ${{ matrix.transport == 'spawn' }}
run: |
if [ ! -f build-split/libMobileGLServer.so ]; then
if [ ! -f "${SPLIT_LIB_DIR}/libMobileGLServer.so" ]; then
echo "::error::the split runtime carries no libMobileGLServer.so, so transport=spawn has no server to launch. ConfigLoader would resolve Spawn, LaunchServer would refuse BY NAME, and this job would red on a message about an unresolvable image rather than about a packaging gap."
exit 1
fi
cp build-split/libMobileGLServer.so build-linux/libMobileGLServer.so
if [ "${SPLIT_LIB_DIR}" != build-linux ]; then
cp "${SPLIT_LIB_DIR}/libMobileGLServer.so" build-linux/libMobileGLServer.so
fi
chmod +x build-linux/libMobileGLServer.so
echo "server image: $(readlink -f build-linux/libMobileGLServer.so)"
@@ -3351,7 +3534,6 @@ jobs:
- name: Test CTest runtime metadata preparation
run: python3 scripts/ci/test_prepare_test_runtime.py
- name: Regenerate the MGPipe interface (G1-G7)
run: |
python3 scripts/gen_pipe.py
@@ -3447,6 +3629,7 @@ jobs:
python3 tools/trace_replay/test_compare_actuals.py
python3 tools/trace_replay/test_run_trace_case.py
python3 scripts/ci/build_fingerprint.py --self-test
python3 MobileGL/MG_IntegrationTest/Harness/test_split_log_paths.py
# Warning only for now: the disaggregation documents are still being written, and a
# lint that fails a rewrite in progress teaches people to ignore it. It becomes