mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-07 01:53:24 +09:00
[CI] (.github/workflows, docs/ci, scripts/ci, tools/trace_replay): drop the retrace-split reruns
- retrace-split loses its two negative controls, which re-ran every leg (~68 runner-min per run), and its dependency on the monolith-control artifact; the in-test evidence check from the previous commit replaces them - build-linux-monolith-control keeps its zero-MG_Remote proof and its upload; on the disaggregation branches the new retrace-shipped job replays OpenRA on both backends through that library (elsewhere the retrace job's library has that shape), so the shipped shape is still replayed on every branch - delete retrace_transport_control.sh, retrace_drop_draw_control.sh, their smoke cases and stub modes, and the uncalled redcheck_control_smoke_test.sh - control-runtime/build-push-control leaves the eleven metadata loops; remove-artifact-clutter waits for retrace-shipped
This commit is contained in:
+157
-92
@@ -220,8 +220,9 @@ jobs:
|
||||
path: ci-artifacts/mobilegl-linux-lib.tgz
|
||||
if-no-files-found: error
|
||||
|
||||
# The pull-library negative control must not consume the primary artifact:
|
||||
# on disaggregation branches that artifact now contains the real transport.
|
||||
# The shipped shape: a library with no MG_Remote, which FCL embeds. On the disaggregation
|
||||
# branches the primary artifact carries the transport, so this is the only build of that shape
|
||||
# there; retrace-shipped replays it.
|
||||
build-linux-monolith-control:
|
||||
name: build push monolith control (no MG_Remote)
|
||||
runs-on: ubuntu-26.04
|
||||
@@ -282,11 +283,9 @@ jobs:
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y ccache clang-20 clang++-20 lld-20 libc++-20-dev libc++abi-20-dev libvulkan-dev libegl1-mesa-dev libgles2-mesa-dev libgl1-mesa-dri mesa-vulkan-drivers ninja-build
|
||||
# THE PUSH MONOLITH WITHOUT MG_Remote - the shape FCL embeds (PLAN-P13 §4.1, §4.3; ID-P13-1).
|
||||
# Two jobs in one: it is the transport-negative control retrace-split swaps in (P13 W2 retired
|
||||
# the pull library that used to play that part), and its runtime_mode_proof is the
|
||||
# zero-MG_Remote-symbol check, one of G1's successors (ID-P13-4): a push library that links
|
||||
# any MG_Remote symbol is red here.
|
||||
# The push monolith without MG_Remote, the shape FCL embeds. Its runtime_mode_proof is the
|
||||
# zero-MG_Remote-symbol check: a push library that links any MG_Remote symbol is red here.
|
||||
# Its library is uploaded for retrace-shipped, a positive replay of that shape.
|
||||
- name: Configure push control without MG_Remote
|
||||
run: |
|
||||
cmake -S . -B "${BUILD_DIR}" -G Ninja \
|
||||
@@ -357,7 +356,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -447,7 +446,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -789,7 +788,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -1034,7 +1033,7 @@ jobs:
|
||||
# otherwise also match this label and the monolith one would match both.
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -1507,7 +1506,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -1745,7 +1744,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -2370,7 +2369,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -2471,7 +2470,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -2746,7 +2745,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -2970,7 +2969,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -3054,23 +3053,18 @@ jobs:
|
||||
# process differs, so a spawn-only failure is a transport fact. It runs the unchanged ctest
|
||||
# names with MOBILEGL_TRANSPORT exported around ctest, since build-retrace does not register the
|
||||
# split variant entries. SSIM cannot catch a monolith run (OpenRA scores 1.000 either way); the
|
||||
# guards are the nm check for MG_Remote and run_trace_case.cmake's transport-resolution check.
|
||||
# guards are the nm check for MG_Remote, run_trace_case.cmake's transport assertions (and, on
|
||||
# OpenRA, its wire-volume assertion), and the per-leg check of the evidence file they leave.
|
||||
retrace-split:
|
||||
name: retrace split (${{ matrix.backend }} / ${{ matrix.transport }}, ${{ matrix.case }})
|
||||
runs-on: ubuntu-26.04
|
||||
timeout-minutes: 240
|
||||
# The setup action selects lavapipe on GitHub-hosted runners.
|
||||
needs:
|
||||
- build-linux-monolith-control
|
||||
- build-linux-split
|
||||
- build-retrace
|
||||
- trace-cases
|
||||
- trace-fixtures
|
||||
# The monolith-control artifact is for the negative controls ONLY: its libraries are what the
|
||||
# controls swap in to prove the transport-resolution assertion is load-bearing. P13 W1: the
|
||||
# legs no longer require that build to have SUCCEEDED - gating on it skipped the whole split
|
||||
# matrix (grey, not red) whenever the control build failed (PLAN-P13 §0-7). The control steps
|
||||
# below download its artifact, and a missing one is a red step on every leg instead.
|
||||
if: ${{ !cancelled() && needs.build-linux-split.result == 'success' && needs.build-retrace.result == 'success' && needs.trace-cases.result == 'success' }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
@@ -3130,19 +3124,6 @@ jobs:
|
||||
name: mobilegl-trace-replay
|
||||
path: .
|
||||
|
||||
# The push monolith without MG_Remote, for the negative control at the end of this job and for
|
||||
# nothing else.
|
||||
- name: Download control runtime (negative control)
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: mobilegl-linux-monolith-control
|
||||
path: control-runtime
|
||||
|
||||
- name: Unpack the control runtime (negative control)
|
||||
run: |
|
||||
tar -xzf control-runtime/mobilegl-linux-monolith-control.tgz -C control-runtime
|
||||
test -f control-runtime/build-push-control/libMobileGL.so
|
||||
|
||||
- name: Unpack split runtime (library under test)
|
||||
# build-retrace's CTestTestfile.cmake has the absolute path
|
||||
# <workspace>/build-linux/libMobileGL.so frozen into every case, so the swap happens here
|
||||
@@ -3162,7 +3143,7 @@ jobs:
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: |
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace control-runtime/build-push-control; do
|
||||
for root in build-linux build-verify build-split build-verify-split build-retrace; do
|
||||
if [ -d "$root" ]; then python3 scripts/ci/prepare_test_runtime.py "$root"; fi
|
||||
done
|
||||
|
||||
@@ -3183,12 +3164,8 @@ jobs:
|
||||
chmod +x build-linux/libMobileGLServer.so
|
||||
echo "server image: $(readlink -f build-linux/libMobileGLServer.so)"
|
||||
|
||||
# THE MAGMA KNOBS GO TO $GITHUB_ENV, NOT TO ONE STEP'S SHELL: the replay below AND both
|
||||
# negative controls after it must run the same repair set. Exported inside "Retrace and
|
||||
# validate" only, they never reached the pull-library control, which replayed
|
||||
# iterationrp x DirectVulkan unrepaired (ssim 0.028 < 0.98), reddened at
|
||||
# run_trace_case.cmake:173 before the transport assertion, and failed as "the failure never
|
||||
# says the library did not resolve the transport" - a red that is not the control's own.
|
||||
# The Magma knobs go to $GITHUB_ENV rather than to one step's shell, so every later step of
|
||||
# this leg runs the same repair set as the replay.
|
||||
- name: Magma repair knobs (all later steps)
|
||||
run: |
|
||||
# P7 package L: THE SAME MAGMA KNOBS THE MONOLITH RETRACE JOB EXPORTS, ALL OF THEM.
|
||||
@@ -3247,47 +3224,6 @@ jobs:
|
||||
--transport '${{ matrix.transport }}' --case '${{ matrix.case }}' \
|
||||
--backend '${{ matrix.backend }}' "${wire[@]}"
|
||||
|
||||
# Negative control: copy a library without MG_Remote (the push monolith FCL embeds) over the
|
||||
# frozen path and require the same replay to fail. OpenRA scores ssim 1.000 either way, so
|
||||
# the picture cannot catch the swap; this proves that run_trace_case.cmake's
|
||||
# transport-resolution check still does. The body is scripts/ci/retrace_transport_control.sh,
|
||||
# which control_smoke_test.sh runs off a runner. It counts the selection first, requires the
|
||||
# red to carry run_trace_case.cmake's own sentence, and restores the split library and the
|
||||
# good run's images, since "Upload actual image" runs `if: always()`.
|
||||
- name: Negative control (library without a transport)
|
||||
working-directory: build-retrace/tools/trace_replay
|
||||
env:
|
||||
CONTROL_TMPDIR: ${{ runner.temp }}
|
||||
CONTROL_LIBRARY: ${{ github.workspace }}/control-runtime/build-push-control/libMobileGL.so
|
||||
FROZEN_LIBRARY: ${{ github.workspace }}/build-linux/libMobileGL.so
|
||||
# The control has to run the arm it is a control FOR. Hard-coded inproc, it would go on
|
||||
# proving something about the other arm while spawn ran unguarded.
|
||||
SPLIT_TRANSPORT: ${{ matrix.transport }}
|
||||
MOBILEGL_IPC_SERVER_PATH: ${{ github.workspace }}/build-linux/libMobileGLServer.so
|
||||
run: >-
|
||||
bash "${GITHUB_WORKSPACE}/scripts/ci/retrace_transport_control.sh"
|
||||
'${{ matrix.case }}' '${{ matrix.backend }}'
|
||||
|
||||
# Negative control for the picture: the step above cannot show that the image came through
|
||||
# the wire, because OpenRA also scores ssim 1.000 on a monolith library. This drops the
|
||||
# DrawVbo records and requires the SSIM to fall. Dropping clears does not work: OpenRA
|
||||
# overdraws every pixel it clears (retrace_drop_draw_control.sh's header has the census).
|
||||
# The body is a script so control_smoke_test.sh can run it against a stubbed ctest, including
|
||||
# a red whose SSIM never fell and a knob that armed but dropped nothing.
|
||||
- name: Negative control (dropped draws)
|
||||
working-directory: build-retrace/tools/trace_replay
|
||||
env:
|
||||
CONTROL_TMPDIR: ${{ runner.temp }}
|
||||
LIBRARY_LOG: ${{ matrix.case }}/${{ matrix.backend }}/output/mobilegl.client.log
|
||||
FROZEN_LIBRARY: ${{ github.workspace }}/build-linux/libMobileGL.so
|
||||
# The control has to run the arm it is a control FOR. Hard-coded inproc, it would go on
|
||||
# proving something about the other arm while spawn ran unguarded.
|
||||
SPLIT_TRANSPORT: ${{ matrix.transport }}
|
||||
MOBILEGL_IPC_SERVER_PATH: ${{ github.workspace }}/build-linux/libMobileGLServer.so
|
||||
run: >-
|
||||
bash "${GITHUB_WORKSPACE}/scripts/ci/retrace_drop_draw_control.sh"
|
||||
'${{ matrix.case }}' '${{ matrix.backend }}'
|
||||
|
||||
# The refusal census, recorded rather than gated. run_trace_case.cmake already REDS the case
|
||||
# on any Fatal{, so reaching here means the count is zero - but the number and the distinct
|
||||
# slot names are what MEASUREMENTS wants from every split run, and reading them out of the
|
||||
@@ -3333,14 +3269,141 @@ jobs:
|
||||
build-retrace/tools/trace_replay/${{ matrix.case }}/${{ matrix.backend }}/output/**
|
||||
if-no-files-found: warn
|
||||
|
||||
# The shipped library, replayed. build-linux-monolith-control's library has no MG_Remote, the
|
||||
# shape FCL embeds; on the disaggregation branches nothing else replays that shape, so it runs
|
||||
# there only (elsewhere the retrace job's library is that shape). One short case on both
|
||||
# backends, with no transport set, must match its golden like any monolith row.
|
||||
retrace-shipped:
|
||||
name: retrace shipped library (${{ matrix.backend }}, ${{ matrix.case }})
|
||||
runs-on: ubuntu-26.04
|
||||
timeout-minutes: 60
|
||||
needs:
|
||||
- build-linux-monolith-control
|
||||
- build-retrace
|
||||
- trace-fixtures
|
||||
if: ${{ !cancelled() && (github.ref_name == 'feat/disaggregated' || github.ref_name == 'codex/fix-github-ci' || github.ref_name == 'cleanup') && needs.build-linux-monolith-control.result == 'success' && needs.build-retrace.result == 'success' && needs.trace-fixtures.result != 'skipped' }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
backend: [DirectGLES, DirectVulkan]
|
||||
case: [OpenRA]
|
||||
|
||||
steps:
|
||||
- name: Set swap space
|
||||
if: runner.environment != 'self-hosted'
|
||||
uses: pierotofy/set-swap-space@v1.0
|
||||
with:
|
||||
swap-size-gb: 16
|
||||
|
||||
- name: Checkout repo
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Configure runner environment
|
||||
uses: ./.github/actions/setup-test-runner
|
||||
with:
|
||||
gpu: "true"
|
||||
|
||||
- name: Download trace fixture
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: trace-fixture-${{ matrix.case }}
|
||||
path: trace-fixture-download
|
||||
|
||||
- name: Install trace fixture
|
||||
run: |
|
||||
mkdir -p tools/trace_replay/fixtures
|
||||
find trace-fixture-download -type f -exec cp {} tools/trace_replay/fixtures/ \;
|
||||
|
||||
- name: Get CMake
|
||||
timeout-minutes: 15
|
||||
uses: lukka/get-cmake@v4.3.3
|
||||
with:
|
||||
cmakeVersion: 3.31.10
|
||||
|
||||
- name: Install runtime dependencies
|
||||
timeout-minutes: 15
|
||||
if: env.MOBILEGL_CI_NATIVE_GPU != '1'
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libvulkan1 libegl1-mesa-dev libgles2-mesa-dev libgl1-mesa-dri mesa-vulkan-drivers
|
||||
test -e /usr/lib/x86_64-linux-gnu/libEGL.so
|
||||
test -e /usr/lib/x86_64-linux-gnu/libGLESv2.so
|
||||
|
||||
- name: Download shipped library
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: mobilegl-linux-monolith-control
|
||||
path: .
|
||||
|
||||
- name: Download trace replay
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: mobilegl-trace-replay
|
||||
path: .
|
||||
|
||||
# build-retrace froze <workspace>/build-linux/libMobileGL.so into every case, so the library
|
||||
# is put where that path points, as retrace-verify does. `nm`, not `nm -D`: the MG_Remote
|
||||
# symbols are hidden in a Release build and only the static table lists them.
|
||||
- name: Unpack shipped library (library under test)
|
||||
run: |
|
||||
tar -xzf mobilegl-linux-monolith-control.tgz
|
||||
tar -xzf mobilegl-trace-replay.tgz
|
||||
test -f build-push-control/libMobileGL.so
|
||||
test -f build-retrace/tools/trace_replay/mobilegl_trace_replay
|
||||
mkdir -p build-linux
|
||||
cp build-push-control/libMobileGL.so build-linux/libMobileGL.so
|
||||
defined=$(nm --defined-only build-linux/libMobileGL.so | wc -l)
|
||||
remote=$(nm --defined-only build-linux/libMobileGL.so | grep -c -i MG_Remote || true)
|
||||
if [ "${defined}" -lt 1000 ] || [ "${remote}" -ne 0 ]; then
|
||||
echo "::error::build-linux/libMobileGL.so defines ${defined} symbols and ${remote} MG_Remote symbol(s): it is not the shipped shape (no MG_Remote, not stripped)"
|
||||
exit 1
|
||||
fi
|
||||
echo "the library at build-linux/libMobileGL.so is the shipped shape (${defined} defined symbols, no MG_Remote)"
|
||||
|
||||
- name: Prepare downloaded test metadata
|
||||
run: python3 scripts/ci/prepare_test_runtime.py build-retrace
|
||||
|
||||
- name: Retrace and validate
|
||||
working-directory: build-retrace/tools/trace_replay
|
||||
run: |
|
||||
ulimit -c unlimited
|
||||
if [ '${{ matrix.backend }}' = 'DirectVulkan' ]; then
|
||||
export MOBILEGL_MAGMA_R11G11B10F_FALLBACK=1
|
||||
fi
|
||||
ctest -V --no-tests=error -R '^MobileGLTraceReplay\.${{ matrix.case }}\.${{ matrix.backend }}$' \
|
||||
--output-junit "${RUNNER_TEMP}/retrace.xml"
|
||||
|
||||
- name: Annotate failed entries
|
||||
if: failure()
|
||||
run: python3 scripts/ci/junit_annotate.py --excerpts 1 "${RUNNER_TEMP}"
|
||||
|
||||
- name: Upload core dumps
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: retrace-shipped-core-dumps-${{ matrix.backend }}-${{ matrix.case }}
|
||||
path: /tmp/core.*
|
||||
if-no-files-found: ignore
|
||||
|
||||
- name: Upload actual image
|
||||
if: ${{ always() }}
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: retrace-shipped-result-${{ matrix.backend }}-${{ matrix.case }}
|
||||
path: |
|
||||
build-retrace/tools/trace_replay/${{ matrix.case }}/actual-images/**
|
||||
build-retrace/tools/trace_replay/${{ matrix.case }}/${{ matrix.backend }}/output/**
|
||||
if-no-files-found: warn
|
||||
|
||||
remove-artifact-clutter:
|
||||
name: remove artifact clutter
|
||||
runs-on: ubuntu-26.04
|
||||
# Every retrace arm downloads the shared fixtures, including split controls.
|
||||
# Every retrace job downloads the shared fixtures, so this waits for all of them.
|
||||
needs:
|
||||
- retrace-summary
|
||||
- retrace-verify
|
||||
- retrace-split
|
||||
- retrace-shipped
|
||||
if: ${{ always() }}
|
||||
permissions:
|
||||
actions: write
|
||||
@@ -3359,14 +3422,15 @@ jobs:
|
||||
# an unchecked process substitution.
|
||||
# 50 rows per page: the jobs endpoint answers HTTP 502 for 90+ rows on a re-run attempt of a run this size.
|
||||
gh api --paginate "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/jobs?per_page=50" \
|
||||
--jq '.jobs[] | select((.name | startswith("retrace (")) or (.name | startswith("retrace verify (")) or (.name | startswith("retrace split ("))) | select(.conclusion == "failure" or .conclusion == "cancelled" or .conclusion == "timed_out" or .conclusion == "action_required") | .name' \
|
||||
--jq '.jobs[] | select((.name | startswith("retrace (")) or (.name | startswith("retrace verify (")) or (.name | startswith("retrace split (")) or (.name | startswith("retrace shipped library ("))) | select(.conclusion == "failure" or .conclusion == "cancelled" or .conclusion == "timed_out" or .conclusion == "action_required") | .name' \
|
||||
> "${RUNNER_TEMP}/cleanup-jobs.txt"
|
||||
gh api --paginate "repos/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}/artifacts?per_page=100" \
|
||||
--jq '.artifacts[] | select(.name | startswith("trace-fixture-") or startswith("retrace-result-") or startswith("retrace-split-result-")) | [.id, .name] | @tsv' \
|
||||
--jq '.artifacts[] | select(.name | startswith("trace-fixture-") or startswith("retrace-result-") or startswith("retrace-split-result-") or startswith("retrace-shipped-result-")) | [.id, .name] | @tsv' \
|
||||
> "${RUNNER_TEMP}/cleanup-artifacts.txt"
|
||||
declare -A failed_cases=()
|
||||
while IFS= read -r job_name; do
|
||||
case_name="${job_name#retrace split (*, }"
|
||||
case_name="${case_name#retrace shipped library (*, }"
|
||||
case_name="${case_name#retrace verify (*, }"
|
||||
case_name="${case_name#retrace (*, }"
|
||||
case_name="${case_name%)}"
|
||||
@@ -3515,10 +3579,11 @@ jobs:
|
||||
python3 scripts/gen_pipe_field_ownership.py --self-test
|
||||
|
||||
# Negative controls must themselves be able to go red (ARCHITECTURE.md §13.2).
|
||||
# control_smoke_test.sh runs the split and retrace control scripts against a stubbed ctest:
|
||||
# an unrelated failure or an empty selection must make each report FAILED, and the
|
||||
# scenarios' real diagnostics must make it report PASSED. The Python tests run the trace and
|
||||
# TCP lane tooling on stubs and synthetic data. No branch guard; no build, GPU or device.
|
||||
# control_smoke_test.sh runs the split control script against a stubbed ctest: an unrelated
|
||||
# failure or an empty selection must make it report FAILED, and the scenarios' real
|
||||
# diagnostics must make it report PASSED. The Python tests run the trace and TCP lane tooling
|
||||
# on stubs and synthetic data; test_run_trace_case.py feeds run_trace_case.cmake canned logs.
|
||||
# No branch guard; no build, GPU or device.
|
||||
- name: Control and trace-tool smoke tests (R-16)
|
||||
run: |
|
||||
bash scripts/ci/control_smoke_test.sh
|
||||
|
||||
@@ -36,6 +36,9 @@ them wrong.
|
||||
| 2026-10-06 | unit `DirectGLESSlotTable.{ANegativeLookupIsNotCachedAcrossAnotherHoldersAcquire, GetOrCreateToleratesANullStateObject}` | Test / unit | P13 (`# P13` ratchet gate): they pinned the slot table's frontend-keyed half - `HandleOf`'s lifetime-id memo and the minting `GetOrCreate(StatePtr)`'s null-object parking - which minted handles off frontend lifetime ids from inside MG_Backend; that half is deleted (the table only receives client-minted handles now). The other ten `DirectGLESSlotTable` mechanics cases were RETARGETED at the handle entries (`GetOrCreate(handle)` / `FindByHandle` / `ReleaseTwinByHandle`), not removed | the retargeted `DirectGLESSlotTable` cases, `SlotAllocatorTest`, and the link ratchet's P13 exit gate (no server object references the client allocator) |
|
||||
| 2026-10-06 | unit `RemoteGuards.{SlotTableHandleOfFromTheApplyThreadIsFatalByName, SlotTableMintingGetOrCreateFromTheApplyThreadIsFatalByName, AnAllocatorProbeInsideAnExemptionScopeFromTheApplyThreadIsFatalEvenAtTheOldDebtSites, BarrieredFrontendRegistryMembersRefuseBothLegacyScopes, LegacyRegistryWrapperCannotExposeFrontendKeysOnApply, AnExemptionScopeHeldOnTheGLThreadDoesNotExemptTheApplyThread}` | Test / unit (split) | P13 (`# P13` ratchet gate): they drove the apply-thread refusals hung on Espryt's frontend-keyed registry entries (`HandleOf`, `GetOrCreate(StatePtr)`, `NoteStateForHandle`, `StateForHandle`, `Find(object)`, `begin()`) and the two no-op marker scopes; the entries, the registry refusal and both scopes are deleted, so there is no apply-thread surface left to refuse | `RemoteGuards.AnAllocatorProbeFromTheApplyThreadOutsideEveryScopeIsFatalByName` / `BarrieredLegacyScopesCannotExemptAllocator` (the allocator's own guard, kept) and the link ratchet's P13 exit gate |
|
||||
| 2026-10-06 | unit `ServerLoopEglTest.FrontendFramebufferDeathDeletesOnTheContextOwner` | Test / unit (split) | P13 (`# P13` ratchet gate): its pin was the refusal of the apply-thread `GetOrCreate(StatePtr)` mint of a framebuffer twin; that overload is deleted | `CtWireScenario`'s object_death cases (framebuffer death by handle on the server) |
|
||||
| 2026-10-17 | `Negative control (library without a transport)` (with its `Download/Unpack control runtime` steps) | Test / `retrace-split` (every leg) | a full replay rerun on all 154 legs (~60 runner-min) that, since the per-role logs, only ever proved "no client log was written"; a broken control build once reddened every leg at the download and hid every real split replay | in-test: `run_trace_case.cmake`'s transport assertions now leave `output/split-evidence.txt`, which `retrace_split_evidence.py` checks on every leg; `test_run_trace_case.py` (pipe-gates) proves each assertion red on canned logs, including the no-client-log and env-dump-line shapes this control used to rely on |
|
||||
| 2026-10-17 | `Negative control (dropped draws)` | Test / `retrace-split` (every leg) | a rerun of every leg with `MOBILEGL_IPC_E2_DROP_DRAW`, a production hook; its knob's own early return produced the DirectVulkan `Fatal{RoleViolation, "gPipeInputs"}` reds of 10-02 to 10-05 | in-test on the OpenRA legs (both backends, inproc and spawn): `MGTRACE_WIRE_EVIDENCE` turns on the library's counters and `run_trace_case.cmake` requires DrawVbo records on the wire and draws on the server role (758 each locally); canned reds in `test_run_trace_case.py` |
|
||||
| 2026-10-17 | `scripts/ci/retrace_transport_control.sh`, `scripts/ci/retrace_drop_draw_control.sh`, the retrace and drop-draw halves of `control_smoke_test.sh` and `stub_ctest.sh`, and `scripts/ci/redcheck_control_smoke_test.sh` | Test / pipe-gates (R-16) | the bodies of the two controls above and their smoke cases; the redcheck script had no caller and mutated the deleted scripts | `test_run_trace_case.py` |
|
||||
|
||||
## Kept after investigation
|
||||
|
||||
|
||||
@@ -7,21 +7,15 @@
|
||||
# SPDX-License-Identifier: LGPL-3.0-only
|
||||
# End of Source File Header
|
||||
|
||||
# R-16 FOR THE CI NEGATIVE CONTROLS THEMSELVES: a control-run smoke test.
|
||||
# A smoke test for the CI negative controls themselves.
|
||||
#
|
||||
# BRIEF-P5 13 (R-16) says a negative control must assert its own failure reason and that every gate
|
||||
# carries a line saying "I made it red once, by doing X". The two controls this file exercises ARE
|
||||
# gates, and until ID-46 finding 8 nobody could make either of them red, because a workflow `run:`
|
||||
# block only executes on a runner. The wave-1 verification agent had to hand-copy the blocks into
|
||||
# throwaway harnesses to show they were broken (wave1-codex-verify.md 8). This file is that
|
||||
# experiment, kept: it runs the REAL control scripts - the same files .github/workflows/test.yml
|
||||
# invokes, not copies of them - against a stubbed ctest, and checks that each one passes exactly
|
||||
# when it should.
|
||||
# A negative control is a gate, and a workflow `run:` block only executes on a runner, so this
|
||||
# file runs the real control script - the same file .github/workflows/test.yml invokes, not a copy
|
||||
# of it - against a stubbed ctest, and checks that it passes exactly when it should.
|
||||
#
|
||||
# The case that matters is the first one. A stubbed ctest reports a NON-EMPTY selection and then
|
||||
# The case that matters is the first one. A stubbed ctest reports a non-empty selection and then
|
||||
# fails with UNRELATED_CONTROL_FAILURE: a reason that has nothing to do with the knob the control
|
||||
# turns. Before ID-48's fix both controls printed their success message and the step exited 0. They
|
||||
# must now report FAILED.
|
||||
# turns. The control must report FAILED on it, and PASSED only on its scenarios' own diagnostic.
|
||||
#
|
||||
# usage: control_smoke_test.sh
|
||||
set -u
|
||||
@@ -61,29 +55,6 @@ run_split() { # $1 = STUB_MODE
|
||||
bash "${HERE}/split_negative_controls.sh"
|
||||
}
|
||||
|
||||
run_retrace() { # $1 = STUB_MODE
|
||||
cd "${WORK}" || return 127
|
||||
mkdir -p "${WORK}/OpenRA"
|
||||
local rc=0
|
||||
env -i PATH="${STUB_DIR}:/usr/bin:/bin" STUB_MODE="$1" \
|
||||
CTEST=ctest CONTROL_TMPDIR="${WORK}/tmp-$1" \
|
||||
CONTROL_LIBRARY="${WORK}/pull.so" FROZEN_LIBRARY="${WORK}/frozen.so" \
|
||||
bash "${HERE}/retrace_transport_control.sh" OpenRA DirectGLES || rc=$?
|
||||
cmp -s "${WORK}/frozen.so" "${WORK}/split.so" || {
|
||||
echo 'F6 FAILED: pull control did not restore the split library'; return 1;
|
||||
}
|
||||
return "${rc}"
|
||||
}
|
||||
|
||||
run_drop_draw() { # $1 = STUB_MODE
|
||||
cd "${WORK}" || return 127
|
||||
mkdir -p "${WORK}/OpenRA"
|
||||
env -i PATH="${STUB_DIR}:/usr/bin:/bin" STUB_MODE="$1" \
|
||||
CTEST=ctest CONTROL_TMPDIR="${WORK}/tmp-$1" \
|
||||
FROZEN_LIBRARY="${WORK}/frozen.so" LIBRARY_LOG="${WORK}/tmp-$1/mobilegl.log" \
|
||||
bash "${HERE}/retrace_drop_draw_control.sh" OpenRA DirectGLES
|
||||
}
|
||||
|
||||
echo "=== the split lane's E1 / E3(a) controls (scripts/ci/split_negative_controls.sh)"
|
||||
# THE FINDING, REPRODUCED. Non-empty selection, green baseline, and a red that is not the knob's.
|
||||
expect FAILED "unrelated failure with a non-empty selection" -- run_split unrelated
|
||||
@@ -102,61 +73,6 @@ expect FAILED "E1 record never reaches the peer" -- run_split e1-no-
|
||||
expect FAILED "E1 observed no real emission" -- run_split e1-no-emit
|
||||
expect FAILED "E1 incorrectly waits every record" -- run_split e1-waitall
|
||||
|
||||
echo
|
||||
echo "=== the retrace lane's pull-library control (scripts/ci/retrace_transport_control.sh)"
|
||||
# A pull-shaped library the nm identity check accepts: a real ELF .so defining no MG_Remote symbol.
|
||||
if command -v cc > /dev/null 2>&1; then
|
||||
printf '%s\n' 'int mobilegl_pull_only(void) { return 1; }' > "${WORK}/pull.c"
|
||||
cc -shared -fPIC -o "${WORK}/pull.so" "${WORK}/pull.c" || { echo "cannot build the stand-in library"; exit 1; }
|
||||
else
|
||||
echo "no cc available; the retrace half of this smoke test needs one" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf '%s\n' 'int MG_Remote_stub(void) { return 1; }' > "${WORK}/split.c"
|
||||
cc -shared -fPIC -o "${WORK}/frozen.so" "${WORK}/split.c" || exit 1
|
||||
cp "${WORK}/frozen.so" "${WORK}/split.so"
|
||||
|
||||
# THE FINDING, part (b): a regex matching no tests. --no-tests=error exits non-zero and the old
|
||||
# control read that as "the pull library turned it red".
|
||||
expect FAILED "empty selection (--no-tests=error exit)" -- run_retrace retrace-noselect
|
||||
# A red that never names the transport: a fixture failure, a loader failure, a timeout.
|
||||
expect FAILED "red without the transport-resolution message" -- run_retrace retrace-unrelated
|
||||
# The real thing - and note the stub emits it CMake-wrapped across two lines, which a line-oriented
|
||||
# grep for the literal sentence would miss.
|
||||
expect PASSED "run_trace_case.cmake's own sentence, wrapped" -- run_retrace retrace-evidence
|
||||
# The sentence a pull library ACTUALLY gets since P6's log rename: it wrote the unsuffixed
|
||||
# mobilegl.log, the runner reads mobilegl.client.log, and the no-log check fires before the marker
|
||||
# search. The control accepted only the marker sentence until B3's fix round 2, and was red on
|
||||
# every real pull-library run (measured on the B3 package tree).
|
||||
expect PASSED "the no-client-log sentence a pull library gets" -- run_retrace retrace-evidence-nolog
|
||||
# Both sentences again as `ctest -V` really prints them, with "1: " on every line: folded
|
||||
# naively that is "never 1: reported resolving it", and the control used to red on it.
|
||||
expect PASSED "the marker sentence under ctest -V's N: prefix" -- run_retrace retrace-evidence-prefixed
|
||||
expect PASSED "the no-log sentence under ctest -V's N: prefix" -- run_retrace retrace-evidence-nolog-prefixed
|
||||
# The pull library replaying green is the failure this control exists to catch.
|
||||
expect FAILED "a pull library passed the split retrace" -- run_retrace retrace-green
|
||||
|
||||
echo
|
||||
echo "=== the retrace lane's draw-drop control (scripts/ci/retrace_drop_draw_control.sh)"
|
||||
# Exit gate E2's picture control. The joint gate's finding is the reason it exists at all: the
|
||||
# CLEAR-drop knob was armed, was read, dropped all 29 of OpenRA's clears - and the retrace still
|
||||
# scored ssim 1.000000, because OpenRA overdraws every pixel it clears. So this control's three
|
||||
# guards are each a different way for "the picture went red" to be someone else's red.
|
||||
expect FAILED "empty selection (--no-tests=error exit)" -- run_drop_draw dropdraw-noselect
|
||||
# The failure the gate exists to catch: the golden survives the loss of every draw.
|
||||
expect FAILED "the retrace passed with every draw dropped" -- run_drop_draw dropdraw-green
|
||||
# A red with no comparator output at all - a loader failure, a missing fixture, a timeout.
|
||||
expect FAILED "red with no ssim summary in the output" -- run_drop_draw dropdraw-nossim
|
||||
# A red whose SSIM is FINE: something else (a Fatal{, a transport assertion) reddened the case.
|
||||
expect FAILED "red but the ssim is above the threshold" -- run_drop_draw dropdraw-ssimhigh
|
||||
# A red picture with no evidence the knob was ever read by the process that produced it.
|
||||
expect FAILED "no 'E2 control armed' line in the library log" -- run_drop_draw dropdraw-nolog
|
||||
# The R-16 case the dropped-record COUNT exists for: the knob armed and dropped nothing, so the
|
||||
# wrong picture came from somewhere else.
|
||||
expect FAILED "the knob armed but dropped zero records" -- run_drop_draw dropdraw-zero
|
||||
# The real thing: a red picture, a fallen SSIM, and N > 0 records the library says it dropped.
|
||||
expect PASSED "ssim below threshold and N records dropped" -- run_drop_draw dropdraw-evidence
|
||||
|
||||
echo
|
||||
echo "smoke test: ${passes} passed, ${failures} failed"
|
||||
# Keep the private-file cases on the entry point used by CI and the local gate.
|
||||
|
||||
@@ -1,174 +0,0 @@
|
||||
#!/bin/bash
|
||||
# MobileGL - scripts/ci/redcheck_control_smoke_test.sh
|
||||
# Copyright (c) 2025-2026 MobileGL-Dev
|
||||
# Licensed under the GNU Lesser General Public License v3.0:
|
||||
# https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
# https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
# SPDX-License-Identifier: LGPL-3.0-only
|
||||
# End of Source File Header
|
||||
|
||||
# R-16's "I made it red once, by doing X" for scripts/ci/control_smoke_test.sh, mechanised so the
|
||||
# claim can be re-checked rather than believed.
|
||||
#
|
||||
# X = revert the message check in each control, i.e. put the controls back in the state ID-46
|
||||
# finding 8 found them in: a non-zero ctest exit is accepted whatever the failure was.
|
||||
#
|
||||
# The smoke test must then FAIL on the cases that exist for this defect -
|
||||
# "unrelated failure with a non-empty selection" and "red without the transport-resolution
|
||||
# message", plus the missing private-file Fatal - and not merely somewhere. A smoke test that goes
|
||||
# red for any other reason when the evidence check is removed would not be pinning the evidence check.
|
||||
set -u
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||
WORK="$(mktemp -d)" || exit 1
|
||||
HELPER="${HERE}/../../MobileGL/MG_IntegrationTest/Harness/split_log_paths.py"
|
||||
WAIT_HELPER="${HERE}/wait_boundary_negative_control.py"
|
||||
trap 'cp "${WORK}/split.orig" "${HERE}/split_negative_controls.sh"; cp "${WORK}/retrace.orig" "${HERE}/retrace_transport_control.sh"; cp "${WORK}/dropdraw.orig" "${HERE}/retrace_drop_draw_control.sh"; cp "${WORK}/helper.orig" "${HELPER}"; cp "${WORK}/wait.orig" "${WAIT_HELPER}"; rm -rf "${WORK}"' EXIT
|
||||
|
||||
cp "${HERE}/split_negative_controls.sh" "${WORK}/split.orig"
|
||||
cp "${HERE}/retrace_transport_control.sh" "${WORK}/retrace.orig"
|
||||
cp "${HERE}/retrace_drop_draw_control.sh" "${WORK}/dropdraw.orig"
|
||||
cp "${HELPER}" "${WORK}/helper.orig"
|
||||
cp "${WAIT_HELPER}" "${WORK}/wait.orig"
|
||||
|
||||
echo "=== baseline: the smoke test must be GREEN before anything is perturbed"
|
||||
if ! bash "${HERE}/control_smoke_test.sh" > "${WORK}/before.log" 2>&1; then
|
||||
echo "the smoke test is ALREADY RED; the red-check below would prove nothing"
|
||||
cat "${WORK}/before.log"
|
||||
exit 1
|
||||
fi
|
||||
tail -1 "${WORK}/before.log"
|
||||
|
||||
echo
|
||||
echo "=== perturbation: remove the evidence check from both controls"
|
||||
python3 - "${HERE}/split_negative_controls.sh" "${HERE}/retrace_transport_control.sh" \
|
||||
"${HERE}/retrace_drop_draw_control.sh" "${WAIT_HELPER}" <<'PY' || exit 1
|
||||
import sys
|
||||
split, retrace, dropdraw, wait = sys.argv[1:]
|
||||
# (file, [(needle, the prefix the line must start with)]). The draw-drop control has THREE
|
||||
# evidence checks rather than one, because "the picture went red" has three different ways of
|
||||
# being somebody else's red: the SSIM never fell, the library never said the knob armed, and the
|
||||
# knob armed and dropped nothing. All three are reverted together, and the smoke cases that pin
|
||||
# each of them must go red below.
|
||||
#
|
||||
# THE PREFIXES ARE A TUPLE, AND THAT IS A FIX, not a generalisation for its own sake. The first
|
||||
# version matched only lines starting with `if ! `, and the split control's evidence check has
|
||||
# been `elif ! tr -s ... | grep -qE "${evidence}"` since it was written - so this red-check found
|
||||
# ZERO checks in it, the SystemExit below was not checked by the caller, and the run went on to
|
||||
# smoke-test the UNPERTURBED controls and report "RED-CHECK FAILED: the controls accept an
|
||||
# unrelated failure again and the smoke test still passed". It failed safe rather than green, so
|
||||
# nothing was ever silently proved - but the one control it was most about was never perturbed.
|
||||
# Reproduced on p5/joint@e61d0012 before this line changed.
|
||||
#
|
||||
# AND THE SPLIT CONTROL HAS THREE EVIDENCE CHECKS, NOT ONE. E1 checks each boundary
|
||||
# probe's own observation and assertion; E3(a) checks its assertion and private log.
|
||||
# All three are reverted: perturbing only one leaves the
|
||||
# other two catching the smoke test's "unrelated failure" case, and the case never flips - which
|
||||
# is what this script measured the first time the perturbation actually applied.
|
||||
rules = [
|
||||
(split, [('LINE', '"${log_helper}" assertion', ('if ! ', 'elif ! ')),
|
||||
('SUBST', '"${private_evidence}" "${name}" || exit 1',
|
||||
'"${private_evidence}" "${name}" || true')]),
|
||||
(wait, [('SUBST', 'if not own_negative(text, observation, diagnostic):',
|
||||
'if False: # E1 evidence check removed by red-check')]),
|
||||
(retrace, [('LINE', 'grep -qF "${EVIDENCE}"', ('if ! ',))]),
|
||||
(dropdraw, [('LINE', 'awk -v a="${ssim}"', ('if ! ',)),
|
||||
('LINE', '[ -z "${armed_line}" ]', ('if ',)),
|
||||
('LINE', '[ "${dropped}" -lt 1 ]', ('if ',))]),
|
||||
]
|
||||
for path, checks in rules:
|
||||
text = open(path).read()
|
||||
for rule in checks:
|
||||
if rule[0] == 'SUBST':
|
||||
_, old, new = rule
|
||||
if text.count(old) != 1:
|
||||
raise SystemExit(f"expected exactly one {old!r} in {path}, found {text.count(old)}")
|
||||
text = text.replace(old, new)
|
||||
continue
|
||||
_, needle, prefixes = rule
|
||||
out, hit = [], 0
|
||||
for line in text.splitlines(keepends=True):
|
||||
if needle in line and line.lstrip().startswith(prefixes):
|
||||
indent = line[:len(line) - len(line.lstrip())]
|
||||
# KEEP THE KEYWORD. Turning an `elif` into an `if` splits the chain in two and
|
||||
# leaves the second half's `fi` dangling - a syntax error, which the smoke test
|
||||
# then reports as rc=2 on every case and which looks nothing like "the control
|
||||
# accepted an unrelated failure".
|
||||
keyword = 'elif' if line.lstrip().startswith('elif') else 'if'
|
||||
out.append(f"{indent}{keyword} false; then\n")
|
||||
hit += 1
|
||||
else:
|
||||
out.append(line)
|
||||
if hit != 1:
|
||||
raise SystemExit(f"expected exactly one {needle!r} check in {path}, found {hit}")
|
||||
text = ''.join(out)
|
||||
open(path, 'w').write(text)
|
||||
print("all seven evidence checks reverted to 'any non-zero ctest exit is accepted'")
|
||||
PY
|
||||
# AND THE PERTURBATION'S OWN FAILURE IS FATAL. Without this, a SystemExit above left the controls
|
||||
# UNTOUCHED and the run continued to smoke-test them, printing "RED-CHECK FAILED: the controls
|
||||
# accept an unrelated failure again and the smoke test still passed" - a true statement about a
|
||||
# perturbation that never happened, and the wrong diagnosis to hand whoever reads it.
|
||||
if [ $? -ne 0 ]; then
|
||||
echo "RED-CHECK ABORTED: the perturbation did not apply, so nothing below would be measuring it."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "=== the smoke test on the reverted controls (it MUST be red on the pinned cases)"
|
||||
bash "${HERE}/control_smoke_test.sh" > "${WORK}/after.log" 2>&1
|
||||
rc=$?
|
||||
cat "${WORK}/after.log"
|
||||
|
||||
if [ "${rc}" -eq 0 ]; then
|
||||
echo
|
||||
echo "RED-CHECK FAILED: the controls accept an unrelated failure again and the smoke test still passed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
missed=0
|
||||
grep -q "NOT OK unrelated failure with a non-empty selection" "${WORK}/after.log" || missed=1
|
||||
grep -q "NOT OK red without the transport-resolution message" "${WORK}/after.log" || missed=1
|
||||
grep -qF "NOT OK missing-fatal: control must report FAILED for its own reason" "${WORK}/after.log" || missed=1
|
||||
for case in \
|
||||
"red but the ssim is above the threshold" \
|
||||
"no 'E2 control armed' line in the library log" \
|
||||
"the knob armed but dropped zero records"; do
|
||||
grep -q "NOT OK ${case}" "${WORK}/after.log" || { echo "still green: ${case}"; missed=1; }
|
||||
done
|
||||
if [ "${missed}" -ne 0 ]; then
|
||||
echo
|
||||
echo "RED-CHECK FAILED: the smoke test did not red every named split, retrace, draw-drop and private-log evidence case."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo
|
||||
cp "${WORK}/split.orig" "${HERE}/split_negative_controls.sh"
|
||||
cp "${WORK}/retrace.orig" "${HERE}/retrace_transport_control.sh"
|
||||
cp "${WORK}/dropdraw.orig" "${HERE}/retrace_drop_draw_control.sh"
|
||||
cp "${WORK}/wait.orig" "${WAIT_HELPER}"
|
||||
echo "=== ID-62 perturbation: remove only the skip check"
|
||||
python3 - "${HELPER}" "${WAIT_HELPER}" <<'PY' || exit 1
|
||||
from pathlib import Path
|
||||
import sys
|
||||
path = Path(sys.argv[1])
|
||||
text = path.read_text()
|
||||
needle = ' if skipped:\n'
|
||||
if text.count(needle) != 1:
|
||||
raise SystemExit('expected exactly one skip check')
|
||||
path.write_text(text.replace(needle, ' if False: # skip check removed by red-check\n'))
|
||||
path = Path(sys.argv[2])
|
||||
text = path.read_text()
|
||||
needle = ' if case.find("skipped") is not None or case.get("status") in {"notrun", "disabled"}:\n'
|
||||
if text.count(needle) != 1:
|
||||
raise SystemExit('expected exactly one E1 skip check')
|
||||
path.write_text(text.replace(needle, ' if False: # E1 skip check removed by red-check\n'))
|
||||
PY
|
||||
bash "${HERE}/control_smoke_test.sh" > "${WORK}/skips.log" 2>&1
|
||||
rc=$?
|
||||
cat "${WORK}/skips.log"
|
||||
if [ "${rc}" -eq 0 ] || ! grep -qFx 'NOT OK skipped-selection: control must report FAILED for its own reason' "${WORK}/skips.log"; then
|
||||
echo 'RED-CHECK FAILED: removing the skip check must red the skipped-selection message assertion'
|
||||
exit 1
|
||||
fi
|
||||
echo 'ID-62 red-once: removing only the skip check made skipped-selection red (smoke rc=1)'
|
||||
echo "P5_T1_CONTROL_SMOKE_REDCHECK_OK - evidence and skipped-selection checks each made every named smoke case red"
|
||||
@@ -1,159 +0,0 @@
|
||||
#!/bin/bash
|
||||
# MobileGL - scripts/ci/retrace_drop_draw_control.sh
|
||||
# Copyright (c) 2025-2026 MobileGL-Dev
|
||||
# Licensed under the GNU Lesser General Public License v3.0:
|
||||
# https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
# https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
# SPDX-License-Identifier: LGPL-3.0-only
|
||||
# End of Source File Header
|
||||
|
||||
# EXIT GATE E2's PICTURE CONTROL: dropping the draws on the wire must redden this retrace, and
|
||||
# it must be THIS control's red.
|
||||
#
|
||||
# WHY THERE IS A SECOND CONTROL ON THIS LANE, and why it is the draws and not the clears.
|
||||
#
|
||||
# The retrace-split job already carries scripts/ci/retrace_transport_control.sh, which swaps a
|
||||
# PULL library in and requires run_trace_case.cmake's transport-resolution assertion to catch it.
|
||||
# That control proves the lane is running a SPLIT library. It does not prove that the PICTURE came
|
||||
# through the wire, and it cannot: OpenRA scores ssim 1.000000 against a monolith library too, so
|
||||
# the transport assertion is what reds it and the comparator is never consulted. BRIEF-P5 7 E2
|
||||
# names a second, sharper statement - "patch an emitter to drop a record and the SSIM must fall
|
||||
# below the threshold" - and until now nothing executed it.
|
||||
#
|
||||
# c1 shipped MOBILEGL_IPC_E2_DROP_CLEAR for exactly that, and the joint gate ran it and it DID NOT
|
||||
# WORK: with the knob armed and its WARN in the library's own log, the OpenRA retrace under inproc
|
||||
# still scored ssim=1.000000 with mismatchPixels=0 (joint-v1.md 3, "E2 clear-drop control - NOT
|
||||
# VERIFIED"). That is not a broken knob. `apitrace dump` over the 31249 replayed calls counts 30
|
||||
# glClear, 30 glXSwapBuffers and 788 glDrawArrays, and the final frame issues its clear at call
|
||||
# 30197 and then covers the surface four times over with a terrain layer before the snapshot at
|
||||
# 31249. A frame that overdraws every pixel it clears has a picture that does not depend on the
|
||||
# clear. Measured here with the dropped-record count published: 29 Clear records dropped, ssim
|
||||
# still 1.000000. The knob worked; the observable was invisible.
|
||||
#
|
||||
# So this control drops every DrawVbo record instead (MOBILEGL_IPC_E2_DROP_DRAW=1): the surface
|
||||
# can then only carry the clear colour, and the golden is made of the geometry. Measured on the
|
||||
# same head: 758 DrawVbo records dropped, ssim=0.000036, mismatchPixels=295296.
|
||||
#
|
||||
# WHAT THE CONTROL ASSERTS, and none of the three is the process exit code alone (R-16):
|
||||
# 1. a NON-EMPTY selection, counted before the run - `--no-tests=error` turns an empty selection
|
||||
# into a non-zero exit, which is how ID-46 finding 8(b) got a control to congratulate itself;
|
||||
# 2. the SSIM ACTUALLY FELL: the numbers are parsed out of the retrace's own summary and
|
||||
# compared, rather than "ctest was non-zero". A loader failure, a missing fixture, a timeout
|
||||
# and a Fatal{ all exit non-zero and none of them is this control's red;
|
||||
# 3. the LIBRARY SAID IT DROPPED SOMETHING: its own "E2 control armed ... N records dropped on
|
||||
# the wire" line, with N > 0, from the log file the replay wrote. Without this, a knob that
|
||||
# was never read - a stale library, a variable that did not reach the process, an emit table
|
||||
# that fell through to the driver - would redden the picture for some other reason and pass.
|
||||
#
|
||||
# Usage: retrace_drop_draw_control.sh <case> <backend>
|
||||
# CTEST ctest binary (default: ctest)
|
||||
# CONTROL_TMPDIR scratch dir (default: ${RUNNER_TEMP:-/tmp})
|
||||
# LIBRARY_LOG the replay's CLIENT library log (default: <case>/<backend>/output/mobilegl.client.log)
|
||||
set -u
|
||||
|
||||
CASE="${1:?usage: retrace_drop_draw_control.sh <case> <backend>}"
|
||||
BACKEND="${2:?usage: retrace_drop_draw_control.sh <case> <backend>}"
|
||||
|
||||
CTEST="${CTEST:-ctest}"
|
||||
CONTROL_TMPDIR="${CONTROL_TMPDIR:-${RUNNER_TEMP:-/tmp}}"
|
||||
# P6: BOTH ROLES HAVE THEIR OWN LOG and neither keeps the old name, so an un-updated default
|
||||
# here would be a file that does not exist rather than half a session read as a whole one. This
|
||||
# control's marker (`E2 control armed`) is a CLIENT-side line.
|
||||
LIBRARY_LOG="${LIBRARY_LOG:-${CASE}/${BACKEND}/output/mobilegl.client.log}"
|
||||
mkdir -p "${CONTROL_TMPDIR}"
|
||||
FROZEN_LIBRARY="${FROZEN_LIBRARY:?FROZEN_LIBRARY must name the split library the replay loads}"
|
||||
symbols=$(nm --defined-only "${FROZEN_LIBRARY}") || exit 1
|
||||
remote_count=$(printf '%s\n' "${symbols}" | grep -ic MG_Remote || true)
|
||||
echo "draw-drop control library: ${FROZEN_LIBRARY}: MG_Remote=${remote_count}"
|
||||
if [ "${remote_count}" -lt 1 ]; then
|
||||
echo '::error::draw-drop control requires a split library: MG_Remote=0'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
selector="^MobileGLTraceReplay\.${CASE}\.${BACKEND}$"
|
||||
|
||||
# The rerun replays into the same case directory, so the good run's images are put aside and
|
||||
# restored whichever way the control goes; "Upload actual image" runs `if: always()` and would
|
||||
# otherwise ship the deliberately-wrong run's output under the good run's name.
|
||||
GOOD_OUTPUT="${CONTROL_TMPDIR}/drop-draw-verified-output"
|
||||
rm -rf "${GOOD_OUTPUT}"
|
||||
if [ -d "${CASE}" ]; then cp -a "${CASE}" "${GOOD_OUTPUT}"; fi
|
||||
|
||||
restore_good_output() {
|
||||
if [ -d "${GOOD_OUTPUT}" ]; then
|
||||
rm -rf "${CASE}"; mv "${GOOD_OUTPUT}" "${CASE}"
|
||||
echo "restored the verified run's output over the control's"
|
||||
fi
|
||||
}
|
||||
trap restore_good_output EXIT
|
||||
trap 'exit 130' INT TERM
|
||||
|
||||
matched=$("${CTEST}" -N -R "${selector}" | grep -cE '^ *Test *#[0-9]+:')
|
||||
if [ "${matched}" -lt 1 ]; then
|
||||
restore_good_output
|
||||
echo "::error::the control selected ${matched} tests with -R '${selector}', so there is nothing for the dropped draws to redden. --no-tests=error would have exited non-zero on the empty selection and a control without this guard reads that as success (ID-46 finding 8b)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A PREVIOUS RUN'S LINE MUST NEVER ARM THIS ONE. The CLIENT truncates this log once at open and
|
||||
# then appends (P6 made the sink O_APPEND, because under spawn a second process writes the same
|
||||
# file and two truncating handles overwrite each other's bytes) - but it only truncates if the
|
||||
# replay gets that far, and a run that died in the loader would leave the baseline's log in place
|
||||
# with a perfectly good "control armed" line in it. Removing it first is the same rule
|
||||
# split_negative_controls.sh's `reset` step follows.
|
||||
rm -f "${LIBRARY_LOG}"
|
||||
|
||||
out="${CONTROL_TMPDIR}/retrace-drop-draw-output.txt"
|
||||
# THE TRANSPORT COMES FROM THE JOB, not from this file. retrace-split is a matrix over
|
||||
# {backend, case, transport} as of P6, and a control that hard-coded `inproc` would have gone on
|
||||
# proving something about the OTHER arm while the spawn arm ran unguarded. Default inproc so a
|
||||
# caller that sets nothing behaves exactly as before.
|
||||
transport="${SPLIT_TRANSPORT:-inproc}"
|
||||
export MOBILEGL_TRANSPORT="${transport}"
|
||||
export MOBILEGL_IPC_E2_DROP_DRAW=1
|
||||
"${CTEST}" -V --no-tests=error --timeout 10800 -R "${selector}" > "${out}" 2>&1
|
||||
control_rc=$?
|
||||
unset MOBILEGL_IPC_E2_DROP_DRAW
|
||||
cat "${out}"
|
||||
|
||||
# Read the library's evidence BEFORE the good output is restored over it.
|
||||
armed_line=""
|
||||
dropped=0
|
||||
if [ -f "${LIBRARY_LOG}" ]; then
|
||||
armed_line=$(grep 'MGPipe: E2 control armed' "${LIBRARY_LOG}" | tail -1)
|
||||
dropped=$(printf '%s' "${armed_line}" | sed -n 's/.*armed[^,]*, \([0-9][0-9]*\) records dropped.*/\1/p')
|
||||
dropped="${dropped:-0}"
|
||||
fi
|
||||
cp -f "${LIBRARY_LOG}" "${CONTROL_TMPDIR}/drop-draw-library.log" 2>/dev/null
|
||||
|
||||
restore_good_output
|
||||
|
||||
if [ "${control_rc}" -eq 0 ]; then
|
||||
echo "::error::the split retrace PASSED with every DrawVbo record dropped on the wire. The golden is made of that geometry, so a green here means the picture did not come from the wire: the emit table fell through to the driver, the library under test is not the one the lane thinks it is, or MOBILEGL_IPC_E2_DROP_DRAW never reached the process. Exit gate E2 is exactly this statement and nothing weaker - MOBILEGL_IPC_E2_DROP_CLEAR is NOT a substitute (measured: 29 clears dropped, ssim still 1.000000, because OpenRA overdraws every pixel it clears)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 2. THE SSIM ACTUALLY FELL. Parsed, not inferred from the exit code.
|
||||
ssim_line=$(grep -o 'ssim=[0-9.]*, ssimThreshold=[0-9.]*' "${out}" | tail -1)
|
||||
if [ -z "${ssim_line}" ]; then
|
||||
echo "::error::the split retrace went red (ctest exit ${control_rc}) with the draws dropped, but its output carries no 'ssim=..., ssimThreshold=...' summary at all, so the comparator never ran. A loader failure, a missing fixture, a timeout or a Fatal{ all land here and none of them is this control's red."
|
||||
exit 1
|
||||
fi
|
||||
ssim=${ssim_line#ssim=}; ssim=${ssim%%,*}
|
||||
threshold=${ssim_line##*ssimThreshold=}
|
||||
if ! awk -v a="${ssim}" -v b="${threshold}" 'BEGIN { exit !(a + 0 < b + 0) }'; then
|
||||
echo "::error::the split retrace went red (ctest exit ${control_rc}) but its ${ssim_line} is NOT below the threshold, so the picture is not what reddened it. This control's whole claim is that the golden is made of the dropped geometry."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 3. THE LIBRARY SAID IT DROPPED SOMETHING.
|
||||
if [ -z "${armed_line}" ]; then
|
||||
echo "::error::the split retrace went red with ssim ${ssim} < ${threshold}, but ${LIBRARY_LOG} carries no 'MGPipe: E2 control armed' line, so there is no evidence the knob was ever read by the process that produced the picture. A library that is not the one under test, an emit table that fell through to the driver, or an environment that did not reach the replay all produce a wrong picture for a reason that has nothing to do with this control."
|
||||
exit 1
|
||||
fi
|
||||
if [ "${dropped}" -lt 1 ]; then
|
||||
echo "::error::the split retrace went red with ssim ${ssim} < ${threshold} and the knob announced itself - '${armed_line}' - but it reports ZERO records dropped. The emitter was never reached, so whatever changed the picture was not this control. This is the R-16 case the dropped-record COUNT exists for: the arming message alone proves only that the knob was read."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "the dropped draws turned the split retrace red for their own reason (ctest exit ${control_rc}): ${matched} selected case(s), ssim ${ssim} < ${threshold}, and the library dropped ${dropped} record(s) on the wire"
|
||||
@@ -1,179 +0,0 @@
|
||||
#!/bin/bash
|
||||
# MobileGL - scripts/ci/retrace_transport_control.sh
|
||||
# Copyright (c) 2025-2026 MobileGL-Dev
|
||||
# Licensed under the GNU Lesser General Public License v3.0:
|
||||
# https://www.gnu.org/licenses/gpl-3.0.txt
|
||||
# https://www.gnu.org/licenses/lgpl-3.0.txt
|
||||
# SPDX-License-Identifier: LGPL-3.0-only
|
||||
# End of Source File Header
|
||||
|
||||
# THE RETRACE-SPLIT LANE'S NEGATIVE CONTROL: a library WITHOUT A TRANSPORT must red this split retrace.
|
||||
#
|
||||
# P13 W2: the library is the push monolith without MG_Remote (the shape FCL embeds). Until W2 it was
|
||||
# the pull build, and this file was retrace_pull_library_control.sh; both libraries were run side by
|
||||
# side for one wave and both went red here for the same sentence. "pull" below is history: what the
|
||||
# control needs is a library with no MG_Remote, and the push monolith is one.
|
||||
#
|
||||
# This file is the body of .github/workflows/test.yml's "Negative control (library without a transport)" step in
|
||||
# the retrace-split job, extracted for the reason given at the top of
|
||||
# scripts/ci/split_negative_controls.sh: a `run:` block is unreviewable and untestable off a
|
||||
# runner, and scripts/ci/control_smoke_test.sh now runs THIS file rather than a hand-made copy.
|
||||
#
|
||||
# WHAT THE CONTROL IS FOR. The retrace-split job replays a trace against the SPLIT runtime under
|
||||
# MOBILEGL_TRANSPORT=inproc. OpenRA scores ssim 1.000000 against a MONOLITH library too - measured -
|
||||
# so the picture is not and cannot be this lane's gate. What stands between the job and a green that
|
||||
# ran monolith end to end is run_trace_case.cmake's transport-resolution assertion
|
||||
# (run_trace_case.cmake:265-289): the library must have logged
|
||||
# "MOBILEGL_TRANSPORT=inproc - the MGPipe record stream", which exists only in
|
||||
# ConfigLoader::InitTransport's InProcess arm, which exists only under MOBILEGL_BUILD_DISAGGREGATED.
|
||||
# This control swaps the pull library over the frozen path and requires the same replay to fail FOR
|
||||
# THAT REASON.
|
||||
#
|
||||
# WHAT THE REVIEW FOUND (ID-46 finding 8 part (b), CONFIRMED by execution against the REAL ctest in
|
||||
# a REAL build tree; ID-48 assigns it here). Two holes, both of which let the control pass while
|
||||
# asserting nothing:
|
||||
#
|
||||
# 1. NO SELECTION GUARD AT ALL - unlike the split lane's run_control, which has had one since
|
||||
# review finding M-4. With a case/backend regex matching no tests, `--no-tests=error` exits 8,
|
||||
# and the old `if [ "${control_rc}" -eq 0 ]` accepted 8 as "the pull library turned it red".
|
||||
# The verifier measured exactly that: "real ctest exit for a regex matching NO tests: 8",
|
||||
# HARNESS_EXIT=0. An empty selection was the one thing this control could not tell apart from
|
||||
# a working transport-identity assertion.
|
||||
# 2. ONLY "non-zero ctest" WAS CHECKED after the nm identity check. The nm check establishes that
|
||||
# the library IS a pull build; it says nothing about why the replay failed. A loader failure, a
|
||||
# missing fixture or a timeout all passed the control.
|
||||
#
|
||||
# Both are closed below: the selection is counted before the run, and the red must carry
|
||||
# run_trace_case.cmake's own words.
|
||||
#
|
||||
# Usage: retrace_transport_control.sh <case> <backend>
|
||||
# CTEST ctest binary (default: ctest)
|
||||
# CONTROL_TMPDIR scratch dir (default: ${RUNNER_TEMP:-/tmp})
|
||||
# CONTROL_LIBRARY the libMobileGL.so without MG_Remote to swap in
|
||||
# FROZEN_LIBRARY the path every case has baked in, which CONTROL_LIBRARY is copied over
|
||||
set -u
|
||||
|
||||
CASE="${1:?usage: retrace_transport_control.sh <case> <backend>}"
|
||||
BACKEND="${2:?usage: retrace_transport_control.sh <case> <backend>}"
|
||||
|
||||
CTEST="${CTEST:-ctest}"
|
||||
CONTROL_TMPDIR="${CONTROL_TMPDIR:-${RUNNER_TEMP:-/tmp}}"
|
||||
CONTROL_LIBRARY="${CONTROL_LIBRARY:?CONTROL_LIBRARY must name a libMobileGL.so built without MG_Remote}"
|
||||
FROZEN_LIBRARY="${FROZEN_LIBRARY:?FROZEN_LIBRARY must name the path the cases have baked in}"
|
||||
mkdir -p "${CONTROL_TMPDIR}"
|
||||
|
||||
# run_trace_case.cmake's own sentences for "this library never resolved the transport", one per
|
||||
# line (grep -F takes its patterns newline-separated). Anchored on the distinctive clause rather
|
||||
# than on the whole paragraph, which carries substituted paths.
|
||||
#
|
||||
# TWO SENTENCES SINCE P6's LOG RENAME, and the second is the one a pull library actually gets.
|
||||
# A pull build has one log role and writes MOBILEGL_LOG_FILE_PATH unchanged - output/mobilegl.log
|
||||
# (MG_Util/Debug/Log.h, the pull arm of RoleLogPath) - while the disaggregated build's readers,
|
||||
# run_trace_case.cmake among them, read output/mobilegl.client.log. So the pull library never
|
||||
# reaches the marker search that says "never reported resolving it": the runner stops one check
|
||||
# earlier, at "the run wrote no <client log>, so there is no evidence the transport ever
|
||||
# resolved". Both are the transport-identity assertion refusing the same library for the same
|
||||
# reason (a pull build has no split log for the marker to be in); neither is a loader failure, a
|
||||
# missing fixture, a timeout or an SSIM drop, which are the reds hole 2 below exists to refuse.
|
||||
# MEASURED on the B3 package tree with the real ctest and the real pull library: with only the
|
||||
# first sentence here, the control exited 1 through hole 2 on every pull-library run - the step
|
||||
# had been red for no product reason since the rename. scripts/ci/testdata/stub_ctest.sh's
|
||||
# retrace-evidence-nolog mode reproduces the shape.
|
||||
EVIDENCE='never reported resolving it
|
||||
no evidence the transport ever resolved'
|
||||
# EVERY LINE OF EVIDENCE IS A PATTERN, SO AN EMPTY LINE IS A PATTERN THAT MATCHES EVERYTHING. A
|
||||
# trailing newline, or a blank line between the two sentences, would turn hole 2's check below
|
||||
# into "any red will do". The smoke test's "red without the transport-resolution message" case
|
||||
# would catch that (22/23), but only once somebody ran it; this refuses it here, before the
|
||||
# selection is counted or the library is swapped.
|
||||
case "${EVIDENCE}" in
|
||||
*$'\n\n'*|*$'\n'|$'\n'*)
|
||||
echo "::error::EVIDENCE carries an empty grep -F pattern (a blank line or a leading/trailing newline), which would match every failure"
|
||||
exit 1;;
|
||||
esac
|
||||
|
||||
selector="^MobileGLTraceReplay\.${CASE}\.${BACKEND}$"
|
||||
|
||||
# The rerun replays into the same case directory, so the good run's images are put aside and
|
||||
# restored whichever way the control goes; "Upload actual image" runs `if: always()` and would
|
||||
# otherwise ship the deliberately-wrong run's output under the good run's name.
|
||||
GOOD_OUTPUT="${CONTROL_TMPDIR}/split-verified-output"
|
||||
rm -rf "${GOOD_OUTPUT}"
|
||||
if [ -d "${CASE}" ]; then cp -a "${CASE}" "${GOOD_OUTPUT}"; fi
|
||||
|
||||
restore_good_output() {
|
||||
if [ -d "${GOOD_OUTPUT}" ]; then
|
||||
rm -rf "${CASE}"; mv "${GOOD_OUTPUT}" "${CASE}"
|
||||
echo "restored the verified run's output over the control's"
|
||||
fi
|
||||
}
|
||||
|
||||
# Restore the exact split library on success, failure, and interruption. The following
|
||||
# draw-drop control uses this same frozen path.
|
||||
saved_library=$(mktemp "${CONTROL_TMPDIR}/split-library.XXXXXX") || exit 1
|
||||
cp -p "${FROZEN_LIBRARY}" "${saved_library}" || exit 1
|
||||
restore_control() {
|
||||
cp -p "${saved_library}" "${FROZEN_LIBRARY}"
|
||||
rm -f "${saved_library}"
|
||||
restore_good_output
|
||||
}
|
||||
trap restore_control EXIT
|
||||
trap 'exit 130' INT TERM
|
||||
|
||||
# HOLE 1: COUNT THE SELECTION FIRST. `--no-tests=error` turns an empty selection into a non-zero
|
||||
# exit, which is indistinguishable from a working control unless the selection is counted.
|
||||
matched=$("${CTEST}" -N -R "${selector}" | grep -cE '^ *Test *#[0-9]+:')
|
||||
if [ "${matched}" -lt 1 ]; then
|
||||
restore_good_output
|
||||
echo "::error::the control selected ${matched} tests with -R '${selector}', so there is nothing for the transport-less library to red. --no-tests=error would have exited non-zero on the empty selection and this control used to read that as success (ID-46 finding 8b, measured: ctest exit 8, step green)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# The pull library, unpacked from build-linux's artifact, over the frozen path every case has baked
|
||||
# in. It defines no MG_Remote symbol, so ConfigLoader has no transport parser and
|
||||
# MOBILEGL_TRANSPORT=inproc is accepted and ignored - the exact shape of "the split lane ran
|
||||
# monolith".
|
||||
cp "${CONTROL_LIBRARY}" "${FROZEN_LIBRARY}" || exit 1
|
||||
symbols=$(nm --defined-only "${FROZEN_LIBRARY}") || exit 1
|
||||
remote_count=$(printf '%s\n' "${symbols}" | grep -ic MG_Remote || true)
|
||||
echo "transport control library: ${FROZEN_LIBRARY}: MG_Remote=${remote_count}"
|
||||
if [ "${remote_count}" -ne 0 ]; then
|
||||
restore_good_output
|
||||
echo "::error::the control's own library defines MG_Remote symbols, so it has a transport and this control would prove nothing"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
out="${CONTROL_TMPDIR}/retrace-control-output.txt"
|
||||
# THE TRANSPORT COMES FROM THE JOB, not from this file. retrace-split is a matrix over
|
||||
# {backend, case, transport} as of P6, and a control that hard-coded `inproc` would have gone on
|
||||
# proving something about the OTHER arm while the spawn arm ran unguarded. Default inproc so a
|
||||
# caller that sets nothing behaves exactly as before.
|
||||
transport="${SPLIT_TRANSPORT:-inproc}"
|
||||
export MOBILEGL_TRANSPORT="${transport}"
|
||||
"${CTEST}" -V --no-tests=error --timeout 10800 -R "${selector}" > "${out}" 2>&1
|
||||
control_rc=$?
|
||||
cat "${out}"
|
||||
|
||||
restore_good_output
|
||||
|
||||
if [ "${control_rc}" -eq 0 ]; then
|
||||
echo "::error::a library WITHOUT A TRANSPORT passed the split retrace. OpenRA scores ssim 1.000000 under a monolith library too (measured), so the picture is not and cannot be this lane's gate - run_trace_case.cmake's transport-resolution assertion is, and it has stopped working. Every green in this job is then a monolith run under a name that says split."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# HOLE 2: THE RED MUST BE THIS CONTROL'S RED.
|
||||
#
|
||||
# Whitespace is normalised across the WHOLE file before the match, newlines included, because the
|
||||
# sentence is emitted by CMake's message(FATAL_ERROR ...) and CMake re-wraps that text to its own
|
||||
# width: "never reported resolving it" arrives split over two lines with a two-space continuation
|
||||
# indent, and a line-oriented grep for the literal finds nothing. That is not hypothetical - it is
|
||||
# the shape the stub reproduces in scripts/ci/testdata/stub_ctest.sh.
|
||||
# `ctest -V` ALSO PREFIXES EVERY OUTPUT LINE WITH "<test number>: ", continuation lines included,
|
||||
# so without stripping it first the folded text reads "never 1: reported resolving it" and the
|
||||
# control reds on the very sentence it is looking for (the *-prefixed stub modes).
|
||||
if ! sed -E 's/^[0-9]+: //' "${out}" | tr -s '[:space:]' ' ' | grep -qF "${EVIDENCE}"; then
|
||||
echo "::error::the split retrace went red (ctest exit ${control_rc}) with the transport-less library in place, but the failure never says the library did not resolve the transport - neither of run_trace_case.cmake's sentences (\"${EVIDENCE//$'\n'/\" / \"}\") is in the output. A loader failure, a missing fixture, a timeout or an SSIM drop all land here, and none of them establishes that the transport-identity assertion is what caught the pull library. Only 'non-zero ctest' used to be checked (ID-46 finding 8b)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "the transport-less library turned the split retrace red for its own reason (ctest exit ${control_rc}): ${matched} selected case(s) named the transport, not the picture"
|
||||
Vendored
-106
@@ -33,28 +33,6 @@
|
||||
# green baseline green; the control's own run PASSES (the knob is not load-bearing)
|
||||
# red-baseline the baseline itself has a failed entry
|
||||
# all-skipped the baseline is entirely skipped (lost implementation, a hard failure)
|
||||
# retrace-noselect `ctest -N` matches nothing; the run exits 8 the way --no-tests=error does
|
||||
# retrace-unrelated one match; the run fails without naming the transport
|
||||
# retrace-evidence one match; the run fails with run_trace_case.cmake's own sentence
|
||||
# retrace-evidence-nolog
|
||||
# one match; the run fails one check earlier, with the runner's "wrote no
|
||||
# <client log> ... no evidence the transport ever resolved" - the shape a
|
||||
# PULL library produces since P6's per-role log rename
|
||||
# retrace-evidence-prefixed, retrace-evidence-nolog-prefixed
|
||||
# the two above with `ctest -V`'s "1: " prefix on every line
|
||||
# retrace-green one match; the run PASSES
|
||||
#
|
||||
# EXIT GATE E2's DRAW-DROP CONTROL (scripts/ci/retrace_drop_draw_control.sh). The library's own
|
||||
# log is modelled as a separate sink from ctest stdout, exactly as it is for the split lane: the
|
||||
# console sink is compiled out of the configurations these lanes run, so the dropped-record count
|
||||
# can only ever arrive through ${LIBRARY_LOG}.
|
||||
# dropdraw-noselect `ctest -N` matches nothing
|
||||
# dropdraw-green the retrace PASSES with the draws dropped
|
||||
# dropdraw-nossim red, but no ssim summary at all (loader failure / timeout shape)
|
||||
# dropdraw-ssimhigh red, and the ssim is ABOVE the threshold: something else reddened it
|
||||
# dropdraw-nolog red with a low ssim, but the library never said the knob armed
|
||||
# dropdraw-zero red with a low ssim, the knob armed, and it dropped NOTHING
|
||||
# dropdraw-evidence red with a low ssim and N > 0 records dropped: the real thing
|
||||
set -u
|
||||
|
||||
mode="${STUB_MODE:?STUB_MODE must be set}"
|
||||
@@ -142,10 +120,6 @@ fi
|
||||
|
||||
emit_listing() {
|
||||
echo "Test project /stub"
|
||||
if [ "${mode}" = "retrace-noselect" ] || [ "${mode}" = "dropdraw-noselect" ]; then
|
||||
echo "Total Tests: 0"
|
||||
return
|
||||
fi
|
||||
echo " Test #1: DirectGLES.Split.ClearThenReadPixelsScenario.ClearWithNoDrawIsVisibleToDefaultFramebufferReadPixels"
|
||||
if [ "${mode}" = partial-fatal ]; then
|
||||
echo " Test #2: DirectGLES.Split.TriangleScenario.SecondEntry"
|
||||
@@ -288,86 +262,6 @@ case "${mode}" in
|
||||
echo "100% tests passed, 0 tests failed out of 1"
|
||||
exit 0
|
||||
;;
|
||||
retrace-noselect)
|
||||
echo "No tests were found!!!"
|
||||
exit 8
|
||||
;;
|
||||
retrace-unrelated)
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "CMake Error: the fixture could not be unpacked"
|
||||
exit 8
|
||||
;;
|
||||
retrace-evidence)
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "CMake Error at run_trace_case.cmake:279 (message):"
|
||||
echo " MOBILEGL_TRANSPORT=inproc is set for OpenRA DirectGLES and the library never"
|
||||
echo " reported resolving it: mobilegl.log carries no"
|
||||
echo ' "MOBILEGL_TRANSPORT=inproc - the MGPipe record stream".'
|
||||
exit 8
|
||||
;;
|
||||
retrace-evidence-nolog)
|
||||
# What a pull library gets since P6: it wrote output/mobilegl.log (one role, no suffix), the
|
||||
# runner reads output/mobilegl.client.log and stops at the no-log check BEFORE the marker
|
||||
# search. CMake-wrapped like the mode above, with the clause split across lines.
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "CMake Error at run_trace_case.cmake:270 (message):"
|
||||
echo " MOBILEGL_TRANSPORT=inproc is set for OpenRA DirectGLES but the run wrote no"
|
||||
echo " OpenRA/DirectGLES/output/mobilegl.client.log, so there is no evidence the"
|
||||
echo " transport ever resolved. A split retrace with no library log cannot be"
|
||||
echo " counted as a split retrace."
|
||||
exit 8
|
||||
;;
|
||||
retrace-evidence-prefixed|retrace-evidence-nolog-prefixed)
|
||||
# The two modes above as `ctest -V` really prints them: EVERY line of a test's output carries
|
||||
# a "<test number>: " prefix, continuation lines included, so after whitespace folding the
|
||||
# wrapped sentence reads "never 1: reported resolving it" / "no evidence the 1: transport ever
|
||||
# resolved". A short case directory breaks the sentence at exactly those words; CI's long
|
||||
# paths happened not to (retrace-split, run 35671704873: 5 false reds from this shape).
|
||||
STUB_MODE="${mode%-prefixed}" bash "$0" "$@" | sed 's/^/1: /'
|
||||
exit "${PIPESTATUS[0]}"
|
||||
;;
|
||||
retrace-green)
|
||||
echo "100% tests passed, 0 tests failed out of 1"
|
||||
exit 0
|
||||
;;
|
||||
dropdraw-*)
|
||||
# The library's own log, written by the replay the way the real one is. The control removes
|
||||
# it before the run, so anything here is this run's.
|
||||
armed=""
|
||||
case "${mode}" in
|
||||
dropdraw-zero)
|
||||
armed='MGPipe: E2 control armed - drop-draw=1 drop-clear=0, 0 records dropped on the wire (draw=0 clear=0), frame 29' ;;
|
||||
dropdraw-nolog) armed="" ;;
|
||||
*)
|
||||
armed='MGPipe: E2 control armed - drop-draw=1 drop-clear=0, 758 records dropped on the wire (draw=758 clear=0), frame 29' ;;
|
||||
esac
|
||||
if [ -n "${LIBRARY_LOG:-}" ] && [ -n "${armed}" ]; then
|
||||
mkdir -p "$(dirname "${LIBRARY_LOG}")"
|
||||
printf '%s\n' "[10:38:30] [Linux mobilegl_trace_/WARN]: ${armed}" > "${LIBRARY_LOG}"
|
||||
fi
|
||||
case "${mode}" in
|
||||
dropdraw-green)
|
||||
echo "100% tests passed, 0 tests failed out of 1"
|
||||
exit 0
|
||||
;;
|
||||
dropdraw-nossim)
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "CMake Error: the replay could not load the library"
|
||||
exit 8
|
||||
;;
|
||||
dropdraw-ssimhigh)
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "-- retrace completed; ssim=1.000000, ssimThreshold=0.990000, mismatchPixels=0"
|
||||
echo "CMake Error at run_trace_case.cmake:301 (message): 3 MGPipe Fatal(s)"
|
||||
exit 8
|
||||
;;
|
||||
*)
|
||||
echo "1/1 Test #1: MobileGLTraceReplay.OpenRA.DirectGLES ...***Failed"
|
||||
echo "-- retrace completed; ssim=0.000036, ssimThreshold=0.990000, mismatchPixels=295296"
|
||||
exit 8
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
*)
|
||||
echo "stub_ctest: unknown STUB_MODE '${mode}'" >&2
|
||||
exit 127
|
||||
|
||||
@@ -394,11 +394,11 @@ def github_transport_matrix(cases, *, apk=False):
|
||||
THE SPAWN LANE IS APK-ONLY, and that is a statement about where the evidence is missing
|
||||
rather than about what spawn can do. On Linux the spawn arm already has a job of its own -
|
||||
test.yml's `retrace-split`, which is {backend, case, transport} over inproc AND spawn and
|
||||
carries the two negative controls that make it mean anything (the pull library must red it,
|
||||
dropping the draws must red it). Adding spawn to THIS matrix as well would pay for a second
|
||||
full Linux sweep that proves nothing the first one does not, and test.yml's `retrace` job
|
||||
has neither the server image staged nor MOBILEGL_IPC_SERVER_PATH set - so the lane would
|
||||
not merely be redundant, it would be red.
|
||||
whose legs prove in-test that the transport resolved and, on OpenRA, that draw records
|
||||
crossed the wire (run_trace_case.cmake's evidence). Adding spawn to THIS matrix would pay
|
||||
for a second full Linux sweep that proves nothing the first one does not, and test.yml's
|
||||
`retrace` job has neither the server image staged nor MOBILEGL_IPC_SERVER_PATH set - so the
|
||||
lane would not merely be redundant, it would be red.
|
||||
|
||||
On Android there is no such job. Until this lane existed, nothing in CI ever started a
|
||||
second MobileGL process on a device.
|
||||
|
||||
Reference in New Issue
Block a user