mirror of
https://github.com/MobileGL-Dev/MobileGL
synced 2026-11-07 01:53:24 +09:00
[tools] (P7): window-2 reducers follow the contract formula, require every block and one boot session
This commit is contained in:
@@ -2,7 +2,11 @@
|
||||
|
||||
- 运行手册：[`tools/RUNBOOK.md`](tools/RUNBOOK.md)（一条命令 `tools/window.sh <pipe-sha>`，detached、可续跑；工作副本在 `~/w7/notes/p7/window2/`）。
|
||||
- 脚本：`tools/{lib.sh,detach.sh,window.sh,10-build.sh,21-preflight.sh,20-install.sh,30-gate3.sh,40-bsl-stats.sh,50-cts-after.sh,60-reduce.py,90-restore.sh}`；
|
||||
`tools/dry-run-p7w6.sh` 是下面 dry run 的原样命令。
|
||||
`tools/dry-run-p7w6.sh` 是下面 dry run 的原样命令。reducer 自检：`python3 tools/60-reduce-selftest.py`（主机，秒级；
|
||||
夹具 `tools/60-reduce-selftest-pre14e1c8b9.json` = pre-14e1c8b9 CTS 五块相对 `$BASE` 的逐例差异）。
|
||||
- 集成者裁定（2026-09-23）已落进脚本与 RUNBOOK §1/§3：CTS rate = Pass/(Pass+Fail)、五块缺一即 INCOMPLETE、UBO 记 unrun；
|
||||
门 3 单一 boot 会话（否则 INVALID-SESSION）、臂内逐位相同是门而跨臂同图只 WARN、monolith 臂身份核验；`.done` 只在产物齐全时写；
|
||||
restore 把 `$BASE` 库推回 `mgcts`。
|
||||
- `$BASE` 的机器可读 JSON 此前缺失，已由 `cts_multi_report.py --adopt-legacy` 从 `~/w7/logs/cts-a64/base/runs` 重生成：
|
||||
`tools/cts/baselines/base-DirectVulkan-monolith-p7w1-9be62cbc.{json,md}`（逐块数字与 `../device-window-1/CTS-base/README.md` 一致）。
|
||||
|
||||
@@ -14,6 +18,6 @@
|
||||
| `dry-run/dry-p7w6-b-bsl/` | 修 supervisor 探测与采样器预停后重跑：bsl inproc + spawn 的 maps / RSS 峰值与 `wbuf[]` |
|
||||
| `dry-run/dry-p7w6-c-cts-ssbo-dsa/` | CTS 计时样本：p7w6 inproc 全 ssbo + dsa |
|
||||
| `dry-run/build-p7w7dry-14e1c8b9/` | `10-build.sh 14e1c8b9 p7w7dry-14e1c8b9`：APK 131 s、主机制品 223 s；**未安装** |
|
||||
| `dry-run/pre-14e1c8b9-cts/` | pipe 头 lib 的 CTS 五块预读（inproc）与红例的 monolith / inproc 逐例归属（`attribution.txt`）——门 5 在该头上会红，见 RUNBOOK §6 |
|
||||
| `dry-run/pre-14e1c8b9-cts/` | pipe 头 lib 的 CTS 五块预读（inproc）与红例的 monolith / inproc 逐例归属（`attribution.txt`）——门 5 在该头上会红，见 RUNBOOK §6；`verdict.txt` 是旧口径读数，`verdict-contract.txt` 是合同口径（dsa −0.272 pp，红在新 crash） |
|
||||
|
||||
原始归档（actual PNG、角色日志、qpa）在 `~/w7/logs/devprep/w2/<stamp>/`。
|
||||
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# 60-reduce.py (contract formula, p7/devprep w2-reducer) on ~/w7/logs/devprep/w2/pre-14e1c8b9 - CTS AFTER dry run of
|
||||
# pipe head 14e1c8b9's arm64 lib; that run had no gate 3 (NOT-RUN below is expected). Pre-ruling reading: verdict.txt.
|
||||
== CTS AFTER (inproc x DirectVulkan) vs BASE (monolith x DirectVulkan, p7w1) ==
|
||||
gate (CONTRACT-P7 7.3): rate = Pass/(Pass+Fail) [NS, warnings W and crashes X are not in the denominator];
|
||||
per block delta >= -0.5 pp and 0 NEW crash; all five blocks required. 'info' = the pre-ruling
|
||||
Pass/(results-NS) delta, printed for comparison only, never gating.
|
||||
block cases BASE P/F/NS/W/X rate AFTER P/F/NS/W/X rate delta pp (info) crash regr fixed verdict
|
||||
shader-image 125 63/6/56/0/0 91.304% 65/3/56/0/1 95.588% +4.284 (+2.899) 1 1 3 FAIL
|
||||
! 1 NEW crash(es)
|
||||
! note: AFTER is 4.284 pp ABOVE BASE (not a regression; review the fixed list)
|
||||
NEW CRASH KHR-GL46.shader_image_load_store.incomplete_textures
|
||||
Pass->not KHR-GL46.shader_image_load_store.advanced-sync-vertexArray
|
||||
not->Pass KHR-GL46.shader_image_load_store.advanced-sync-drawIndirect
|
||||
not->Pass KHR-GL46.shader_image_load_store.advanced-sync-textureUpdate
|
||||
not->Pass KHR-GL46.shader_image_load_store.non-layered_binding
|
||||
ssbo 124 124/0/0/0/0 100.000% 124/0/0/0/0 100.000% +0.000 (+0.000) 0 0 0 PASS
|
||||
dsa 371 368/2/0/1/0 99.459% 366/3/0/1/1 99.187% -0.272 (-0.539) 1 2 0 FAIL
|
||||
! 1 NEW crash(es)
|
||||
NEW CRASH KHR-GL46.direct_state_access.renderbuffers_storage
|
||||
Pass->not KHR-GL46.direct_state_access.renderbuffers_storage
|
||||
Pass->not KHR-GL46.direct_state_access.renderbuffers_storage_multisample
|
||||
texture 1054 815/18/221/0/0 97.839% 823/10/221/0/0 98.800% +0.960 (+0.960) 0 1 9 PASS
|
||||
! note: AFTER is 0.960 pp ABOVE BASE (not a regression; review the fixed list)
|
||||
Pass->not KHR-GL46.texture_border_clamp.Texture2DArrayCompressed
|
||||
not->Pass KHR-GL46.texture_barrier.disjoint-texels
|
||||
not->Pass KHR-GL46.texture_barrier.overlapping-texels
|
||||
not->Pass KHR-GL46.texture_barrier.same-texel-rw
|
||||
not->Pass KHR-GL46.texture_barrier.same-texel-rw-multipass
|
||||
not->Pass KHR-GL46.texture_size_promotion.functional
|
||||
not->Pass KHR-GL46.texture_swizzle.functional_format_idx_19_target_idx_7
|
||||
not->Pass KHR-GL46.texture_swizzle.functional_format_idx_19_target_idx_8
|
||||
not->Pass KHR-GL46.texture_swizzle.functional_format_idx_8_target_idx_7
|
||||
not->Pass KHR-GL46.texture_swizzle.functional_format_idx_8_target_idx_8
|
||||
packed-pixels 4732 4732/0/0/0/0 100.000% 4732/0/0/0/0 100.000% +0.000 (+0.000) 0 0 0 PASS
|
||||
ubo unrun (not in this glcts): the device glcts has no GTF module (ID-P7-16); recorded, not gated (GTF-GL46.gtf31.GL3Tests.uniform_buffer_object*)
|
||||
CTS FAIL
|
||||
|
||||
== TIMING ==
|
||||
preflight 5s 2026-09-23T09:59:01 -> 2026-09-23T09:59:06 reboot=0 pin=1
|
||||
cts-shader-image 9s 2026-09-23T09:59:10 -> 2026-09-23T09:59:19 limit=0
|
||||
cts-ssbo 3s 2026-09-23T09:59:20 -> 2026-09-23T09:59:23 limit=0
|
||||
cts-dsa 9s 2026-09-23T09:59:24 -> 2026-09-23T09:59:33 limit=0
|
||||
cts-texture 108s 2026-09-23T09:59:34 -> 2026-09-23T10:01:22 limit=0
|
||||
cts-packed-pixels 220s 2026-09-23T10:01:23 -> 2026-09-23T10:05:03 limit=0
|
||||
cts 358s 2026-09-23T09:59:06 -> 2026-09-23T10:05:04 blocks=[shader-image ssbo dsa texture packed-pixels] limit=0
|
||||
restore 5s 2026-09-23T10:05:04 -> 2026-09-23T10:05:09
|
||||
restore 6s 2026-09-23T10:08:05 -> 2026-09-23T10:08:11
|
||||
restore 11s 2026-09-23T10:08:36 -> 2026-09-23T10:08:47
|
||||
|
||||
WINDOW2 GATE3 NOT-RUN | CTS FAIL -> exit 1
|
||||
@@ -4,7 +4,11 @@
|
||||
# Session start (runbook 1 section 2, CONTRACT-P7 7.2 "same reboot-clean session"):
|
||||
# --reboot records boot_id, reboots, waits for boot_completed, records boot_id again; the two
|
||||
# must differ or the window is NOT reboot-clean. Gate 3 requires it; the dry run
|
||||
# does not reboot.
|
||||
# does not reboot. Refused while gate3/state holds finished pairs (they belong to
|
||||
# the old session). session/boot-id.txt is THE session every gate-3 record must match;
|
||||
# without --reboot on a stamp that already has session/boot-id.txt: a resume - the phone must
|
||||
# still be in that boot session (else stop), and boot-id.txt / reboot-clean.txt are
|
||||
# kept, not rewritten (session/resumed.txt logs the re-entry);
|
||||
# records the state 90-restore.sh puts back: TCP supervisor up or not, stay-on setting, pin
|
||||
# state, Doze whitelist line, focused activity;
|
||||
# wakes the phone and sets `svc power stayon true` (息屏 cut child TCP in window 1b, ID-P7-48);
|
||||
@@ -42,6 +46,10 @@ EOF
|
||||
log "found: supervisor=$sup stayon=$stayon pin=$pinstate"
|
||||
fi
|
||||
|
||||
SESSION_BOOT=$(w2_session_boot_id "$STAMP")
|
||||
if [ $REBOOT -eq 1 ] && compgen -G "$(w2_out "$STAMP")/gate3/state/*/*.done" > /dev/null; then
|
||||
die "gate 3 has finished pairs from boot session ${SESSION_BOOT:-?}; --reboot starts a NEW session and would void them (60-reduce.py: INVALID-SESSION). Delete them first (RUNBOOK section 1 'phone rebooted') or use a new stamp"
|
||||
fi
|
||||
if [ $REBOOT -eq 1 ]; then
|
||||
Ash "cat /proc/sys/kernel/random/boot_id" > "$OUT/boot-id-before.txt"
|
||||
log "rebooting for a reboot-clean session"
|
||||
@@ -61,12 +69,20 @@ if [ $REBOOT -eq 1 ]; then
|
||||
done
|
||||
[ "$(Ash 'getprop sys.user.0.ce_available')" = true ] || die "user 0 storage still locked after 10 min"
|
||||
fi
|
||||
Ash "cat /proc/sys/kernel/random/boot_id" > "$OUT/boot-id.txt"
|
||||
if [ $REBOOT -eq 1 ]; then
|
||||
if cmp -s "$OUT/boot-id-before.txt" "$OUT/boot-id.txt"; then die "boot_id unchanged: the reboot did not happen"; fi
|
||||
echo "reboot-clean: $(cat "$OUT/boot-id-before.txt") -> $(cat "$OUT/boot-id.txt")" | tee "$OUT/reboot-clean.txt"
|
||||
if [ $REBOOT -eq 0 ] && [ -n "$SESSION_BOOT" ]; then
|
||||
# Re-entering this stamp's session (window.sh after 90-restore.sh ran with steps still open, or
|
||||
# by hand): the phone must still be in it, and boot-id.txt / reboot-clean.txt stay as the
|
||||
# session's first preflight wrote them. Stay-on and the pin are set again below.
|
||||
w2_same_session "$SESSION_BOOT" "preflight (resume of $STAMP)"
|
||||
echo "resumed $(date -Is): boot_id $SESSION_BOOT unchanged ($(cat "$OUT/reboot-clean.txt" 2>/dev/null))" | tee -a "$OUT/resumed.txt"
|
||||
else
|
||||
echo "NOT reboot-clean (no --reboot): boot_id $(cat "$OUT/boot-id.txt"), up $(Ash 'cat /proc/uptime' | cut -d' ' -f1)s" | tee "$OUT/reboot-clean.txt"
|
||||
Ash "cat /proc/sys/kernel/random/boot_id" > "$OUT/boot-id.txt"
|
||||
if [ $REBOOT -eq 1 ]; then
|
||||
if cmp -s "$OUT/boot-id-before.txt" "$OUT/boot-id.txt"; then die "boot_id unchanged: the reboot did not happen"; fi
|
||||
echo "reboot-clean: $(cat "$OUT/boot-id-before.txt") -> $(cat "$OUT/boot-id.txt")" | tee "$OUT/reboot-clean.txt"
|
||||
else
|
||||
echo "NOT reboot-clean (no --reboot): boot_id $(cat "$OUT/boot-id.txt"), up $(Ash 'cat /proc/uptime' | cut -d' ' -f1)s" | tee "$OUT/reboot-clean.txt"
|
||||
fi
|
||||
fi
|
||||
|
||||
w2_wake
|
||||
|
||||
@@ -11,9 +11,18 @@
|
||||
# apk.yml:460-462 Magma knobs on every arm. Every repeat's result.json, actual PNG, both role logs,
|
||||
# transport proof and logcat are archived under <out>/gate3/archive/<arm>/<case>-DirectVulkan/.
|
||||
#
|
||||
# RESUMABLE: one runner invocation per (arm, case); a finished pair writes
|
||||
# <out>/gate3/state/<arm>/<case>.done and is skipped on the next start. An interrupted pair is
|
||||
# re-run from scratch (its partial archive is removed first). Run it detached:
|
||||
# ONE SESSION (CONTRACT-P7 7.2 "same reboot-clean session"): the phone's boot_id must equal
|
||||
# <out>/session/boot-id.txt (21-preflight.sh) before and after every pair; the driver stops the
|
||||
# moment it does not. Every pair's .done and every archived repeat (repeat-NN/boot_id.txt) records
|
||||
# boot_id / boot_id_before, and 60-reduce.py calls the gate INVALID-SESSION unless all of them,
|
||||
# over all arms, are that one boot_id.
|
||||
#
|
||||
# RESUMABLE: one runner invocation per (arm, case); a pair writes
|
||||
# <out>/gate3/state/<arm>/<case>.done ONLY when every repeat it asked for left result.json AND an
|
||||
# actual PNG in the archive - otherwise it is logged INCOMPLETE, left without a .done, and the
|
||||
# next start re-runs it from scratch (its partial archive is removed first). Exit 0 = every pair
|
||||
# done; 3 = the loop finished but some pairs are INCOMPLETE (window.sh carries on and leaves the
|
||||
# step for the next run); anything else = stopped. Run it detached:
|
||||
# bash detach.sh ~/w7/logs/p7w7/<stamp>/gate3.log 30-gate3.sh <stamp>
|
||||
set -uo pipefail
|
||||
. "$(dirname "$0")/lib.sh"
|
||||
@@ -34,10 +43,14 @@ OUT=$(w2_out "$STAMP")/gate3
|
||||
mkdir -p "$OUT"
|
||||
w2_require_device
|
||||
w2_lock
|
||||
SESSION_BOOT=$(w2_session_boot_id "$STAMP")
|
||||
[ -n "$SESSION_BOOT" ] || die "no $(w2_out "$STAMP")/session/boot-id.txt: run 21-preflight.sh first"
|
||||
w2_same_session "$SESSION_BOOT" "gate 3 start"
|
||||
export MOBILEGL_TRACE_PACKAGE=$W2_PKG MOBILEGL_TRACE_SKIP_INSTALL=1 MOBILEGL_TRACE_APK=$W2_APK
|
||||
RUNNER="$W2_TOOLS/tools/trace_replay/run_android_retrace_local.py"
|
||||
|
||||
# ---- the case list is frozen on first start so a resume cannot change the denominator ----------
|
||||
# ---- the case list (and below, the repeat count) is frozen on first start: a resume cannot change
|
||||
# the denominator ------------------------------------------------------------------------------------
|
||||
if [ ! -s "$OUT/cases.txt" ]; then
|
||||
if [ -n "$CASES_ARG" ]; then
|
||||
tr ',' '\n' <<<"$CASES_ARG" | sed '/^$/d' > "$OUT/cases.txt"
|
||||
@@ -57,6 +70,13 @@ fi
|
||||
mapfile -t CASES < "$OUT/cases.txt"
|
||||
[ ${#CASES[@]} -gt 0 ] || die "empty case list"
|
||||
[ -f "$OUT/arms.txt" ] || printf 'arms=%s\nrepeat=%s\napk=%s\ntools=%s\n' "$ARMS" "$REPEAT" "$W2_APK" "$W2_TOOLS" > "$OUT/arms.txt"
|
||||
# The repeat count is frozen with the case list: a resume started without the first run's --repeat
|
||||
# (RUNBOOK: "30-gate3.sh <stamp>") must not re-run a pair with another count than its siblings'.
|
||||
FROZEN_REPEAT=$(sed -n 's/^repeat=//p' "$OUT/arms.txt")
|
||||
if [ -n "$FROZEN_REPEAT" ] && [ "$FROZEN_REPEAT" != "$REPEAT" ]; then
|
||||
log "repeat frozen at $FROZEN_REPEAT by $OUT/arms.txt (asked: $REPEAT)"
|
||||
REPEAT=$FROZEN_REPEAT
|
||||
fi
|
||||
log "gate3 stamp=$STAMP cases=${#CASES[@]} arms=[$ARMS] repeat=$REPEAT apk=$W2_APK tools=$W2_TOOLS"
|
||||
|
||||
arm_args() { # <arm> -> runner arguments
|
||||
@@ -68,8 +88,20 @@ arm_args() { # <arm> -> runner arguments
|
||||
*) die "unknown arm $1";;
|
||||
esac
|
||||
}
|
||||
arm_repeats() { case "$1" in inproc|spawn) echo "$REPEAT";; *) echo 1;; esac; }
|
||||
|
||||
# <case-archive> <n> -> the repeats (of 1..n) that lack result.json or an actual PNG; empty = complete
|
||||
missing_repeats() {
|
||||
local i d miss=()
|
||||
for i in $(seq 1 "$2"); do
|
||||
d=$(printf '%s/repeat-%02d' "$1" "$i")
|
||||
if [ ! -s "$d/result.json" ] || ! compgen -G "$d/*-actual.png" > /dev/null; then miss+=("repeat-$(printf %02d "$i")"); fi
|
||||
done
|
||||
echo "${miss[*]}"
|
||||
}
|
||||
|
||||
t_all=$(date +%s)
|
||||
INCOMPLETE=0
|
||||
for arm in $ARMS; do
|
||||
mkdir -p "$OUT/state/$arm" "$OUT/logs/$arm" "$OUT/archive/$arm"
|
||||
pending=0
|
||||
@@ -92,6 +124,8 @@ for arm in $ARMS; do
|
||||
attempt=1
|
||||
while :; do
|
||||
w2_require_device
|
||||
boot0=$(w2_boot_id)
|
||||
w2_same_session "$SESSION_BOOT" "before $arm $c" "$boot0"
|
||||
w2_wake
|
||||
t0=$(date +%s)
|
||||
# shellcheck disable=SC2046
|
||||
@@ -109,16 +143,41 @@ for arm in $ARMS; do
|
||||
done
|
||||
cp "$OUT/logs/$arm/$c.attempt$attempt.log" "$OUT/logs/$arm/$c.log"
|
||||
# The runner rewrites <archive>/run.json per invocation; keep each case's copy beside it.
|
||||
[ -f "$OUT/archive/$arm/run.json" ] && mv "$OUT/archive/$arm/run.json" "$OUT/archive/$arm/$c-DirectVulkan/run.json" 2>/dev/null
|
||||
[ -f "$OUT/archive/$arm/run.json" ] && mkdir -p "$OUT/archive/$arm/$c-DirectVulkan" \
|
||||
&& mv "$OUT/archive/$arm/run.json" "$OUT/archive/$arm/$c-DirectVulkan/run.json" 2>/dev/null
|
||||
# Session stamp on every repeat the runner archived (complete or not), read AFTER the run: a
|
||||
# reboot inside the invocation shows as boot_id != boot_id_before.
|
||||
boot1=$(w2_boot_id)
|
||||
for d in "$OUT/archive/$arm/$c-DirectVulkan"/repeat-*/; do
|
||||
[ -d "$d" ] && printf 'boot_id=%s boot_id_before=%s stamped=%s\n' "$boot1" "$boot0" "$(date -Is)" > "$d/boot_id.txt"
|
||||
done
|
||||
passes=$(grep -c '"passed": true' "$OUT/logs/$arm/$c.log" || true)
|
||||
printf 'rc=%s seconds=%s attempts=%s passed_lines=%s finished=%s\n' "$rc" "$((t1 - t0))" "$attempt" "$passes" "$(date -Is)" \
|
||||
> "$OUT/state/$arm/$c.done"
|
||||
printf '%s\t%s\t%s\t%s\t%s\n' "$arm" "$c" "$rc" "$((t1 - t0))" "$(date -Is)" >> "$OUT/progress.tsv"
|
||||
miss=$(missing_repeats "$OUT/archive/$arm/$c-DirectVulkan" "$(arm_repeats "$arm")")
|
||||
if [ -n "$boot1" ] && [ "$boot1" != "$SESSION_BOOT" ]; then
|
||||
printf '%s\t%s\t%s\t%s\t%s\tSESSION-BROKEN\n' "$arm" "$c" "$rc" "$((t1 - t0))" "$(date -Is)" >> "$OUT/progress.tsv"
|
||||
w2_same_session "$SESSION_BOOT" "after $arm $c" "$boot1"
|
||||
fi
|
||||
if [ -n "$miss" ] || [ -z "$boot1" ]; then
|
||||
# No .done: the next start of this script re-runs the pair from scratch.
|
||||
printf '%s\t%s\t%s\t%s\t%s\tINCOMPLETE\n' "$arm" "$c" "$rc" "$((t1 - t0))" "$(date -Is)" >> "$OUT/progress.tsv"
|
||||
why=${miss:+$miss lack result.json / actual PNG}
|
||||
[ -z "$boot1" ] && why="${why:+$why; }boot_id unreadable after the run"
|
||||
log " $arm $c rc=$rc $((t1 - t0))s INCOMPLETE ($why): no .done, the next run retries the pair"
|
||||
INCOMPLETE=$((INCOMPLETE + 1))
|
||||
continue
|
||||
fi
|
||||
printf 'rc=%s seconds=%s attempts=%s passed_lines=%s boot_id=%s boot_id_before=%s finished=%s\n' \
|
||||
"$rc" "$((t1 - t0))" "$attempt" "$passes" "$boot1" "$boot0" "$(date -Is)" > "$OUT/state/$arm/$c.done"
|
||||
printf '%s\t%s\t%s\t%s\t%s\tdone\n' "$arm" "$c" "$rc" "$((t1 - t0))" "$(date -Is)" >> "$OUT/progress.tsv"
|
||||
log " $arm $c rc=$rc $((t1 - t0))s"
|
||||
done
|
||||
pin=$(w2_pin_check "$W2_TOOLS" "$OUT/pin-after-$arm.txt"); log "pin after $arm: $pin"
|
||||
w2_timing "$(w2_out "$STAMP")" "gate3-$arm" "$t_arm" "$(date +%s)" "${#CASES[@]} cases"
|
||||
log "=== arm $arm done in $(( $(date +%s) - t_arm ))s ==="
|
||||
done
|
||||
w2_timing "$(w2_out "$STAMP")" gate3 "$t_all" "$(date +%s)"
|
||||
w2_timing "$(w2_out "$STAMP")" gate3 "$t_all" "$(date +%s)" "incomplete=$INCOMPLETE"
|
||||
if [ $INCOMPLETE -gt 0 ]; then
|
||||
log "=== GATE3 INCOMPLETE $STAMP: $INCOMPLETE pair(s) without .done (progress.tsv 'INCOMPLETE'); run the same command again to retry them ==="
|
||||
exit 3
|
||||
fi
|
||||
log "=== GATE3 DONE $STAMP ==="
|
||||
|
||||
@@ -14,8 +14,13 @@
|
||||
# Arm proof before any block: mgprobe (the SKILL step-5 preflight) and one glcts case, each under
|
||||
# the AFTER environment, must log ConfigLoader's "Config: MOBILEGL_TRANSPORT=inproc - the MGPipe
|
||||
# record stream" and a "Config: IPC" line with strict=1 role-split-state=1 run-ahead=1.
|
||||
# RESUMABLE per block (<out>/cts/runs/<block>/.done). --limit N runs the first N cases of each
|
||||
# block (the dry run). --restore-base-lib re-pushes the $BASE library afterwards (dry run only).
|
||||
# RESUMABLE per block (<out>/cts/runs/<block>/.done), and a block's .done is written ONLY when its
|
||||
# report-<block>.json exists and holds results: a block without one is logged INCOMPLETE and redone
|
||||
# from scratch by the next run (exit 3 = the loop finished with such blocks; window.sh carries on
|
||||
# and leaves the step for the next run). 60-reduce.py requires all five blocks. The UBO block
|
||||
# (GTF-GL46 uniform_buffer_object) is not run: this glcts has no GTF module (ID-P7-16).
|
||||
# --limit N runs the first N cases of each block (the dry run). --restore-base-lib re-pushes the
|
||||
# $BASE library at once; 90-restore.sh does it at the end of every window anyway.
|
||||
set -uo pipefail
|
||||
. "$(dirname "$0")/lib.sh"
|
||||
|
||||
@@ -92,10 +97,12 @@ if [ ! -f "$OUT/preflight.done" ]; then
|
||||
fi
|
||||
|
||||
# ---- blocks -------------------------------------------------------------------------------------
|
||||
INCOMPLETE=0
|
||||
for b in $BLOCKS; do
|
||||
run="$OUT/runs/$b"
|
||||
[ -f "$run/.done" ] && { log "block $b already done"; continue; }
|
||||
rm -rf "$run"; mkdir -p "$run"
|
||||
# A report left by an interrupted attempt must not stand in for this one.
|
||||
rm -rf "$run"; rm -f "$OUT/report-$b.json" "$OUT/report-$b.txt"; mkdir -p "$run"
|
||||
caselist="$W2_TOOLS/tools/cts/caselists/p7-$b-gl46.txt"
|
||||
[ -f "$caselist" ] || die "no caselist $caselist"
|
||||
if [ "$LIMIT" -gt 0 ]; then
|
||||
@@ -115,12 +122,19 @@ for b in $BLOCKS; do
|
||||
t1=$(date +%s)
|
||||
python3 "$W2_TOOLS/tools/cts/scripts/qpa_report.py" "$run" --label "DirectVulkan-inproc-$b" \
|
||||
--json "$OUT/report-$b.json" > "$OUT/report-$b.txt" 2>&1
|
||||
qrc=$?
|
||||
pin=$(w2_pin_check "$W2_TOOLS" "$OUT/pin-after-$b.txt")
|
||||
for f in unrun hung crashed; do [ -s "$run/$f.txt" ] && log " [$b] $f.txt: $(wc -l < "$run/$f.txt") lines"; done
|
||||
grep -E 'Pass|Fail|NotSupported|Crash|rate' "$OUT/report-$b.txt" | head -8 | sed "s/^/ [$b] /"
|
||||
printf 'rc=%s seconds=%s pin_after=%s finished=%s\n' "$rc" "$((t1 - t0))" "$pin" "$(date -Is)" > "$run/.done"
|
||||
w2_timing "$(w2_out "$STAMP")" "cts-$b" "$t0" "$t1" "limit=$LIMIT"
|
||||
log "block $b rc=$rc $((t1 - t0))s"
|
||||
if [ -s "$OUT/report-$b.json" ] && python3 -c 'import json, sys; sys.exit(0 if json.load(open(sys.argv[1]))["results"] else 1)' \
|
||||
"$OUT/report-$b.json" 2> /dev/null; then
|
||||
printf 'rc=%s seconds=%s pin_after=%s finished=%s\n' "$rc" "$((t1 - t0))" "$pin" "$(date -Is)" > "$run/.done"
|
||||
log "block $b rc=$rc $((t1 - t0))s"
|
||||
else
|
||||
INCOMPLETE=$((INCOMPLETE + 1))
|
||||
log "block $b rc=$rc $((t1 - t0))s INCOMPLETE: no report-$b.json with results (qpa_report rc=$qrc, see $OUT/report-$b.txt); no .done, the next run redoes the block"
|
||||
fi
|
||||
done
|
||||
|
||||
# The same machine-readable shape as $BASE (tools/cts/baselines/base-DirectVulkan-monolith-p7w1-
|
||||
@@ -136,11 +150,11 @@ if [ ${#suites[@]} -gt 0 ]; then
|
||||
fi
|
||||
|
||||
if [ $RESTORE_BASE -eq 1 ]; then
|
||||
BASE_LIB="$W2_CTS_BUNDLE/libMobileGL.so"
|
||||
BASE_SHA=$(awk '/^libMobileGL.so *sha256:/ {print $3}' "$W2_CTS_BUNDLE/IDENTITY.txt")
|
||||
A push "$BASE_LIB" "$W2_CTS_DEV/libMobileGL.so" >/dev/null 2>&1
|
||||
GOT=$(Ash "sha256sum $W2_CTS_DEV/libMobileGL.so" | cut -d' ' -f1)
|
||||
[ "$GOT" = "$BASE_SHA" ] && log "restored the \$BASE library ($BASE_SHA)" || log "WARN: restore mismatch $GOT != $BASE_SHA"
|
||||
w2_restore_base_lib | tee "$OUT/deploy/restored-base.txt" || log "WARN: the \$BASE library is NOT back in $W2_CTS_DEV"
|
||||
fi
|
||||
w2_timing "$(w2_out "$STAMP")" cts "$t_all" "$(date +%s)" "blocks=[$BLOCKS] limit=$LIMIT incomplete=$INCOMPLETE"
|
||||
if [ $INCOMPLETE -gt 0 ]; then
|
||||
log "=== CTS AFTER INCOMPLETE $STAMP: $INCOMPLETE block(s) without a report; run the same command again to redo them ==="
|
||||
exit 3
|
||||
fi
|
||||
w2_timing "$(w2_out "$STAMP")" cts "$t_all" "$(date +%s)" "blocks=[$BLOCKS] limit=$LIMIT"
|
||||
log "=== CTS AFTER DONE $STAMP ==="
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"what": "CTS AFTER dry run of pipe head 14e1c8b9 (arm64 lib, inproc x DirectVulkan, Redmi 2f7cbe2e, 2026-09-23), stored as its differences from $BASE (device-window-1/CTS-base/report-<block>.json): AFTER results = BASE results updated with after_differences. after_counts are the real report's counts (a check that the rebuild is exact).",
|
||||
"source": "~/w7/logs/devprep/w2/pre-14e1c8b9/cts/report-<block>.json (qpa_report.py --json)",
|
||||
"evidence": "docs/Disaggregated/notes/p7/device-window-2/dry-run/pre-14e1c8b9-cts/",
|
||||
"blocks": {
|
||||
"shader-image": {
|
||||
"after_differences": {
|
||||
"KHR-GL46.shader_image_load_store.advanced-sync-drawIndirect": "Pass",
|
||||
"KHR-GL46.shader_image_load_store.advanced-sync-textureUpdate": "Pass",
|
||||
"KHR-GL46.shader_image_load_store.advanced-sync-vertexArray": "Fail",
|
||||
"KHR-GL46.shader_image_load_store.incomplete_textures": "Crash",
|
||||
"KHR-GL46.shader_image_load_store.non-layered_binding": "Pass"
|
||||
},
|
||||
"after_counts": {
|
||||
"Crash": 1,
|
||||
"Fail": 3,
|
||||
"NotSupported": 56,
|
||||
"Pass": 65
|
||||
},
|
||||
"after_sha256_of_sorted_results": "06052360198585ad9b8784d39081a0072c05045bac569df2a3d5a24fea811c47"
|
||||
},
|
||||
"ssbo": {
|
||||
"after_differences": {},
|
||||
"after_counts": {
|
||||
"Pass": 124
|
||||
},
|
||||
"after_sha256_of_sorted_results": "753fce812b22c65e341c3a7c147f3431b9d514e5801a185205fb6340bc71eae2"
|
||||
},
|
||||
"dsa": {
|
||||
"after_differences": {
|
||||
"KHR-GL46.direct_state_access.renderbuffers_storage": "Crash",
|
||||
"KHR-GL46.direct_state_access.renderbuffers_storage_multisample": "Fail"
|
||||
},
|
||||
"after_counts": {
|
||||
"CompatibilityWarning": 1,
|
||||
"Crash": 1,
|
||||
"Fail": 3,
|
||||
"Pass": 366
|
||||
},
|
||||
"after_sha256_of_sorted_results": "ae644a78b6dccd831b3baabac45f05308f348584f70d4f3a0abb2dbedee9ce25"
|
||||
},
|
||||
"texture": {
|
||||
"after_differences": {
|
||||
"KHR-GL46.texture_barrier.disjoint-texels": "Pass",
|
||||
"KHR-GL46.texture_barrier.overlapping-texels": "Pass",
|
||||
"KHR-GL46.texture_barrier.same-texel-rw": "Pass",
|
||||
"KHR-GL46.texture_barrier.same-texel-rw-multipass": "Pass",
|
||||
"KHR-GL46.texture_border_clamp.Texture2DArrayCompressed": "Fail",
|
||||
"KHR-GL46.texture_size_promotion.functional": "Pass",
|
||||
"KHR-GL46.texture_swizzle.functional_format_idx_19_target_idx_7": "Pass",
|
||||
"KHR-GL46.texture_swizzle.functional_format_idx_19_target_idx_8": "Pass",
|
||||
"KHR-GL46.texture_swizzle.functional_format_idx_8_target_idx_7": "Pass",
|
||||
"KHR-GL46.texture_swizzle.functional_format_idx_8_target_idx_8": "Pass"
|
||||
},
|
||||
"after_counts": {
|
||||
"Fail": 10,
|
||||
"NotSupported": 221,
|
||||
"Pass": 823
|
||||
},
|
||||
"after_sha256_of_sorted_results": "dae6aac473ee5ecce31eaf49217f44f017a10190233d4965fac8b1205e1e3f55"
|
||||
},
|
||||
"packed-pixels": {
|
||||
"after_differences": {},
|
||||
"after_counts": {
|
||||
"Pass": 4732
|
||||
},
|
||||
"after_sha256_of_sorted_results": "d373e2b192baa92bc2a9f42e0e5fb682dad1719331149bf6ae05c35ab7fcaa38"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,449 @@
|
||||
#!/usr/bin/env python3
|
||||
"""60-reduce-selftest.py [--tools TREE] [--dry-run-dir DIR] [--keep]
|
||||
|
||||
Self-test of 60-reduce.py: builds canned window output directories in a temp dir and runs the
|
||||
reducer on each exactly as window.sh does (a subprocess, <stamp> --logroot --tools), then checks
|
||||
the printed verdict, verdict.json and the exit status. The scenarios:
|
||||
|
||||
control gate 3 (2 cases x monolith/inproc x3/spawn x3/ra0, one boot_id) and all five CTS
|
||||
blocks equal to $BASE -> GATE3 PASS, CTS PASS, exit 0
|
||||
missing-block ssbo never finished, dsa's report is empty, texture's has no results; and a
|
||||
window with no cts/ at all -> CTS INCOMPLETE, exit != 0
|
||||
boot-id-change a spawn pair's .done / one repeat's boot_id.txt names another boot; a .done
|
||||
in the pre-ruling format (no boot_id); no session/boot-id.txt
|
||||
-> GATE3 INVALID-SESSION
|
||||
monolith-arm the monolith repeat's log carries Config: IPC / has no log; a split repeat's
|
||||
transport-proof did not pass -> that case FAIL
|
||||
cross-arm spawn's pictures differ from inproc's and monolith's, each arm self-identical
|
||||
-> WARN line, GATE3 still PASS
|
||||
one inproc repeat differs from its siblings -> FAIL (within-arm identity)
|
||||
unfinished a pair without .done; a repeat without an actual PNG -> GATE3 INCOMPLETE
|
||||
dry-run the pre-14e1c8b9 CTS dry run rebuilt from $BASE + its recorded differences
|
||||
(60-reduce-selftest-pre14e1c8b9.json): contract rate Pass/(Pass+Fail) gives
|
||||
dsa -0.272 pp (the pre-ruling Pass/(results-NS) -0.539 pp is printed as info
|
||||
only), shader-image +4.284, texture +0.960, ssbo/packed-pixels 0; dsa and
|
||||
shader-image FAIL on one NEW crash each, not on the rate -> CTS FAIL
|
||||
When --dry-run-dir (default ~/w7/logs/devprep/w2/pre-14e1c8b9) exists, the reducer also runs on
|
||||
that real output (verdict JSON to the temp dir) and must give the same CTS numbers.
|
||||
TREE (compare_actuals.py + device-window-1/CTS-base + tools/cts/caselists) defaults to the
|
||||
MobileGL tree this file sits in, else $W2_TOOLS_OVERRIDE, else $W2_PIPE / ~/w7/pipe (only read).
|
||||
Exit 0 when every check holds.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
REDUCER = HERE / "60-reduce.py"
|
||||
FIXTURE = HERE / "60-reduce-selftest-pre14e1c8b9.json"
|
||||
CTS_BASE_REL = "docs/Disaggregated/notes/p7/device-window-1/CTS-base"
|
||||
BLOCKS = ("shader-image", "ssbo", "dsa", "texture", "packed-pixels")
|
||||
BOOT = "11111111-2222-3333-4444-555555555555"
|
||||
OTHER_BOOT = "99999999-8888-7777-6666-555555555555"
|
||||
CASES = ("caseA-in-world", "caseB-main-menu")
|
||||
REPEAT = 3
|
||||
ARMS = (("monolith", 1), ("inproc", REPEAT), ("spawn", REPEAT), ("inproc-ra0", 1))
|
||||
DSA_STORAGE = "KHR-GL46.direct_state_access.renderbuffers_storage"
|
||||
SI_INCOMPLETE = "KHR-GL46.shader_image_load_store.incomplete_textures"
|
||||
IPC = ("Config: IPC ring=8MiB stage=32MiB wire-deferred=64MiB spin=50us persistent-block=64KiB adopt-tier=2 "
|
||||
"verb-barrier=1 run-ahead=%d present-credit=1 strict=1 audit=0 role-split-state=1 affinity='auto'")
|
||||
TAG = "[12:00:00] [Android MobileGLTraceRe/INFO]: "
|
||||
LOGS = {
|
||||
"monolith": TAG + "Config: Active backend type set to DirectVulkan\n" + TAG + "Config loaded\n",
|
||||
"inproc": TAG + "Config: Active backend type set to DirectVulkan\n" + TAG + "Config: MOBILEGL_TRANSPORT=inproc"
|
||||
" - the MGPipe record stream crosses a real ring to an apply thread\n" + TAG + IPC % 1 + "\n",
|
||||
"spawn": TAG + "Config: Active backend type set to DirectVulkan\n" + TAG + "Config: MOBILEGL_TRANSPORT=spawn"
|
||||
" - the MGPipe record stream crosses a real process boundary\n" + TAG + IPC % 1 + "\n",
|
||||
"inproc-ra0": TAG + "Config: MOBILEGL_TRANSPORT=inproc - the MGPipe record stream crosses a real ring to an"
|
||||
" apply thread\n" + TAG + IPC % 0 + "\n",
|
||||
}
|
||||
|
||||
|
||||
def write(path, data):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if isinstance(data, bytes):
|
||||
path.write_bytes(data)
|
||||
else:
|
||||
path.write_text(data, encoding="utf-8")
|
||||
|
||||
|
||||
def read_list(path):
|
||||
if not path.is_file():
|
||||
return []
|
||||
return [l.strip() for l in path.read_text(encoding="utf-8").splitlines() if l.strip() and not l.startswith("#")]
|
||||
|
||||
|
||||
def find_tree(arg):
|
||||
# The repo copy finds its own tree; the working copy (~/w7/notes/p7/window2) falls back to the
|
||||
# tools override, then the integration tree lib.sh names (read only).
|
||||
candidates = [arg] + [str(p) for p in HERE.parents] + [
|
||||
os.environ.get("W2_TOOLS_OVERRIDE"), os.environ.get("W2_PIPE"), str(Path.home() / "w7/pipe")]
|
||||
for c in candidates:
|
||||
if c and (Path(c) / "tools/trace_replay/compare_actuals.py").is_file() and (Path(c) / CTS_BASE_REL).is_dir():
|
||||
return Path(c)
|
||||
return None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
# canned window directories
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
class Canned:
|
||||
def __init__(self, root, tree, ca):
|
||||
self.root, self.tree, self.ca = root, tree, ca
|
||||
|
||||
def png(self, value):
|
||||
return self.ca._png_bytes(8, 8, self.ca._solid(8, 8, value))
|
||||
|
||||
def gate3(self, out, picture=None, boot=BOOT):
|
||||
picture = picture or {}
|
||||
g = out / "gate3"
|
||||
write(g / "cases.txt", "\n".join(CASES) + "\n")
|
||||
write(g / "excluded.txt", "x-red-on-monolith-case\n")
|
||||
write(g / "arms.txt", "arms=monolith inproc spawn inproc-ra0\nrepeat=%d\n" % REPEAT)
|
||||
write(out / "session/boot-id.txt", boot + "\n")
|
||||
write(out / "session/reboot-clean.txt", "reboot-clean: 00000000-0000-0000-0000-000000000000 -> %s\n" % boot)
|
||||
for arm, n in ARMS:
|
||||
for case in CASES:
|
||||
name = case + "-DirectVulkan"
|
||||
for i in range(1, n + 1):
|
||||
rep = g / "archive" / arm / name / ("repeat-%02d" % i)
|
||||
write(rep / "result.json", json.dumps({
|
||||
"passed": True, "statusCode": 0, "backend": "DirectVulkan", "ssim": 0.998,
|
||||
"ssimThreshold": 0.99, "mismatchPixels": 17, "matchedGoldenPath": "/in/golden.png",
|
||||
"cropX": 0, "cropY": 0, "cropWidth": 0, "cropHeight": 0}))
|
||||
write(rep / (name + "-actual.png"), self.png(picture.get(arm, 128)))
|
||||
write(rep / "logcat.txt", "canned logcat %s %s repeat %d\n" % (arm, case, i))
|
||||
write(rep / "mobilegl.log", LOGS[arm])
|
||||
if arm in ("inproc", "spawn"):
|
||||
write(rep / "transport-proof.json", json.dumps({"required_transport": arm, "passed": True}))
|
||||
write(rep / "boot_id.txt", "boot_id=%s boot_id_before=%s stamped=2026-09-23T12:00:00-04:00\n" % (boot, boot))
|
||||
write(g / "state" / arm / (case + ".done"),
|
||||
"rc=0 seconds=7 attempts=1 passed_lines=%d boot_id=%s boot_id_before=%s "
|
||||
"finished=2026-09-23T12:00:00-04:00\n" % (n, boot, boot))
|
||||
return g
|
||||
|
||||
def cts(self, out, differences=None, drop=(), empty=(), no_results=()):
|
||||
differences = differences or {}
|
||||
base_dir = self.tree / CTS_BASE_REL
|
||||
skip = set(read_list(self.tree / "tools/cts/caselists/p7-skip.txt"))
|
||||
c = out / "cts"
|
||||
c.mkdir(parents=True, exist_ok=True)
|
||||
rebuilt = {}
|
||||
for b in BLOCKS:
|
||||
if b in drop:
|
||||
continue
|
||||
caselist = self.tree / "tools/cts/caselists" / ("p7-%s-gl46.txt" % b)
|
||||
run = c / "runs" / b
|
||||
write(run / "caselist.path", str(caselist) + "\n")
|
||||
skipped = [x for x in read_list(caselist) if x in skip]
|
||||
if skipped:
|
||||
write(run / "skipped.txt", "\n".join(skipped) + "\n")
|
||||
write(run / "unrun.txt", "")
|
||||
write(run / "hung.txt", "")
|
||||
results = dict(json.loads((base_dir / ("report-%s.json" % b)).read_text(encoding="utf-8"))["results"])
|
||||
results.update(differences.get(b, {}))
|
||||
rebuilt[b] = results
|
||||
write(run / "crashed.txt", "".join(k + "\n" for k, v in sorted(results.items()) if v == "Crash"))
|
||||
write(run / ".done", "rc=0 seconds=1 pin_after=PINNED finished=2026-09-23T12:00:00-04:00\n")
|
||||
report = c / ("report-%s.json" % b)
|
||||
if b in empty:
|
||||
write(report, "")
|
||||
elif b in no_results:
|
||||
write(report, json.dumps({"label": "DirectVulkan-inproc-" + b, "results": {}}))
|
||||
else:
|
||||
write(report, json.dumps({"label": "DirectVulkan-inproc-" + b, "total": len(results),
|
||||
"counts": dict(Counter(results.values())), "results": results}, indent=1))
|
||||
return rebuilt
|
||||
|
||||
def window(self, stamp, gate3=True, cts=True, **cts_args):
|
||||
out = self.root / stamp
|
||||
out.mkdir(parents=True)
|
||||
if gate3:
|
||||
self.gate3(out)
|
||||
else:
|
||||
write(out / "session/boot-id.txt", BOOT + "\n")
|
||||
if cts:
|
||||
self.cts(out, **cts_args)
|
||||
return out
|
||||
|
||||
|
||||
def run_reducer(logroot, stamp, tree, json_out=None):
|
||||
cmd = [sys.executable, str(REDUCER), stamp, "--logroot", str(logroot), "--tools", str(tree)]
|
||||
if json_out:
|
||||
cmd += ["--json", str(json_out)]
|
||||
p = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True)
|
||||
vpath = Path(json_out) if json_out else Path(logroot) / stamp / "verdict.json"
|
||||
verdict = json.loads(vpath.read_text(encoding="utf-8")) if vpath.is_file() else {}
|
||||
return p.returncode, p.stdout, verdict
|
||||
|
||||
|
||||
class Checks:
|
||||
def __init__(self):
|
||||
self.total, self.failed = 0, []
|
||||
|
||||
def expect(self, name, cond, detail=""):
|
||||
self.total += 1
|
||||
print(" %s %s%s" % ("ok " if cond else "FAIL", name, "" if cond else " <-- " + str(detail)[:300]))
|
||||
if not cond:
|
||||
self.failed.append(name)
|
||||
|
||||
|
||||
def block(verdict, name):
|
||||
for b in verdict.get("cts", {}).get("blocks", []):
|
||||
if b["block"] == name:
|
||||
return b
|
||||
return {}
|
||||
|
||||
|
||||
def case(verdict, name):
|
||||
for e in verdict.get("gate3", {}).get("cases", []):
|
||||
if e["case"] == name:
|
||||
return e
|
||||
return {}
|
||||
|
||||
|
||||
def r3(v):
|
||||
return None if v is None else round(v, 3)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
# scenarios
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
def scenario_control(k, cn, tree):
|
||||
print("[control] complete clean window")
|
||||
cn.window("control")
|
||||
rc, text, v = run_reducer(cn.root, "control", tree)
|
||||
k.expect("exit 0", rc == 0, "rc=%d\n%s" % (rc, text))
|
||||
k.expect("GATE3 PASS", v.get("gate3", {}).get("overall") == "PASS", text)
|
||||
k.expect("CTS PASS, all five blocks", v.get("cts", {}).get("overall") == "PASS"
|
||||
and [b["block"] for b in v["cts"]["blocks"]] == list(BLOCKS), text)
|
||||
k.expect("every block delta 0.000 pp", all(r3(block(v, b).get("delta_pp")) == 0.0 for b in BLOCKS))
|
||||
k.expect("UBO recorded 'unrun (not in this glcts)'", v.get("cts", {}).get("ubo", {}).get("verdict") == "UNRUN"
|
||||
and "unrun (not in this glcts)" in text)
|
||||
s = v.get("gate3", {}).get("session", {})
|
||||
records = 2 * len(CASES) * (len(ARMS) + sum(n for _, n in ARMS)) # (.done + repeats) x (boot_id, before)
|
||||
k.expect("session valid: one boot_id over %d records" % records,
|
||||
s.get("valid") is True and s.get("boot_ids") == [BOOT] and s.get("records") == records, s)
|
||||
k.expect("verdict line", "WINDOW2 GATE3 PASS | CTS PASS -> exit 0" in text, text[-400:])
|
||||
|
||||
|
||||
def scenario_missing_block(k, cn, tree):
|
||||
print("[missing-block] a block without a report makes CTS INCOMPLETE")
|
||||
cn.window("missing", drop=("ssbo",), empty=("dsa",), no_results=("texture",))
|
||||
rc, text, v = run_reducer(cn.root, "missing", tree)
|
||||
k.expect("exit != 0", rc != 0, "rc=%d" % rc)
|
||||
k.expect("CTS INCOMPLETE", v.get("cts", {}).get("overall") == "INCOMPLETE" and "CTS INCOMPLETE" in text, text)
|
||||
k.expect("ssbo MISSING (never finished)", block(v, "ssbo").get("verdict") == "MISSING"
|
||||
and "no runs/ssbo/.done" in " ".join(block(v, "ssbo").get("reasons", [])), block(v, "ssbo"))
|
||||
k.expect("dsa MISSING (.done but empty report)", block(v, "dsa").get("verdict") == "MISSING"
|
||||
and "report-dsa.json is empty" in " ".join(block(v, "dsa").get("reasons", [])), block(v, "dsa"))
|
||||
k.expect("texture MISSING (report without results)", block(v, "texture").get("verdict") == "MISSING"
|
||||
and "has no results" in " ".join(block(v, "texture").get("reasons", [])), block(v, "texture"))
|
||||
k.expect("the two complete blocks still PASS", block(v, "shader-image").get("verdict") == "PASS"
|
||||
and block(v, "packed-pixels").get("verdict") == "PASS")
|
||||
k.expect("GATE3 unaffected (PASS)", v.get("gate3", {}).get("overall") == "PASS")
|
||||
cn.window("no-cts", cts=False)
|
||||
rc, text, v = run_reducer(cn.root, "no-cts", tree)
|
||||
k.expect("no cts/ at all -> CTS INCOMPLETE, exit != 0", rc != 0 and v.get("cts", {}).get("overall") == "INCOMPLETE"
|
||||
and all(b["verdict"] == "MISSING" for b in v["cts"]["blocks"]), text)
|
||||
|
||||
|
||||
def scenario_boot_id(k, cn, tree):
|
||||
print("[boot-id-change] every gate-3 record must carry the session's one boot_id")
|
||||
out = cn.window("boot-done")
|
||||
done = out / "gate3/state/spawn" / (CASES[1] + ".done")
|
||||
done.write_text(done.read_text().replace(BOOT, OTHER_BOOT))
|
||||
rc, text, v = run_reducer(cn.root, "boot-done", tree)
|
||||
s = v.get("gate3", {}).get("session", {})
|
||||
k.expect("spawn .done on another boot -> GATE3 INVALID-SESSION, exit != 0",
|
||||
rc != 0 and v.get("gate3", {}).get("overall") == "INVALID-SESSION" and "GATE3 INVALID-SESSION" in text, text)
|
||||
k.expect("the other boot_id is named", any(OTHER_BOOT in p for p in s.get("problems", [])), s)
|
||||
k.expect("...although every case's pictures PASS", all(e["verdict"] == "PASS" for e in v["gate3"]["cases"]))
|
||||
|
||||
out = cn.window("boot-repeat")
|
||||
rep = out / "gate3/archive/inproc" / (CASES[0] + "-DirectVulkan") / "repeat-02" / "boot_id.txt"
|
||||
rep.write_text("boot_id=%s boot_id_before=%s\n" % (OTHER_BOOT, BOOT))
|
||||
rc, text, v = run_reducer(cn.root, "boot-repeat", tree)
|
||||
k.expect("one repeat's boot_id.txt on another boot -> INVALID-SESSION",
|
||||
rc != 0 and v.get("gate3", {}).get("overall") == "INVALID-SESSION", text)
|
||||
|
||||
out = cn.window("boot-old-format")
|
||||
done = out / "gate3/state/monolith" / (CASES[0] + ".done")
|
||||
done.write_text("rc=0 seconds=8 attempts=1 passed_lines=1 finished=2026-09-23T09:40:14-04:00\n")
|
||||
rc, text, v = run_reducer(cn.root, "boot-old-format", tree)
|
||||
k.expect(".done without boot_id (pre-ruling format) -> INVALID-SESSION",
|
||||
rc != 0 and v.get("gate3", {}).get("overall") == "INVALID-SESSION"
|
||||
and any("carry no boot_id" in p for p in v["gate3"]["session"]["problems"]), text)
|
||||
|
||||
out = cn.window("boot-no-session")
|
||||
(out / "session/boot-id.txt").unlink()
|
||||
rc, text, v = run_reducer(cn.root, "boot-no-session", tree)
|
||||
k.expect("no session/boot-id.txt -> INVALID-SESSION",
|
||||
rc != 0 and v.get("gate3", {}).get("overall") == "INVALID-SESSION", text)
|
||||
|
||||
|
||||
def scenario_monolith_arm(k, cn, tree):
|
||||
print("[monolith-arm] the monolith arm must be the monolith; split arms must prove theirs")
|
||||
out = cn.window("mono-ipc")
|
||||
log = out / "gate3/archive/monolith" / (CASES[0] + "-DirectVulkan") / "repeat-01" / "mobilegl.log"
|
||||
log.write_text(LOGS["monolith"] + TAG + IPC % 1 + "\n")
|
||||
rc, text, v = run_reducer(cn.root, "mono-ipc", tree)
|
||||
e = case(v, CASES[0])
|
||||
k.expect("monolith log with Config: IPC -> case FAIL, GATE3 FAIL",
|
||||
e.get("verdict") == "FAIL" and v["gate3"]["overall"] == "FAIL" and rc != 0
|
||||
and any("NOT proven monolith" in r for r in e.get("reasons", [])), e)
|
||||
k.expect("the other case still PASS", case(v, CASES[1]).get("verdict") == "PASS")
|
||||
k.expect("arm identity line counts 1/2 monolith", "monolith 1/2 repeat(s)" in text, text)
|
||||
|
||||
out = cn.window("mono-nolog")
|
||||
(out / "gate3/archive/monolith" / (CASES[1] + "-DirectVulkan") / "repeat-01" / "mobilegl.log").unlink()
|
||||
rc, text, v = run_reducer(cn.root, "mono-nolog", tree)
|
||||
e = case(v, CASES[1])
|
||||
k.expect("monolith repeat without mobilegl.log -> FAIL (identity unproven)",
|
||||
e.get("verdict") == "FAIL" and any("unproven" in r for r in e.get("reasons", [])), e)
|
||||
|
||||
out = cn.window("split-proof")
|
||||
p = out / "gate3/archive/spawn" / (CASES[0] + "-DirectVulkan") / "repeat-02" / "transport-proof.json"
|
||||
p.write_text(json.dumps({"required_transport": "spawn", "passed": False}))
|
||||
rc, text, v = run_reducer(cn.root, "split-proof", tree)
|
||||
e = case(v, CASES[0])
|
||||
k.expect("spawn transport-proof not passed -> FAIL",
|
||||
e.get("verdict") == "FAIL" and any("spawn rep2 arm proof" in r for r in e.get("reasons", [])), e)
|
||||
|
||||
|
||||
def scenario_cross_arm(k, cn, tree):
|
||||
print("[cross-arm] cross-arm identity is reported (WARN), within-arm identity is the gate")
|
||||
out = cn.root / "cross"
|
||||
out.mkdir()
|
||||
cn.gate3(out, picture={"spawn": 129})
|
||||
cn.cts(out)
|
||||
rc, text, v = run_reducer(cn.root, "cross", tree)
|
||||
k.expect("spawn != inproc != monolith pictures -> GATE3 still PASS, exit 0",
|
||||
rc == 0 and v.get("gate3", {}).get("overall") == "PASS", text)
|
||||
k.expect("a WARN cross-arm line per case", all(("WARN cross-arm %s:" % c) in text for c in CASES), text)
|
||||
k.expect("cross_arm.identical false in the JSON", all(case(v, c).get("cross_arm", {}).get("identical") is False for c in CASES))
|
||||
k.expect("the ruling is named", "ruling: integrator 2026-09-23" in text)
|
||||
|
||||
out = cn.window("within")
|
||||
rep = out / "gate3/archive/inproc" / (CASES[1] + "-DirectVulkan") / "repeat-03"
|
||||
(rep / (CASES[1] + "-DirectVulkan-actual.png")).write_bytes(cn.png(127))
|
||||
rc, text, v = run_reducer(cn.root, "within", tree)
|
||||
e = case(v, CASES[1])
|
||||
k.expect("one inproc repeat differs from its siblings -> FAIL",
|
||||
e.get("verdict") == "FAIL" and any("inproc repeats NOT bit-identical" in r for r in e.get("reasons", [])), e)
|
||||
|
||||
|
||||
def scenario_unfinished(k, cn, tree):
|
||||
print("[unfinished] a pair without .done or with a repeat lacking its picture is INCOMPLETE")
|
||||
out = cn.window("nodone")
|
||||
(out / "gate3/state/inproc" / (CASES[1] + ".done")).unlink()
|
||||
rc, text, v = run_reducer(cn.root, "nodone", tree)
|
||||
e = case(v, CASES[1])
|
||||
k.expect("pair without .done -> case INCOMPLETE, GATE3 INCOMPLETE",
|
||||
e.get("verdict") == "INCOMPLETE" and v["gate3"]["overall"] == "INCOMPLETE" and rc != 0
|
||||
and any("no state .done" in r for r in e.get("reasons", [])), e)
|
||||
out = cn.window("nopng")
|
||||
(out / "gate3/archive/spawn" / (CASES[0] + "-DirectVulkan") / "repeat-03" / (CASES[0] + "-DirectVulkan-actual.png")).unlink()
|
||||
rc, text, v = run_reducer(cn.root, "nopng", tree)
|
||||
e = case(v, CASES[0])
|
||||
k.expect("repeat without actual PNG -> INCOMPLETE", e.get("verdict") == "INCOMPLETE"
|
||||
and any("no *-actual.png" in r for r in e.get("reasons", [])), e)
|
||||
|
||||
|
||||
def check_dryrun_numbers(k, v, text, label):
|
||||
dsa, si, tex = block(v, "dsa"), block(v, "shader-image"), block(v, "texture")
|
||||
k.expect("%s: dsa contract delta -0.272 pp" % label, r3(dsa.get("delta_pp")) == -0.272, dsa.get("delta_pp"))
|
||||
k.expect("%s: dsa BASE 368/(368+2) = 99.459%%, AFTER 366/(366+3) = 99.187%%" % label,
|
||||
r3(100 * dsa["base"]["rate"]) == 99.459 and r3(100 * dsa["after"]["rate"]) == 99.187
|
||||
if dsa.get("base") else False, dsa)
|
||||
k.expect("%s: dsa pre-ruling delta -0.539 pp kept as info" % label,
|
||||
r3(dsa.get("delta_pp_results_info")) == -0.539 and "(-0.539)" in text, dsa.get("delta_pp_results_info"))
|
||||
k.expect("%s: dsa FAIL on the new crash only, not on the rate" % label,
|
||||
dsa.get("verdict") == "FAIL" and dsa.get("reasons") == ["1 NEW crash(es)"]
|
||||
and dsa.get("new_crash") == [DSA_STORAGE], dsa.get("reasons"))
|
||||
k.expect("%s: dsa counts P/F/NS/W/X 366/3/0/1/1" % label,
|
||||
[dsa.get("after", {}).get(x) for x in ("pass", "fail", "ns", "warn", "crash")] == [366, 3, 0, 1, 1])
|
||||
k.expect("%s: shader-image +4.284 pp, FAIL on new crash incomplete_textures" % label,
|
||||
r3(si.get("delta_pp")) == 4.284 and si.get("verdict") == "FAIL" and si.get("new_crash") == [SI_INCOMPLETE],
|
||||
(si.get("delta_pp"), si.get("verdict"), si.get("new_crash")))
|
||||
k.expect("%s: texture +0.960 pp PASS" % label, r3(tex.get("delta_pp")) == 0.96 and tex.get("verdict") == "PASS",
|
||||
(tex.get("delta_pp"), tex.get("verdict")))
|
||||
k.expect("%s: ssbo, packed-pixels 0.000 pp PASS" % label,
|
||||
all(r3(block(v, b).get("delta_pp")) == 0.0 and block(v, b).get("verdict") == "PASS"
|
||||
for b in ("ssbo", "packed-pixels")))
|
||||
k.expect("%s: CTS FAIL" % label, v.get("cts", {}).get("overall") == "FAIL" and "\nCTS FAIL" in text)
|
||||
k.expect("%s: printed '-0.272'" % label, "-0.272" in text)
|
||||
|
||||
|
||||
def scenario_dryrun(k, cn, tree, dry_dir):
|
||||
print("[dry-run] contract formula on the pre-14e1c8b9 dry-run data")
|
||||
fixture = json.loads(FIXTURE.read_text(encoding="utf-8"))
|
||||
diffs = {b: fixture["blocks"][b]["after_differences"] for b in BLOCKS}
|
||||
out = cn.root / "dryrun"
|
||||
out.mkdir()
|
||||
rebuilt = cn.cts(out, differences=diffs)
|
||||
for b in BLOCKS:
|
||||
want = fixture["blocks"][b]
|
||||
got_sha = hashlib.sha256(json.dumps(rebuilt[b], sort_keys=True).encode()).hexdigest()
|
||||
k.expect("rebuild of %s is the recorded report (counts + sha256)" % b,
|
||||
dict(Counter(rebuilt[b].values())) == want["after_counts"]
|
||||
and got_sha == want["after_sha256_of_sorted_results"], (Counter(rebuilt[b].values()), got_sha))
|
||||
rc, text, v = run_reducer(cn.root, "dryrun", tree)
|
||||
print(" --- reducer output (canned rebuild), CTS section ---")
|
||||
start = text.find("== CTS AFTER")
|
||||
for line in text[start:].splitlines():
|
||||
if line.strip():
|
||||
print(" | " + line)
|
||||
check_dryrun_numbers(k, v, text, "canned")
|
||||
k.expect("canned: no gate3/ -> GATE3 NOT-RUN, exit != 0",
|
||||
rc != 0 and v.get("gate3", {}).get("overall") == "NOT-RUN")
|
||||
if dry_dir and (Path(dry_dir) / "cts").is_dir():
|
||||
dry = Path(dry_dir)
|
||||
rc2, text2, v2 = run_reducer(dry.parent, dry.name, tree, json_out=cn.root / "real-dry-run-verdict.json")
|
||||
check_dryrun_numbers(k, v2, text2, "real %s" % dry.name)
|
||||
else:
|
||||
print(" (skip) real dry-run output not found at %s" % dry_dir)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--tools", help="MobileGL tree (default: the tree holding this file, else $W2_TOOLS_OVERRIDE)")
|
||||
ap.add_argument("--dry-run-dir", default=str(Path.home() / "w7/logs/devprep/w2/pre-14e1c8b9"))
|
||||
ap.add_argument("--keep", action="store_true", help="keep the canned directories")
|
||||
args = ap.parse_args()
|
||||
tree = find_tree(args.tools)
|
||||
if tree is None:
|
||||
print("no MobileGL tree with tools/trace_replay/compare_actuals.py and %s: pass --tools" % CTS_BASE_REL,
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
sys.path.insert(0, str(tree / "tools" / "trace_replay"))
|
||||
import compare_actuals as ca
|
||||
root = Path(tempfile.mkdtemp(prefix="w2-reduce-selftest-"))
|
||||
print("60-reduce self-test: reducer %s, tree %s, canned windows in %s" % (REDUCER, tree, root))
|
||||
k = Checks()
|
||||
cn = Canned(root, tree, ca)
|
||||
try:
|
||||
scenario_control(k, cn, tree)
|
||||
scenario_missing_block(k, cn, tree)
|
||||
scenario_boot_id(k, cn, tree)
|
||||
scenario_monolith_arm(k, cn, tree)
|
||||
scenario_cross_arm(k, cn, tree)
|
||||
scenario_unfinished(k, cn, tree)
|
||||
scenario_dryrun(k, cn, tree, args.dry_run_dir)
|
||||
finally:
|
||||
if not args.keep:
|
||||
shutil.rmtree(root, ignore_errors=True)
|
||||
print("60-reduce self-test: %d/%d checks passed%s" % (
|
||||
k.total - len(k.failed), k.total, "" if not k.failed else "; FAILED: " + "; ".join(k.failed)))
|
||||
return 0 if not k.failed else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -1,24 +1,39 @@
|
||||
#!/usr/bin/env python3
|
||||
"""60-reduce.py <stamp> [--logroot DIR] [--tools TREE] [--json OUT]
|
||||
|
||||
Device window 2 verdicts, read from the window's own output directory (<logroot>/<stamp>):
|
||||
Device window 2 verdicts, read from the window's own output directory (<logroot>/<stamp>).
|
||||
|
||||
GATE 3 (CONTRACT-P7 7.2), per case of gate3/cases.txt:
|
||||
* monolith x1 is the same-session control; it must itself pass;
|
||||
* inproc and spawn: every expected repeat present and error-free, each repeat's arm proof
|
||||
(transport-proof.json passed), the device's own pass verdict (ssim >= case threshold),
|
||||
ALL repeats bit-identical (actual PNG SHA-256), and |ssim - monolith ssim| <= 0.0005 on every
|
||||
repeat; a repeat whose logcat is byte-identical to the previous repeat's is flagged STALE;
|
||||
* ONE SESSION: every gate3/state/<arm>/<case>.done and every archived repeat's boot_id.txt, of
|
||||
every arm, carries one boot_id, equal to session/boot-id.txt. A second boot_id, a record that
|
||||
carries none, or no session/boot-id.txt makes the whole gate GATE3 INVALID-SESSION, whatever
|
||||
the pictures say (the gate is a same-session reading);
|
||||
* monolith x1 is the same-session control: it must itself pass, and its role logs must show no
|
||||
'Config: IPC' / MOBILEGL_TRANSPORT line (the arm really was the monolith);
|
||||
* inproc and spawn: the pair finished (state .done), every expected repeat present and
|
||||
error-free, each repeat's arm proof (transport-proof.json passed), the device's own pass
|
||||
verdict (ssim >= case threshold), ALL repeats OF THE ARM bit-identical (actual PNG SHA-256;
|
||||
within-arm identity is the gate), and |ssim - monolith ssim| <= 0.0005 on every repeat; a
|
||||
repeat whose logcat is byte-identical to the previous repeat's is flagged STALE;
|
||||
* OpenRA: 1.000000 and mismatch 0 on every repeat of both split arms AND on monolith;
|
||||
* cross-arm picture identity (inproc vs spawn vs monolith) is REPORTED per case and a mismatch
|
||||
prints a WARN line - information, not a gate (ruling: integrator 2026-09-23);
|
||||
* inproc-ra0 (RUN_AHEAD=0): recorded (ssim, sha, run-ahead=0 proof), never gating.
|
||||
BSL STATS: per arm, maps-line / VmRSS / VmHWM peaks and the wbuf[] gauges.
|
||||
BSL STATS: per arm, maps-line / VmRSS / VmHWM peaks and the wbuf[] gauges (information).
|
||||
CTS AFTER (CONTRACT-P7 7.3), per block vs device-window-1/CTS-base/report-<block>.json, over the
|
||||
cases the AFTER run was asked to run (so a --limit subset is compared like for like):
|
||||
rate = Pass / (results - NotSupported) [every non-NS outcome - Fail, Crash, warnings - is in the
|
||||
denominator; NS excluded], delta in percentage points, PASS iff delta >= -0.5 pp, no new crash
|
||||
(Crash/Timeout/InternalError/ResourceError/DeviceHang/Incomplete not already so in BASE) and the
|
||||
block is complete (unrun.txt empty).
|
||||
Exit status: 0 when every section that has data is PASS, 1 otherwise, 2 on bad input.
|
||||
* all five blocks (shader-image ssbo dsa texture packed-pixels) are REQUIRED, each with a
|
||||
finished run (cts/runs/<block>/.done) and a non-empty cts/report-<block>.json; a missing one
|
||||
makes CTS INCOMPLETE. The UBO block (GTF-GL46 uniform_buffer_object) is recorded as
|
||||
'unrun (not in this glcts)': the device glcts carries no GTF module (ID-P7-16);
|
||||
* rate = Pass / (Pass + Fail), the contract formula: NotSupported, the warnings and the crashes
|
||||
are NOT in the denominator; gate: delta >= -0.5 pp on this rate;
|
||||
* crashes (Crash/Timeout/InternalError/ResourceError/DeviceHang/Incomplete, plus hung.txt) are
|
||||
counted in their own column and are non-Pass; one that BASE does not already have is a NEW
|
||||
crash, a hard red of its own (a Fail that became a Crash is a new crash);
|
||||
* information only, never gating: the pre-ruling rate Pass / (results - NotSupported), delta.
|
||||
Exit status: 0 only when GATE 3 and CTS are both PASS; 1 otherwise (a section that did not run,
|
||||
is INCOMPLETE / INVALID-SESSION, or FAILs); 2 on bad input.
|
||||
"""
|
||||
import argparse
|
||||
import hashlib
|
||||
@@ -26,13 +41,25 @@ import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
TOL = 0.0005
|
||||
CTS_TOL_PP = 0.5
|
||||
HARD = {"Crash", "Timeout", "InternalError", "ResourceError", "DeviceHang", "Incomplete"}
|
||||
HARD = ("Crash", "Timeout", "InternalError", "ResourceError", "DeviceHang", "Incomplete")
|
||||
SPLIT_ARMS = ("inproc", "spawn")
|
||||
OPENRA = "OpenRA"
|
||||
CTS_BLOCKS = ("shader-image", "ssbo", "dsa", "texture", "packed-pixels")
|
||||
UBO_BLOCK = {"block": "ubo", "caselist": "tools/cts/caselists/p7-ubo-gl46.txt",
|
||||
"cases": "GTF-GL46.gtf31.GL3Tests.uniform_buffer_object*", "verdict": "UNRUN",
|
||||
"reasons": ["unrun (not in this glcts): the device glcts has no GTF module (ID-P7-16);"
|
||||
" recorded, not gated"]}
|
||||
REPEAT_DIR = re.compile(r"^repeat-(\d+)$")
|
||||
KV = re.compile(r"([A-Za-z_][A-Za-z0-9_]*)=(\S*)")
|
||||
# ConfigLoader.cpp: "Config: MOBILEGL_TRANSPORT=inproc|spawn - ..." (and its refusals) and
|
||||
# "Config: IPC ring=..." are printed only when a transport resolves; the monolith prints neither.
|
||||
NOT_MONOLITH = re.compile(r"Config: IPC|MOBILEGL_TRANSPORT=")
|
||||
RULING = "ruling: integrator 2026-09-23"
|
||||
|
||||
|
||||
def sha(path):
|
||||
@@ -53,9 +80,81 @@ def read_kv(path):
|
||||
return out
|
||||
|
||||
|
||||
def read_record(path):
|
||||
"""A one-line 'k=v k=v ...' record (gate3 .done, boot_id.txt) -> dict, or None if absent."""
|
||||
try:
|
||||
return dict(KV.findall(path.read_text(encoding="utf-8", errors="replace")))
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
|
||||
def first_line(path):
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
|
||||
except OSError:
|
||||
return ""
|
||||
return lines[0].strip() if lines else ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
# gate 3
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
def gate3_session(out, g):
|
||||
"""One boot session for every record of every arm (ruling c)."""
|
||||
want = first_line(out / "session" / "boot-id.txt")
|
||||
problems, seen, unstamped = [], {}, []
|
||||
|
||||
def note(value, where):
|
||||
if value:
|
||||
seen.setdefault(value, []).append(where)
|
||||
else:
|
||||
unstamped.append(where)
|
||||
|
||||
state = g / "state"
|
||||
for arm_dir in sorted(p for p in state.iterdir() if p.is_dir()) if state.is_dir() else []:
|
||||
for done in sorted(arm_dir.glob("*.done")):
|
||||
rec = read_record(done) or {}
|
||||
where = "state/%s/%s" % (arm_dir.name, done.name)
|
||||
note(rec.get("boot_id"), where)
|
||||
if "boot_id_before" in rec:
|
||||
note(rec["boot_id_before"], where + " (boot_id_before)")
|
||||
archive = g / "archive"
|
||||
for arm_dir in sorted(p for p in archive.iterdir() if p.is_dir()) if archive.is_dir() else []:
|
||||
for case_dir in sorted(p for p in arm_dir.iterdir() if p.is_dir()):
|
||||
for rep in sorted(p for p in case_dir.iterdir() if p.is_dir() and REPEAT_DIR.match(p.name)):
|
||||
where = "archive/%s/%s/%s/boot_id.txt" % (arm_dir.name, case_dir.name, rep.name)
|
||||
rec = read_record(rep / "boot_id.txt") or {}
|
||||
note(rec.get("boot_id"), where)
|
||||
if "boot_id_before" in rec:
|
||||
note(rec["boot_id_before"], where + " (boot_id_before)")
|
||||
if not want:
|
||||
problems.append("no session/boot-id.txt: 21-preflight.sh records the session's boot_id there")
|
||||
if unstamped:
|
||||
problems.append("%d record(s) carry no boot_id (pre-ruling 30-gate3.sh, or a repeat the driver did "
|
||||
"not stamp), e.g. %s" % (len(unstamped), unstamped[0]))
|
||||
for boot in sorted(b for b in seen if b != want):
|
||||
problems.append("boot_id %s != session %s in %d record(s), e.g. %s"
|
||||
% (boot, want or "?", len(seen[boot]), seen[boot][0]))
|
||||
return {"session_boot_id": want, "boot_ids": sorted(seen), "records": sum(len(v) for v in seen.values()),
|
||||
"unstamped": len(unstamped), "problems": problems, "valid": not problems,
|
||||
"reboot_clean": first_line(out / "session" / "reboot-clean.txt")}
|
||||
|
||||
|
||||
def monolith_identity(row):
|
||||
"""None when the repeat's role logs show a monolith, else what says otherwise (ruling f)."""
|
||||
d = Path(row["dir"])
|
||||
if not (d / "mobilegl.log").is_file():
|
||||
return "no mobilegl.log archived: the arm's identity is unproven"
|
||||
for name in ("mobilegl.log", "mobilegl.client.log", "mobilegl.server.log"):
|
||||
p = d / name
|
||||
if not p.is_file():
|
||||
continue
|
||||
for line in p.read_text(encoding="utf-8", errors="replace").splitlines():
|
||||
if NOT_MONOLITH.search(line):
|
||||
return "%s: %s" % (name, line.strip()[:140])
|
||||
return None
|
||||
|
||||
|
||||
def gate3(out, ca):
|
||||
g = out / "gate3"
|
||||
cases_file = g / "cases.txt"
|
||||
@@ -67,6 +166,8 @@ def gate3(out, ca):
|
||||
repeat = int(meta.get("repeat", "3"))
|
||||
arms_present = [a for a in ("monolith", "inproc", "spawn", "inproc-ra0") if (g / "archive" / a).is_dir()]
|
||||
summaries = {a: ca.summarize(g / "archive" / a) for a in arms_present}
|
||||
session = gate3_session(out, g)
|
||||
identity = {"monolith_repeats": 0, "monolith_proven": 0, "split_repeats": 0, "split_proofs_passed": 0}
|
||||
|
||||
def rows(arm, case):
|
||||
s = summaries.get(arm)
|
||||
@@ -74,6 +175,9 @@ def gate3(out, ca):
|
||||
return []
|
||||
return [r for r in s["rows"] if r["arm"] == case + "-DirectVulkan"]
|
||||
|
||||
def finished(arm, case):
|
||||
return (g / "state" / arm / (case + ".done")).is_file()
|
||||
|
||||
def proof(row):
|
||||
p = Path(row["dir"]) / "transport-proof.json"
|
||||
try:
|
||||
@@ -99,14 +203,21 @@ def gate3(out, ca):
|
||||
reasons, incomplete = [], []
|
||||
entry = {"case": case, "arms": {}}
|
||||
mono = rows("monolith", case)
|
||||
mono_ssim = None
|
||||
mono_ssim = mono_sha = None
|
||||
if "monolith" in summaries and not finished("monolith", case):
|
||||
incomplete.append("monolith: no state .done (pair unfinished; 30-gate3.sh retries it)")
|
||||
if len(mono) != 1 or mono[0].get("error"):
|
||||
incomplete.append("monolith: %s" % (mono[0].get("error") if mono else "not run"))
|
||||
else:
|
||||
m = mono[0]
|
||||
mono_ssim = m.get("ssim_vs_golden")
|
||||
entry["arms"]["monolith"] = {"ssim": mono_ssim, "sha": m.get("actual_sha256"),
|
||||
mono_ssim, mono_sha = m.get("ssim_vs_golden"), m.get("actual_sha256")
|
||||
not_mono = monolith_identity(m)
|
||||
identity["monolith_repeats"] += 1
|
||||
identity["monolith_proven"] += not_mono is None
|
||||
entry["arms"]["monolith"] = {"ssim": mono_ssim, "sha": mono_sha, "identity": not_mono or "monolith",
|
||||
"mismatch": m.get("mismatch_pixels_vs_golden"), "passed": m.get("passed")}
|
||||
if not_mono:
|
||||
reasons.append("monolith arm is NOT proven monolith (%s)" % not_mono)
|
||||
if m.get("passed") is not True:
|
||||
reasons.append("monolith control did not pass (ssim %s)" % mono_ssim)
|
||||
if case == OPENRA and not (m.get("mismatch_pixels_vs_golden") == 0 and (mono_ssim or 0) >= 0.9999995):
|
||||
@@ -114,11 +225,13 @@ def gate3(out, ca):
|
||||
for arm in SPLIT_ARMS:
|
||||
rs = rows(arm, case)
|
||||
a = {"repeats": len(rs), "ssim": [r.get("ssim_vs_golden") for r in rs],
|
||||
"sha": sorted({r.get("actual_sha256") for r in rs if r.get("actual_sha256")})}
|
||||
"sha": sorted({r.get("actual_sha256") for r in rs if r.get("actual_sha256")}), "proofs_passed": 0}
|
||||
entry["arms"][arm] = a
|
||||
if arm not in summaries:
|
||||
incomplete.append(arm + ": not run")
|
||||
continue
|
||||
if not finished(arm, case):
|
||||
incomplete.append("%s: no state .done (pair unfinished; 30-gate3.sh retries it)" % arm)
|
||||
if len(rs) != repeat:
|
||||
incomplete.append("%s: %d/%d repeats" % (arm, len(rs), repeat))
|
||||
for r in rs:
|
||||
@@ -128,8 +241,12 @@ def gate3(out, ca):
|
||||
if r.get("passed") is not True:
|
||||
reasons.append("%s rep%d below threshold (ssim %s)" % (arm, r["repeat"], r.get("ssim_vs_golden")))
|
||||
p = proof(r)
|
||||
identity["split_repeats"] += 1
|
||||
if not p or p.get("passed") is not True:
|
||||
reasons.append("%s rep%d arm proof not passed" % (arm, r["repeat"]))
|
||||
reasons.append("%s rep%d arm proof (transport-proof.json) not passed" % (arm, r["repeat"]))
|
||||
else:
|
||||
a["proofs_passed"] += 1
|
||||
identity["split_proofs_passed"] += 1
|
||||
if mono_ssim is not None and r.get("ssim_vs_golden") is not None:
|
||||
d = abs(r["ssim_vs_golden"] - mono_ssim)
|
||||
if d > TOL:
|
||||
@@ -148,22 +265,36 @@ def gate3(out, ca):
|
||||
r = ra0[0]
|
||||
entry["arms"]["inproc-ra0"] = {"ssim": r.get("ssim_vs_golden"), "sha": r.get("actual_sha256"),
|
||||
"passed": r.get("passed"), "proof": ra0_proof(r), "error": r.get("error")}
|
||||
# Cross-arm identity: reported, never gating (ruling d).
|
||||
ip = set(entry["arms"].get("inproc", {}).get("sha", []))
|
||||
sp = set(entry["arms"].get("spawn", {}).get("sha", []))
|
||||
entry["inproc_eq_spawn"] = bool(ip) and ip == sp
|
||||
entry["split_eq_monolith"] = bool(ip) and ip == sp == {entry["arms"].get("monolith", {}).get("sha")}
|
||||
entry["split_eq_monolith"] = bool(ip) and ip == sp == {mono_sha}
|
||||
entry["cross_arm"] = {"monolith": mono_sha, "inproc": sorted(ip), "spawn": sorted(sp),
|
||||
"identical": (entry["split_eq_monolith"] if (ip and sp and mono_sha) else None)}
|
||||
entry["verdict"] = "INCOMPLETE" if incomplete else ("FAIL" if reasons else "PASS")
|
||||
entry["reasons"] = incomplete + reasons
|
||||
results.append(entry)
|
||||
verdicts = [e["verdict"] for e in results]
|
||||
overall = "PASS" if verdicts and all(v == "PASS" for v in verdicts) else (
|
||||
"INCOMPLETE" if "INCOMPLETE" in verdicts and "FAIL" not in verdicts else "FAIL")
|
||||
if not session["valid"]:
|
||||
overall = "INVALID-SESSION"
|
||||
elif verdicts and all(v == "PASS" for v in verdicts):
|
||||
overall = "PASS"
|
||||
elif "FAIL" in verdicts:
|
||||
overall = "FAIL"
|
||||
else:
|
||||
overall = "INCOMPLETE"
|
||||
return {"cases": results, "excluded": excluded, "repeat": repeat, "denominator": len(cases),
|
||||
"overall": overall, "arms_present": arms_present}
|
||||
"overall": overall, "arms_present": arms_present, "session": session, "identity": identity}
|
||||
|
||||
|
||||
def print_gate3(g3):
|
||||
print("== GATE 3 (DirectVulkan x pbuffer; split repeat=%d; tol |ssim-monolith| <= %.4f) ==" % (g3["repeat"], TOL))
|
||||
s = g3["session"]
|
||||
print("-- session: boot_id %s; %d record(s) stamped; %s" % (s["session_boot_id"] or "?", s["records"],
|
||||
s["reboot_clean"] or "no session/reboot-clean.txt"))
|
||||
for problem in s["problems"]:
|
||||
print(" ! INVALID-SESSION: " + problem)
|
||||
w = max([len(e["case"]) for e in g3["cases"]] + [4])
|
||||
print("%-*s %-10s %-11s %-24s %-24s %-11s %s" % (w, "case", "verdict", "monolith", "inproc(min..max) #sha", "spawn(min..max) #sha", "ra0", "i==s==m"))
|
||||
for e in g3["cases"]:
|
||||
@@ -187,11 +318,28 @@ def print_gate3(g3):
|
||||
if ra0s:
|
||||
print("-- RUN_AHEAD=0 control (recorded, not gating): %d case(s); proofs: %s" % (
|
||||
len(ra0s), ", ".join(sorted({r["proof"] for _, r in ra0s}))))
|
||||
i = g3["identity"]
|
||||
print("-- arm identity: monolith %d/%d repeat(s) with no Config: IPC / MOBILEGL_TRANSPORT line; "
|
||||
"inproc+spawn transport-proof passed %d/%d" % (i["monolith_proven"], i["monolith_repeats"],
|
||||
i["split_proofs_passed"], i["split_repeats"]))
|
||||
judged = [e for e in g3["cases"] if e["cross_arm"]["identical"] is not None]
|
||||
print("-- cross-arm picture identity (reported, NOT a gate; %s): %d/%d case(s) inproc==spawn==monolith" % (
|
||||
RULING, sum(e["cross_arm"]["identical"] for e in judged), len(judged)))
|
||||
for e in judged:
|
||||
if not e["cross_arm"]["identical"]:
|
||||
x = e["cross_arm"]
|
||||
print("WARN cross-arm %s: monolith %s | inproc %s | spawn %s" % (
|
||||
e["case"], (x["monolith"] or "-")[:12], ",".join(v[:12] for v in x["inproc"]) or "-",
|
||||
",".join(v[:12] for v in x["spawn"]) or "-"))
|
||||
print("-- excluded (red on monolith, CONTRACT-P7 7.1): " + (", ".join(g3["excluded"]) or "none"))
|
||||
n = len(g3["cases"])
|
||||
npass = sum(e["verdict"] == "PASS" for e in g3["cases"])
|
||||
note = "" if n == 36 else " (denominator is %d, not the gate's 36: a subset run is NOT a gate verdict)" % n
|
||||
print("GATE3 %s: %d/%d cases PASS%s" % (g3["overall"], npass, n, note))
|
||||
notes = []
|
||||
if n != 36:
|
||||
notes.append("denominator is %d, not the gate's 36: a subset run is NOT a gate verdict" % n)
|
||||
if not s["reboot_clean"].startswith("reboot-clean:"):
|
||||
notes.append("session not reboot-clean: NOT a gate verdict")
|
||||
print("GATE3 %s: %d/%d cases PASS%s" % (g3["overall"], npass, n, "".join(" (%s)" % x for x in notes)))
|
||||
|
||||
|
||||
# ------------------------------------------------------------------------------------------------
|
||||
@@ -236,79 +384,121 @@ def read_list(path):
|
||||
|
||||
|
||||
def rate(statuses):
|
||||
p = sum(1 for s in statuses if s == "Pass")
|
||||
ns = sum(1 for s in statuses if s == "NotSupported")
|
||||
f = sum(1 for s in statuses if s == "Fail")
|
||||
d = len(statuses) - ns
|
||||
return {"pass": p, "ns": ns, "fail": f, "other": d - p - f, "denominator": d,
|
||||
"rate": (p / d) if d else None, "literal_p_over_p_plus_f": (p / (p + f)) if (p + f) else None}
|
||||
"""CONTRACT-P7 7.3: rate = Pass / (Pass + Fail). NotSupported, the warnings (Compatibility /
|
||||
Quality) and the crashes stay out of the denominator; crashes get their own count. The
|
||||
pre-ruling Pass / (results - NotSupported) is kept as `results_rate`, information only."""
|
||||
c = Counter(statuses)
|
||||
p, f, ns = c["Pass"], c["Fail"], c["NotSupported"]
|
||||
x = sum(c[s] for s in HARD)
|
||||
n = len(statuses)
|
||||
return {"pass": p, "fail": f, "ns": ns, "crash": x, "warn": n - p - f - ns - x, "results": n,
|
||||
"rate": (p / (p + f)) if (p + f) else None,
|
||||
"results_rate": (p / (n - ns)) if (n - ns) else None}
|
||||
|
||||
|
||||
def load_results(path):
|
||||
"""-> (results dict, None) or (None, why the report does not count)."""
|
||||
if not path.is_file():
|
||||
return None, "no %s" % path.name
|
||||
if path.stat().st_size == 0:
|
||||
return None, "%s is empty" % path.name
|
||||
try:
|
||||
results = json.loads(path.read_text(encoding="utf-8"))["results"]
|
||||
except (OSError, ValueError, KeyError, TypeError) as error:
|
||||
return None, "%s unreadable (%s)" % (path.name, error)
|
||||
if not isinstance(results, dict) or not results:
|
||||
return None, "%s has no results" % path.name
|
||||
return results, None
|
||||
|
||||
|
||||
def cts_block(c, b, base_dir):
|
||||
run = c / "runs" / b
|
||||
if not c.is_dir():
|
||||
return {"block": b, "verdict": "MISSING", "reasons": ["no cts/ directory: 50-cts-after.sh did not run"]}
|
||||
if not (run / ".done").is_file():
|
||||
return {"block": b, "verdict": "MISSING", "reasons": ["block not finished (no runs/%s/.done)" % b]}
|
||||
after, why = load_results(c / ("report-%s.json" % b))
|
||||
if why:
|
||||
return {"block": b, "verdict": "MISSING", "reasons": [why]}
|
||||
base, why = load_results(base_dir / ("report-%s.json" % b))
|
||||
if why:
|
||||
return {"block": b, "verdict": "NO-BASE", "reasons": ["$BASE: " + why + " in " + str(base_dir)]}
|
||||
caselist = Path((run / "caselist.path").read_text().strip()) if (run / "caselist.path").is_file() else None
|
||||
universe = read_list(caselist) if caselist else sorted(after)
|
||||
skipped = set(read_list(run / "skipped.txt"))
|
||||
universe = [x for x in universe if x not in skipped]
|
||||
unrun = read_list(run / "unrun.txt")
|
||||
hung = read_list(run / "hung.txt")
|
||||
a_st = [after[x] for x in universe if x in after]
|
||||
b_st = [base[x] for x in universe if x in base]
|
||||
ra, rb = rate(a_st), rate(b_st)
|
||||
delta = None if ra["rate"] is None or rb["rate"] is None else (ra["rate"] - rb["rate"]) * 100.0
|
||||
delta_results = (None if ra["results_rate"] is None or rb["results_rate"] is None
|
||||
else (ra["results_rate"] - rb["results_rate"]) * 100.0)
|
||||
new_crash = sorted(x for x in universe if after.get(x) in HARD and base.get(x) not in HARD)
|
||||
new_crash += sorted(x for x in hung if x not in new_crash)
|
||||
regress = sorted(x for x in universe if base.get(x) == "Pass" and after.get(x) not in (None, "Pass"))
|
||||
fixed = sorted(x for x in universe if base.get(x) not in (None, "Pass") and after.get(x) == "Pass")
|
||||
missing_base = sorted(x for x in universe if x not in base)
|
||||
reasons = []
|
||||
verdict = "PASS"
|
||||
if unrun or len(a_st) < len(universe):
|
||||
verdict = "INCOMPLETE"
|
||||
reasons.append("%d case(s) without an AFTER result (unrun.txt %d)" % (len(universe) - len(a_st), len(unrun)))
|
||||
if new_crash:
|
||||
verdict = "FAIL"
|
||||
reasons.append("%d NEW crash(es)" % len(new_crash))
|
||||
if delta is not None and delta < -CTS_TOL_PP:
|
||||
verdict = "FAIL"
|
||||
reasons.append("delta %.3f pp < -%.1f pp (Pass/(Pass+Fail))" % (delta, CTS_TOL_PP))
|
||||
if delta is not None and delta > CTS_TOL_PP:
|
||||
reasons.append("note: AFTER is %.3f pp ABOVE BASE (not a regression; review the fixed list)" % delta)
|
||||
return {"block": b, "cases": len(universe), "after": ra, "base": rb, "delta_pp": delta,
|
||||
"delta_pp_results_info": delta_results, "new_crash": new_crash, "regressed": regress,
|
||||
"fixed": fixed, "missing_in_base": missing_base, "unrun": len(unrun), "hung": hung,
|
||||
"verdict": verdict, "reasons": reasons, "subset": bool(caselist and "first" in caselist.name)}
|
||||
|
||||
|
||||
def cts(out, base_dir):
|
||||
c = out / "cts"
|
||||
if not c.is_dir():
|
||||
return None
|
||||
blocks = []
|
||||
for report in sorted(c.glob("report-*.json")):
|
||||
b = report.stem[len("report-"):]
|
||||
run = c / "runs" / b
|
||||
after = json.loads(report.read_text(encoding="utf-8"))["results"]
|
||||
base_path = base_dir / ("report-%s.json" % b)
|
||||
if not base_path.is_file():
|
||||
blocks.append({"block": b, "verdict": "NO-BASE", "reasons": ["no " + str(base_path)]})
|
||||
continue
|
||||
base = json.loads(base_path.read_text(encoding="utf-8"))["results"]
|
||||
caselist = Path((run / "caselist.path").read_text().strip()) if (run / "caselist.path").is_file() else None
|
||||
universe = read_list(caselist) if caselist else sorted(after)
|
||||
skipped = set(read_list(run / "skipped.txt"))
|
||||
universe = [x for x in universe if x not in skipped]
|
||||
unrun = read_list(run / "unrun.txt")
|
||||
hung = read_list(run / "hung.txt")
|
||||
a_st = [after[x] for x in universe if x in after]
|
||||
b_st = [base[x] for x in universe if x in base]
|
||||
ra, rb = rate(a_st), rate(b_st)
|
||||
delta = None if ra["rate"] is None or rb["rate"] is None else (ra["rate"] - rb["rate"]) * 100.0
|
||||
new_crash = sorted(x for x in universe if after.get(x) in HARD and base.get(x) not in HARD)
|
||||
new_crash += sorted(x for x in hung if x not in new_crash)
|
||||
regress = sorted(x for x in universe if base.get(x) == "Pass" and after.get(x) not in (None, "Pass"))
|
||||
fixed = sorted(x for x in universe if base.get(x) not in (None, "Pass") and after.get(x) == "Pass")
|
||||
missing_base = sorted(x for x in universe if x not in base)
|
||||
reasons = []
|
||||
verdict = "PASS"
|
||||
if unrun or len(a_st) < len(universe):
|
||||
verdict = "INCOMPLETE"
|
||||
reasons.append("%d case(s) without an AFTER result (unrun.txt %d)" % (len(universe) - len(a_st), len(unrun)))
|
||||
if new_crash:
|
||||
verdict = "FAIL"
|
||||
reasons.append("%d NEW crash(es)" % len(new_crash))
|
||||
if delta is not None and delta < -CTS_TOL_PP:
|
||||
verdict = "FAIL"
|
||||
reasons.append("delta %.3f pp < -%.1f pp" % (delta, CTS_TOL_PP))
|
||||
if delta is not None and delta > CTS_TOL_PP:
|
||||
reasons.append("note: AFTER is %.3f pp ABOVE BASE (not a regression; review the fixed list)" % delta)
|
||||
blocks.append({"block": b, "cases": len(universe), "after": ra, "base": rb, "delta_pp": delta,
|
||||
"new_crash": new_crash, "regressed": regress, "fixed": fixed, "missing_in_base": missing_base,
|
||||
"unrun": len(unrun), "hung": hung, "verdict": verdict, "reasons": reasons,
|
||||
"subset": bool(caselist and "first" in caselist.name)})
|
||||
blocks = [cts_block(c, b, base_dir) for b in CTS_BLOCKS]
|
||||
extra = sorted(p.stem[len("report-"):] for p in c.glob("report-*.json")
|
||||
if p.stem[len("report-"):] not in CTS_BLOCKS) if c.is_dir() else []
|
||||
verdicts = [b["verdict"] for b in blocks]
|
||||
overall = "PASS" if blocks and all(v == "PASS" for v in verdicts) else ("FAIL" if "FAIL" in verdicts else "INCOMPLETE")
|
||||
return {"blocks": blocks, "overall": overall}
|
||||
if "FAIL" in verdicts:
|
||||
overall = "FAIL"
|
||||
elif all(v == "PASS" for v in verdicts):
|
||||
overall = "PASS"
|
||||
else:
|
||||
overall = "INCOMPLETE"
|
||||
return {"blocks": blocks, "overall": overall, "ubo": dict(UBO_BLOCK), "ignored_reports": extra,
|
||||
"incomplete_blocks": [b["block"] for b in blocks if b["verdict"] not in ("PASS", "FAIL")]}
|
||||
|
||||
|
||||
def print_cts(c):
|
||||
print("== CTS AFTER (inproc x DirectVulkan) vs BASE (monolith x DirectVulkan, p7w1) ==")
|
||||
print("rate = Pass/(results-NotSupported); gate: delta >= -%.1f pp and 0 new crash per block" % CTS_TOL_PP)
|
||||
print("%-14s %6s %-26s %-26s %9s %5s %5s %5s %s" % ("block", "cases", "BASE P/NS/F/X rate", "AFTER P/NS/F/X rate", "delta pp", "crash", "regr", "fixed", "verdict"))
|
||||
print("gate (CONTRACT-P7 7.3): rate = Pass/(Pass+Fail) [NS, warnings W and crashes X are not in the denominator];")
|
||||
print(" per block delta >= -%.1f pp and 0 NEW crash; all five blocks required. 'info' = the pre-ruling" % CTS_TOL_PP)
|
||||
print(" Pass/(results-NS) delta, printed for comparison only, never gating.")
|
||||
print("%-14s %6s %-20s %8s %-20s %8s %9s %9s %5s %4s %5s %s" % (
|
||||
"block", "cases", "BASE P/F/NS/W/X", "rate", "AFTER P/F/NS/W/X", "rate", "delta pp", "(info)",
|
||||
"crash", "regr", "fixed", "verdict"))
|
||||
for b in c["blocks"]:
|
||||
if "after" not in b:
|
||||
print("%-14s %s %s" % (b["block"], b["verdict"], "; ".join(b["reasons"])))
|
||||
print("%-14s %s: %s" % (b["block"], b["verdict"], "; ".join(b["reasons"])))
|
||||
continue
|
||||
def cell(r):
|
||||
return "%d/%d/%d/%d %s" % (r["pass"], r["ns"], r["fail"], r["other"],
|
||||
"%.3f%%" % (100 * r["rate"]) if r["rate"] is not None else "n/a")
|
||||
print("%-14s %6d %-26s %-26s %9s %5d %5d %5d %s%s" % (
|
||||
b["block"], b["cases"], cell(b["base"]), cell(b["after"]),
|
||||
"%+.3f" % b["delta_pp"] if b["delta_pp"] is not None else "n/a",
|
||||
|
||||
def counts(r):
|
||||
return "%d/%d/%d/%d/%d" % (r["pass"], r["fail"], r["ns"], r["warn"], r["crash"])
|
||||
|
||||
def pct(v):
|
||||
return "%.3f%%" % (100 * v) if v is not None else "n/a"
|
||||
|
||||
def pp(v):
|
||||
return "%+.3f" % v if v is not None else "n/a"
|
||||
print("%-14s %6d %-20s %8s %-20s %8s %9s %9s %5d %4d %5d %s%s" % (
|
||||
b["block"], b["cases"], counts(b["base"]), pct(b["base"]["rate"]), counts(b["after"]),
|
||||
pct(b["after"]["rate"]), pp(b["delta_pp"]), "(%s)" % pp(b["delta_pp_results_info"]),
|
||||
len(b["new_crash"]), len(b["regressed"]), len(b["fixed"]), b["verdict"],
|
||||
" (SUBSET)" if b["subset"] else ""))
|
||||
for reason in b["reasons"]:
|
||||
@@ -321,7 +511,17 @@ def print_cts(c):
|
||||
print(" ... %d more regressions in the JSON" % (len(b["regressed"]) - 40))
|
||||
for x in b["fixed"][:20]:
|
||||
print(" not->Pass " + x)
|
||||
print("CTS %s" % c["overall"] + (" (SUBSET run: not a gate verdict)" if any(b.get("subset") for b in c["blocks"]) else ""))
|
||||
u = c["ubo"]
|
||||
print("%-14s %s (%s)" % (u["block"], u["reasons"][0], u["cases"]))
|
||||
if c["ignored_reports"]:
|
||||
print(" note: report(s) outside the five gate blocks ignored: " + ", ".join(c["ignored_reports"]))
|
||||
tail = ""
|
||||
if c["incomplete_blocks"]:
|
||||
tail += " (INCOMPLETE: %s)" % ", ".join("%s %s" % (b["block"], b["verdict"]) for b in c["blocks"]
|
||||
if b["block"] in c["incomplete_blocks"])
|
||||
if any(b.get("subset") for b in c["blocks"]):
|
||||
tail += " (SUBSET run: not a gate verdict)"
|
||||
print("CTS %s%s" % (c["overall"], tail))
|
||||
|
||||
|
||||
def print_timing(out):
|
||||
@@ -361,20 +561,25 @@ def main():
|
||||
import compare_actuals as ca
|
||||
base_dir = Path(tools) / "docs/Disaggregated/notes/p7/device-window-1/CTS-base"
|
||||
report = {"stamp": args.stamp, "out": str(out)}
|
||||
ok = True
|
||||
g3 = gate3(out, ca)
|
||||
if g3:
|
||||
print_gate3(g3); print()
|
||||
report["gate3"] = g3; ok &= g3["overall"] == "PASS"
|
||||
print_gate3(g3)
|
||||
else:
|
||||
g3 = {"overall": "NOT-RUN"}
|
||||
print("== GATE 3 ==\nGATE3 NOT-RUN (no gate3/cases.txt)")
|
||||
print()
|
||||
report["gate3"] = g3
|
||||
b = bsl(out, ca)
|
||||
if b:
|
||||
print_bsl(b); print()
|
||||
report["bsl"] = b
|
||||
c = cts(out, base_dir)
|
||||
if c:
|
||||
print_cts(c); print()
|
||||
report["cts"] = c; ok &= c["overall"] == "PASS"
|
||||
print_cts(c); print()
|
||||
report["cts"] = c
|
||||
print_timing(out)
|
||||
ok = g3["overall"] == "PASS" and c["overall"] == "PASS"
|
||||
report["window"] = {"gate3": g3["overall"], "cts": c["overall"], "exit": 0 if ok else 1}
|
||||
print("\nWINDOW2 GATE3 %s | CTS %s -> exit %d" % (g3["overall"], c["overall"], 0 if ok else 1))
|
||||
Path(args.json or out / "verdict.json").write_text(json.dumps(report, indent=1) + "\n", encoding="utf-8")
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
@@ -4,8 +4,10 @@
|
||||
# Puts the phone back the way 21-preflight.sh found it (<out>/session/found.env): Home screen,
|
||||
# the TCP supervisor restarted on the INSTALLED package if it was running (every replay's
|
||||
# `am force-stop` kills it; tcp_device_server.py start with the same listen/token and the saved
|
||||
# Doze state file, which it never overwrites), the pin released if this window set it, and the
|
||||
# stay-on setting as found. Prints the final state; never uninstalls anything.
|
||||
# Doze state file, which it never overwrites), the pin released if this window set it, the
|
||||
# stay-on setting as found, and the $BASE library back in /data/local/tmp/mgcts (50-cts-after.sh
|
||||
# leaves the AFTER one there; pushed and verified by sha256 only when the device differs).
|
||||
# Prints the final state; never uninstalls anything. Idempotent: window.sh runs it on every pass.
|
||||
set -uo pipefail
|
||||
. "$(dirname "$0")/lib.sh"
|
||||
|
||||
@@ -34,6 +36,7 @@ if [ "${FOUND_SUPERVISOR:-0}" = 1 ]; then
|
||||
w2_supervisor_running && log "supervisor restarted on $W2_LISTEN" || log "WARN: supervisor NOT listening (see $OUT/supervisor-restart.txt)"
|
||||
Ash "input keyevent KEYCODE_HOME"
|
||||
fi
|
||||
BASE_LIB_STATE=$(w2_restore_base_lib) || log "WARN: $BASE_LIB_STATE"
|
||||
if [ -f "$OUT/pinned-by-window" ]; then
|
||||
bash "$W2_TOOLS/tools/device_bench/pin_device.sh" "$W2_SERIAL" unpin > "$OUT/pin-session-end.txt" 2>&1
|
||||
rm -f "$OUT/pinned-by-window"
|
||||
@@ -55,6 +58,7 @@ sleep 2
|
||||
echo "pin : $pin (found: ${FOUND_PIN:-?})"
|
||||
echo "stayon : $(Ash 'settings get global stay_on_while_plugged_in') (found: ${FOUND_STAYON:-?})"
|
||||
echo "doze wl : $(Ash 'dumpsys deviceidle whitelist' | grep -i mobilegl | tr '\n' ' ')"
|
||||
echo "$BASE_LIB_STATE"
|
||||
} | tee "$OUT/restored.txt"
|
||||
w2_timing "$(w2_out "$STAMP")" restore "$t0" "$(date +%s)"
|
||||
log "=== RESTORE DONE $STAMP ==="
|
||||
|
||||
@@ -28,8 +28,28 @@ tail -f ~/w7/logs/p7w7/window-<sha8>.log # 结束行 "=== WINDOW 2 DONE <
|
||||
```
|
||||
|
||||
`detach.sh` = `setsid nohup … < /dev/null &`，与终端 / 工具 10 min 上限脱钩。**中断后原命令重跑即续**：
|
||||
每步完成写 `<out>/steps/<step>.done`；门 3 逐 (arm, case) 写 `gate3/state/<arm>/<case>.done`（未完成的那一对整对重跑）；
|
||||
CTS 逐块写 `cts/runs/<block>/.done`。只重跑一例：删它的 `.done` 再跑 `30-gate3.sh <stamp>`。
|
||||
每步完成写 `<out>/steps/<step>.done`；门 3 逐 (arm, case) 写 `gate3/state/<arm>/<case>.done`——**只在该对要求的每一遍都留下
|
||||
`result.json` 与 actual PNG 时才写**，否则日志记 `INCOMPLETE`、`gate3/progress.tsv` 第 6 列 `INCOMPLETE`、不写 `.done`，下次整对重跑
|
||||
（残档先删；遍数按首跑冻结在 `gate3/arms.txt` 的 `repeat=`）；CTS 逐块写 `cts/runs/<block>/.done`——**只在 `cts/report-<block>.json`
|
||||
存在且有结果时才写**。循环跑完仍有 INCOMPLETE 时 `30-gate3.sh` / `50-cts-after.sh` 退出 3：`window.sh` 不给该步写 `.done`、
|
||||
照走后面各步（reduce 与 restore 每次都跑），结尾打 `WINDOW 2 INCOMPLETE … left open`；原命令再跑一次只补缺的——若上一遍已 restore，
|
||||
它先以 `21-preflight.sh <stamp>`（不重启）回到**同一** boot 会话（重新钉频、stay-on；boot_id 变了就停；`reboot-clean.txt` 不改写，
|
||||
`session/resumed.txt` 记一笔）。
|
||||
|
||||
**会话**（CONTRACT-P7 §7.2「同一 reboot-clean 会话」）：`session/boot-id.txt` 是本窗口唯一的会话。`30-gate3.sh` 每对前后读手机
|
||||
boot_id，与它不同立即停；每对的 `.done` 与每遍归档的 `repeat-NN/boot_id.txt` 都记 `boot_id=… boot_id_before=…`；`60-reduce.py`
|
||||
要求四臂所有记录是同一个 boot_id 且等于 `session/boot-id.txt`，否则 `GATE3 INVALID-SESSION`。
|
||||
|
||||
**中断后要删什么**（先比 `adb -s 2f7cbe2e shell cat /proc/sys/kernel/random/boot_id` 与 `<out>/session/boot-id.txt`）：
|
||||
- **boot_id 没变**（主机 / WSL / adb 断、脚本被杀、某步 INCOMPLETE）：**什么都不删**，原命令重跑。没 `.done` 的对 / 块整对 / 整块重跑，
|
||||
残档由脚本先删。只想重跑某个已完成的对：删 `gate3/state/<arm>/<case>.done` 与 `steps/gate3.done`（CTS 块同理：
|
||||
`cts/runs/<block>/.done` 与 `steps/cts.done`），再原命令重跑。
|
||||
- **boot_id 变了（手机重启过）**：门 3 已完成的对全部作废（`30-gate3.sh` 在下一对之前就停；`21-preflight.sh` 不带 `--reboot`
|
||||
也停；带 `--reboot` 时只要 `gate3/state` 里还有 `.done` 就拒绝，防漏删）。删 `gate3/state/`、`gate3/archive/`、`gate3/logs/`、
|
||||
`gate3/progress.tsv`、`gate3/pin-*.txt` 与 `steps/preflight.done`、`steps/gate3.done`，然后原命令重跑（preflight 带 `--reboot`
|
||||
起新会话；`session/found.env`、`gate3/cases.txt`、`gate3/arms.txt` 保留——「找到时」状态、分母与遍数不变）。bsl 与 CTS 不受会话约束，保留。
|
||||
- 或者换 stamp 从头开（`window.sh <pipe-sha> <new-stamp>`），旧目录留作记录。
|
||||
|
||||
不重启：`window.sh <pipe-sha> --no-reboot`（门 3 要 reboot-clean，只在复跑时用）。
|
||||
|
||||
## 2. 步骤（等价的逐条命令）与耗时
|
||||
@@ -39,26 +59,43 @@ CTS 逐块写 `cts/runs/<block>/.done`。只重跑一例：删它的 `.done` 再
|
||||
| 1 | `10-build.sh <sha> [<stamp>]` | `~/w7/p7w7-tree-<sha8>`（pipe 的 detached worktree，子模块从 pipe 拷，LFS fixture 随 checkout）里构建 APK（`assembleTraceRelease`，`.p7w1` 并排 id，debuggable，显式 stamp，debug keystore 签名）与主机制品（`build-host`：`libMobileGL.so`/`libMobileGLServer.so`/`mobilegl_trace_replay` + catalog），两端 stamp 字符串各 ≥1 | 6 min（dry：APK 131 s、host 223 s） |
|
||||
| 2 | `21-preflight.sh <stamp> --reboot` | 记录「找到时」的状态（supervisor、stay-on、钉频、Doze 白名单）→ reboot，前后 boot_id 必须不同 → 唤醒、`svc power stayon true` → `pin_device.sh pin`，`check` 为 `PINNED` 或 `PINNED-1100`（本板 pwrlevel 0 实跑 1100 MHz，窗口 1 已记，脚本期望值过期） | 2–3 min |
|
||||
| 3 | `20-install.sh <stamp>` | `adb install -r`；**设备上 `base.apk` 的 sha256 必须等于签名 APK**，versionName 带 `<sha7>`；等 dex2oat 空闲 | ~1 min |
|
||||
| 4 | `30-gate3.sh <stamp>` | 36 例（`--matrix` 的 DirectVulkan 集减 `create-indirect` / `1.21.11-main-menu` / `photon-v1.3b`，冻结在 `gate3/cases.txt`）× 四臂；iterationrp 带 `apk.yml:460-462` 三个 Magma 旋钮；每遍 result.json / actual PNG / 双角色日志 / transport-proof / logcat 归档；每臂前后 pin check | 55–65 min |
|
||||
| 4 | `30-gate3.sh <stamp>` | 36 例（`--matrix` 的 DirectVulkan 集减 `create-indirect` / `1.21.11-main-menu` / `photon-v1.3b`，冻结在 `gate3/cases.txt`）× 四臂；iterationrp 带 `apk.yml:460-462` 三个 Magma 旋钮；每遍 result.json / actual PNG / 双角色日志 / transport-proof / logcat 归档；每臂前后 pin check；每对前后核 boot_id（§1「会话」），`.done` 只在每遍都有 result.json + actual PNG 时写，否则退出 3 留待续跑 | 55–65 min |
|
||||
| 5 | `40-bsl-stats.sh <stamp>` | bsl-esc-menu × {inproc, spawn} 各 1 遍，`MOBILEGL_PIPE_STATS=1 PERIOD=1`；先 force-stop 再起设备端 root 采样器（0.25 s：maps 行数、VmRSS、VmHWM；排除 argv 带 `tcp://` 的 supervisor server）；`wbuf[]` 从归档日志取 | ~1 min |
|
||||
| 6 | `50-cts-after.sh <stamp>` | APK 的 arm64 `libMobileGL.so` 推到 `mgcts` 并在设备上核 sha256；AFTER 环境 = `$BASE` 的 flag + `MOBILEGL_TRANSPORT=inproc` + `ROLE_SPLIT_STATE=1 STRICT_ERRORS=1 RUN_AHEAD=1`；臂证明（mgprobe PASS 且 logcat 有 inproc 解析句与 `Config: IPC … strict=1 role-split-state=1 run-ahead=1`、0 Fatal；再用一个 glcts 用例证一次）；五块依次跑（顺序同 `$BASE`），每块 `qpa_report --json`，最后 `cts_multi_report` 出与 `$BASE` 同形的 JSON | 6–7 min（pipe 头 lib inproc 实测 358 s；`$BASE` monolith 372 s） |
|
||||
| 7 | `60-reduce.py <stamp>` | 写 `verdict.txt` / `verdict.json`（见 §3） | <10 s |
|
||||
| 8 | `90-restore.sh <stamp>` | Home；`am force-stop` 包（回放后缓存的 app 进程与 spawn 回放留下的 `libMobileGLServer.so @mgl-…` 子进程会一直占着几百 MB）；supervisor 若开窗时在跑则以 `tcp_device_server.py start --allow-idle`（同 listen / token / Doze 状态文件）在**新 APK**上重启并核实在听；解钉频（MIUI 启动 boost 会让 `check` 短暂读成 DRIFT，重读至多 20 s）；stay-on 复原 | ~15 s |
|
||||
| 6 | `50-cts-after.sh <stamp>` | APK 的 arm64 `libMobileGL.so` 推到 `mgcts` 并在设备上核 sha256；AFTER 环境 = `$BASE` 的 flag + `MOBILEGL_TRANSPORT=inproc` + `ROLE_SPLIT_STATE=1 STRICT_ERRORS=1 RUN_AHEAD=1`；臂证明（mgprobe PASS 且 logcat 有 inproc 解析句与 `Config: IPC … strict=1 role-split-state=1 run-ahead=1`、0 Fatal；再用一个 glcts 用例证一次）；五块依次跑（顺序同 `$BASE`），每块 `qpa_report --json`（块的 `.done` 只在 `report-<block>.json` 有结果时写，否则退出 3 留待续跑），最后 `cts_multi_report` 出与 `$BASE` 同形的 JSON；UBO（GTF）块不跑：本 glcts 无 GTF 模块（ID-P7-16） | 6–7 min（pipe 头 lib inproc 实测 358 s；`$BASE` monolith 372 s） |
|
||||
| 7 | `60-reduce.py <stamp>` | 写 `verdict.txt` / `verdict.json`（见 §3）；每遍都跑。自检：`python3 60-reduce-selftest.py`（主机，不占设备） | <10 s |
|
||||
| 8 | `90-restore.sh <stamp>` | Home；`am force-stop` 包（回放后缓存的 app 进程与 spawn 回放留下的 `libMobileGLServer.so @mgl-…` 子进程会一直占着几百 MB）；**把 `$BASE` 库推回 `/data/local/tmp/mgcts`**（设备 sha256 ≠ `$BASE` = `IDENTITY.txt` 的 `7f58aa0b…` 时才推，推后在设备上核 sha，结果行进 `restored.txt`）；supervisor 若开窗时在跑则以 `tcp_device_server.py start --allow-idle`（同 listen / token / Doze 状态文件）在**新 APK**上重启并核实在听；解钉频（MIUI 启动 boost 会让 `check` 短暂读成 DRIFT，重读至多 20 s）；stay-on 复原；每遍都跑（幂等） | ~15 s |
|
||||
|
||||
合计：构建 ~6 min + 设备 ~65–80 min ≈ **1.3–1.5 h**。外推依据见 §5。
|
||||
|
||||
## 3. 判读 `verdict.txt`
|
||||
|
||||
- **门 3**，逐例 `PASS / FAIL / INCOMPLETE` + 原因：monolith 本身通过；inproc 与 spawn 各 3 遍齐全、每遍臂证明通过、
|
||||
每遍 ssim ≥ 阈值、**三遍 actual PNG SHA-256 相同**、每遍 `|ssim − monolith| ≤ 0.0005`；OpenRA 在 monolith / inproc / spawn
|
||||
每遍都 `1.000000` 且 mismatch 0；一遍的 logcat 与上一遍逐字节相同记 `STALE`。`RUN_AHEAD=0` 臂只记录
|
||||
- **门 3**，先看 `-- session` 行：`session/boot-id.txt` 与四臂的每条记录（`gate3/state/<arm>/<case>.done` 的 `boot_id` / `boot_id_before`、
|
||||
每遍 `repeat-NN/boot_id.txt`）必须是**同一个** boot_id；有第二个值、有记录不带 boot_id（含本规则之前的旧 `.done`）、或缺 `session/boot-id.txt`，
|
||||
整门判 **`GATE3 INVALID-SESSION`**，与图无关。会话不是 reboot-clean（`session/reboot-clean.txt`）时末行注明「不是判据」。
|
||||
逐例 `PASS / FAIL / INCOMPLETE` + 原因：monolith 本身通过且**确是 monolith**（该遍 `mobilegl.log`（及 client / server 日志）没有
|
||||
`Config: IPC` 也没有 `MOBILEGL_TRANSPORT=` 行；没有日志即「未证」）；inproc 与 spawn 各 3 遍齐全、该对有 `.done`、每遍臂证明
|
||||
（`transport-proof.json` passed）通过、每遍 ssim ≥ 阈值、**同一臂内三遍 actual PNG SHA-256 相同**、每遍 `|ssim − monolith| ≤ 0.0005`；
|
||||
OpenRA 在 monolith / inproc / spawn 每遍都 `1.000000` 且 mismatch 0；一遍的 logcat 与上一遍逐字节相同记 `STALE`。`-- arm identity`
|
||||
行汇总 monolith 身份与 split 臂 transport-proof 的通过数。`RUN_AHEAD=0` 臂只记录
|
||||
（ssim、SHA、`Config: IPC` 全是 `run-ahead=0` 的证明）；它的 runner rc=1 是 `--require-inproc` 要求 `run-ahead=1` 的已知副作用（E1）。
|
||||
末行 `GATE3 PASS: 36/36` 才是门 3；列 `i==s==m` 表示三臂同图（信息，不设门）。
|
||||
末行 `GATE3 PASS: 36/36` 才是门 3。
|
||||
- **跨臂同图不是门（ruling: integrator 2026-09-23）**：§7.2 的「三遍逐位相同」是**臂内**判据——inproc 自己三遍、spawn 自己三遍各自逐位相同。
|
||||
inproc / spawn / monolith 三臂之间 actual PNG 是否相同只**报告**：逐例列 `i==s==m`（`yes` / `i==s` / `no`），`-- cross-arm` 行给同图例数，
|
||||
不同的例各打一行 `WARN cross-arm <case>: monolith <sha> | inproc <sha> | spawn <sha>`；WARN 不改变该例与门 3 的判定
|
||||
（跨臂的门是每遍 `|ssim − monolith| ≤ 0.0005`）。
|
||||
- **BSL**：每臂每进程（app / server）的 maps 行数峰值、VmRSS / VmHWM 峰值、`wbuf[wbufs wlivepk wdefpk wdefsync]`。
|
||||
- **CTS**：每块 BASE 与 AFTER 的 `P/NS/F/X` 与 rate = Pass / (结果数 − NotSupported)（Fail、Crash、warning 全在分母里），
|
||||
delta pp；块判据 delta ≥ −0.5 pp、**新增 crash = 0**（AFTER 为 Crash/Timeout/InternalError/ResourceError/DeviceHang/Incomplete 而 BASE 不是，另加 hung.txt）、
|
||||
`unrun.txt` 为空。另列 Pass→非 Pass 与非 Pass→Pass 的逐例名单；AFTER 高出 0.5 pp 以上只标注不判红。
|
||||
- `reduce` 的退出码只作记录：红的门也要走完 restore。
|
||||
- **CTS**：五块（shader-image / ssbo / dsa / texture / packed-pixels）**缺一不可**：每块要有 `cts/runs/<block>/.done` 与非空、有结果的
|
||||
`cts/report-<block>.json`，否则该块记 `MISSING`、整体 `CTS INCOMPLETE`、退出码 ≠ 0。UBO 块（`GTF-GL46.gtf31.GL3Tests.uniform_buffer_object*`，
|
||||
`p7-ubo-gl46.txt` 90 例）显式记 `unrun (not in this glcts)`（设备 glcts 无 GTF 模块，ID-P7-16），不入判据。
|
||||
每块 BASE 与 AFTER 的 `P/F/NS/W/X`（W = CompatibilityWarning / QualityWarning 等，X = Crash / Timeout / InternalError / ResourceError / DeviceHang / Incomplete），
|
||||
**rate = Pass / (Pass + Fail)**（§7.3 原文：NS 不入分母；warning 与 crash 也不在分母里），delta pp；块判据 delta ≥ −0.5 pp、
|
||||
**新增 crash = 0 单独硬红**（AFTER 为 X 而 BASE 不是——Fail→Crash 也算——另加 hung.txt；X 也是非 Pass，进 Pass→非 Pass 名单）、`unrun.txt` 为空。
|
||||
`(info)` 列是旧口径 Pass / (结果数 − NS) 的 delta，只供对照，**不判**。另列 Pass→非 Pass 与非 Pass→Pass 的逐例名单；AFTER 高出 0.5 pp 以上只标注不判红。
|
||||
- 末行 `WINDOW2 GATE3 <v> | CTS <v> -> exit <n>`：两门都 PASS 才是 0；门 3 没跑（无 `gate3/cases.txt`）记 `NOT-RUN`，同样非 0。
|
||||
`reduce` 的退出码只作记录：红的门也要走完 restore。
|
||||
- 自检（主机，秒级）：`python3 60-reduce-selftest.py`——造目录验：缺块 → `CTS INCOMPLETE`；boot_id 变 / 旧 `.done` / 缺 `boot-id.txt` →
|
||||
`GATE3 INVALID-SESSION`；monolith 日志带 `Config: IPC` → FAIL；跨臂不同 → WARN 仍 PASS；臂内不同 → FAIL；dry run 数据上的合同口径
|
||||
（`60-reduce-selftest-pre14e1c8b9.json` = pre-14e1c8b9 五块相对 `$BASE` 的差异）dsa −0.272 pp；`~/w7/logs/devprep/w2/pre-14e1c8b9` 在时再对原始输出验一遍。
|
||||
|
||||
## 4. 入库清单 → `docs/Disaggregated/notes/p7/device-window-2/`
|
||||
|
||||
@@ -81,7 +118,7 @@ CTS 逐块写 `cts/runs/<block>/.done`。只重跑一例：删它的 `.done` 再
|
||||
| CTS AFTER dsa 前 20 例 | 部署 p7w6 lib `12861aa1…` 并核 sha；mgprobe PASS；mgprobe 与 glcts 的臂证明 PROVEN；BASE 19/0/0/1 vs AFTER 19/0/0/1，+0.000 pp，0 新 crash；之后 `$BASE` lib `7f58aa0b…` 推回 | 1–2 s/块 |
|
||||
| restore | Home、supervisor 在新起的 `tcp://0.0.0.0:40613` 在听、UNPINNED、stay-on 15、Doze 白名单同开窗 | 5 s |
|
||||
| 构建脚本 @ `14e1c8b9`，stamp `p7w7dry-14e1c8b9`（**未安装**） | APK `f2ca6b02…`（id `.p7w1`、versionName `26.09.14e1c8b-trace`、同 debug 证书），lib stamp 串 1；主机制品 stamp 串 1 | 358 s |
|
||||
| CTS 计时样本：p7w6 inproc 全 ssbo + dsa | ssbo 124/124；dsa 366/0/3/2，−0.539 pp，1 新 crash（见 §6） | 3 s + 9 s |
|
||||
| CTS 计时样本：p7w6 inproc 全 ssbo + dsa | ssbo 124/124；dsa P/F/NS/W/X 366/3/0/1/1，合同口径 −0.272 pp（旧口径 −0.539），1 新 crash（见 §6） | 3 s + 9 s |
|
||||
| CTS 预读：pipe 头 `14e1c8b9` 的 arm64 lib（取自上行未安装的 APK，只推到 `mgcts`）五块全跑 | 见 §6：dsa 与 shader-image 各 1 个新 crash；之后 `$BASE` lib 推回并核 sha | 358 s |
|
||||
|
||||
**外推**：窗口 1 同机同口径的逐例间隔（E0a monolith 39 例 360 s、E0a inproc 342 s、W4 inproc 412 s；最慢 derivative 30 s、
|
||||
@@ -95,12 +132,14 @@ bsl-esc-menu 15–30 s）与本次 dry run 的 7–8 s/例一致，36 例一遍
|
||||
- `KHR-GL46.direct_state_access.renderbuffers_storage`：monolith Pass，inproc **SIGABRT** `Fatal{ReplyTooLarge, "ReadPixels 256x512 0x1908/0x1406 2097152 > 2097136"}`
|
||||
——RGBA/FLOAT 回读恰 2 MiB，比一个 reply 槽的载荷上限（槽 − 16 B 头，`MG_Remote/Transport/ReplySlot.h:166-169`）多 16 B；客户端在发射前具名拒绝
|
||||
（`MG_Remote/Client/ClientSession.cpp:2319`，ID-47：「chunked readback 是 P6 债」）。
|
||||
- `…renderbuffers_storage_multisample`：monolith Pass，inproc Fail。两例合计 dsa −0.539 pp（门限 0.5）+ 1 新 crash。
|
||||
- `…renderbuffers_storage_multisample`：monolith Pass，inproc Fail。两例合计 dsa 按合同口径 Pass/(Pass+Fail) 为 **−0.272 pp**
|
||||
（368/370 → 366/369，在 0.5 pp 内；旧口径 Pass/(结果数 − NS) 为 −0.539 pp）——dsa 红在 1 个新 crash，不在 rate。
|
||||
新 reducer 在该数据上的读数：`docs/Disaggregated/notes/p7/device-window-2/dry-run/pre-14e1c8b9-cts/verdict-contract.txt`。
|
||||
- `KHR-GL46.shader_image_load_store.incomplete_textures`：monolith Fail（`$BASE` 亦非 Pass），inproc **SIGABRT** `Fatal{UnmigratedVerb, "Magma:image-view-window"}`
|
||||
（`MG_Backend/DirectVulkan/Renderer/UniformManager.cpp:151`）——Fail 变 Crash，按门 5 是新 crash。
|
||||
- `…shader_image_load_store.advanced-sync-vertexArray`：`$BASE` Pass，pipe 头 monolith 与 inproc 都 Fail——不是分离缺陷，是 p7w1 之后两臂共有的漂移。
|
||||
- `KHR-GL46.texture_border_clamp.Texture2DArrayCompressed`：整块里 Fail 一次，单跑三臂都 Pass——抖动。
|
||||
- 其余：ssbo、packed-pixels 与 `$BASE` 相同；texture +0.96 pp；shader-image 整体 +2.9 pp（3 例修好）。
|
||||
- 其余：ssbo、packed-pixels 与 `$BASE` 相同；texture +0.960 pp；shader-image 整体 +4.284 pp（合同口径；旧口径 +2.9，3 例修好）。
|
||||
终局窗口之前要么修（ReadPixels 分块或放大 reply 槽；`image-view-window` 的 incomplete-texture 形状 decline 成 GL 语义而不是 Fatal），要么由集成者裁定。
|
||||
- dry run 暴露并已修的两处：`run_android_retrace_local.py` 只会用 Git Bash 起跑（WSL 下每例 FileNotFoundError），且一遍没产出时会归档上一遍的图（假「逐位相同」）——`fe4c8745`，带 red-once 单测；
|
||||
脚本自身的 `pipefail` + `grep -q` 把在跑的 supervisor 读成「不在」——已改为先取输出再匹配。
|
||||
|
||||
@@ -92,6 +92,40 @@ w2_supervisor_running() {
|
||||
return 1
|
||||
}
|
||||
|
||||
# ---- one boot session (CONTRACT-P7 7.2; ruling: integrator 2026-09-23) ------------------------
|
||||
# The phone's current boot_id ('' when adb cannot read it).
|
||||
w2_boot_id() { timeout 20 adb -s "$W2_SERIAL" shell cat /proc/sys/kernel/random/boot_id 2>/dev/null | tr -d '\r\n '; }
|
||||
# The window's session boot_id, as 21-preflight.sh recorded it.
|
||||
w2_session_boot_id() {
|
||||
local f; f="$(w2_out "$1")/session/boot-id.txt"
|
||||
if [ -f "$f" ]; then tr -d '\r\n ' < "$f"; fi
|
||||
}
|
||||
# <session-boot-id> <where> [<current>]: die unless the phone is still in that boot session.
|
||||
w2_same_session() {
|
||||
local now=${3-$(w2_boot_id)}
|
||||
[ -n "$now" ] || die "$2: cannot read the phone's boot_id (adb)"
|
||||
[ "$now" = "$1" ] || die "$2: the phone left this window's session (boot_id $1 -> $now; it rebooted). Gate 3 is a one-session reading: its finished pairs are void. RUNBOOK section 1 'phone rebooted' says what to delete before a new --reboot session"
|
||||
}
|
||||
|
||||
# ---- the $BASE library back into mgcts ---------------------------------------------------------
|
||||
# 50-cts-after.sh leaves the AFTER library in $W2_CTS_DEV; whoever uses mgcts next expects $BASE
|
||||
# (device-window-1/CTS-base, the p7w1 monolith). Push the bundle's lib unless the device already has
|
||||
# it, verify by sha256 on the device, print one status line. 0 = the device carries $BASE.
|
||||
w2_restore_base_lib() {
|
||||
local want have got
|
||||
want=$(awk '/^libMobileGL.so *sha256:/ {print $3}' "$W2_CTS_BUNDLE/IDENTITY.txt" 2>/dev/null)
|
||||
[ -n "$want" ] || { echo "mgcts lib : NOT restored - no \$BASE sha256 in $W2_CTS_BUNDLE/IDENTITY.txt"; return 1; }
|
||||
[ "$(Ash "test -d $W2_CTS_DEV && echo ok")" = ok ] || { echo "mgcts lib : no $W2_CTS_DEV on the device, nothing to restore"; return 0; }
|
||||
got=$(Ash "sha256sum $W2_CTS_DEV/libMobileGL.so 2>/dev/null" | cut -d' ' -f1)
|
||||
if [ "$got" = "$want" ]; then echo "mgcts lib : \$BASE $want (already)"; return 0; fi
|
||||
have=$(sha256sum "$W2_CTS_BUNDLE/libMobileGL.so" 2>/dev/null | cut -d' ' -f1)
|
||||
[ "$have" = "$want" ] || { echo "mgcts lib : NOT restored - bundle lib ${have:-missing} != \$BASE $want"; return 1; }
|
||||
A push "$W2_CTS_BUNDLE/libMobileGL.so" "$W2_CTS_DEV/libMobileGL.so" > /dev/null 2>&1
|
||||
local now; now=$(Ash "sha256sum $W2_CTS_DEV/libMobileGL.so 2>/dev/null" | cut -d' ' -f1)
|
||||
if [ "$now" = "$want" ]; then echo "mgcts lib : \$BASE $want restored (was ${got:-absent})"; return 0; fi
|
||||
echo "mgcts lib : RESTORE FAILED (${now:-absent} != \$BASE $want)"; return 1
|
||||
}
|
||||
|
||||
w2_timing() { # <out> <step> <start-epoch> <end-epoch> [note]
|
||||
printf '%s\t%s\t%s\t%s\t%s\n' "$2" "$(date -d @"$3" +%FT%T)" "$(date -d @"$4" +%FT%T)" \
|
||||
"$(( $4 - $3 ))" "${5:-}" >> "$1/timing.tsv"
|
||||
|
||||
@@ -9,9 +9,13 @@
|
||||
# 40-bsl-stats.sh bsl-esc-menu x {inproc, spawn} with MOBILEGL_PIPE_STATS + maps sampler
|
||||
# 50-cts-after.sh five KHR-GL46 blocks, inproc x DirectVulkan, AFTER lib deployed to mgcts
|
||||
# 60-reduce.py gate-3 verdict per case, bsl peaks, CTS delta table + new-crash list
|
||||
# 90-restore.sh Home, supervisor as found, unpin, stay-on as found
|
||||
# 90-restore.sh Home, supervisor as found, unpin, stay-on as found, $BASE lib back in mgcts
|
||||
# Each finished step leaves <out>/steps/<step>.done; a re-run skips it (gate3 / cts also resume
|
||||
# inside a step). A step that fails stops the sequence; fix the cause and run the same command.
|
||||
# inside a step). A step that exits 3 (gate3 / cts: loop finished, some pairs / blocks INCOMPLETE)
|
||||
# is NOT marked done and the sequence carries on; a re-run of the same command retries only what
|
||||
# is missing (after re-entering the same boot session with 21-preflight.sh, no reboot, if the
|
||||
# phone was already restored). reduce and restore run on every pass. Any other failure stops the
|
||||
# sequence; fix the cause and run the same command (RUNBOOK section 1).
|
||||
set -euo pipefail
|
||||
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
||||
. "$HERE/lib.sh"
|
||||
@@ -33,13 +37,21 @@ OUT=$(w2_out "$STAMP")
|
||||
mkdir -p "$OUT/steps"
|
||||
log "=== WINDOW 2 START sha=$SHA stamp=$STAMP out=$OUT ==="
|
||||
|
||||
PARTIAL=""
|
||||
skipped() { case "$SKIP" in *",$1,"*) return 0;; esac; return 1; }
|
||||
step() { # <name> <command...>
|
||||
local name=$1; shift
|
||||
if [ -f "$OUT/steps/$name.done" ]; then log "--- $name: done earlier, skipping"; return 0; fi
|
||||
case "$SKIP" in *",$name,"*) log "--- $name: skipped by --skip"; return 0;; esac
|
||||
if skipped "$name"; then log "--- $name: skipped by --skip"; return 0; fi
|
||||
log "--- $name: $*"
|
||||
local t0; t0=$(date +%s)
|
||||
"$@"
|
||||
local t0 rc=0; t0=$(date +%s)
|
||||
"$@" || rc=$?
|
||||
if [ $rc -eq 3 ]; then
|
||||
PARTIAL="$PARTIAL $name"
|
||||
log "--- $name: INCOMPLETE after $(( $(date +%s) - t0 ))s - not marked done; the next run of this command retries what is missing"
|
||||
return 0
|
||||
fi
|
||||
[ $rc -eq 0 ] || die "--- $name failed (rc=$rc); fix the cause and run the same command"
|
||||
date -Is > "$OUT/steps/$name.done"
|
||||
log "--- $name: OK in $(( $(date +%s) - t0 ))s"
|
||||
}
|
||||
@@ -47,11 +59,29 @@ step() { # <name> <command...>
|
||||
step build bash "$HERE/10-build.sh" "$SHA" "$STAMP"
|
||||
step preflight bash "$HERE/21-preflight.sh" "$STAMP" $REBOOT
|
||||
step install bash "$HERE/20-install.sh" "$STAMP"
|
||||
# A previous pass restored the phone (unpinned, stay-on as found) with device steps still open:
|
||||
# re-enter the SAME boot session first (no reboot; 21-preflight.sh stops if the phone rebooted).
|
||||
if [ -f "$OUT/steps/restore.done" ]; then
|
||||
for s in gate3 bsl cts; do
|
||||
if [ ! -f "$OUT/steps/$s.done" ] && ! skipped "$s"; then
|
||||
log "--- resume: $s still open after a restore; re-entering the session"
|
||||
bash "$HERE/21-preflight.sh" "$STAMP"
|
||||
rm -f "$OUT/steps/restore.done"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
step gate3 bash "$HERE/30-gate3.sh" "$STAMP"
|
||||
step bsl bash "$HERE/40-bsl-stats.sh" "$STAMP"
|
||||
step cts bash "$HERE/50-cts-after.sh" "$STAMP"
|
||||
# The verdict is data, not a stop condition: a red gate still has to restore the phone.
|
||||
# The verdict is data, not a stop condition: a red gate still has to restore the phone. Both run on
|
||||
# every pass (cheap, idempotent), so a resumed window re-reduces and re-restores.
|
||||
rm -f "$OUT/steps/reduce.done" "$OUT/steps/restore.done"
|
||||
step reduce bash -c "python3 '$HERE/60-reduce.py' '$STAMP' > '$OUT/verdict.txt' 2>&1; echo reduce-rc=\$? >> '$OUT/verdict.txt'"
|
||||
step restore bash "$HERE/90-restore.sh" "$STAMP"
|
||||
log "=== WINDOW 2 DONE $STAMP: verdict in $OUT/verdict.txt ==="
|
||||
if [ -n "$PARTIAL" ]; then
|
||||
log "=== WINDOW 2 INCOMPLETE $STAMP: step(s)$PARTIAL left open; run the same command again (RUNBOOK section 1). verdict so far in $OUT/verdict.txt ==="
|
||||
else
|
||||
log "=== WINDOW 2 DONE $STAMP: verdict in $OUT/verdict.txt ==="
|
||||
fi
|
||||
cat "$OUT/verdict.txt"
|
||||
|
||||
Reference in New Issue
Block a user