Files
MobileGL/docs/Disaggregated/notes/p1/p1-results/verify-review-v2.md
T
swung0x48 aba329cc53 [Docs] (Disaggregated): P6's plan and contract draft; the working notes move from a local directory into the repository
Two things land together: the plan and contract draft P6 needs before it starts, and the
working notes that until now existed on exactly one machine.

1. P6's plan and contract draft (neither started; not one line of code)

     docs/Disaggregated/P6-SPAWN-PLAN.md      the package plan
     docs/Disaggregated/P6-CONTRACT-DRAFT.md  the contract draft (moves to
                                              MG_Remote/CONTRACT-P6.md when c6 lands)

   The three findings that mattered, all written into the documents:

   a. P6 is SMALLER than its roadmap cell suggests. The transport primitives are already in
      the tree and already tested: SocketDoorbell including hangup detection and the two
      lost-wakeup fences (Doorbell.cpp:211-360, four cases in FdPassingTest), SCM_RIGHTS fd
      passing, ShmSegment::Adopt, and the SurfaceOp / WindowKind schema already defined in
      protocol.fbs. inproc uses a real second shm mapping rather than an alias, and
      SessionRings.h:22-28 says that was done so today is not the first time it runs. P6 is
      assembly plus a control plane plus a process, not "write a socket transport".

   b. The evidence for "P6 is only a transport swap" has EXPIRED. It was bought by P5c, whose
      audit found 59 direct memory accesses in a tree that already looked clean. P5e changed
      the tree again. So the first package, a6, is a read-only audit that writes no code, and
      every contract row that depends on it is marked PENDING a6 rather than decided in
      advance.

   c. P6 does not get a game running on the phone. ANativeWindow* is meaningless in the child,
      and protocol.fbs:186-196's own comment says Android transfers the window out of band -
      that band is P12. P6 lands pbuffer / surfaceless / offscreen, which is what HeadlessGL,
      the split-equivalent lane and trace replay need, and which is exactly what P6's own exit
      gate measures. A real window reaching the server is a named refusal. This gets its own
      section because "P6 is done" would otherwise read as "the game runs now".

   The contract adds rule G: a role may not name a process-local handle. It extends P5c's
   rule E (a role may not name the other role's memory) from memory to identity - EGLDisplay /
   EGLSurface / EGLContext / ANativeWindow* / fd number / pid / mapped address either cross as
   a token this contract defines or they do not cross, and a descriptor crosses only through
   SCM_RIGHTS, where the kernel renumbers it.

   What P5e left for P6 gets its own section. The latch semantics are already designed
   (ARCHITECTURE.md:504) and are copied unchanged. What is new is that after run-ahead the
   client no longer waits while the peer process can DIE: the client holds records that were
   published and will never be applied, and Android's freezer suspends the child without
   closing anything. So slow and dead must be distinguishable WITHOUT a timeout threshold.
   That is P6's only new design; everything else is carriage. It shares a predicate with E1's
   outstanding re-specification (ID-122), and the two should be designed together.

   Two gates carried over from P5e's lessons: G2 requires integration-spawn's test-name set to
   match integration-split's name for name, because a differing set means the lane is quietly
   running less; and the arm-proof gate requires every entry to record the child pid and
   transport=spawn in its OWN private log - ID-124 established that this kind of false green
   really happens, and a spawn lane that actually ran monolith would pass every other gate.
   S2's shape must be settled before it is written, and the document says why: E1 is the
   cautionary case.

2. The working notes move into the repository (docs/Disaggregated/notes/, 415 .md files)

   ROADMAP / CURRENT_STAGE_PROGRESS / MEASUREMENTS cite ruling numbers, audit line numbers and
   the provenance of gate figures from ~/w7/notes/, a directory that exists on one machine and
   is under no version control. A record cited hundreds of times that lives only locally is not
   a record, so it moves in whole, with its directory structure intact.

   Only .md moves. Scripts, run logs, census and gate result JSON, patches, APKs and nm dumps
   stay where they are: they are evidence DATA rather than documents - large, regenerated per
   run, and mostly carrying absolute paths and device serials that mean something only on the
   machine that produced them. The mapping and this split are written in
   docs/Disaggregated/notes/README.md.

   Two exceptions are recorded there too: P6's BRIEF has been promoted to a first-class
   document and keeps no copy under notes/, so the two cannot drift apart; and recovered/ holds
   material salvaged from failed workflow runs, which may duplicate the like-named phase
   directories and is kept because a few of those were the only complete version at the time.

   CI's citation lint takes docs/Disaggregated/*.md only (non-recursive), and notes/ is
   deliberately outside it: those notes' file:line citations were written against their own
   code heads, and resolving them against today's HEAD would fail wholesale - which is not an
   error, it is their correct state as a historical record. To check one, use the revision it
   declares (--rev).

3. Inconsistencies fixed in passing

   - README.md's status line still said P5e was in progress while the stage table already said
     closed.
   - CURRENT_STAGE_PROGRESS's next-step item 1 still listed the exit gate as pending, though
     (a) and (b) were run in the previous commit; rewritten to closed plus the one item that
     actually remains (E1).
   - ROADMAP's P6 row still had the pre-phase plan in three of its cells; it now carries the
     package order, the scope boundary and the two new gates.

Checked with check_doc_citations.py: 125 citations across docs/Disaggregated/*.md, 0 problems.
2026-09-18 23:31:39 -04:00

70 lines
15 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Adversarial review — package E `p1-verify` (v2)
Tree `~/w7/p1-verify`, branch `p1/verify` = `eec8cf2c`, eight commits on `p1/contract` (`bf86b1ed`): the four of v1 plus `6060690a`, `01300e6a`, `ce26029c`, `eec8cf2c`. Tree left exactly as found (the four untracked `build-*` directories the implementer created; no file touched; the only side effect of this review is a rebuilt-and-relinked `MobileGLIntegrationTest` in `build-linux`/`build-push`, see minor 4, and fresh `pipe-*.log*` files under `build-verify/MobileGL/MG_IntegrationTest/`). Scripts and raw outputs: `scratchpad/wf4/verify-review2/s01.sh … s11.sh`, `jobdiff.py`, and `~/w7/p1-verify-review2/`.
**Verdict: APPROVED — 0 majors, 5 minors.** The three v1 majors are fixed and the fixes were re-derived independently (not just re-read): the static-symtab gate finds `MGPipeFillForVerb` today and `MGPipeVerifyInputs` exists in `p1/core`; the arming assertion has a one-process lane and the CI step reads only those logs; `gate_failures` now fails on all four buckets and on a `.text` that moved either way. Everything else the package claims holds under re-run, and the deeper hunts (the `.so` swap's linkage, `ctest --timeout` vs a `TIMEOUT` property, log flushing before `std::abort()`, both backends reading `GetActiveTextureUnit` at `GenerateMipmap`, the ambient `Verify.` lanes being functionally green on this tree, the per-job YAML diffs against their originals) found nothing that can make a lane red or green for the wrong reason.
## 1. What was re-run independently (command → observed)
All in `~/w7/p1-verify` unless stated; `CCACHE_BASEDIR=/home/swung/w7`.
| # | command | observed |
|---|---|---|
| 1 | `git status --short`; `git log --oneline p1/contract..HEAD`; `git diff --stat p1/contract..HEAD` | clean apart from `build-*`; 8 commits; 9 files, +1738/−14 — exactly E's row of C.5. Nothing under `MG_Backend/**`, `MG_Test/**`, `MG_Pipe/**`, root `CMakeLists.txt`, `Config.h`, `ConfigLoader.cpp`. Every touched file is LF (`grep -c $'\r'` = 0 ×9). |
| 2 | `cmake --build build-{linux,push,verify} -j 12` | `build-verify`: no work; `build-linux`/`build-push`: `[2/2] Linking CXX executable MobileGL/MG_IntegrationTest/MobileGLIntegrationTest` (minor 4). The library itself did not change (row 3). |
| 3 | `symbol_report.py --before ~/w7/p1-before-libMobileGL.so --after build-linux/libMobileGL.so --threshold 0 --fail-on-symbol-set-change --fail-on-added-bytes 0` | rc 0; `.text 10792579 -> 10792579 (+0)`; `27799 -> 27799: 0 added, 0 removed, 0 resized, 0 renamed`. G1 holds under the now-strict gate. |
| 4 | `grep -rc pGLContext MobileGL/MG_Backend \| sort \| diff - ~/w7/p1-before-pglcontext.txt` | only the two `:0` lines for the contract's `MGPipe/PipeInputs.{h,cpp}`. E converts no site, touches no site (G2 is B/C's). |
| 5 | `python3 scripts/gen_pipe.py --check` | rc 0: `63 verify payloads, 63 PipeInputs fields (7 sticky), 69 verbs, 9 classes`, `0 UNMAPPED`, up to date. `--self-test` → rc 2 here (contract tree); on `~/w7/p1-core` (read-only, `d9f4698d`) → `6 negative-control trip(s), positive control OK`, rc 0 — the `pipe-gates` step E added will be green once core lands. |
| 6 | `python3 scripts/symbol_report.py --self-test` | `OK (2 canned transcripts, 5 buckets, 3 gates)`. |
| 7 | `ctest -L unit --no-tests=error -j 12` in build-linux / build-push / build-verify | `100% tests passed, 0 tests failed out of 1466` ×3 (after the relink of row 2). |
| 8 | `ctest --test-dir build-linux -N` vs `~/w7/p1-before-ctest-names.txt` (`comm`) | removed 0, added 10 (the five new cases × 2 backends). G7 additions only. |
| 9 | `ctest --test-dir build-verify -N -L integration-verify` / `-L integration-gpu`; build-linux `-L integration-gpu` | **818** / 1696 / 878. The 16 non-ambient-or-new names listed: `Verify.` ×10, `VerifyArming.` ×2, `VerifyCorrupted.` ×2, `PoisonOmitted.` ×2. |
| 10 | `ctest --test-dir build-linux -L integration-verify --no-tests=error` | `Total Tests: 0`, rc 8 — G8 half two holds. |
| 11 | `ctest --test-dir build-retrace -N -L retrace` | 80. |
| 12 | env-knob census over `build-verify/MobileGL/MG_IntegrationTest/*.cmake` | `818 MOBILEGL_PIPE_VERIFY=1`, `2 …_CORRUPT=GetRenderStateParameters`, `2 …_FATAL=0`, `2 …_POISON_OMIT=GenerateMipmap:GetActiveTextureUnit`, `2 MGITEST_PIPE_ARMING_LANE=1`. No ambient entry names a control knob or the arming marker — the C.4 ENVIRONMENT constraint holds. |
| 13 | `ctest --test-dir build-verify -R 'PipeVerifyArmingScenario\|PoisonOmissionScenario'` (26 entries) | pull-prefixed ×10: all ***Skipped**; ambient `Verify.` ×10: 8 ***Skipped**, both `WithoutOmissionCompletes` **Passed** (0.44/0.39 s); `VerifyArming.` ×2, `VerifyCorrupted.` ×2, `PoisonOmitted.` ×2: all ***Failed**. **The arming case skips or fails on the contract tree; it never passes.** |
| 14 | `ctest … -R '^DirectGLES\.VerifyArming\.' --output-on-failure` | fails at `PipeVerifyArmingScenario.cpp:211` with "the library never reported arming the comparator" — the right reason (no comparator until core c4), not a GPU/harness failure. `pipe-verify-arming-{DirectGLES,DirectVulkan}.log` exist (318/330 lines, 0 arming lines) — so the CI arming step would red this tree for the right reason too. |
| 15 | `ctest --test-dir build-verify -L integration-verify -E 'VerifyArming\.\|VerifyCorrupted\.\|PoisonOmitted\.' -j 8 --no-tests=error` (the two ambient `Verify.` lanes, 812 entries, **not run by the implementer**) | rc 0, `100% tests passed, 0 tests failed out of 812`, 77 skipped. The registration itself (per-lane `MOBILEGL_LOG_FILE_PATH`, `TIMEOUT 900`, `-j 8` on a shared log path) breaks no existing scenario. |
| 16 | control filters: `ctest -N -L integration-verify -R 'DirectGLES\.Verify\..*ClearThenReadPixels'` / `-R 'DirectGLES\.Verify\.PoisonOmissionScenario\.WithoutOmissionCompletes'` | 5 entries (all five `ClearThenReadPixelsScenario` cases, each issues a clear on DirectGLES) / 1 entry. Neither control can pass on an empty selection. |
| 17 | `trace_cases.py --ci --format github-verify-matrix` | 16 entries over exactly the 8 named cases; `github-test-matrix` 77. |
| 18 | `readelf -d build-verify/libMobileGL.so \| grep -E NEEDED\|RUNPATH` | NEEDED = `libvulkan.so.1 libstdc++.so.6 libm.so.6 libgcc_s.so.1 libc.so.6 ld-linux-x86-64.so.2`; no RUNPATH. The `cp build-verify/libMobileGL.so build-linux/libMobileGL.so` swap in `retrace-verify` is sound — the library pulls nothing from its own build tree (`libSPIRV-Tools-shared.so`/`libEGL.so` in the tree are not NEEDED). `build-retrace`'s `CTestTestfile.cmake` freezes `MOBILEGL_LIBRARY=<tree>/build-verify/libMobileGL.so` here (local configure) / `build-linux/` in CI, as the step assumes. |
| 19 | `grep -c TIMEOUT build-retrace/tools/trace_replay/CTestTestfile.cmake` → 0; empirical: a scratch project with `TIMEOUT 1` on a 3 s test under `ctest --timeout 100` → `***Timeout 1.00 sec`, the property-less sibling passes | `--timeout` only governs tests WITHOUT a `TIMEOUT` property, and the retrace tests have none (`timeout_seconds` in the manifest is not emitted as a property), so `--timeout 10800` in `retrace-verify` is effective. Not a defect; recorded because the opposite would have been a lane that cannot pass. |
| 20 | `MG_Util/Debug/Log.cpp:81-84` | every write is `fputs` + `fflush(s_logFile)`, so the `MGLOG_F` line precedes `std::abort()` in the child log the omission scenario reads. `p1/core` prints `MGPipe: Fatal{PipeVerifyDiffer, "%s@%s", verb=%llu, where=%s}` unconditionally and aborts only when `g_verify.Fatal` (`PipeFill.cpp:384-391`), `Fatal{UnmigratedPipeInput, "%s@%s"}` at `PipeFilled.inc:308`, `MGPipe: verify armed - …` at `PipeFill.cpp:377`, `poison omission armed - %s@%s` at `:452` — every string E greps for is the string core prints. |
| 21 | `VulkanRenderer.cpp:10963` and `DirectGLES.cpp:6932` | both backends' `GenerateMipmap` read `pGLContext->GetActiveTextureUnit()` first, so `<backend>.PoisonOmitted.` can go green on both once the poison exists; `grep -rln glGenerateMipmap MobileGL/MG_IntegrationTest` → only `PoisonOmissionScenario.cpp` (the "only possible target" claim holds). |
| 22 | `jobdiff.py`: per-job YAML diffs `build-linux`→`build-linux-verify`, `integration`→`integration-verify`, `retrace`→`retrace-verify`; `yaml.safe_load` with BOM | only the declared changes: BUILD_DIR/artifact names, `timeout-minutes`, `-DMOBILEGL_PIPE_VERIFY=ON` + `BENCHMARK=OFF` + Release-always, the nm gate, the arming step, the two controls, the log upload; `retrace-verify` drops only the iterationrp exports (that case is not in the verify set) and adds `MOBILEGL_PIPE_VERIFY=1` + `--timeout 10800`. `retrace` (pull) is byte-identical to the contract's. `trace-cases`, `remove-artifact-clutter`, `pipe-gates` changed as declared. 17 jobs. |
| 23 | `bash -eo pipefail -c 'matched=$(echo "" \| grep -cE "^ *Test *#[0-9]+:"); echo reached'` | rc 1, nothing printed (minor 2). |
| 24 | `git ls-tree p1/contract scripts/symbol_report.py` vs `git ls-files -s` | `100755` → `100644` (minor 1). |
## 2. Majors
None.
The three v1 majors, each re-tested rather than re-read:
- **M1 (nm -D)** — `test.yml` `build-linux-verify` "The verify library really carries the comparator" and `retrace-verify` "Unpack the VERIFY runtime" now use `nm --defined-only`; `nm --defined-only build-verify/libMobileGL.so | grep -c MGPipeFillForVerb` → 1 here, and `MGPipeVerifyInputs` is a defined function in `p1/core` (`PipeInputs.cpp:160`), so both greps turn green at integration. The added "looks stripped" floor (`< 1000` defined symbols) closes the third silent way the gate could red.
- **M2 (arming step)** — `PipeVerifyArmingScenario.Armed` now runs only where `MGITEST_PIPE_ARMING_LANE` is set (`DirectGLES.VerifyArming.` / `DirectVulkan.VerifyArming.`, own log path), skips in the ambient lanes (row 13), and the CI step greps only `pipe-verify-arming-*.log` and requires one per backend. Reproduced the fix's premise (row 14) and the old failure (the ambient lane's last entry writes a 317-line log with no verb). The step's name and text now claim per-lane arming, which is what it proves.
- **M3 (gate_failures)** — `gate_failures([],[],[],[],-4096,0,True)` → one reason; resized-only and renamed-only under `--fail-on-symbol-set-change` → one reason each; the gates read threshold-0 buckets regardless of `--threshold`; the self-test drives all three shapes (row 6). The CI invocation is unchanged and now means G1.
## 3. Minors
1. **`scripts/symbol_report.py` lost its executable bit** (`100755` at `p1/contract` → `100644` at HEAD, introduced in `09804491`, carried by `01300e6a`). Undeclared; inert for CI (`python3 scripts/symbol_report.py` everywhere, `test.yml:1398,1506`) but a mode regression the integrator's rebase will carry into `feat/disaggregated`. `git update-index --chmod=+x` before or at integration.
2. **The two negative-control steps die before their diagnostic when the filter matches nothing.** `test.yml` control A/B: `matched=$(ctest -N … | grep -cE '^ *Test *#[0-9]+:')` under GitHub's default `bash -eo pipefail` — `grep -c` exits 1 on zero matches, the assignment fails, `set -e` ends the step before `if [ "${matched}" -lt 1 ]` prints `::error::negative control … selected 0 tests`. Reproduced (row 23). The outcome is still red (correct), so this is a message defect only; `grep -cE … || true` fixes it.
3. **The full 79-case verify sweep "on `workflow_dispatch`" (brief E, "Verify cost" row) is not implemented and not declared.** `retrace-verify`'s matrix is always `verify-matrix` (8 cases); there is no dispatch input or job that runs the 77-entry matrix under the verify library. C.4 itself asks only for the subset, so this is a documentation gap against the risk table rather than against the file list; the D.3 WSL run covers the P1 exit.
4. **`build-linux` and `build-push` were not current with HEAD on arrival** — `cmake --build` relinked `MobileGLIntegrationTest` in both (row 2), so the result file's row 1 "rc 0 ×3" and the unit rows were measured before the final scenario edit landed in those two trees (the library was unchanged, and the unit suites pass after the relink, row 7). Cosmetic; noted so the integrator does not read the implementer's table as the final-tree state for those two builds.
5. **Verify-cost arithmetic on the retrace subset.** The eight `"verify": true` cases include two `timeout_seconds: 1800` cases (`minecraft-1.21.4-fabric-sodium-in-world`, `improved-transparency-minecraft-26.3`) and one default-900 (`minecraft-1.21.4-in-world`); at the design's 5-10× a 1800 s case is 9000-18000 s against `--timeout 10800` and `timeout-minutes: 240` (14400 s). Both numbers are the brief's (C.4/E), and `--timeout` is effective here (row 19), so nothing is mis-wired — but the first real run may show the budget itself is short for those two cases, and the fix then is a `trace_cases.json` decision, not E's plumbing.
Carried from v1 as observations, not defects: deviation 5 (`integration-verify` does not repeat the `MOBILEGL_ESPRYT_DISABLE_INVALIDATE_FLUSH=1` pass) is now written into the step comment; the `Fatal{PipeVerifyBadKnob` regex addition in `run_trace_case.cmake` is correct and matches core's spelling (`PipeFill.cpp:284`).
## 4. Brief vs tree (facts recorded; tree wins)
- `integration-verify` is **818** entries (2 × 406 ambient + 2 arming + 2 corrupted + 2 omitted), not the brief's 742/752; the pull `integration-gpu` set is 878; the fold parity holds trivially.
- `LABELS "integration-gpu\;integration-verify"` escaping, the verify block after `include(GoogleTest)` at `:395`, the `'^ *Test *#[0-9]+:'` count regex — as the result file states.
- The brief's D.3 G5 line `-R 'Mipmap'` selects nothing (row 16 shows the only matching name is `…WithoutOmissionCompletes`); the result file hands the integrator the corrected line.
- The brief's C.4 has the arming case ride the ambient lanes; the tree gives it a lane (deviation 3 in the result file). Justified: the ambient log path is truncated by every process, so the brief's own design would have been unsound.
- `gen_pipe.py --self-test` is a core c5 deliverable; present and green in `~/w7/p1-core` today (row 5).
- `timeout_seconds` in `trace_cases.json` is not a ctest `TIMEOUT` property (row 19) — the brief's "1800 s cases already sit above ctest's 1500 s default" is the right mental model and `--timeout 10800` is the right knob.
## 5. What was not re-run
The full 818-entry lane with the controls (six entries are red by design on this tree; the 812 ambient entries were run, row 15), any real retrace under `MOBILEGL_PIPE_VERIFY` (no comparator on the contract tree; `run_trace_case.cmake`'s block was driven by the implementer on synthetic logs and its `pipe_verify_fatals` regex and arming `string(FIND)` were re-read), and any CI dispatch (no push). None of the approved claims depends on those: the CI mode's correctness on the integrated tree is D.3's job, and every mechanism it needs from E has been shown here to red for its stated reason and to green under the strings core actually prints.