[MG_IntegrationTest, MG_Util, MG_Backend, CI, Docs] (Disaggregated): P5e ID-119/ID-115 - the lane reads every log, counts, and has a positive control

Three measured defects in the strict lane, one commit each in spirit and one in fact because
they are the same step.

1. THE LOG SET CAME FROM A DIRECTORY. The CI step grepped MG_IntegrationTest/split-logs/, which
   holds 90 of the lane's entries. The ones it could not see were the F1. readback block, both
   NamedBlit pairs, the four Ct. entries and PersistentMapArm - precisely the readback population
   the allowlist is ABOUT, plus the only DirectVulkan entries in the lane. split_log_paths.py
   gains a `markers` mode that walks ctest's own {entry: MOBILEGL_LOG_FILE_PATH} map and
   classifies every Fatal{ and Admitted{ line. Measured on this head: 113 of 113 entries read.
   The lane is now a TWO-SIDED RATCHET - an unadmitted marker fails, an admitted pair not in
   Harness/strict-expected-markers.txt fails, AND an expected pair that no longer appears fails,
   so the lane cannot rot green by outliving its own debts.

2. THE `rsp` CHECK GREPPED FOR A COUNTER NO SPLIT ENTRY EMITS, and the pin behind it was vacuous:
   CountBarrierPull's unbarriered arm is [[noreturn]] and runs BEFORE ++g_residualPulls, so
   "rsp = 0 on unbarriered records" was true of every possible implementation. Replaced by an
   assertion where the counter actually is - one entry, DirectGLES.Split.StrictArming., with
   MOBILEGL_PIPE_STATS=1, MOBILEGL_PIPE_STATS_PERIOD=1 and a private log path of its own (a
   whole-lane env flip races under -j: the library opens that path "w"). It asserts
   rsp == 0 || rsp >= draws, which is the device evidence's shape - rsp ~= the draw count under
   inproc, 0 under monolith - so the draw path's retirement is visible as rsp ceasing to scale.

3. THE LANE HAD NO POSITIVE CONTROL (ID-115). Of the seven entries that passed strict, three were
   monolith transport - where nothing stamps a verb boundary, so the whole mechanism is
   structurally unreachable - two self-skipped, one was a death test and one was a Python check.
   Not one was a record-carrying split GL scenario, so "green" and "strict was never armed" were
   the same observation. New push-only counter `vbs` (PipeStats::ServerVerbBoundaries), counted
   at MGPipeServerStampVerbBoundary behind Enabled(), and the same entry asserts vbs > 0 on a
   frame that drew. It is deliberately NOT rsp: rsp reaches zero when the phase SUCCEEDS, so a
   control built on it would start failing on the day the debt is paid.

- Measured on this head (base 7c6f6886, before pa and mv): isplit 119/119; under strict the
  census is Fatal{ GetProgramForDraw@DrawArrays 52, GetBoundVertexArray@DrawArrays 9,
  GetProgramForDispatch@DispatchCompute 7 - exactly what pa and mv owe - and Admitted{ on ten
  pairs, three of them ADMITTED-ESCALATED. The expected-marker file carries all ten and says in
  its header that the wave-3 merge retires three of them.
- CONTRACT-P5E §7 rewritten around what is now checkable: the derived allowlist and its three
  disjuncts, the two-sided ratchet, the restated rsp pin, and the positive control.
This commit is contained in:
2026-09-18 12:30:10 -04:00
parent a0836568ac
commit bc6e24ff30
9 changed files with 507 additions and 43 deletions
@@ -33,6 +33,150 @@ def paths(document):
return split
# P5e (gl), ID-119: THE MARKER GRAMMAR, PARSED WHERE IT IS WRITTEN.
#
# MG_Backend/MGPipe/PipeInputs.cpp writes exactly two shapes and they differ only in the leading
# tag and the reason, which is what lets every filter written since P5c keep meaning "red":
#
# MGPipe: Fatal{UnmigratedPipeInput, "<F>@<V>"} [BARRIER-PULLED, <why>, retires in <phase>]
# MGPipe: Admitted{UnmigratedPipeInput, "<F>@<V>"} [BARRIER-PULLED, ADMITTED[-ESCALATED], retires in <phase>]
MARKER_RE = re.compile(
r"(Fatal|Admitted)\{UnmigratedPipeInput,\s*\"([^\"@]+)@([^\"]+)\"\}"
r"(?:\s*\[BARRIER-PULLED,\s*([^,\]]+))?")
def marker_log_paths(document):
"""{entry: absolute MOBILEGL_LOG_FILE_PATH} for EVERY entry that declares one.
P5e (gl), ID-119: THE LOG SET COMES FROM ctest, NOT FROM A DIRECTORY. The CI step used to
grep MobileGL/MG_IntegrationTest/split-logs/, which holds 90 of the lane's entries. The ones
it missed were the F1. readback block, both NamedBlit pairs, the four Ct. entries and
PersistentMapArm - i.e. precisely the readback population the allowlist is ABOUT, plus the
only DirectVulkan entries in the lane. A directory is a guess about where the lane put its
logs; `ctest --show-only=json-v1` is the lane saying where it put them.
Unlike paths() above this does not restrict itself to DirectGLES.Split. entries and does not
validate ownership - that is paths()' job and SplitLogPaths.PrivateAndDistinct runs it. This
one answers "which files might carry a marker", for every backend prefix the lane has."""
logs = {}
for test in document["tests"]:
props = {p["name"]: p["value"] for p in test.get("properties", [])}
values = [v.split("=", 1)[1] for v in props.get("ENVIRONMENT", [])
if v.startswith("MOBILEGL_LOG_FILE_PATH=")]
if len(values) != 1 or not values[0]:
continue
logs[test["name"]] = str(Path(values[0]).resolve())
return logs
def read_pair_set(path):
"""A committed `<field>@<verb>` set, one per line; blank lines and # comments ignored."""
pairs = set()
for line in Path(path).read_text().splitlines():
line = line.split("#", 1)[0].strip()
if line:
pairs.add(line)
return pairs
def classify_markers(logs):
"""(fatal, admitted, escalated, scanned) - each a {pair: sorted entry names}.
Counted BY ENTRY and not by line: the runtime dedupes each (field, verb) once per process,
so a line count would only ever say how many processes ran."""
fatal, admitted, escalated = {}, {}, {}
scanned = []
for name in sorted(logs):
path = logs[name]
if not Path(path).is_file():
continue
scanned.append(name)
text = Path(path).read_text(errors="replace")
for tag, field, verb, why in MARKER_RE.findall(text):
pair = f"{field}@{verb}"
if tag == "Fatal":
fatal.setdefault(pair, set()).add(name)
elif (why or "").strip() == "ADMITTED-ESCALATED":
escalated.setdefault(pair, set()).add(name)
else:
admitted.setdefault(pair, set()).add(name)
return ({p: sorted(e) for p, e in fatal.items()},
{p: sorted(e) for p, e in admitted.items()},
{p: sorted(e) for p, e in escalated.items()},
scanned)
def print_marker_table(title, table):
print(f" {title}: {len(table)} distinct pair(s)")
for pair in sorted(table, key=lambda p: (-len(table[p]), p)):
print(f" {len(table[pair]):4d} {pair}")
def markers(document, selector, allowed_path, expected_path, require_no_fatal):
"""THE TWO-SIDED RATCHET (ID-119).
One side: a marker that is not admitted fails the lane. That half existed in spirit but was
unreachable code, because strict aborted on every barrier-pulled read and the run's rc took
the step out before the comparison (ID-117 fixed the knob; this reads the result).
The other side, which is the new half: an admitted pair that NO LONGER APPEARS must be
removed from the expected set. Without it the lane rots green - a debt retires, its marker
stops being written, and the expected list quietly becomes a list of things that used to
happen, so the next regression to re-introduce one of them reads as "expected"."""
logs = {name: path for name, path in marker_log_paths(document).items()
if re.search(selector, name)}
if not logs:
raise ValueError(f"no lane entry matching /{selector}/ declares a "
"MOBILEGL_LOG_FILE_PATH, so there is nothing to read markers out of")
fatal, admitted, escalated, scanned = classify_markers(logs)
print(f"SplitLogPaths markers: {len(scanned)} of {len(logs)} selected entries wrote a log")
print_marker_table("Fatal{UnmigratedPipeInput", fatal)
print_marker_table("Admitted{UnmigratedPipeInput", admitted)
print_marker_table("Admitted{UnmigratedPipeInput (ESCALATED)", escalated)
if not scanned:
raise ValueError("not one selected entry wrote its private log: the marker census is "
"EMPTY because nothing was read, which is not the same statement as "
"'no marker fired' and must never be reported as one")
problems = []
if require_no_fatal and fatal:
problems.append("%d Fatal marker(s) - each is a field an apply still reads out of client "
"memory with nothing admitting it: %s"
% (len(fatal), "; ".join(f"{p} ({len(e)} entries, e.g. {e[0]})"
for p, e in sorted(fatal.items()))))
# An ADMITTED (non-escalated) marker claims the GENERATED table admitted it, so it must be in
# that table. An ADMITTED-ESCALATED one does not: it is a runtime fact about the record's
# payload (an open XFB span, a draw with client arrays) that no table indexed by
# (field, verb) can carry, so looking for it in a static list would mean widening that list
# with every pair that could ever escalate - which would forgive the ordinary draw path too.
allowed = read_pair_set(allowed_path)
outside = sorted(set(admitted) - allowed)
if outside:
problems.append("%d marker(s) tagged ADMITTED are not in the generated allowlist, so the "
"committed PipeFieldOwnership.inc and the generator disagree: %s"
% (len(outside), ", ".join(outside)))
expected = read_pair_set(expected_path)
observed = set(admitted) | set(escalated)
appeared = sorted(observed - expected)
vanished = sorted(expected - observed)
if appeared:
problems.append("%d admitted pair(s) the lane has not seen before: %s. Each is a debt "
"somebody now owes - add it to %s with the phase that retires it, or "
"retire it."
% (len(appeared), ", ".join(appeared), Path(expected_path).name))
if vanished:
problems.append("%d expected pair(s) no longer appear: %s. Remove them from %s in the "
"commit that retired them - an expected set that keeps rows nothing "
"writes any more is how this lane rots green."
% (len(vanished), ", ".join(vanished), Path(expected_path).name))
if problems:
raise ValueError("the strict lane's marker ratchet: " + " | ".join(problems))
print(f"SplitLogPaths markers: ratchet OK - {len(observed)} admitted pair(s), "
f"{len(fatal)} fatal pair(s)")
def main():
mode = sys.argv[1]
if mode == "check":
@@ -41,6 +185,10 @@ def main():
selected = paths(json.loads(data))
print(f"SplitLogPaths: {len(selected)} entries, {len(set(selected.values()))} distinct private paths")
return
if mode == "markers":
markers(json.loads(Path(sys.argv[2]).read_text()), sys.argv[3], sys.argv[4], sys.argv[5],
len(sys.argv) > 6 and sys.argv[6] == "no-fatal")
return
selected = paths(json.loads(Path(sys.argv[2]).read_text()))
selected = {name: path for name, path in selected.items() if re.search(sys.argv[3], name)}
if not selected:
@@ -0,0 +1,45 @@
# The admitted barrier-pull markers the integration-split lane ACTUALLY WRITES under
# MOBILEGL_IPC_STRICT_ERRORS=1, one `<field>@<verb>` per line.
#
# P5e (gl), ruling ID-119. THIS IS THE OTHER SIDE OF THE RATCHET, and it is the side that was
# missing. The generated allowlist (scripts/gen_pipe_field_ownership.py --print-admitted) says
# which pairs MAY be admitted - it is a statement about the tables. This file says which ones the
# lane DOES produce - a statement about the run. A lane checked only against the first kind
# ROTS GREEN: a debt retires, its marker stops being written, and nothing notices, so the day a
# regression re-introduces that read the lane calls it "expected".
#
# So Harness/split_log_paths.py's `markers` mode fails BOTH ways:
# - a pair here that no longer appears -> delete it, in the commit that retired it;
# - a pair that appears and is not here -> add it with the phase that owes it, or retire it.
#
# HOW TO REFRESH (never by hand from a guess):
# ctest -L integration-split --show-only=json-v1 > lane.json
# MOBILEGL_IPC_STRICT_ERRORS=1 ctest -L integration-split -j 4
# python3 MobileGL/MG_IntegrationTest/Harness/split_log_paths.py markers lane.json . \
# <(python3 scripts/gen_pipe_field_ownership.py --print-admitted) this-file
# and put the census it prints here.
#
# ---------------------------------------------------------------------------------------------
# MEASURED AT p5e/gl's BASE, 7c6f6886, WHICH IS BEFORE pa AND mv LANDED. That is stated rather
# than hidden: the wave-3 integration commit retires GetProgramForDraw@DrawArrays,
# GetProgramForDispatch@DispatchCompute and the ordinary GetBoundVertexArray@DrawArrays, so the
# three escalated rows below WILL stop appearing and the ratchet will say so by name. Removing
# them is part of that merge, not a failure of it.
# ---------------------------------------------------------------------------------------------
#
# Admitted by the generated table (disjuncts 1 and 2 of ID-116 / ID-125):
GetBufferBindingSlot@ReadPixels
GetFramebufferBindingSlot@ReadPixels
GetProgramObject@ShaderStorageBlockBinding
GetTextureObject@CopyImageSubData
GetTextureUnitObject@CopyTexImage2D
GetTransformFeedbackProgram@DrawArrays
ValidateProgramName@ShaderStorageBlockBinding
#
# Admitted by ESCALATION only (disjunct 3, ID-128): the record was barriered because its payload
# said so - an open transform-feedback span, or a draw carrying client vertex arrays - which no
# table indexed by (field, verb) can see. All three are pulls on the draw verb by entries that
# are NOT the ordinary draw path.
GetBoundVertexArray@DrawArrays
GetBufferBindingPoint@DrawArrays
GetProgramForDraw@DrawArrays